2024-04-26T04:25:57.180 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-26T04:25:57.180 DEBUG:teuthology.run:Teuthology command: teuthology --name teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi --description upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} --verbose --archive /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558 --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558/orig.config.yaml 2024-04-26T04:25:57.236 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T04:25:57.318 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558 branch: squid description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7674558' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi meta: - desc: 'setup ceph/v18.2.0 ' name: teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: mon pg warn min per osd: 0 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - OSD_DOWN - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - MON_DOWN sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a kclient: syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - host.a - client.0 - osd.0 - osd.1 - osd.2 - - host.b - client.1 - osd.3 - osd.4 - osd.5 seed: 8320 sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a sleep_before_teardown: 0 subset: 10/32 suite: upgrade suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a targets: smithi090.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKHvUOgerVAl90xiwIzvDp+nuUqtVXE1yLDbcIeLEIUHwetE3XyAa5a+ASLhlDKWIr/Rnj6I21VQQbXcc1cVz0o= smithi104.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKK9lyyZR7LZtO6iswJJ+sZtpIzaBpQ6RIdU3ckT5N1Y0XyA6PEr2lSO+4gLdNQ5UvJYOoE5wDf1b932mwtckaY= tasks: - install: exclude_packages: - ceph-volume tag: v18.2.0 - print: '**** done install task...' - cephadm: compiled_cephadm_branch: reef conf: osd: osd_class_default_list: '*' osd_class_load_list: '*' image: quay.io/ceph/ceph:v18.2.0 roleless: true - print: '**** done end installing v18.2.0 cephadm ...' - cephadm.shell: host.a: - ceph config set mgr mgr/cephadm/use_repo_digest true --force - print: '**** done cephadm.shell ceph config set mgr...' - cephadm.shell: host.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: host.a: - ceph fs volume create cephfs --placement=4 - ceph fs dump - cephadm.shell: host.a: - ceph fs set cephfs max_mds 2 - cephadm.shell: host.a: - ceph fs set cephfs allow_standby_replay false - cephadm.shell: host.a: - ceph fs set cephfs inline_data false - cephadm.shell: host.a: - ceph fs dump - ceph --format=json fs dump | jq -e ".filesystems | length == 1" - while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done - fs.pre_upgrade_save: null - kclient: null - print: '**** done client' - parallel: - upgrade-tasks - workload-tasks - cephadm.shell: host.a: - ceph fs dump - fs.post_upgrade_checks: null teuthology: fragments_dropped: [] meta: {} postmerge: [] variables: fail_fs: true teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_02:08:13 tube: smithi upgrade-tasks: sequential: - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done - ceph versions | jq -e '.mgr | length == 1' - ceph versions | jq -e '.mgr | keys' | grep $sha1 - ceph versions | jq -e '.overall | length == 2' - ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '.up_to_date | length == 2' - ceph orch ps - cephadm.shell: env: - sha1 host.a: - ceph config set mgr mgr/orchestrator/fail_fs true - cephadm.shell: env: - sha1 host.a: - ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force - ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force - ceph config set global log_to_journald false --force - ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 - cephadm.shell: env: - sha1 host.a: - while ceph orch upgrade status | jq '.in_progress' | grep true && ! ceph orch upgrade status | jq '.message' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done - ceph orch ps - ceph orch upgrade status - ceph health detail - ceph versions - echo "wait for servicemap items w/ changing names to refresh" - sleep 60 - ceph orch ps - ceph versions - ceph versions | jq -e '.overall | length == 1' - ceph versions | jq -e '.overall | keys' | grep $sha1 user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 workload-tasks: sequential: - workunit: clients: all: - suites/fsstress.sh 2024-04-26T04:25:57.318 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa; will attempt to use it 2024-04-26T04:25:57.319 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks 2024-04-26T04:25:57.319 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-26T04:25:57.321 INFO:teuthology.task.internal:Checking packages... 2024-04-26T04:25:57.344 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a' 2024-04-26T04:25:57.344 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-26T04:25:57.344 INFO:teuthology.packaging:ref: None 2024-04-26T04:25:57.344 INFO:teuthology.packaging:tag: None 2024-04-26T04:25:57.344 INFO:teuthology.packaging:branch: squid 2024-04-26T04:25:57.344 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:25:57.345 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-26T04:25:57.670 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2447.gb22e2ebd 2024-04-26T04:25:57.672 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-26T04:25:57.678 INFO:teuthology.task.internal:no buildpackages task found 2024-04-26T04:25:57.678 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-26T04:25:57.701 INFO:teuthology.task.internal:Saving configuration 2024-04-26T04:25:57.725 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-26T04:25:57.730 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-26T04:25:57.753 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi090.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:20:15.852683', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKHvUOgerVAl90xiwIzvDp+nuUqtVXE1yLDbcIeLEIUHwetE3XyAa5a+ASLhlDKWIr/Rnj6I21VQQbXcc1cVz0o='} 2024-04-26T04:25:57.777 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi104.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-26 04:20:15.854152', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKK9lyyZR7LZtO6iswJJ+sZtpIzaBpQ6RIdU3ckT5N1Y0XyA6PEr2lSO+4gLdNQ5UvJYOoE5wDf1b932mwtckaY='} 2024-04-26T04:25:57.778 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-26T04:25:57.783 INFO:teuthology.task.internal:roles: ubuntu@smithi090.front.sepia.ceph.com - ['host.a', 'client.0', 'osd.0', 'osd.1', 'osd.2'] 2024-04-26T04:25:57.783 INFO:teuthology.task.internal:roles: ubuntu@smithi104.front.sepia.ceph.com - ['host.b', 'client.1', 'osd.3', 'osd.4', 'osd.5'] 2024-04-26T04:25:57.783 INFO:teuthology.run_tasks:Running task console_log... 2024-04-26T04:25:57.855 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f04aeb050d0>, signals=[15]) 2024-04-26T04:25:57.855 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-26T04:25:57.863 INFO:teuthology.task.internal:Opening connections... 2024-04-26T04:25:57.863 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:25:57.867 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:25:57.951 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi104.front.sepia.ceph.com 2024-04-26T04:25:57.952 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:25:58.025 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-26T04:25:58.030 DEBUG:teuthology.orchestra.run.smithi090:> uname -m 2024-04-26T04:25:58.048 INFO:teuthology.orchestra.run.smithi090.stdout:x86_64 2024-04-26T04:25:58.048 DEBUG:teuthology.orchestra.run.smithi090:> cat /etc/os-release 2024-04-26T04:25:58.105 INFO:teuthology.orchestra.run.smithi090.stdout:NAME="CentOS Stream" 2024-04-26T04:25:58.105 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION="9" 2024-04-26T04:25:58.105 INFO:teuthology.orchestra.run.smithi090.stdout:ID="centos" 2024-04-26T04:25:58.105 INFO:teuthology.orchestra.run.smithi090.stdout:ID_LIKE="rhel fedora" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:VERSION_ID="9" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:PLATFORM_ID="platform:el9" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:ANSI_COLOR="0;31" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:LOGO="fedora-logo-icon" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:HOME_URL="https://centos.org/" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T04:25:58.106 INFO:teuthology.orchestra.run.smithi090.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T04:25:58.107 INFO:teuthology.lock.ops:Updating smithi090.front.sepia.ceph.com on lock server 2024-04-26T04:25:58.129 DEBUG:teuthology.orchestra.run.smithi104:> uname -m 2024-04-26T04:25:58.147 INFO:teuthology.orchestra.run.smithi104.stdout:x86_64 2024-04-26T04:25:58.147 DEBUG:teuthology.orchestra.run.smithi104:> cat /etc/os-release 2024-04-26T04:25:58.204 INFO:teuthology.orchestra.run.smithi104.stdout:NAME="CentOS Stream" 2024-04-26T04:25:58.204 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION="9" 2024-04-26T04:25:58.204 INFO:teuthology.orchestra.run.smithi104.stdout:ID="centos" 2024-04-26T04:25:58.204 INFO:teuthology.orchestra.run.smithi104.stdout:ID_LIKE="rhel fedora" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:VERSION_ID="9" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:PLATFORM_ID="platform:el9" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:ANSI_COLOR="0;31" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:LOGO="fedora-logo-icon" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:HOME_URL="https://centos.org/" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-26T04:25:58.205 INFO:teuthology.orchestra.run.smithi104.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-26T04:25:58.206 INFO:teuthology.lock.ops:Updating smithi104.front.sepia.ceph.com on lock server 2024-04-26T04:25:58.229 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-26T04:25:58.237 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-26T04:25:58.241 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-26T04:25:58.242 DEBUG:teuthology.orchestra.run.smithi090:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:25:58.244 DEBUG:teuthology.orchestra.run.smithi104:> test '!' -e /home/ubuntu/cephtest 2024-04-26T04:25:58.260 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-26T04:25:58.265 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-26T04:25:58.265 DEBUG:teuthology.orchestra.run.smithi090:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:25:58.301 DEBUG:teuthology.orchestra.run.smithi104:> test -z $(ls -A /var/lib/ceph) 2024-04-26T04:25:58.434 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-26T04:25:58.489 INFO:teuthology.run_tasks:Running task kernel... 2024-04-26T04:25:58.502 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-26T04:25:58.502 INFO:teuthology.task.kernel:config {'host.a': {'kdb': 1, 'sha1': 'distro'}, 'host.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-26T04:25:58.502 DEBUG:teuthology.orchestra.run.smithi090:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:25:58.503 DEBUG:teuthology.orchestra.run.smithi104:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-26T04:25:58.519 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:25:58.519 DEBUG:teuthology.orchestra.run.smithi090:> uname -r 2024-04-26T04:25:58.520 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:25:58.520 DEBUG:teuthology.orchestra.run.smithi104:> uname -r 2024-04-26T04:25:58.577 INFO:teuthology.orchestra.run.smithi090.stdout:5.14.0-437.el9.x86_64 2024-04-26T04:25:58.577 INFO:teuthology.task.kernel:Running kernel on smithi090: 5.14.0-437.el9.x86_64 2024-04-26T04:25:58.577 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum install -y kernel 2024-04-26T04:25:58.578 INFO:teuthology.orchestra.run.smithi104.stdout:5.14.0-437.el9.x86_64 2024-04-26T04:25:58.578 INFO:teuthology.task.kernel:Running kernel on smithi104: 5.14.0-437.el9.x86_64 2024-04-26T04:25:58.578 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum install -y kernel 2024-04-26T04:26:01.180 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.0 MB 00:00 2024-04-26T04:26:01.474 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.0 MB 00:00 2024-04-26T04:26:03.098 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-26T04:26:03.468 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 20 MB/s | 19 MB 00:00 2024-04-26T04:26:07.390 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 8.3 MB/s | 6.2 MB 00:00 2024-04-26T04:26:07.585 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 11 MB/s | 6.2 MB 00:00 2024-04-26T04:26:08.952 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-04-26T04:26:09.046 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-26T04:26:10.206 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-26T04:26:15.263 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T04:26:15.687 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 3.2 MB/s | 21 MB 00:06 2024-04-26T04:26:17.144 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T04:26:17.145 INFO:teuthology.orchestra.run.smithi104.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T04:26:17.233 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:26:17.233 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:26:17.234 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:26:17.357 DEBUG:teuthology.orchestra.run.smithi104:> echo no | sudo yum reinstall kernel || true 2024-04-26T04:26:17.853 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:02 ago on Fri 26 Apr 2024 04:26:15 AM UTC. 2024-04-26T04:26:17.972 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout:Reinstalling: 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:26:17.981 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:26:17.982 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:26:17.982 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 255 k 2024-04-26T04:26:17.982 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 0 2024-04-26T04:26:17.982 INFO:teuthology.orchestra.run.smithi104.stderr:Operation aborted. 2024-04-26T04:26:18.040 INFO:teuthology.orchestra.run.smithi104.stdout:Is this ok [y/N]: 2024-04-26T04:26:18.040 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum reinstall -y kernel || true 2024-04-26T04:26:18.536 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 04:26:15 AM UTC. 2024-04-26T04:26:18.654 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:26:18.654 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:26:18.654 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout:Reinstalling: 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 255 k 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 0 2024-04-26T04:26:18.655 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:26:18.849 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.0 MB/s | 255 kB 00:00 2024-04-26T04:26:18.849 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:26:18.849 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.3 MB/s | 255 kB 00:00 2024-04-26T04:26:18.868 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:26:19.075 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:26:19.075 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:26:19.080 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:26:19.080 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:26:19.209 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:26:19.291 INFO:teuthology.orchestra.run.smithi104.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:26:19.291 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:26:19.916 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:26:19.916 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:26:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:26:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:26:20.577 INFO:teuthology.orchestra.run.smithi104.stdout:Reinstalled: 2024-04-26T04:26:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T04:26:20.577 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:26:20.577 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:26:20.744 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T04:26:20.754 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-26T04:26:20.791 INFO:teuthology.orchestra.run.smithi104.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T04:26:20.792 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T04:26:20.792 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T04:26:20.792 INFO:teuthology.task.kernel:Enabling kdb on host.b... 2024-04-26T04:26:20.792 DEBUG:teuthology.orchestra.run.smithi104:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:26:20.826 INFO:teuthology.orchestra.run.smithi104.stdout:ttyS1 2024-04-26T04:26:20.847 DEBUG:teuthology.parallel:result is None 2024-04-26T04:26:22.661 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-26T04:26:22.661 INFO:teuthology.orchestra.run.smithi090.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-26T04:26:22.768 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:26:22.769 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:26:22.769 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:26:22.871 DEBUG:teuthology.orchestra.run.smithi090:> echo no | sudo yum reinstall kernel || true 2024-04-26T04:26:23.369 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:03 ago on Fri 26 Apr 2024 04:26:20 AM UTC. 2024-04-26T04:26:23.494 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:26:23.511 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalling: 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 255 k 2024-04-26T04:26:23.512 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 0 2024-04-26T04:26:23.513 INFO:teuthology.orchestra.run.smithi090.stderr:Operation aborted. 2024-04-26T04:26:23.599 INFO:teuthology.orchestra.run.smithi090.stdout:Is this ok [y/N]: 2024-04-26T04:26:23.600 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum reinstall -y kernel || true 2024-04-26T04:26:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:04 ago on Fri 26 Apr 2024 04:26:20 AM UTC. 2024-04-26T04:26:24.208 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:26:24.209 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:26:24.209 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalling: 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 255 k 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 0 2024-04-26T04:26:24.210 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:26:24.465 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.9 MB/s | 255 kB 00:00 2024-04-26T04:26:24.466 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:26:24.466 INFO:teuthology.orchestra.run.smithi090.stdout:Total 994 kB/s | 255 kB 00:00 2024-04-26T04:26:24.485 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:26:24.689 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:26:24.689 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:26:24.694 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:26:24.694 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:26:24.833 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:26:24.917 INFO:teuthology.orchestra.run.smithi090.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:26:24.917 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:26:25.772 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:26:25.773 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-26T04:26:26.534 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-26T04:26:26.534 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:26:26.534 INFO:teuthology.orchestra.run.smithi090.stdout:Reinstalled: 2024-04-26T04:26:26.534 INFO:teuthology.orchestra.run.smithi090.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-26T04:26:26.534 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:26:26.534 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:26:26.749 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q kernel | sort -rV | head -n 1 2024-04-26T04:26:26.837 INFO:teuthology.orchestra.run.smithi090.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-26T04:26:26.837 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-26T04:26:26.837 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-26T04:26:26.838 INFO:teuthology.task.kernel:Enabling kdb on host.a... 2024-04-26T04:26:26.838 DEBUG:teuthology.orchestra.run.smithi090:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-26T04:26:26.871 INFO:teuthology.orchestra.run.smithi090.stdout:ttyS1 2024-04-26T04:26:26.893 DEBUG:teuthology.parallel:result is None 2024-04-26T04:26:26.893 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-26T04:26:26.900 INFO:teuthology.task.internal:Creating test directory... 2024-04-26T04:26:26.901 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:26:26.935 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-26T04:26:26.953 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-26T04:26:26.960 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-26T04:26:26.967 INFO:teuthology.task.internal:Creating archive directory... 2024-04-26T04:26:26.967 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:26:26.990 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-26T04:26:27.023 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-26T04:26:27.030 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-26T04:26:27.030 DEBUG:teuthology.orchestra.run.smithi090:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:26:27.055 DEBUG:teuthology.orchestra.run.smithi104:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-26T04:26:27.095 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:26:27.106 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:26:27.108 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:26:27.118 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-26T04:26:27.119 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-26T04:26:27.126 INFO:teuthology.task.internal:Configuring sudo... 2024-04-26T04:26:27.127 DEBUG:teuthology.orchestra.run.smithi090:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:26:27.129 DEBUG:teuthology.orchestra.run.smithi104:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-26T04:26:27.179 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-26T04:26:27.188 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-26T04:26:27.188 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:26:27.192 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-26T04:26:27.237 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:26:27.284 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:26:27.329 DEBUG:teuthology.orchestra.run.smithi090:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:26:27.409 DEBUG:teuthology.orchestra.run.smithi090:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:26:27.449 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:26:27.449 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:26:27.512 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:26:27.550 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-26T04:26:27.610 DEBUG:teuthology.orchestra.run.smithi104:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:26:27.689 DEBUG:teuthology.orchestra.run.smithi104:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-26T04:26:27.735 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:26:27.735 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-26T04:26:27.800 DEBUG:teuthology.orchestra.run.smithi090:> sudo service rsyslog restart 2024-04-26T04:26:27.802 DEBUG:teuthology.orchestra.run.smithi104:> sudo service rsyslog restart 2024-04-26T04:26:27.836 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T04:26:27.877 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T04:26:28.299 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-26T04:26:28.306 INFO:teuthology.task.internal:Starting timer... 2024-04-26T04:26:28.307 INFO:teuthology.run_tasks:Running task pcp... 2024-04-26T04:26:28.323 INFO:teuthology.run_tasks:Running task selinux... 2024-04-26T04:26:28.333 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-04-26T04:26:28.334 DEBUG:teuthology.orchestra.run.smithi090:> sudo service auditd rotate 2024-04-26T04:26:28.390 INFO:teuthology.orchestra.run.smithi090.stdout:Rotating logs: 2024-04-26T04:26:28.392 DEBUG:teuthology.orchestra.run.smithi104:> sudo service auditd rotate 2024-04-26T04:26:28.443 INFO:teuthology.orchestra.run.smithi104.stdout:Rotating logs: 2024-04-26T04:26:28.445 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-26T04:26:28.445 DEBUG:teuthology.orchestra.run.smithi090:> /usr/sbin/getenforce 2024-04-26T04:26:28.479 INFO:teuthology.orchestra.run.smithi090.stdout:Permissive 2024-04-26T04:26:28.479 DEBUG:teuthology.orchestra.run.smithi104:> /usr/sbin/getenforce 2024-04-26T04:26:28.501 INFO:teuthology.orchestra.run.smithi104.stdout:Permissive 2024-04-26T04:26:28.501 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi090.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi104.front.sepia.ceph.com': 'permissive'} 2024-04-26T04:26:28.501 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T04:26:28.546 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:26:28.546 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T04:26:28.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:26:28.571 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-26T04:26:28.572 DEBUG:teuthology.orchestra.run.smithi090:> sudo /usr/sbin/setenforce permissive 2024-04-26T04:26:28.621 DEBUG:teuthology.orchestra.run.smithi104:> sudo /usr/sbin/setenforce permissive 2024-04-26T04:26:28.644 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-26T04:26:28.654 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-26T04:26:28.673 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-26T04:26:28.674 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi090.front.sepia.ceph.com,smithi104.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-26T04:30:00.314 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi090.front.sepia.ceph.com'), Remote(name='ubuntu@smithi104.front.sepia.ceph.com')] 2024-04-26T04:30:00.315 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-26T04:30:00.316 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi090.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:30:00.392 DEBUG:teuthology.orchestra.run.smithi090:> true 2024-04-26T04:30:00.469 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi090.front.sepia.ceph.com' 2024-04-26T04:30:00.469 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-26T04:30:00.470 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi104.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-26T04:30:00.544 DEBUG:teuthology.orchestra.run.smithi104:> true 2024-04-26T04:30:00.625 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi104.front.sepia.ceph.com' 2024-04-26T04:30:00.625 INFO:teuthology.run_tasks:Running task clock... 2024-04-26T04:30:00.636 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-26T04:30:00.636 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:30:00.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:30:00.639 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-26T04:30:00.639 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:30:00.664 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T04:30:00.678 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T04:30:00.693 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-26T04:30:00.700 INFO:teuthology.orchestra.run.smithi090.stderr:sudo: ntpd: command not found 2024-04-26T04:30:00.705 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-26T04:30:00.710 INFO:teuthology.orchestra.run.smithi090.stdout:506 Cannot talk to daemon 2024-04-26T04:30:00.722 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T04:30:00.727 INFO:teuthology.orchestra.run.smithi104.stderr:sudo: ntpd: command not found 2024-04-26T04:30:00.735 INFO:teuthology.orchestra.run.smithi090.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T04:30:00.738 INFO:teuthology.orchestra.run.smithi104.stdout:506 Cannot talk to daemon 2024-04-26T04:30:00.751 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-26T04:30:00.764 INFO:teuthology.orchestra.run.smithi104.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-26T04:30:00.792 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-26T04:30:00.797 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T04:30:00.797 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-26T04:30:00.797 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.797 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.797 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.797 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.815 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-04-26T04:30:00.818 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T04:30:00.818 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-04-26T04:30:00.818 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.818 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.818 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.818 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:30:00.819 INFO:teuthology.run_tasks:Running task install... 2024-04-26T04:30:00.828 DEBUG:teuthology.task.install:project ceph 2024-04-26T04:30:00.828 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}} 2024-04-26T04:30:00.828 DEBUG:teuthology.task.install:config {'exclude_packages': ['ceph-volume'], 'tag': 'v18.2.0', 'flavor': 'default', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'} 2024-04-26T04:30:00.828 INFO:teuthology.task.install:Using flavor: default 2024-04-26T04:30:00.835 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-26T04:30:00.835 INFO:teuthology.task.install:extra packages: [] 2024-04-26T04:30:00.835 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-04-26T04:30:00.836 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-26T04:30:00.836 INFO:teuthology.packaging:ref: None 2024-04-26T04:30:00.836 INFO:teuthology.packaging:tag: v18.2.0 2024-04-26T04:30:00.836 INFO:teuthology.packaging:branch: None 2024-04-26T04:30:00.836 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:30:03.225 DEBUG:teuthology.repo_utils:git ls-remote https://git.ceph.com/ceph v18.2.0^{} -> 5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-26T04:30:03.226 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-26T04:30:03.229 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': ['ceph-volume'], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'tag': 'v18.2.0', 'wait_for_package': False} 2024-04-26T04:30:03.229 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-26T04:30:03.229 INFO:teuthology.packaging:ref: None 2024-04-26T04:30:03.229 INFO:teuthology.packaging:tag: v18.2.0 2024-04-26T04:30:03.229 INFO:teuthology.packaging:branch: None 2024-04-26T04:30:03.229 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:30:03.230 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-26T04:30:03.379 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-04-26T04:30:03.379 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-04-26T04:30:03.383 INFO:teuthology.task.install.rpm:Pulling from https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/ 2024-04-26T04:30:03.383 INFO:teuthology.task.install.rpm:Package version is 18.2.0-0 2024-04-26T04:30:03.430 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T04:30:03.430 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:30:03.430 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T04:30:03.438 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://chacra.ceph.com/r/ceph/reef/5dd24139a1eada541a3bc16b6941c5dde975e26d/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-26T04:30:03.438 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:30:03.438 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-26T04:30:03.460 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T04:30:03.460 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-26T04:30:03.460 INFO:teuthology.packaging:ref: None 2024-04-26T04:30:03.460 INFO:teuthology.packaging:tag: v18.2.0 2024-04-26T04:30:03.461 INFO:teuthology.packaging:branch: None 2024-04-26T04:30:03.461 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:30:03.461 DEBUG:teuthology.orchestra.run.smithi090:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T04:30:03.467 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-26T04:30:03.467 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-26T04:30:03.467 INFO:teuthology.packaging:ref: None 2024-04-26T04:30:03.467 INFO:teuthology.packaging:tag: v18.2.0 2024-04-26T04:30:03.467 INFO:teuthology.packaging:branch: None 2024-04-26T04:30:03.467 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:30:03.467 DEBUG:teuthology.orchestra.run.smithi104:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;ref/v18.2.0/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-26T04:30:03.529 DEBUG:teuthology.orchestra.run.smithi090:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T04:30:03.536 DEBUG:teuthology.orchestra.run.smithi104:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-26T04:30:03.606 DEBUG:teuthology.orchestra.run.smithi090:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T04:30:03.613 DEBUG:teuthology.orchestra.run.smithi104:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-26T04:30:03.671 INFO:teuthology.orchestra.run.smithi090.stdout:check_obsoletes = 1 2024-04-26T04:30:03.672 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum clean all 2024-04-26T04:30:03.678 INFO:teuthology.orchestra.run.smithi104.stdout:check_obsoletes = 1 2024-04-26T04:30:03.680 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum clean all 2024-04-26T04:30:03.898 INFO:teuthology.orchestra.run.smithi090.stdout:48 files removed 2024-04-26T04:30:03.909 INFO:teuthology.orchestra.run.smithi104.stdout:48 files removed 2024-04-26T04:30:03.925 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-radosgw 2024-04-26T04:30:03.935 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-radosgw 2024-04-26T04:30:04.374 INFO:teuthology.orchestra.run.smithi090.stdout:ceph packages for x86_64 400 kB/s | 76 kB 00:00 2024-04-26T04:30:04.394 INFO:teuthology.orchestra.run.smithi104.stdout:ceph packages for x86_64 379 kB/s | 76 kB 00:00 2024-04-26T04:30:04.583 INFO:teuthology.orchestra.run.smithi090.stdout:ceph noarch packages 56 kB/s | 9.3 kB 00:00 2024-04-26T04:30:04.595 INFO:teuthology.orchestra.run.smithi104.stdout:ceph noarch packages 59 kB/s | 9.3 kB 00:00 2024-04-26T04:30:04.775 INFO:teuthology.orchestra.run.smithi090.stdout:ceph source packages 14 kB/s | 2.2 kB 00:00 2024-04-26T04:30:04.795 INFO:teuthology.orchestra.run.smithi104.stdout:ceph source packages 13 kB/s | 2.2 kB 00:00 2024-04-26T04:30:05.498 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.0 MB 00:00 2024-04-26T04:30:05.704 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - BaseOS 9.0 MB/s | 8.0 MB 00:00 2024-04-26T04:30:07.789 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:01 2024-04-26T04:30:07.882 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - AppStream 14 MB/s | 19 MB 00:01 2024-04-26T04:30:12.024 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - CRB 9.3 MB/s | 6.2 MB 00:00 2024-04-26T04:30:12.249 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.2 MB 00:00 2024-04-26T04:30:13.706 INFO:teuthology.orchestra.run.smithi090.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T04:30:13.914 INFO:teuthology.orchestra.run.smithi104.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 16 kB 00:00 2024-04-26T04:30:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-04-26T04:30:16.398 INFO:teuthology.orchestra.run.smithi104.stdout:Extra Packages for Enterprise Linux 8.8 MB/s | 21 MB 00:02 2024-04-26T04:30:20.117 INFO:teuthology.orchestra.run.smithi090.stdout:lab-extras 26 kB/s | 1.7 kB 00:00 2024-04-26T04:30:21.506 INFO:teuthology.orchestra.run.smithi104.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-26T04:30:22.257 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrading: 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-04-26T04:30:22.259 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-26T04:30:22.260 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:30:22.261 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:22.262 INFO:teuthology.orchestra.run.smithi090.stdout:Install 28 Packages 2024-04-26T04:30:22.262 INFO:teuthology.orchestra.run.smithi090.stdout:Upgrade 2 Packages 2024-04-26T04:30:22.262 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:22.262 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 53 M 2024-04-26T04:30:22.262 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:30:22.986 INFO:teuthology.orchestra.run.smithi090.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 21 MB/s | 5.2 MB 00:00 2024-04-26T04:30:23.028 INFO:teuthology.orchestra.run.smithi090.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 578 kB/s | 24 kB 00:00 2024-04-26T04:30:23.095 INFO:teuthology.orchestra.run.smithi090.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 9.6 MB/s | 653 kB 00:00 2024-04-26T04:30:23.262 INFO:teuthology.orchestra.run.smithi090.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 2.8 MB/s | 474 kB 00:00 2024-04-26T04:30:23.496 INFO:teuthology.orchestra.run.smithi090.stdout:(5/30): ceph-common-18.2.0-0.el9.x86_64.rpm 24 MB/s | 18 MB 00:00 2024-04-26T04:30:23.604 INFO:teuthology.orchestra.run.smithi090.stdout:(6/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 8.8 MB/s | 7.6 MB 00:00 2024-04-26T04:30:23.627 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw x86_64 2:18.2.0-0.el9 ceph 7.6 M 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrading: 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout: librados2 x86_64 2:18.2.0-0.el9 ceph 3.3 M 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout: librbd1 x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-26T04:30:23.629 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base x86_64 2:18.2.0-0.el9 ceph 5.2 M 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux x86_64 2:18.2.0-0.el9 ceph 24 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: fmt x86_64 8.1.1-5.el9 epel 111 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2 x86_64 2:18.2.0-0.el9 ceph 653 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1 x86_64 2:18.2.0-0.el9 ceph 474 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2 x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-26T04:30:23.630 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse x86_64 2:18.2.0-0.el9 ceph 45 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common x86_64 2:18.2.0-0.el9 ceph 119 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs x86_64 2:18.2.0-0.el9 ceph 155 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados x86_64 2:18.2.0-0.el9 ceph 321 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd x86_64 2:18.2.0-0.el9 ceph 297 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw x86_64 2:18.2.0-0.el9 ceph 99 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:30:23.631 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:23.632 INFO:teuthology.orchestra.run.smithi104.stdout:Install 28 Packages 2024-04-26T04:30:23.632 INFO:teuthology.orchestra.run.smithi104.stdout:Upgrade 2 Packages 2024-04-26T04:30:23.632 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:23.632 INFO:teuthology.orchestra.run.smithi090.stdout:(7/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 336 kB/s | 45 kB 00:00 2024-04-26T04:30:23.632 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 53 M 2024-04-26T04:30:23.633 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:30:23.655 INFO:teuthology.orchestra.run.smithi090.stdout:(8/30): python3-ceph-common-18.2.0-0.el9.x86_64 2.3 MB/s | 119 kB 00:00 2024-04-26T04:30:23.681 INFO:teuthology.orchestra.run.smithi090.stdout:(9/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-26T04:30:23.714 INFO:teuthology.orchestra.run.smithi090.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 5.3 MB/s | 321 kB 00:00 2024-04-26T04:30:23.739 INFO:teuthology.orchestra.run.smithi090.stdout:(11/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 5.0 MB/s | 297 kB 00:00 2024-04-26T04:30:23.773 INFO:teuthology.orchestra.run.smithi090.stdout:(12/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 1.7 MB/s | 99 kB 00:00 2024-04-26T04:30:23.840 INFO:teuthology.orchestra.run.smithi090.stdout:(13/30): librgw2-18.2.0-0.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-04-26T04:30:23.982 INFO:teuthology.orchestra.run.smithi090.stdout:(14/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 387 kB/s | 45 kB 00:00 2024-04-26T04:30:24.007 INFO:teuthology.orchestra.run.smithi090.stdout:(15/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 684 kB/s | 160 kB 00:00 2024-04-26T04:30:24.124 INFO:teuthology.orchestra.run.smithi090.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 4.6 MB/s | 662 kB 00:00 2024-04-26T04:30:24.150 INFO:teuthology.orchestra.run.smithi090.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-04-26T04:30:24.232 INFO:teuthology.orchestra.run.smithi104.stdout:(1/30): ceph-base-18.2.0-0.el9.x86_64.rpm 29 MB/s | 5.2 MB 00:00 2024-04-26T04:30:24.265 INFO:teuthology.orchestra.run.smithi104.stdout:(2/30): ceph-selinux-18.2.0-0.el9.x86_64.rpm 723 kB/s | 24 kB 00:00 2024-04-26T04:30:24.308 INFO:teuthology.orchestra.run.smithi104.stdout:(3/30): libcephfs2-18.2.0-0.el9.x86_64.rpm 15 MB/s | 653 kB 00:00 2024-04-26T04:30:24.341 INFO:teuthology.orchestra.run.smithi104.stdout:(4/30): libradosstriper1-18.2.0-0.el9.x86_64.rp 14 MB/s | 474 kB 00:00 2024-04-26T04:30:24.466 INFO:teuthology.orchestra.run.smithi104.stdout:(5/30): ceph-radosgw-18.2.0-0.el9.x86_64.rpm 18 MB/s | 7.6 MB 00:00 2024-04-26T04:30:24.491 INFO:teuthology.orchestra.run.smithi104.stdout:(6/30): python3-ceph-argparse-18.2.0-0.el9.x86_ 1.8 MB/s | 45 kB 00:00 2024-04-26T04:30:24.508 INFO:teuthology.orchestra.run.smithi090.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 309 kB/s | 111 kB 00:00 2024-04-26T04:30:24.550 INFO:teuthology.orchestra.run.smithi104.stdout:(7/30): librgw2-18.2.0-0.el9.x86_64.rpm 21 MB/s | 4.4 MB 00:00 2024-04-26T04:30:24.558 INFO:teuthology.orchestra.run.smithi090.stdout:(19/30): boost-program-options-1.75.0-8.el9.x86 130 kB/s | 107 kB 00:00 2024-04-26T04:30:24.575 INFO:teuthology.orchestra.run.smithi104.stdout:(8/30): python3-ceph-common-18.2.0-0.el9.x86_64 1.4 MB/s | 119 kB 00:00 2024-04-26T04:30:24.601 INFO:teuthology.orchestra.run.smithi104.stdout:(9/30): python3-cephfs-18.2.0-0.el9.x86_64.rpm 3.0 MB/s | 155 kB 00:00 2024-04-26T04:30:24.626 INFO:teuthology.orchestra.run.smithi104.stdout:(10/30): python3-rados-18.2.0-0.el9.x86_64.rpm 6.3 MB/s | 321 kB 00:00 2024-04-26T04:30:24.651 INFO:teuthology.orchestra.run.smithi104.stdout:(11/30): python3-rbd-18.2.0-0.el9.x86_64.rpm 5.8 MB/s | 297 kB 00:00 2024-04-26T04:30:24.667 INFO:teuthology.orchestra.run.smithi090.stdout:(20/30): python3-prettytable-0.7.2-27.el9.noarc 78 kB/s | 42 kB 00:00 2024-04-26T04:30:24.676 INFO:teuthology.orchestra.run.smithi104.stdout:(12/30): python3-rgw-18.2.0-0.el9.x86_64.rpm 1.9 MB/s | 99 kB 00:00 2024-04-26T04:30:24.692 INFO:teuthology.orchestra.run.smithi090.stdout:(21/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.6 MB/s | 309 kB 00:00 2024-04-26T04:30:24.776 INFO:teuthology.orchestra.run.smithi090.stdout:(22/30): liboath-2.6.7-2.el9.x86_64.rpm 588 kB/s | 49 kB 00:00 2024-04-26T04:30:24.851 INFO:teuthology.orchestra.run.smithi090.stdout:(23/30): libunwind-1.6.2-1.el9.x86_64.rpm 896 kB/s | 67 kB 00:00 2024-04-26T04:30:24.902 INFO:teuthology.orchestra.run.smithi104.stdout:(13/30): ceph-common-18.2.0-0.el9.x86_64.rpm 22 MB/s | 18 MB 00:00 2024-04-26T04:30:24.927 INFO:teuthology.orchestra.run.smithi090.stdout:(24/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 100 kB/s | 26 kB 00:00 2024-04-26T04:30:24.994 INFO:teuthology.orchestra.run.smithi104.stdout:(14/30): libpmemobj-1.12.1-1.el9.x86_64.rpm 505 kB/s | 160 kB 00:00 2024-04-26T04:30:25.019 INFO:teuthology.orchestra.run.smithi104.stdout:(15/30): librabbitmq-0.11.0-7.el9.x86_64.rpm 492 kB/s | 45 kB 00:00 2024-04-26T04:30:25.027 INFO:teuthology.orchestra.run.smithi090.stdout:(25/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.7 MB/s | 839 kB 00:00 2024-04-26T04:30:25.154 INFO:teuthology.orchestra.run.smithi090.stdout:(26/30): re2-20211101-3.el9.x86_64.rpm 846 kB/s | 192 kB 00:00 2024-04-26T04:30:25.194 INFO:teuthology.orchestra.run.smithi104.stdout:(16/30): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-04-26T04:30:25.195 INFO:teuthology.orchestra.run.smithi090.stdout:(27/30): thrift-0.15.0-2.el9.x86_64.rpm 9.4 MB/s | 1.6 MB 00:00 2024-04-26T04:30:25.253 INFO:teuthology.orchestra.run.smithi104.stdout:(17/30): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.2 MB/s | 292 kB 00:00 2024-04-26T04:30:25.279 INFO:teuthology.orchestra.run.smithi090.stdout:(28/30): libarrow-9.0.0-11.el9.x86_64.rpm 6.1 MB/s | 4.4 MB 00:00 2024-04-26T04:30:25.363 INFO:teuthology.orchestra.run.smithi090.stdout:(29/30): librbd1-18.2.0-0.el9.x86_64.rpm 18 MB/s | 3.0 MB 00:00 2024-04-26T04:30:25.421 INFO:teuthology.orchestra.run.smithi090.stdout:(30/30): librados2-18.2.0-0.el9.x86_64.rpm 12 MB/s | 3.3 MB 00:00 2024-04-26T04:30:25.428 INFO:teuthology.orchestra.run.smithi104.stdout:(18/30): fmt-8.1.1-5.el9.x86_64.rpm 633 kB/s | 111 kB 00:00 2024-04-26T04:30:25.428 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:30:25.429 INFO:teuthology.orchestra.run.smithi090.stdout:Total 17 MB/s | 53 MB 00:03 2024-04-26T04:30:25.493 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:30:25.528 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:30:25.528 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:30:25.528 INFO:teuthology.orchestra.run.smithi104.stdout:(19/30): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.0 MB/s | 309 kB 00:00 2024-04-26T04:30:25.728 INFO:teuthology.orchestra.run.smithi104.stdout:(20/30): libarrow-9.0.0-11.el9.x86_64.rpm 22 MB/s | 4.4 MB 00:00 2024-04-26T04:30:25.754 INFO:teuthology.orchestra.run.smithi104.stdout:(21/30): boost-program-options-1.75.0-8.el9.x86 97 kB/s | 107 kB 00:01 2024-04-26T04:30:25.779 INFO:teuthology.orchestra.run.smithi104.stdout:(22/30): python3-prettytable-0.7.2-27.el9.noarc 72 kB/s | 42 kB 00:00 2024-04-26T04:30:25.821 INFO:teuthology.orchestra.run.smithi104.stdout:(23/30): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-26T04:30:25.846 INFO:teuthology.orchestra.run.smithi104.stdout:(24/30): liboath-2.6.7-2.el9.x86_64.rpm 729 kB/s | 49 kB 00:00 2024-04-26T04:30:25.863 INFO:teuthology.orchestra.run.smithi104.stdout:(25/30): libunwind-1.6.2-1.el9.x86_64.rpm 808 kB/s | 67 kB 00:00 2024-04-26T04:30:25.896 INFO:teuthology.orchestra.run.smithi104.stdout:(26/30): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-04-26T04:30:25.922 INFO:teuthology.orchestra.run.smithi104.stdout:(27/30): re2-20211101-3.el9.x86_64.rpm 3.3 MB/s | 192 kB 00:00 2024-04-26T04:30:25.950 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:30:25.955 INFO:teuthology.orchestra.run.smithi104.stdout:(28/30): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-04-26T04:30:25.970 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:30:26.030 INFO:teuthology.orchestra.run.smithi104.stdout:(29/30): librbd1-18.2.0-0.el9.x86_64.rpm 28 MB/s | 3.0 MB 00:00 2024-04-26T04:30:26.090 INFO:teuthology.orchestra.run.smithi104.stdout:(30/30): librados2-18.2.0-0.el9.x86_64.rpm 17 MB/s | 3.3 MB 00:00 2024-04-26T04:30:26.093 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:30:26.094 INFO:teuthology.orchestra.run.smithi104.stdout:Total 21 MB/s | 53 MB 00:02 2024-04-26T04:30:26.160 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:30:26.195 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:30:26.195 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:30:26.613 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:30:26.613 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:30:27.372 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:30:27.465 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-26T04:30:27.502 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-26T04:30:27.560 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-26T04:30:27.919 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-26T04:30:28.019 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:30:28.098 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/32 2024-04-26T04:30:28.127 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/32 2024-04-26T04:30:28.152 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-26T04:30:28.176 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/32 2024-04-26T04:30:28.253 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-26T04:30:28.341 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-04-26T04:30:28.372 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-26T04:30:28.434 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-26T04:30:28.471 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-26T04:30:28.485 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/32 2024-04-26T04:30:28.514 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-04-26T04:30:28.580 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-26T04:30:28.593 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-26T04:30:28.608 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-26T04:30:28.657 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-26T04:30:28.685 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:18.2.0-0.el9.x86_64 5/32 2024-04-26T04:30:28.764 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rados-2:18.2.0-0.el9.x86_64 6/32 2024-04-26T04:30:28.770 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-26T04:30:28.787 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-26T04:30:28.798 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-04-26T04:30:28.838 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-26T04:30:28.848 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs2-2:18.2.0-0.el9.x86_64 7/32 2024-04-26T04:30:28.872 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-26T04:30:28.877 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : liboath-2.6.7-2.el9.x86_64 8/32 2024-04-26T04:30:28.908 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-04-26T04:30:28.912 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 9/32 2024-04-26T04:30:28.944 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-26T04:30:28.978 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-26T04:30:28.978 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-26T04:30:29.005 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-26T04:30:29.070 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libradosstriper1-2:18.2.0-0.el9.x86_64 11/32 2024-04-26T04:30:29.177 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : re2-1:20211101-3.el9.x86_64 12/32 2024-04-26T04:30:29.212 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 13/32 2024-04-26T04:30:29.251 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/32 2024-04-26T04:30:29.274 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-26T04:30:29.277 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/32 2024-04-26T04:30:29.304 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 16/32 2024-04-26T04:30:29.333 INFO:teuthology.orchestra.run.smithi090.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-26T04:30:29.340 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fmt-8.1.1-5.el9.x86_64 17/32 2024-04-26T04:30:29.373 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 18/32 2024-04-26T04:30:29.447 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-26T04:30:29.479 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-04-26T04:30:29.515 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-26T04:30:29.670 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 19/32 2024-04-26T04:30:29.711 INFO:teuthology.orchestra.run.smithi104.stdout: Upgrading : librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-26T04:30:29.801 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:18.2.0-0.el9.x86_64 20/32 2024-04-26T04:30:29.832 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rbd-2:18.2.0-0.el9.x86_64 21/32 2024-04-26T04:30:29.868 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 22/32 2024-04-26T04:30:29.909 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-04-26T04:30:29.935 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-26T04:30:30.007 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-26T04:30:30.082 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-04-26T04:30:30.113 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-04-26T04:30:30.260 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 23/32 2024-04-26T04:30:30.288 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-26T04:30:30.358 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librgw2-2:18.2.0-0.el9.x86_64 24/32 2024-04-26T04:30:30.426 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rgw-2:18.2.0-0.el9.x86_64 25/32 2024-04-26T04:30:30.457 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ceph-common-2:18.2.0-0.el9.x86_64 26/32 2024-04-26T04:30:32.207 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-26T04:30:32.254 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-26T04:30:32.500 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-26T04:30:32.539 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-26T04:30:32.792 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-26T04:30:32.884 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-26T04:30:32.986 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-26T04:30:32.986 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T04:30:32.986 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T04:30:32.986 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:33.018 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-26T04:30:33.078 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-common-2:18.2.0-0.el9.x86_64 27/32 2024-04-26T04:30:33.117 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-26T04:30:33.212 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-base-2:18.2.0-0.el9.x86_64 28/32 2024-04-26T04:30:33.213 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-26T04:30:33.213 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-26T04:30:33.213 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:33.243 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /sys 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /proc 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /mnt 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /var/tmp 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /home 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /root 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout:skipping the directory /tmp 2024-04-26T04:30:42.709 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:43.246 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-selinux-2:18.2.0-0.el9.x86_64 29/32 2024-04-26T04:30:43.246 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /sys 2024-04-26T04:30:43.246 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /proc 2024-04-26T04:30:43.246 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /mnt 2024-04-26T04:30:43.247 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /var/tmp 2024-04-26T04:30:43.247 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /home 2024-04-26T04:30:43.247 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /root 2024-04-26T04:30:43.247 INFO:teuthology.orchestra.run.smithi090.stdout:skipping the directory /tmp 2024-04-26T04:30:43.247 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:43.456 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-26T04:30:43.481 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-26T04:30:43.481 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:30:43.481 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T04:30:43.481 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:43.481 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:43.481 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:43.483 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-26T04:30:43.592 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-26T04:30:43.592 INFO:teuthology.orchestra.run.smithi104.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-26T04:30:43.993 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-26T04:30:44.017 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-radosgw-2:18.2.0-0.el9.x86_64 30/32 2024-04-26T04:30:44.017 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:30:44.017 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-26T04:30:44.017 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:44.017 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-26T04:30:44.017 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:44.020 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-26T04:30:44.106 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 31/32 2024-04-26T04:30:44.106 INFO:teuthology.orchestra.run.smithi090.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-26T04:30:44.733 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-04-26T04:30:44.734 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-26T04:30:44.735 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-26T04:30:44.735 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-04-26T04:30:44.735 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-26T04:30:44.735 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-26T04:30:44.735 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-04-26T04:30:44.735 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-26T04:30:44.735 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-04-26T04:30:45.466 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-26T04:30:45.466 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout:Upgraded: 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.467 INFO:teuthology.orchestra.run.smithi104.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.468 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:45.469 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:30:45.491 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 32/32 2024-04-26T04:30:45.491 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-base-2:18.2.0-0.el9.x86_64 1/32 2024-04-26T04:30:45.491 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-common-2:18.2.0-0.el9.x86_64 2/32 2024-04-26T04:30:45.491 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-radosgw-2:18.2.0-0.el9.x86_64 3/32 2024-04-26T04:30:45.491 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-selinux-2:18.2.0-0.el9.x86_64 4/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs2-2:18.2.0-0.el9.x86_64 5/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libradosstriper1-2:18.2.0-0.el9.x86_64 6/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librgw2-2:18.2.0-0.el9.x86_64 7/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-argparse-2:18.2.0-0.el9.x86_64 8/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ceph-common-2:18.2.0-0.el9.x86_64 9/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cephfs-2:18.2.0-0.el9.x86_64 10/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rados-2:18.2.0-0.el9.x86_64 11/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rbd-2:18.2.0-0.el9.x86_64 12/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rgw-2:18.2.0-0.el9.x86_64 13/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/32 2024-04-26T04:30:45.492 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/32 2024-04-26T04:30:45.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/32 2024-04-26T04:30:45.493 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/32 2024-04-26T04:30:45.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fmt-8.1.1-5.el9.x86_64 20/32 2024-04-26T04:30:45.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 21/32 2024-04-26T04:30:45.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 22/32 2024-04-26T04:30:45.494 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 23/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 24/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 25/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 26/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : re2-1:20211101-3.el9.x86_64 27/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 28/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:18.2.0-0.el9.x86_64 29/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 30/32 2024-04-26T04:30:45.495 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:18.2.0-0.el9.x86_64 31/32 2024-04-26T04:30:45.743 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-test 2024-04-26T04:30:46.236 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 32/32 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout:Upgraded: 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: librados2-2:18.2.0-0.el9.x86_64 librbd1-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-base-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-common-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-radosgw-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-selinux-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: fmt-8.1.1-5.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-26T04:30:46.237 INFO:teuthology.orchestra.run.smithi090.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs2-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: libradosstriper1-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: librgw2-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-argparse-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ceph-common-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.238 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cephfs-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rados-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rbd-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rgw-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout: re2-1:20211101-3.el9.x86_64 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:46.239 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:30:46.272 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:25 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:30:46.392 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:46.393 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:30:46.394 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:46.394 INFO:teuthology.orchestra.run.smithi104.stdout:Install 4 Packages 2024-04-26T04:30:46.394 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:46.394 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 40 M 2024-04-26T04:30:46.394 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 165 M 2024-04-26T04:30:46.394 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:30:46.566 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-test 2024-04-26T04:30:47.172 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:27 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:30:47.198 INFO:teuthology.orchestra.run.smithi104.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 306 kB/s | 161 kB 00:00 2024-04-26T04:30:47.292 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:30:47.293 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:47.293 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:30:47.293 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:47.293 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:30:47.293 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test x86_64 2:18.2.0-0.el9 ceph 40 M 2024-04-26T04:30:47.294 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:30:47.294 INFO:teuthology.orchestra.run.smithi090.stdout: libcephsqlite x86_64 2:18.2.0-0.el9 ceph 161 k 2024-04-26T04:30:47.294 INFO:teuthology.orchestra.run.smithi090.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-26T04:30:47.294 INFO:teuthology.orchestra.run.smithi090.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-26T04:30:47.294 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:47.295 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:30:47.295 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:47.295 INFO:teuthology.orchestra.run.smithi090.stdout:Install 4 Packages 2024-04-26T04:30:47.295 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:47.295 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 40 M 2024-04-26T04:30:47.296 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 165 M 2024-04-26T04:30:47.296 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:30:47.518 INFO:teuthology.orchestra.run.smithi090.stdout:(1/4): libcephsqlite-18.2.0-0.el9.x86_64.rpm 1.6 MB/s | 161 kB 00:00 2024-04-26T04:30:47.867 INFO:teuthology.orchestra.run.smithi104.stdout:(2/4): ceph-test-18.2.0-0.el9.x86_64.rpm 33 MB/s | 40 MB 00:01 2024-04-26T04:30:47.984 INFO:teuthology.orchestra.run.smithi104.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 232 kB/s | 305 kB 00:01 2024-04-26T04:30:48.026 INFO:teuthology.orchestra.run.smithi104.stdout:(4/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 77 kB/s | 64 kB 00:00 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:30:48.029 INFO:teuthology.orchestra.run.smithi104.stdout:Total 25 MB/s | 40 MB 00:01 2024-04-26T04:30:48.056 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:30:48.075 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:30:48.075 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:30:48.226 INFO:teuthology.orchestra.run.smithi090.stdout:(2/4): ceph-test-18.2.0-0.el9.x86_64.rpm 49 MB/s | 40 MB 00:00 2024-04-26T04:30:48.397 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:30:48.398 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:30:48.586 INFO:teuthology.orchestra.run.smithi090.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 60 kB/s | 64 kB 00:01 2024-04-26T04:30:48.861 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:30:48.945 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T04:30:49.002 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T04:30:49.030 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-26T04:30:49.193 INFO:teuthology.orchestra.run.smithi090.stdout:(4/4): socat-1.7.4.1-5.el9.x86_64.rpm 172 kB/s | 305 kB 00:01 2024-04-26T04:30:49.198 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:30:49.198 INFO:teuthology.orchestra.run.smithi090.stdout:Total 21 MB/s | 40 MB 00:01 2024-04-26T04:30:49.224 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:30:49.246 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:30:49.246 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:30:49.568 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:30:49.568 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:30:50.034 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:30:50.121 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-26T04:30:50.171 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-26T04:30:50.198 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-26T04:30:52.693 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-26T04:30:52.767 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-26T04:30:53.931 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephsqlite-2:18.2.0-0.el9.x86_64 3/4 2024-04-26T04:30:54.047 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-26T04:30:54.746 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-26T04:30:54.746 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-04-26T04:30:54.746 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-04-26T04:30:54.746 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T04:30:54.991 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T04:30:54.992 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:54.992 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:30:54.992 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:54.992 INFO:teuthology.orchestra.run.smithi104.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T04:30:54.992 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:54.992 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:30:55.267 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph 2024-04-26T04:30:55.773 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:34 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:30:55.900 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:30:55.902 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T04:30:55.903 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T04:30:55.904 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:30:55.905 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:30:55.906 INFO:teuthology.orchestra.run.smithi104.stdout:Install 33 Packages 2024-04-26T04:30:55.906 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:30:55.906 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 30 M 2024-04-26T04:30:55.906 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 109 M 2024-04-26T04:30:55.906 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:30:56.588 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-test-2:18.2.0-0.el9.x86_64 4/4 2024-04-26T04:30:56.588 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-test-2:18.2.0-0.el9.x86_64 1/4 2024-04-26T04:30:56.588 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephsqlite-2:18.2.0-0.el9.x86_64 2/4 2024-04-26T04:30:56.588 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-26T04:30:56.753 INFO:teuthology.orchestra.run.smithi104.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 50 kB/s | 6.4 kB 00:00 2024-04-26T04:30:56.820 INFO:teuthology.orchestra.run.smithi104.stdout:(2/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 11 MB/s | 2.1 MB 00:00 2024-04-26T04:30:56.843 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-26T04:30:56.843 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:56.844 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:30:56.844 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-test-2:18.2.0-0.el9.x86_64 libcephsqlite-2:18.2.0-0.el9.x86_64 2024-04-26T04:30:56.844 INFO:teuthology.orchestra.run.smithi090.stdout: socat-1.7.4.1-5.el9.x86_64 xmlstarlet-1.6.1-20.el9.x86_64 2024-04-26T04:30:56.844 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:56.844 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:30:56.870 INFO:teuthology.orchestra.run.smithi104.stdout:(3/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 5.9 MB/s | 1.4 MB 00:00 2024-04-26T04:30:57.161 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph 2024-04-26T04:30:57.370 INFO:teuthology.orchestra.run.smithi104.stdout:(4/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 32 MB/s | 18 MB 00:00 2024-04-26T04:30:57.395 INFO:teuthology.orchestra.run.smithi104.stdout:(5/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 457 kB/s | 240 kB 00:00 2024-04-26T04:30:57.487 INFO:teuthology.orchestra.run.smithi104.stdout:(6/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 6.0 MB/s | 4.4 MB 00:00 2024-04-26T04:30:57.629 INFO:teuthology.orchestra.run.smithi104.stdout:(7/33): python3-ply-3.11-14.el9.noarch.rpm 753 kB/s | 106 kB 00:00 2024-04-26T04:30:57.667 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:37 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:30:57.671 INFO:teuthology.orchestra.run.smithi104.stdout:(8/33): python3-pycparser-2.20-6.el9.noarch.rpm 3.2 MB/s | 135 kB 00:00 2024-04-26T04:30:57.713 INFO:teuthology.orchestra.run.smithi104.stdout:(9/33): python3-requests-2.25.1-8.el9.noarch.rp 2.9 MB/s | 125 kB 00:00 2024-04-26T04:30:57.795 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout: ceph x86_64 2:18.2.0-0.el9 ceph 6.4 k 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds x86_64 2:18.2.0-0.el9 ceph 2.1 M 2024-04-26T04:30:57.797 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr x86_64 2:18.2.0-0.el9 ceph 1.4 M 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core noarch 2:18.2.0-0.el9 ceph-noarch 240 k 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon x86_64 2:18.2.0-0.el9 ceph 4.4 M 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd x86_64 2:18.2.0-0.el9 ceph 18 M 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-26T04:30:57.798 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-26T04:30:57.799 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout:Install 33 Packages 2024-04-26T04:30:57.800 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:30:57.801 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 30 M 2024-04-26T04:30:57.801 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 109 M 2024-04-26T04:30:57.801 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:30:58.488 INFO:teuthology.orchestra.run.smithi104.stdout:(10/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 227 kB/s | 253 kB 00:01 2024-04-26T04:30:58.621 INFO:teuthology.orchestra.run.smithi104.stdout:(11/33): python3-urllib3-1.26.5-5.el9.noarch.rp 236 kB/s | 215 kB 00:00 2024-04-26T04:30:58.652 INFO:teuthology.orchestra.run.smithi090.stdout:(1/33): ceph-18.2.0-0.el9.x86_64.rpm 53 kB/s | 6.4 kB 00:00 2024-04-26T04:30:58.703 INFO:teuthology.orchestra.run.smithi090.stdout:(2/33): ceph-mgr-18.2.0-0.el9.x86_64.rpm 8.4 MB/s | 1.4 MB 00:00 2024-04-26T04:30:58.747 INFO:teuthology.orchestra.run.smithi104.stdout:(12/33): python3-mako-1.1.4-6.el9.noarch.rpm 667 kB/s | 172 kB 00:00 2024-04-26T04:30:58.754 INFO:teuthology.orchestra.run.smithi090.stdout:(3/33): ceph-mds-18.2.0-0.el9.x86_64.rpm 9.4 MB/s | 2.1 MB 00:00 2024-04-26T04:30:58.788 INFO:teuthology.orchestra.run.smithi090.stdout:(4/33): ceph-mgr-modules-core-18.2.0-0.el9.noar 7.0 MB/s | 240 kB 00:00 2024-04-26T04:30:58.814 INFO:teuthology.orchestra.run.smithi104.stdout:(13/33): python3-markupsafe-1.1.1-12.el9.x86_64 181 kB/s | 35 kB 00:00 2024-04-26T04:30:58.864 INFO:teuthology.orchestra.run.smithi104.stdout:(14/33): python3-cryptography-36.0.1-4.el9.x86_ 855 kB/s | 1.2 MB 00:01 2024-04-26T04:30:58.880 INFO:teuthology.orchestra.run.smithi090.stdout:(5/33): ceph-mon-18.2.0-0.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2024-04-26T04:30:58.964 INFO:teuthology.orchestra.run.smithi104.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 235 kB/s | 51 kB 00:00 2024-04-26T04:30:59.006 INFO:teuthology.orchestra.run.smithi104.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 217 kB/s | 42 kB 00:00 2024-04-26T04:30:59.181 INFO:teuthology.orchestra.run.smithi104.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-04-26T04:30:59.205 INFO:teuthology.orchestra.run.smithi090.stdout:(6/33): ceph-osd-18.2.0-0.el9.x86_64.rpm 35 MB/s | 18 MB 00:00 2024-04-26T04:30:59.282 INFO:teuthology.orchestra.run.smithi104.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 107 kB/s | 11 kB 00:00 2024-04-26T04:30:59.399 INFO:teuthology.orchestra.run.smithi104.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 151 kB/s | 18 kB 00:00 2024-04-26T04:30:59.414 INFO:teuthology.orchestra.run.smithi090.stdout:(7/33): python3-cryptography-36.0.1-4.el9.x86_6 2.3 MB/s | 1.2 MB 00:00 2024-04-26T04:30:59.457 INFO:teuthology.orchestra.run.smithi104.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 349 kB/s | 172 kB 00:00 2024-04-26T04:30:59.516 INFO:teuthology.orchestra.run.smithi104.stdout:(21/33): python3-jaraco-collections-3.0.0-8.el9 198 kB/s | 23 kB 00:00 2024-04-26T04:30:59.649 INFO:teuthology.orchestra.run.smithi104.stdout:(22/33): python3-cherrypy-18.6.1-2.el9.noarch.r 558 kB/s | 358 kB 00:00 2024-04-26T04:30:59.674 INFO:teuthology.orchestra.run.smithi104.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 90 kB/s | 19 kB 00:00 2024-04-26T04:30:59.700 INFO:teuthology.orchestra.run.smithi104.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 107 kB/s | 20 kB 00:00 2024-04-26T04:30:59.733 INFO:teuthology.orchestra.run.smithi104.stdout:(25/33): python3-logutils-0.3.5-21.el9.noarch.r 556 kB/s | 46 kB 00:00 2024-04-26T04:30:59.834 INFO:teuthology.orchestra.run.smithi104.stdout:(26/33): python3-portend-3.1.0-2.el9.noarch.rpm 163 kB/s | 16 kB 00:00 2024-04-26T04:30:59.892 INFO:teuthology.orchestra.run.smithi104.stdout:(27/33): python3-more-itertools-8.12.0-2.el9.no 410 kB/s | 79 kB 00:00 2024-04-26T04:30:59.934 INFO:teuthology.orchestra.run.smithi104.stdout:(28/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.1 MB/s | 272 kB 00:00 2024-04-26T04:30:59.979 INFO:teuthology.orchestra.run.smithi104.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 620 kB/s | 90 kB 00:00 2024-04-26T04:31:00.018 INFO:teuthology.orchestra.run.smithi104.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 285 kB/s | 36 kB 00:00 2024-04-26T04:31:00.060 INFO:teuthology.orchestra.run.smithi104.stdout:(31/33): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-26T04:31:00.118 INFO:teuthology.orchestra.run.smithi104.stdout:(32/33): python3-zc-lockfile-2.0-10.el9.noarch. 199 kB/s | 20 kB 00:00 2024-04-26T04:31:00.152 INFO:teuthology.orchestra.run.smithi104.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.4 MB/s | 427 kB 00:00 2024-04-26T04:31:00.159 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:00.160 INFO:teuthology.orchestra.run.smithi104.stdout:Total 7.0 MB/s | 30 MB 00:04 2024-04-26T04:31:00.189 INFO:teuthology.orchestra.run.smithi090.stdout:(8/33): python3-cffi-1.14.5-5.el9.x86_64.rpm 181 kB/s | 253 kB 00:01 2024-04-26T04:31:00.276 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:00.290 INFO:teuthology.orchestra.run.smithi090.stdout:(9/33): python3-ply-3.11-14.el9.noarch.rpm 98 kB/s | 106 kB 00:01 2024-04-26T04:31:00.315 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:00.315 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:00.374 INFO:teuthology.orchestra.run.smithi090.stdout:(10/33): python3-pycparser-2.20-6.el9.noarch.rp 141 kB/s | 135 kB 00:00 2024-04-26T04:31:00.424 INFO:teuthology.orchestra.run.smithi090.stdout:(11/33): python3-requests-2.25.1-8.el9.noarch.r 534 kB/s | 125 kB 00:00 2024-04-26T04:31:00.532 INFO:teuthology.orchestra.run.smithi090.stdout:(12/33): python3-urllib3-1.26.5-5.el9.noarch.rp 886 kB/s | 215 kB 00:00 2024-04-26T04:31:00.599 INFO:teuthology.orchestra.run.smithi090.stdout:(13/33): python3-mako-1.1.4-6.el9.noarch.rpm 763 kB/s | 172 kB 00:00 2024-04-26T04:31:00.612 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:00.614 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:00.625 INFO:teuthology.orchestra.run.smithi090.stdout:(14/33): python3-markupsafe-1.1.1-12.el9.x86_64 173 kB/s | 35 kB 00:00 2024-04-26T04:31:00.725 INFO:teuthology.orchestra.run.smithi090.stdout:(15/33): python3-pytz-2021.1-5.el9.noarch.rpm 265 kB/s | 51 kB 00:00 2024-04-26T04:31:00.784 INFO:teuthology.orchestra.run.smithi090.stdout:(16/33): python3-toml-0.10.2-6.el9.noarch.rpm 226 kB/s | 42 kB 00:00 2024-04-26T04:31:01.130 INFO:teuthology.orchestra.run.smithi090.stdout:(17/33): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 86 kB/s | 43 kB 00:00 2024-04-26T04:31:01.260 INFO:teuthology.orchestra.run.smithi090.stdout:(18/33): python3-jaraco-8.2.1-3.el9.noarch.rpm 83 kB/s | 11 kB 00:00 2024-04-26T04:31:01.385 INFO:teuthology.orchestra.run.smithi090.stdout:(19/33): python3-jaraco-classes-3.2.1-5.el9.noa 141 kB/s | 18 kB 00:00 2024-04-26T04:31:01.418 INFO:teuthology.orchestra.run.smithi090.stdout:(20/33): python3-cheroot-8.6.0-4.el9.noarch.rpm 257 kB/s | 172 kB 00:00 2024-04-26T04:31:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:(21/33): python3-cherrypy-18.6.1-2.el9.noarch.r 499 kB/s | 358 kB 00:00 2024-04-26T04:31:01.529 INFO:teuthology.orchestra.run.smithi090.stdout:(22/33): python3-jaraco-collections-3.0.0-8.el9 162 kB/s | 23 kB 00:00 2024-04-26T04:31:01.553 INFO:teuthology.orchestra.run.smithi090.stdout:(23/33): python3-jaraco-functools-3.5.0-2.el9.n 145 kB/s | 19 kB 00:00 2024-04-26T04:31:01.611 INFO:teuthology.orchestra.run.smithi090.stdout:(24/33): python3-jaraco-text-3.2.0-6.el9.noarch 180 kB/s | 20 kB 00:00 2024-04-26T04:31:01.687 INFO:teuthology.orchestra.run.smithi090.stdout:(25/33): python3-more-itertools-8.12.0-2.el9.no 589 kB/s | 79 kB 00:00 2024-04-26T04:31:01.713 INFO:teuthology.orchestra.run.smithi090.stdout:(26/33): python3-logutils-0.3.5-21.el9.noarch.r 250 kB/s | 46 kB 00:00 2024-04-26T04:31:01.764 INFO:teuthology.orchestra.run.smithi090.stdout:(27/33): python3-pecan-1.4.2-3.el9.noarch.rpm 1.8 MB/s | 272 kB 00:00 2024-04-26T04:31:01.765 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:01.806 INFO:teuthology.orchestra.run.smithi090.stdout:(28/33): python3-portend-3.1.0-2.el9.noarch.rpm 138 kB/s | 16 kB 00:00 2024-04-26T04:31:01.838 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-26T04:31:01.856 INFO:teuthology.orchestra.run.smithi090.stdout:(29/33): python3-pyOpenSSL-21.0.0-1.el9.noarch. 631 kB/s | 90 kB 00:00 2024-04-26T04:31:01.870 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-26T04:31:01.873 INFO:teuthology.orchestra.run.smithi090.stdout:(30/33): python3-tempora-5.0.0-2.el9.noarch.rpm 327 kB/s | 36 kB 00:00 2024-04-26T04:31:01.903 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-26T04:31:01.982 INFO:teuthology.orchestra.run.smithi090.stdout:(31/33): python3-zc-lockfile-2.0-10.el9.noarch. 184 kB/s | 20 kB 00:00 2024-04-26T04:31:01.994 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-26T04:31:02.073 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-26T04:31:02.113 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-26T04:31:02.132 INFO:teuthology.orchestra.run.smithi090.stdout:(32/33): python3-webob-1.8.7-6.el9.noarch.rpm 704 kB/s | 230 kB 00:00 2024-04-26T04:31:02.149 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-26T04:31:02.166 INFO:teuthology.orchestra.run.smithi090.stdout:(33/33): python3-werkzeug-2.0.3-3.el9.1.noarch. 1.3 MB/s | 427 kB 00:00 2024-04-26T04:31:02.172 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:02.172 INFO:teuthology.orchestra.run.smithi090.stdout:Total 6.8 MB/s | 30 MB 00:04 2024-04-26T04:31:02.178 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-26T04:31:02.213 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-26T04:31:02.246 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-26T04:31:02.286 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:02.287 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-26T04:31:02.322 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-26T04:31:02.324 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:02.324 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:02.356 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-26T04:31:02.394 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-26T04:31:02.467 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-26T04:31:02.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-26T04:31:02.609 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:02.618 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:02.684 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-26T04:31:02.746 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-26T04:31:02.785 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-26T04:31:02.842 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-26T04:31:02.897 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-26T04:31:03.082 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-26T04:31:03.131 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-26T04:31:03.208 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-26T04:31:03.296 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-26T04:31:03.337 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-26T04:31:03.453 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-26T04:31:03.606 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-04-26T04:31:03.632 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-26T04:31:03.657 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-26T04:31:03.657 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:03.657 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T04:31:03.657 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:31:03.657 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:31:03.657 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:03.784 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:03.852 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/33 2024-04-26T04:31:03.884 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/33 2024-04-26T04:31:03.926 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/33 2024-04-26T04:31:04.017 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/33 2024-04-26T04:31:04.095 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/33 2024-04-26T04:31:04.145 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 6/33 2024-04-26T04:31:04.180 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/33 2024-04-26T04:31:04.210 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/33 2024-04-26T04:31:04.238 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/33 2024-04-26T04:31:04.271 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/33 2024-04-26T04:31:04.311 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/33 2024-04-26T04:31:04.355 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/33 2024-04-26T04:31:04.390 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/33 2024-04-26T04:31:04.428 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/33 2024-04-26T04:31:04.501 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 15/33 2024-04-26T04:31:04.647 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 16/33 2024-04-26T04:31:04.729 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 17/33 2024-04-26T04:31:04.782 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 18/33 2024-04-26T04:31:04.829 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 19/33 2024-04-26T04:31:04.887 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-ply-3.11-14.el9.noarch 20/33 2024-04-26T04:31:04.951 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 21/33 2024-04-26T04:31:05.135 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 22/33 2024-04-26T04:31:05.165 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-26T04:31:05.191 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-26T04:31:05.191 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:05.191 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T04:31:05.191 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:31:05.191 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:31:05.192 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:05.201 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 23/33 2024-04-26T04:31:05.280 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 24/33 2024-04-26T04:31:05.367 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 25/33 2024-04-26T04:31:05.402 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 26/33 2024-04-26T04:31:05.517 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 27/33 2024-04-26T04:31:05.633 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-26T04:31:05.655 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-26T04:31:05.655 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:05.655 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T04:31:05.656 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:31:05.656 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:31:05.656 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:05.679 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 28/33 2024-04-26T04:31:05.713 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-26T04:31:05.737 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-2:18.2.0-0.el9.x86_64 29/33 2024-04-26T04:31:05.737 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:05.737 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-26T04:31:05.737 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:31:05.738 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-26T04:31:05.738 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:05.867 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-26T04:31:05.890 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-26T04:31:05.890 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:05.890 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T04:31:05.890 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:31:05.890 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:31:05.890 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:05.954 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-26T04:31:07.256 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-26T04:31:07.281 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-osd-2:18.2.0-0.el9.x86_64 30/33 2024-04-26T04:31:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-26T04:31:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:31:07.282 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-26T04:31:07.282 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:07.517 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-26T04:31:07.517 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-04-26T04:31:07.517 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-04-26T04:31:07.517 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-04-26T04:31:07.517 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-04-26T04:31:07.517 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-04-26T04:31:07.517 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-26T04:31:07.518 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-26T04:31:07.519 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-26T04:31:07.520 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-26T04:31:07.748 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-26T04:31:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mon-2:18.2.0-0.el9.x86_64 31/33 2024-04-26T04:31:07.771 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:07.771 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-26T04:31:07.771 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:31:07.771 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-26T04:31:07.771 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:07.974 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-26T04:31:07.997 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mds-2:18.2.0-0.el9.x86_64 32/33 2024-04-26T04:31:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-26T04:31:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:31:07.997 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-26T04:31:07.997 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:08.061 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T04:31:08.409 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T04:31:08.410 INFO:teuthology.orchestra.run.smithi104.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:08.411 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:08.752 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-base 2024-04-26T04:31:09.269 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:48 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:09.353 INFO:teuthology.orchestra.run.smithi104.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:09.388 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:09.389 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:09.389 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:09.455 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install cephadm 2024-04-26T04:31:09.970 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:48 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-2:18.2.0-0.el9.x86_64 33/33 2024-04-26T04:31:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-2:18.2.0-0.el9.x86_64 1/33 2024-04-26T04:31:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mds-2:18.2.0-0.el9.x86_64 2/33 2024-04-26T04:31:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-2:18.2.0-0.el9.x86_64 3/33 2024-04-26T04:31:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mon-2:18.2.0-0.el9.x86_64 4/33 2024-04-26T04:31:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-osd-2:18.2.0-0.el9.x86_64 5/33 2024-04-26T04:31:09.998 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 6/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 15/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 16/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 17/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 18/33 2024-04-26T04:31:09.999 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 19/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 20/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 21/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 22/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 23/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 24/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 25/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 26/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 27/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 28/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 29/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 30/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 31/33 2024-04-26T04:31:10.000 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 32/33 2024-04-26T04:31:10.090 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:10.090 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:10.090 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 209 k 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 200 k 2024-04-26T04:31:10.091 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:10.190 INFO:teuthology.orchestra.run.smithi104.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.1 MB/s | 209 kB 00:00 2024-04-26T04:31:10.190 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:10.191 INFO:teuthology.orchestra.run.smithi104.stdout:Total 2.0 MB/s | 209 kB 00:00 2024-04-26T04:31:10.191 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:10.196 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:10.196 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:10.198 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:10.199 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:10.278 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:10.675 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:10.759 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:10.919 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 33/33 2024-04-26T04:31:10.919 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:10.919 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:10.919 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:10.919 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mds-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:10.919 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:10.919 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-modules-core-2:18.2.0-0.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mon-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-osd-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-26T04:31:10.920 INFO:teuthology.orchestra.run.smithi090.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-ply-3.11-14.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-26T04:31:10.921 INFO:teuthology.orchestra.run.smithi090.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-26T04:31:10.922 INFO:teuthology.orchestra.run.smithi090.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-26T04:31:10.922 INFO:teuthology.orchestra.run.smithi090.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-26T04:31:10.922 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:10.922 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:11.114 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:11.341 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-base 2024-04-26T04:31:11.359 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:11.359 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:11.359 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:11.359 INFO:teuthology.orchestra.run.smithi104.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-04-26T04:31:11.359 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:11.359 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:11.568 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T04:31:11.862 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:51 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:11.943 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-base-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:11.984 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:11.985 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:11.985 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:12.055 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install cephadm 2024-04-26T04:31:12.079 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:51 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:12.201 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:12.202 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repo Size 2024-04-26T04:31:12.202 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 142 k 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 399 k 2024-04-26T04:31:12.203 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:12.303 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 1.4 MB/s | 142 kB 00:00 2024-04-26T04:31:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.4 MB/s | 142 kB 00:00 2024-04-26T04:31:12.304 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:12.312 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:12.371 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:12.506 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:12.564 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:52 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:12.605 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:12.629 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:12.629 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:12.629 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T04:31:12.629 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:12.682 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:12.683 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:12.683 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:12.683 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:12.683 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm noarch 2:18.2.0-0.el9 ceph-noarch 209 k 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 209 k 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 200 k 2024-04-26T04:31:12.684 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:12.755 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm-18.2.0-0.el9.noarch.rpm 2.9 MB/s | 209 kB 00:00 2024-04-26T04:31:12.755 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:12.755 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.8 MB/s | 209 kB 00:00 2024-04-26T04:31:12.755 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:12.760 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:12.760 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:12.763 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:12.763 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:12.829 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:13.282 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:13.375 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:13.552 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:13.552 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:13.552 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:13.552 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:13.553 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:13.553 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:13.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr 2024-04-26T04:31:13.786 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:14.047 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : cephadm-2:18.2.0-0.el9.noarch 1/1 2024-04-26T04:31:14.047 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:14.047 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:14.047 INFO:teuthology.orchestra.run.smithi090.stdout: cephadm-2:18.2.0-0.el9.noarch 2024-04-26T04:31:14.047 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:14.047 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:14.260 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-immutable-object-cache 2024-04-26T04:31:14.288 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:53 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:14.370 INFO:teuthology.orchestra.run.smithi104.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:14.412 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:14.412 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:14.413 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:14.473 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T04:31:14.767 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:54 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:14.887 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repo Size 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache x86_64 2:18.2.0-0.el9 ceph 142 k 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:14.888 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 142 k 2024-04-26T04:31:14.889 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 399 k 2024-04-26T04:31:14.889 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:14.954 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-immutable-object-cache-18.2.0-0.el9.x86_64 2.2 MB/s | 142 kB 00:00 2024-04-26T04:31:14.955 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:14.956 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.1 MB/s | 142 kB 00:00 2024-04-26T04:31:14.956 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:14.963 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:14.963 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:14.981 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:53 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:15.019 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:15.019 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:15.100 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:15.101 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:15.101 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-26T04:31:15.101 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:15.101 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout:Installing weak dependencies: 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:15.102 INFO:teuthology.orchestra.run.smithi104.stdout:Install 7 Packages 2024-04-26T04:31:15.103 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:15.103 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 2.0 M 2024-04-26T04:31:15.103 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 8.1 M 2024-04-26T04:31:15.103 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:15.180 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:15.258 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:15.282 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:15.282 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:15.282 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-26T04:31:15.283 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:15.340 INFO:teuthology.orchestra.run.smithi104.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 185 kB/s | 23 kB 00:00 2024-04-26T04:31:15.390 INFO:teuthology.orchestra.run.smithi104.stdout:(2/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 9.7 MB/s | 1.7 MB 00:00 2024-04-26T04:31:15.424 INFO:teuthology.orchestra.run.smithi104.stdout:(3/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 70 kB/s | 15 kB 00:00 2024-04-26T04:31:15.458 INFO:teuthology.orchestra.run.smithi104.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 77 kB/s | 9.0 kB 00:00 2024-04-26T04:31:15.558 INFO:teuthology.orchestra.run.smithi104.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 244 kB/s | 41 kB 00:00 2024-04-26T04:31:15.583 INFO:teuthology.orchestra.run.smithi104.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 194 kB/s | 31 kB 00:00 2024-04-26T04:31:15.700 INFO:teuthology.orchestra.run.smithi104.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 780 kB/s | 188 kB 00:00 2024-04-26T04:31:15.701 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:15.702 INFO:teuthology.orchestra.run.smithi104.stdout:Total 3.3 MB/s | 2.0 MB 00:00 2024-04-26T04:31:15.702 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:15.710 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:15.710 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:15.761 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:15.761 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:16.042 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:16.126 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-26T04:31:16.160 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-26T04:31:16.179 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:16.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:16.179 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:16.179 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-immutable-object-cache-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:16.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:16.179 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:16.214 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-26T04:31:16.247 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-26T04:31:16.283 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-04-26T04:31:16.418 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr 2024-04-26T04:31:16.513 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-04-26T04:31:16.545 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-26T04:31:16.935 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:56 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:17.016 INFO:teuthology.orchestra.run.smithi090.stdout:Package ceph-mgr-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:17.048 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-26T04:31:17.048 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-04-26T04:31:17.048 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-04-26T04:31:17.048 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-04-26T04:31:17.048 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-26T04:31:17.048 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-26T04:31:17.048 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-26T04:31:17.058 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:17.059 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:17.059 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:17.129 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-dashboard 2024-04-26T04:31:17.285 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:17.286 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:17.475 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T04:31:17.629 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:00:57 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:17.747 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard noarch 2:18.2.0-0.el9 ceph-noarch 1.7 M 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards noarch 2:18.2.0-0.el9 ceph-noarch 23 k 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts noarch 2:18.2.0-0.el9 ceph-noarch 15 k 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-26T04:31:17.748 INFO:teuthology.orchestra.run.smithi090.stdout:Installing weak dependencies: 2024-04-26T04:31:17.749 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-04-26T04:31:17.749 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:17.749 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:17.749 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:17.749 INFO:teuthology.orchestra.run.smithi090.stdout:Install 7 Packages 2024-04-26T04:31:17.749 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:17.749 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 2.0 M 2024-04-26T04:31:17.750 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 8.1 M 2024-04-26T04:31:17.750 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:17.977 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:00:56 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:18.096 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:18.097 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T04:31:18.098 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout:Install 12 Packages 2024-04-26T04:31:18.099 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:18.100 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 38 M 2024-04-26T04:31:18.100 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 209 M 2024-04-26T04:31:18.100 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:18.281 INFO:teuthology.orchestra.run.smithi090.stdout:(1/7): ceph-grafana-dashboards-18.2.0-0.el9.noa 134 kB/s | 23 kB 00:00 2024-04-26T04:31:18.306 INFO:teuthology.orchestra.run.smithi090.stdout:(2/7): ceph-prometheus-alerts-18.2.0-0.el9.noar 74 kB/s | 15 kB 00:00 2024-04-26T04:31:18.357 INFO:teuthology.orchestra.run.smithi090.stdout:(3/7): ceph-mgr-dashboard-18.2.0-0.el9.noarch.r 6.8 MB/s | 1.7 MB 00:00 2024-04-26T04:31:18.500 INFO:teuthology.orchestra.run.smithi090.stdout:(4/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 41 kB/s | 9.0 kB 00:00 2024-04-26T04:31:18.575 INFO:teuthology.orchestra.run.smithi090.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 152 kB/s | 41 kB 00:00 2024-04-26T04:31:18.617 INFO:teuthology.orchestra.run.smithi090.stdout:(6/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 118 kB/s | 31 kB 00:00 2024-04-26T04:31:18.722 INFO:teuthology.orchestra.run.smithi104.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 922 kB/s | 195 kB 00:00 2024-04-26T04:31:18.789 INFO:teuthology.orchestra.run.smithi104.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 803 kB 00:00 2024-04-26T04:31:18.814 INFO:teuthology.orchestra.run.smithi104.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 358 kB/s | 33 kB 00:00 2024-04-26T04:31:18.923 INFO:teuthology.orchestra.run.smithi104.stdout:(4/12): ceph-mgr-diskprediction-local-18.2.0-0. 18 MB/s | 7.4 MB 00:00 2024-04-26T04:31:18.934 INFO:teuthology.orchestra.run.smithi090.stdout:(7/7): python3-routes-2.5.1-5.el9.noarch.rpm 435 kB/s | 188 kB 00:00 2024-04-26T04:31:18.936 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:18.937 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.7 MB/s | 2.0 MB 00:01 2024-04-26T04:31:18.937 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:18.945 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:18.945 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:18.948 INFO:teuthology.orchestra.run.smithi104.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 136 kB/s | 18 kB 00:00 2024-04-26T04:31:18.997 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:18.998 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:19.024 INFO:teuthology.orchestra.run.smithi104.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 13 MB/s | 3.0 MB 00:00 2024-04-26T04:31:19.040 INFO:teuthology.orchestra.run.smithi104.stdout:(7/12): openblas-0.3.21-2.el9.x86_64.rpm 303 kB/s | 35 kB 00:00 2024-04-26T04:31:19.183 INFO:teuthology.orchestra.run.smithi104.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 20 MB/s | 4.7 MB 00:00 2024-04-26T04:31:19.208 INFO:teuthology.orchestra.run.smithi104.stdout:(9/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.3 MB/s | 244 kB 00:00 2024-04-26T04:31:19.286 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:19.352 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-04-26T04:31:19.366 INFO:teuthology.orchestra.run.smithi104.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 16 MB/s | 5.1 MB 00:00 2024-04-26T04:31:19.386 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-04-26T04:31:19.391 INFO:teuthology.orchestra.run.smithi104.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 1.2 MB/s | 250 kB 00:00 2024-04-26T04:31:19.449 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-04-26T04:31:19.490 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-04-26T04:31:19.535 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 5/7 2024-04-26T04:31:19.764 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 6/7 2024-04-26T04:31:19.792 INFO:teuthology.orchestra.run.smithi104.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-04-26T04:31:19.794 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:19.795 INFO:teuthology.orchestra.run.smithi104.stdout:Total 22 MB/s | 38 MB 00:01 2024-04-26T04:31:19.797 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-26T04:31:20.140 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:20.167 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:20.167 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:20.376 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 7/7 2024-04-26T04:31:20.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 1/7 2024-04-26T04:31:20.376 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2/7 2024-04-26T04:31:20.377 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 3/7 2024-04-26T04:31:20.377 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-04-26T04:31:20.377 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-04-26T04:31:20.377 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-04-26T04:31:20.570 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:20.571 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:20.619 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-04-26T04:31:20.619 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:20.619 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:20.619 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-grafana-dashboards-2:18.2.0-0.el9.noarch 2024-04-26T04:31:20.619 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-dashboard-2:18.2.0-0.el9.noarch 2024-04-26T04:31:20.620 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-prometheus-alerts-2:18.2.0-0.el9.noarch 2024-04-26T04:31:20.620 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-04-26T04:31:20.620 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-04-26T04:31:20.620 INFO:teuthology.orchestra.run.smithi090.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-26T04:31:20.620 INFO:teuthology.orchestra.run.smithi090.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-26T04:31:20.620 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:20.620 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:20.844 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-26T04:31:21.313 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:21.354 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:01 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:21.371 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T04:31:21.414 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:31:21.475 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local noarch 2:18.2.0-0.el9 ceph-noarch 7.4 M 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-26T04:31:21.476 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout:Install 12 Packages 2024-04-26T04:31:21.477 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:21.478 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 38 M 2024-04-26T04:31:21.478 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 209 M 2024-04-26T04:31:21.478 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:21.513 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:31:21.557 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T04:31:21.734 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T04:31:21.765 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T04:31:21.841 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T04:31:21.952 INFO:teuthology.orchestra.run.smithi090.stdout:(1/12): ceph-mgr-diskprediction-local-18.2.0-0. 37 MB/s | 7.4 MB 00:00 2024-04-26T04:31:22.086 INFO:teuthology.orchestra.run.smithi090.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 382 kB/s | 33 kB 00:00 2024-04-26T04:31:22.244 INFO:teuthology.orchestra.run.smithi090.stdout:(3/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 19 MB/s | 3.0 MB 00:00 2024-04-26T04:31:22.480 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T04:31:22.602 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T04:31:22.886 INFO:teuthology.orchestra.run.smithi090.stdout:(4/12): libquadmath-11.4.1-3.el9.x86_64.rpm 172 kB/s | 195 kB 00:01 2024-04-26T04:31:23.187 INFO:teuthology.orchestra.run.smithi090.stdout:(5/12): libgfortran-11.4.1-3.el9.x86_64.rpm 559 kB/s | 803 kB 00:01 2024-04-26T04:31:23.212 INFO:teuthology.orchestra.run.smithi090.stdout:(6/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 19 kB/s | 18 kB 00:00 2024-04-26T04:31:23.237 INFO:teuthology.orchestra.run.smithi090.stdout:(7/12): openblas-0.3.21-2.el9.x86_64.rpm 100 kB/s | 35 kB 00:00 2024-04-26T04:31:23.638 INFO:teuthology.orchestra.run.smithi090.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 610 kB/s | 244 kB 00:00 2024-04-26T04:31:23.820 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:31:24.055 INFO:teuthology.orchestra.run.smithi090.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 5.5 MB/s | 4.7 MB 00:00 2024-04-26T04:31:24.138 INFO:teuthology.orchestra.run.smithi090.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.7 MB/s | 5.1 MB 00:00 2024-04-26T04:31:24.163 INFO:teuthology.orchestra.run.smithi090.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 477 kB/s | 250 kB 00:00 2024-04-26T04:31:24.838 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T04:31:24.863 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-26T04:31:25.130 INFO:teuthology.orchestra.run.smithi090.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 15 MB/s | 16 MB 00:01 2024-04-26T04:31:25.134 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:25.135 INFO:teuthology.orchestra.run.smithi090.stdout:Total 10 MB/s | 38 MB 00:03 2024-04-26T04:31:25.491 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:25.518 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:25.518 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:25.928 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:25.929 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:26.705 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:26.794 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-26T04:31:26.837 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:31:26.940 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:31:26.980 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-26T04:31:27.158 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-26T04:31:27.189 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-26T04:31:27.273 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T04:31:27.338 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T04:31:27.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T04:31:27.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:31:27.339 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T04:31:27.742 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T04:31:27.742 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:27.742 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T04:31:27.743 INFO:teuthology.orchestra.run.smithi104.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T04:31:27.744 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:27.744 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:27.923 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-26T04:31:28.046 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-26T04:31:28.089 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-rook 2024-04-26T04:31:28.604 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:07 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:28.727 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:28.728 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:28.728 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-26T04:31:28.728 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T04:31:28.729 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout:Install 13 Packages 2024-04-26T04:31:28.730 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:28.731 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 2.2 M 2024-04-26T04:31:28.731 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 26 M 2024-04-26T04:31:28.731 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:29.284 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:31:29.462 INFO:teuthology.orchestra.run.smithi104.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 436 kB/s | 47 kB 00:00 2024-04-26T04:31:29.479 INFO:teuthology.orchestra.run.smithi104.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 153 kB/s | 19 kB 00:00 2024-04-26T04:31:29.529 INFO:teuthology.orchestra.run.smithi104.stdout:(3/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 3.3 MB/s | 222 kB 00:00 2024-04-26T04:31:29.579 INFO:teuthology.orchestra.run.smithi104.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.5 MB/s | 159 kB 00:00 2024-04-26T04:31:29.605 INFO:teuthology.orchestra.run.smithi104.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 5.3 MB/s | 279 kB 00:00 2024-04-26T04:31:29.630 INFO:teuthology.orchestra.run.smithi104.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 1.1 MB/s | 54 kB 00:00 2024-04-26T04:31:29.963 INFO:teuthology.orchestra.run.smithi104.stdout:(7/13): python3-certifi-2021.10.8-2.el9.noarch. 45 kB/s | 15 kB 00:00 2024-04-26T04:31:29.990 INFO:teuthology.orchestra.run.smithi104.stdout:(8/13): python3-jsonpatch-1.21-16.el9.noarch.rp 40 kB/s | 26 kB 00:00 2024-04-26T04:31:30.014 INFO:teuthology.orchestra.run.smithi104.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 79 kB/s | 32 kB 00:00 2024-04-26T04:31:30.264 INFO:teuthology.orchestra.run.smithi104.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 683 kB/s | 206 kB 00:00 2024-04-26T04:31:30.337 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-26T04:31:30.373 INFO:teuthology.orchestra.run.smithi104.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 828 kB/s | 90 kB 00:00 2024-04-26T04:31:30.406 INFO:teuthology.orchestra.run.smithi104.stdout:(12/13): python3-kubernetes-26.1.0-2.el9.noarch 2.4 MB/s | 1.0 MB 00:00 2024-04-26T04:31:30.410 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-26T04:31:30.431 INFO:teuthology.orchestra.run.smithi104.stdout:(13/13): python3-rsa-4.9-2.el9.noarch.rpm 142 kB/s | 59 kB 00:00 2024-04-26T04:31:30.436 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:30.437 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.3 MB/s | 2.2 MB 00:01 2024-04-26T04:31:30.498 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:30.513 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:30.513 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:30.597 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:30.597 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:31.185 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:31.354 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T04:31:31.408 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T04:31:31.471 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T04:31:31.509 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T04:31:31.551 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T04:31:31.644 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T04:31:31.753 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T04:31:31.802 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T04:31:32.512 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T04:31:32.565 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T04:31:32.599 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T04:31:32.644 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T04:31:32.669 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T04:31:33.464 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T04:31:33.465 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T04:31:33.465 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T04:31:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T04:31:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:33.775 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-04-26T04:31:33.775 INFO:teuthology.orchestra.run.smithi104.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T04:31:33.776 INFO:teuthology.orchestra.run.smithi104.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T04:31:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T04:31:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T04:31:33.777 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:33.777 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:33.850 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 12/12 2024-04-26T04:31:33.850 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarc 1/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-26T04:31:33.851 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-26T04:31:34.014 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T04:31:34.226 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-26T04:31:34.226 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:34.226 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:34.226 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-diskprediction-local-2:18.2.0-0.el9.noarch 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:34.227 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:34.535 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:13 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:34.655 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:34.656 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:34.656 INFO:teuthology.orchestra.run.smithi104.stdout: Package Arch Version Repository Size 2024-04-26T04:31:34.656 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:34.656 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:34.656 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-04-26T04:31:34.656 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout:Install 6 Packages 2024-04-26T04:31:34.657 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:34.658 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 7.0 M 2024-04-26T04:31:34.658 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 33 M 2024-04-26T04:31:34.658 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:34.696 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-rook 2024-04-26T04:31:35.084 INFO:teuthology.orchestra.run.smithi104.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.1 MB/s | 127 kB 00:00 2024-04-26T04:31:35.167 INFO:teuthology.orchestra.run.smithi104.stdout:(2/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.2 MB/s | 248 kB 00:00 2024-04-26T04:31:35.219 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:15 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:35.339 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:35.340 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:35.340 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-26T04:31:35.340 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:35.340 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:35.340 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook noarch 2:18.2.0-0.el9 ceph-noarch 47 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-26T04:31:35.341 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-26T04:31:35.342 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-26T04:31:35.342 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-26T04:31:35.342 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:35.342 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:35.342 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:35.342 INFO:teuthology.orchestra.run.smithi090.stdout:Install 13 Packages 2024-04-26T04:31:35.342 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:35.343 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 2.2 M 2024-04-26T04:31:35.343 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 26 M 2024-04-26T04:31:35.343 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:35.717 INFO:teuthology.orchestra.run.smithi104.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 110 kB/s | 58 kB 00:00 2024-04-26T04:31:35.901 INFO:teuthology.orchestra.run.smithi104.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 278 kB/s | 51 kB 00:00 2024-04-26T04:31:35.977 INFO:teuthology.orchestra.run.smithi090.stdout:(1/13): ceph-mgr-rook-18.2.0-0.el9.noarch.rpm 369 kB/s | 47 kB 00:00 2024-04-26T04:31:36.002 INFO:teuthology.orchestra.run.smithi090.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 167 kB/s | 26 kB 00:00 2024-04-26T04:31:36.027 INFO:teuthology.orchestra.run.smithi090.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 107 kB/s | 19 kB 00:00 2024-04-26T04:31:36.061 INFO:teuthology.orchestra.run.smithi090.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.6 MB/s | 222 kB 00:00 2024-04-26T04:31:36.086 INFO:teuthology.orchestra.run.smithi090.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-04-26T04:31:36.112 INFO:teuthology.orchestra.run.smithi090.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-04-26T04:31:36.137 INFO:teuthology.orchestra.run.smithi090.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 713 kB/s | 54 kB 00:00 2024-04-26T04:31:36.201 INFO:teuthology.orchestra.run.smithi104.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 491 kB/s | 548 kB 00:01 2024-04-26T04:31:36.229 INFO:teuthology.orchestra.run.smithi090.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 163 kB/s | 15 kB 00:00 2024-04-26T04:31:36.254 INFO:teuthology.orchestra.run.smithi090.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 192 kB/s | 32 kB 00:00 2024-04-26T04:31:36.313 INFO:teuthology.orchestra.run.smithi090.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 1.1 MB/s | 206 kB 00:00 2024-04-26T04:31:36.347 INFO:teuthology.orchestra.run.smithi090.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 641 kB/s | 59 kB 00:00 2024-04-26T04:31:36.372 INFO:teuthology.orchestra.run.smithi090.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.5 MB/s | 90 kB 00:00 2024-04-26T04:31:36.447 INFO:teuthology.orchestra.run.smithi090.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 4.7 MB/s | 1.0 MB 00:00 2024-04-26T04:31:36.449 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:36.449 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.0 MB/s | 2.2 MB 00:01 2024-04-26T04:31:36.511 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:36.525 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:36.526 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:36.609 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:36.609 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:36.818 INFO:teuthology.orchestra.run.smithi104.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.2 MB/s | 6.0 MB 00:01 2024-04-26T04:31:36.822 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:36.822 INFO:teuthology.orchestra.run.smithi104.stdout:Total 3.2 MB/s | 7.0 MB 00:02 2024-04-26T04:31:36.895 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:36.907 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:36.907 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:37.003 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:37.003 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:37.259 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:37.302 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:37.429 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-26T04:31:37.465 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T04:31:37.484 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-26T04:31:37.511 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T04:31:37.547 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-26T04:31:37.585 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-26T04:31:37.626 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-26T04:31:37.720 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-26T04:31:37.829 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-26T04:31:37.886 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-26T04:31:38.033 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T04:31:38.113 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T04:31:38.172 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T04:31:38.198 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-26T04:31:38.604 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-26T04:31:38.678 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-26T04:31:38.711 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-26T04:31:38.758 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-26T04:31:38.782 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-26T04:31:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-26T04:31:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-04-26T04:31:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T04:31:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T04:31:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T04:31:38.942 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T04:31:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T04:31:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:39.180 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:39.404 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install ceph-fuse 2024-04-26T04:31:39.657 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-rook-2:18.2.0-0.el9.noarch 13/13 2024-04-26T04:31:39.657 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-rook-2:18.2.0-0.el9.noarch 1/13 2024-04-26T04:31:39.657 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-26T04:31:39.657 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-26T04:31:39.657 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-26T04:31:39.658 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-26T04:31:39.913 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:18 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:39.960 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-rook-2:18.2.0-0.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-26T04:31:39.961 INFO:teuthology.orchestra.run.smithi090.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-26T04:31:39.962 INFO:teuthology.orchestra.run.smithi090.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-26T04:31:39.962 INFO:teuthology.orchestra.run.smithi090.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-26T04:31:39.962 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:39.962 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:40.032 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:40.032 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:40.032 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:40.032 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:40.032 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:40.032 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-04-26T04:31:40.033 INFO:teuthology.orchestra.run.smithi104.stdout:Installing dependencies: 2024-04-26T04:31:40.033 INFO:teuthology.orchestra.run.smithi104.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T04:31:40.033 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:40.033 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:40.033 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:40.033 INFO:teuthology.orchestra.run.smithi104.stdout:Install 2 Packages 2024-04-26T04:31:40.033 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:40.034 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 915 k 2024-04-26T04:31:40.034 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 2.7 M 2024-04-26T04:31:40.034 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:40.225 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-mgr-cephadm 2024-04-26T04:31:40.552 INFO:teuthology.orchestra.run.smithi104.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 7.4 MB/s | 835 kB 00:00 2024-04-26T04:31:40.585 INFO:teuthology.orchestra.run.smithi104.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 559 kB/s | 80 kB 00:00 2024-04-26T04:31:40.586 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:40.587 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.6 MB/s | 915 kB 00:00 2024-04-26T04:31:40.598 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:40.619 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:40.620 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:40.700 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:40.701 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:40.734 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:20 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:40.854 INFO:teuthology.orchestra.run.smithi090.stdout: Package Arch Version Repository Size 2024-04-26T04:31:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:40.854 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm noarch 2:18.2.0-0.el9 ceph-noarch 127 k 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout:Install 6 Packages 2024-04-26T04:31:40.855 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 7.0 M 2024-04-26T04:31:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 33 M 2024-04-26T04:31:40.856 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:40.883 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:41.024 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T04:31:41.092 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-26T04:31:41.239 INFO:teuthology.orchestra.run.smithi090.stdout:(1/6): ceph-mgr-cephadm-18.2.0-0.el9.noarch.rpm 1.4 MB/s | 127 kB 00:00 2024-04-26T04:31:41.712 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-26T04:31:41.712 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-04-26T04:31:41.939 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T04:31:41.940 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:41.940 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:41.940 INFO:teuthology.orchestra.run.smithi104.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T04:31:41.940 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:41.940 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:42.163 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librados-devel 2024-04-26T04:31:42.215 INFO:teuthology.orchestra.run.smithi090.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 562 kB/s | 548 kB 00:00 2024-04-26T04:31:42.248 INFO:teuthology.orchestra.run.smithi090.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 226 kB/s | 248 kB 00:01 2024-04-26T04:31:42.340 INFO:teuthology.orchestra.run.smithi090.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 556 kB/s | 51 kB 00:00 2024-04-26T04:31:42.366 INFO:teuthology.orchestra.run.smithi090.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 383 kB/s | 58 kB 00:00 2024-04-26T04:31:42.674 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:21 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:42.793 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:42.794 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:42.794 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:42.794 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:42.794 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:42.794 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-04-26T04:31:42.794 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:42.794 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:42.795 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:42.795 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-04-26T04:31:42.795 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:42.795 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 126 k 2024-04-26T04:31:42.795 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 456 k 2024-04-26T04:31:42.795 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:42.890 INFO:teuthology.orchestra.run.smithi104.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.3 MB/s | 126 kB 00:00 2024-04-26T04:31:42.890 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:42.890 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.3 MB/s | 126 kB 00:00 2024-04-26T04:31:42.890 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:42.896 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:42.896 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:42.932 INFO:teuthology.orchestra.run.smithi090.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 3.3 MB/s | 6.0 MB 00:01 2024-04-26T04:31:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:Total 3.4 MB/s | 7.0 MB 00:02 2024-04-26T04:31:42.961 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:42.961 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:43.004 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:43.015 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:43.015 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:43.104 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:43.110 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:43.111 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:43.235 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:43.412 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:43.580 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-26T04:31:43.612 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:43.642 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-26T04:31:43.877 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:43.878 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:43.878 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:43.878 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:44.113 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install libcephfs2 2024-04-26T04:31:44.166 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-26T04:31:44.264 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-26T04:31:44.323 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-26T04:31:44.357 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-26T04:31:44.647 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:23 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:44.726 INFO:teuthology.orchestra.run.smithi104.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:44.767 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:44.768 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:44.768 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:44.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install libcephfs-devel 2024-04-26T04:31:45.354 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:24 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:45.456 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 6/6 2024-04-26T04:31:45.456 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 1/6 2024-04-26T04:31:45.457 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-26T04:31:45.457 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-26T04:31:45.457 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-26T04:31:45.457 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-26T04:31:45.476 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-04-26T04:31:45.477 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:45.478 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 30 k 2024-04-26T04:31:45.478 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 137 k 2024-04-26T04:31:45.478 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:45.570 INFO:teuthology.orchestra.run.smithi104.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 330 kB/s | 30 kB 00:00 2024-04-26T04:31:45.571 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:45.571 INFO:teuthology.orchestra.run.smithi104.stdout:Total 325 kB/s | 30 kB 00:00 2024-04-26T04:31:45.571 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:45.575 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:45.575 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:45.577 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:45.577 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:45.662 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:45.692 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-mgr-cephadm-2:18.2.0-0.el9.noarch 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:45.693 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:45.774 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:45.948 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install ceph-fuse 2024-04-26T04:31:46.094 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:46.293 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:46.293 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:46.293 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:46.293 INFO:teuthology.orchestra.run.smithi104.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:46.293 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:46.294 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:46.450 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:26 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:46.479 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librados2 2024-04-26T04:31:46.569 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:46.569 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:46.569 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:46.569 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:46.569 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:46.569 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse x86_64 2:18.2.0-0.el9 ceph 835 k 2024-04-26T04:31:46.569 INFO:teuthology.orchestra.run.smithi090.stdout:Installing dependencies: 2024-04-26T04:31:46.570 INFO:teuthology.orchestra.run.smithi090.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-26T04:31:46.570 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:46.570 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:46.570 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:46.570 INFO:teuthology.orchestra.run.smithi090.stdout:Install 2 Packages 2024-04-26T04:31:46.570 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:46.571 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 915 k 2024-04-26T04:31:46.571 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 2.7 M 2024-04-26T04:31:46.571 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:46.971 INFO:teuthology.orchestra.run.smithi090.stdout:(1/2): ceph-fuse-18.2.0-0.el9.x86_64.rpm 6.7 MB/s | 835 kB 00:00 2024-04-26T04:31:46.983 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:25 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:47.061 INFO:teuthology.orchestra.run.smithi104.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:47.101 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:47.102 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:47.103 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:47.162 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install librbd1 2024-04-26T04:31:47.664 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:26 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:47.680 INFO:teuthology.orchestra.run.smithi090.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 96 kB/s | 80 kB 00:00 2024-04-26T04:31:47.682 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:47.682 INFO:teuthology.orchestra.run.smithi090.stdout:Total 823 kB/s | 915 kB 00:01 2024-04-26T04:31:47.694 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:47.716 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:47.717 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:47.742 INFO:teuthology.orchestra.run.smithi104.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:47.782 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:47.782 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:47.783 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:47.797 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:47.798 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:47.845 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rados 2024-04-26T04:31:47.976 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:48.121 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-26T04:31:48.188 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-26T04:31:48.347 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:27 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:48.425 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:48.465 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:48.466 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:48.466 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:48.528 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rgw 2024-04-26T04:31:48.877 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: ceph-fuse-2:18.2.0-0.el9.x86_64 2/2 2024-04-26T04:31:48.877 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : ceph-fuse-2:18.2.0-0.el9.x86_64 1/2 2024-04-26T04:31:49.042 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:28 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:49.120 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:49.151 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-26T04:31:49.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:49.151 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:49.151 INFO:teuthology.orchestra.run.smithi090.stdout: ceph-fuse-2:18.2.0-0.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-26T04:31:49.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:49.152 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:49.161 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:49.162 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:49.162 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:49.230 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-cephfs 2024-04-26T04:31:49.532 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados-devel 2024-04-26T04:31:49.738 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:28 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:49.816 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:49.858 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:49.859 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:49.859 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:49.924 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install python3-rbd 2024-04-26T04:31:50.050 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:30 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:50.171 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel x86_64 2:18.2.0-0.el9 ceph 126 k 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 126 k 2024-04-26T04:31:50.172 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 456 k 2024-04-26T04:31:50.173 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:50.239 INFO:teuthology.orchestra.run.smithi090.stdout:librados-devel-18.2.0-0.el9.x86_64.rpm 1.9 MB/s | 126 kB 00:00 2024-04-26T04:31:50.239 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:50.240 INFO:teuthology.orchestra.run.smithi090.stdout:Total 1.8 MB/s | 126 kB 00:00 2024-04-26T04:31:50.240 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:50.246 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:50.246 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:50.309 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:50.310 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:50.431 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:29 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:50.467 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:50.510 INFO:teuthology.orchestra.run.smithi104.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:50.551 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:50.552 INFO:teuthology.orchestra.run.smithi104.stdout:Nothing to do. 2024-04-26T04:31:50.552 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:50.594 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:50.618 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-fuse 2024-04-26T04:31:51.080 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:51.121 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:30 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:51.239 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-04-26T04:31:51.240 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:51.241 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 86 k 2024-04-26T04:31:51.241 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 230 k 2024-04-26T04:31:51.241 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:51.323 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 1.0 MB/s | 86 kB 00:00 2024-04-26T04:31:51.323 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:51.323 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.0 MB/s | 86 kB 00:00 2024-04-26T04:31:51.324 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:51.330 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : librados-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:51.330 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:51.330 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:51.330 INFO:teuthology.orchestra.run.smithi090.stdout: librados-devel-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:51.330 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:51.330 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:51.331 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:51.331 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:51.394 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:51.394 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:51.541 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:51.641 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs2 2024-04-26T04:31:51.644 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:52.015 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:32 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:52.236 INFO:teuthology.orchestra.run.smithi090.stdout:Package libcephfs2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:52.264 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:52.264 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:52.265 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:52.265 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:52.265 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:52.265 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:52.277 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:52.279 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:52.279 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:52.397 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install libcephfs-devel 2024-04-26T04:31:52.505 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-mirror 2024-04-26T04:31:52.908 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:32 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:53.023 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:32 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:53.026 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:53.026 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:53.026 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:53.026 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:53.026 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:53.026 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel x86_64 2:18.2.0-0.el9 ceph 30 k 2024-04-26T04:31:53.027 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:53.027 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:53.027 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:53.027 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-26T04:31:53.027 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:53.027 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 30 k 2024-04-26T04:31:53.027 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 137 k 2024-04-26T04:31:53.028 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:53.114 INFO:teuthology.orchestra.run.smithi090.stdout:libcephfs-devel-18.2.0-0.el9.x86_64.rpm 353 kB/s | 30 kB 00:00 2024-04-26T04:31:53.115 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:53.115 INFO:teuthology.orchestra.run.smithi090.stdout:Total 347 kB/s | 30 kB 00:00 2024-04-26T04:31:53.115 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:53.118 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:53.118 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:53.120 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:53.120 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:53.143 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:53.143 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:53.143 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:53.143 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:53.144 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:53.144 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-26T04:31:53.144 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:53.144 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:53.144 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:53.144 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-04-26T04:31:53.144 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:53.145 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 3.0 M 2024-04-26T04:31:53.145 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 12 M 2024-04-26T04:31:53.145 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:53.208 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:53.288 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 21 MB/s | 3.0 MB 00:00 2024-04-26T04:31:53.289 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:53.289 INFO:teuthology.orchestra.run.smithi104.stdout:Total 21 MB/s | 3.0 MB 00:00 2024-04-26T04:31:53.289 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:53.298 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:53.299 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:53.326 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:53.378 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:53.378 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:53.712 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:53.804 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:53.918 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : libcephfs-devel-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:53.918 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:53.919 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:31:53.919 INFO:teuthology.orchestra.run.smithi090.stdout: libcephfs-devel-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:53.919 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:53.919 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:53.952 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:53.976 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:53.976 INFO:teuthology.orchestra.run.smithi104.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:31:53.977 INFO:teuthology.orchestra.run.smithi104.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T04:31:53.977 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:31:53.977 INFO:teuthology.orchestra.run.smithi104.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:31:53.977 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:54.151 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librados2 2024-04-26T04:31:54.655 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:34 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:54.733 INFO:teuthology.orchestra.run.smithi090.stdout:Package librados2-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:54.773 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:54.774 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:54.774 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:54.857 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install librbd1 2024-04-26T04:31:54.860 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:54.860 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:54.860 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:54.860 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:54.860 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:54.860 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:55.122 DEBUG:teuthology.orchestra.run.smithi104:> sudo yum -y install rbd-nbd 2024-04-26T04:31:55.357 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:35 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:55.435 INFO:teuthology.orchestra.run.smithi090.stdout:Package librbd1-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:55.475 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:55.476 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:55.476 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:55.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rados 2024-04-26T04:31:55.635 INFO:teuthology.orchestra.run.smithi104.stdout:Last metadata expiration check: 0:01:34 ago on Fri 26 Apr 2024 04:30:21 AM UTC. 2024-04-26T04:31:55.756 INFO:teuthology.orchestra.run.smithi104.stdout:Dependencies resolved. 2024-04-26T04:31:55.757 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:55.757 INFO:teuthology.orchestra.run.smithi104.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:55.757 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:55.757 INFO:teuthology.orchestra.run.smithi104.stdout:Installing: 2024-04-26T04:31:55.757 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction Summary 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout:================================================================================ 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout:Install 1 Package 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout:Total download size: 169 k 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout:Installed size: 490 k 2024-04-26T04:31:55.758 INFO:teuthology.orchestra.run.smithi104.stdout:Downloading Packages: 2024-04-26T04:31:55.854 INFO:teuthology.orchestra.run.smithi104.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 1.7 MB/s | 169 kB 00:00 2024-04-26T04:31:55.855 INFO:teuthology.orchestra.run.smithi104.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:55.855 INFO:teuthology.orchestra.run.smithi104.stdout:Total 1.7 MB/s | 169 kB 00:00 2024-04-26T04:31:55.855 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction check 2024-04-26T04:31:55.862 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction check succeeded. 2024-04-26T04:31:55.862 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction test 2024-04-26T04:31:55.925 INFO:teuthology.orchestra.run.smithi104.stdout:Transaction test succeeded. 2024-04-26T04:31:55.925 INFO:teuthology.orchestra.run.smithi104.stdout:Running transaction 2024-04-26T04:31:56.053 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:36 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:56.074 INFO:teuthology.orchestra.run.smithi104.stdout: Preparing : 1/1 2024-04-26T04:31:56.131 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rados-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:56.171 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:56.172 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:56.172 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:56.174 INFO:teuthology.orchestra.run.smithi104.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:56.260 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rgw 2024-04-26T04:31:56.583 INFO:teuthology.orchestra.run.smithi104.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:56.767 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:36 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:56.816 INFO:teuthology.orchestra.run.smithi104.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:56.817 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:56.817 INFO:teuthology.orchestra.run.smithi104.stdout:Installed: 2024-04-26T04:31:56.817 INFO:teuthology.orchestra.run.smithi104.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-04-26T04:31:56.817 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:31:56.817 INFO:teuthology.orchestra.run.smithi104.stdout:Complete! 2024-04-26T04:31:56.846 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rgw-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:56.886 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:56.887 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:56.887 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:56.975 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-cephfs 2024-04-26T04:31:57.065 DEBUG:teuthology.parallel:result is None 2024-04-26T04:31:57.474 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:37 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:57.552 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-cephfs-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:57.592 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:57.593 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:57.593 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:57.671 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install python3-rbd 2024-04-26T04:31:58.171 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:38 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:58.248 INFO:teuthology.orchestra.run.smithi090.stdout:Package python3-rbd-2:18.2.0-0.el9.x86_64 is already installed. 2024-04-26T04:31:58.288 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:58.289 INFO:teuthology.orchestra.run.smithi090.stdout:Nothing to do. 2024-04-26T04:31:58.289 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:31:58.415 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-fuse 2024-04-26T04:31:58.917 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:38 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:31:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:31:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:59.037 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse x86_64 2:18.2.0-0.el9 ceph 86 k 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 86 k 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 230 k 2024-04-26T04:31:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:31:59.161 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-fuse-18.2.0-0.el9.x86_64.rpm 696 kB/s | 86 kB 00:00 2024-04-26T04:31:59.162 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:31:59.162 INFO:teuthology.orchestra.run.smithi090.stdout:Total 688 kB/s | 86 kB 00:00 2024-04-26T04:31:59.162 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:31:59.169 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:31:59.169 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:31:59.231 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:31:59.231 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:31:59.372 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:31:59.474 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:31:59.860 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-fuse-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:00.078 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:32:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-fuse-2:18.2.0-0.el9.x86_64 2024-04-26T04:32:00.078 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:00.078 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:32:00.345 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-mirror 2024-04-26T04:32:00.872 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:40 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:32:00.997 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:32:00.998 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:32:00.998 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:32:00.998 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror x86_64 2:18.2.0-0.el9 ceph 3.0 M 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 3.0 M 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 12 M 2024-04-26T04:32:00.999 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:32:01.109 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-mirror-18.2.0-0.el9.x86_64.rpm 27 MB/s | 3.0 MB 00:00 2024-04-26T04:32:01.110 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:32:01.110 INFO:teuthology.orchestra.run.smithi090.stdout:Total 27 MB/s | 3.0 MB 00:00 2024-04-26T04:32:01.111 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:32:01.120 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:32:01.120 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:32:01.203 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:32:01.203 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:32:01.634 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:32:01.837 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:01.863 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:01.863 INFO:teuthology.orchestra.run.smithi090.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-26T04:32:01.863 INFO:teuthology.orchestra.run.smithi090.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-26T04:32:01.863 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:32:01.863 INFO:teuthology.orchestra.run.smithi090.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-26T04:32:01.863 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:02.911 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-mirror-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:02.912 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:02.912 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:32:02.912 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-mirror-2:18.2.0-0.el9.x86_64 2024-04-26T04:32:02.912 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:02.912 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:32:03.235 DEBUG:teuthology.orchestra.run.smithi090:> sudo yum -y install rbd-nbd 2024-04-26T04:32:03.752 INFO:teuthology.orchestra.run.smithi090.stdout:Last metadata expiration check: 0:01:43 ago on Fri 26 Apr 2024 04:30:20 AM UTC. 2024-04-26T04:32:03.877 INFO:teuthology.orchestra.run.smithi090.stdout:Dependencies resolved. 2024-04-26T04:32:03.877 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:32:03.877 INFO:teuthology.orchestra.run.smithi090.stdout: Package Architecture Version Repository Size 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:Installing: 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd x86_64 2:18.2.0-0.el9 ceph 169 k 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction Summary 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:================================================================================ 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:Install 1 Package 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:Total download size: 169 k 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:Installed size: 490 k 2024-04-26T04:32:03.878 INFO:teuthology.orchestra.run.smithi090.stdout:Downloading Packages: 2024-04-26T04:32:03.944 INFO:teuthology.orchestra.run.smithi090.stdout:rbd-nbd-18.2.0-0.el9.x86_64.rpm 2.6 MB/s | 169 kB 00:00 2024-04-26T04:32:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:32:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:Total 2.5 MB/s | 169 kB 00:00 2024-04-26T04:32:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction check 2024-04-26T04:32:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction check succeeded. 2024-04-26T04:32:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction test 2024-04-26T04:32:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:Transaction test succeeded. 2024-04-26T04:32:04.108 INFO:teuthology.orchestra.run.smithi090.stdout:Running transaction 2024-04-26T04:32:04.164 INFO:teuthology.orchestra.run.smithi090.stdout: Preparing : 1/1 2024-04-26T04:32:04.289 INFO:teuthology.orchestra.run.smithi090.stdout: Installing : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:04.753 INFO:teuthology.orchestra.run.smithi090.stdout: Running scriptlet: rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:04.993 INFO:teuthology.orchestra.run.smithi090.stdout: Verifying : rbd-nbd-2:18.2.0-0.el9.x86_64 1/1 2024-04-26T04:32:04.993 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:04.993 INFO:teuthology.orchestra.run.smithi090.stdout:Installed: 2024-04-26T04:32:04.993 INFO:teuthology.orchestra.run.smithi090.stdout: rbd-nbd-2:18.2.0-0.el9.x86_64 2024-04-26T04:32:04.993 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:04.993 INFO:teuthology.orchestra.run.smithi090.stdout:Complete! 2024-04-26T04:32:05.322 DEBUG:teuthology.parallel:result is None 2024-04-26T04:32:05.322 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-26T04:32:05.322 INFO:teuthology.packaging:ref: None 2024-04-26T04:32:05.322 INFO:teuthology.packaging:tag: v18.2.0 2024-04-26T04:32:05.322 INFO:teuthology.packaging:branch: None 2024-04-26T04:32:05.322 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:32:05.323 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-26T04:32:05.549 DEBUG:teuthology.orchestra.run.smithi090:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T04:32:05.570 INFO:teuthology.orchestra.run.smithi090.stdout:18.2.0-0.el9 2024-04-26T04:32:05.571 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-04-26T04:32:05.571 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-04-26T04:32:05.572 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using tag 2024-04-26T04:32:05.573 INFO:teuthology.packaging:ref: None 2024-04-26T04:32:05.573 INFO:teuthology.packaging:tag: v18.2.0 2024-04-26T04:32:05.573 INFO:teuthology.packaging:branch: None 2024-04-26T04:32:05.573 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:32:05.573 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5dd24139a1eada541a3bc16b6941c5dde975e26d 2024-04-26T04:32:05.790 DEBUG:teuthology.orchestra.run.smithi104:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-26T04:32:05.815 INFO:teuthology.orchestra.run.smithi104.stdout:18.2.0-0.el9 2024-04-26T04:32:05.815 INFO:teuthology.packaging:The installed version of ceph is 18.2.0-0.el9 2024-04-26T04:32:05.815 INFO:teuthology.task.install:The correct ceph version 18.2.0-0 is installed. 2024-04-26T04:32:05.817 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-26T04:32:05.817 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:32:05.817 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:32:05.842 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:32:05.842 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-26T04:32:05.884 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-26T04:32:05.884 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:32:05.884 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:32:05.907 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:32:05.970 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:32:05.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/daemon-helper 2024-04-26T04:32:05.998 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-26T04:32:06.063 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-26T04:32:06.064 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:32:06.064 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:32:06.087 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:32:06.159 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:32:06.159 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-26T04:32:06.185 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-26T04:32:06.250 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-26T04:32:06.250 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:32:06.250 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:32:06.274 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:32:06.337 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:32:06.337 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/usr/bin/stdin-killer 2024-04-26T04:32:06.360 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-26T04:32:06.424 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:32:06.438 INFO:teuthology.task.print:**** done install task... 2024-04-26T04:32:06.438 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-26T04:32:06.546 INFO:tasks.cephadm:Config: {'compiled_cephadm_branch': 'reef', 'conf': {'osd': {'osd_class_default_list': '*', 'osd_class_load_list': '*', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'mon pg warn min per osd': 0}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}}, 'image': 'quay.io/ceph/ceph:v18.2.0', 'roleless': True, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'OSD_DOWN', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'MON_DOWN'], 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'} 2024-04-26T04:32:06.546 INFO:tasks.cephadm:Cluster image is quay.io/ceph/ceph:v18.2.0 2024-04-26T04:32:06.547 INFO:tasks.cephadm:Cluster fsid is f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:32:06.547 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-26T04:32:06.547 INFO:tasks.cephadm:No mon roles; fabricating mons 2024-04-26T04:32:06.547 INFO:tasks.cephadm:Monitor IPs: {'mon.smithi090': '172.21.15.90', 'mon.smithi104': '172.21.15.104'} 2024-04-26T04:32:06.547 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-26T04:32:06.548 DEBUG:teuthology.orchestra.run.smithi090:> sudo hostname $(hostname -s) 2024-04-26T04:32:06.571 DEBUG:teuthology.orchestra.run.smithi104:> sudo hostname $(hostname -s) 2024-04-26T04:32:06.596 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra for reef 2024-04-26T04:32:06.596 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:32:06.740 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/54991/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-2447-gb22e2ebd', 'node_name': '172.21.5.36+adami06', 'job_name': 'ceph-dev-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-2447.gb22e2ebd'}, 'url': 'https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-25 23:00:56.935300', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'squid', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-26T04:32:06.980 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref reef, sha1 7d2137f0a666c92a9411c1af5c33e4968be54da6 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&ref=reef 2024-04-26T04:32:06.981 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/x86_64/flavors/default/cephadm 2024-04-26T04:32:06.981 DEBUG:teuthology.orchestra.run.smithi090:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T04:32:07.335 INFO:teuthology.orchestra.run.smithi090.stdout:-rw-r--r--. 1 ubuntu ubuntu 215336 Apr 26 04:32 /home/ubuntu/cephtest/cephadm 2024-04-26T04:32:07.336 DEBUG:teuthology.orchestra.run.smithi104:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/reef/7d2137f0a666c92a9411c1af5c33e4968be54da6/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-26T04:32:07.686 INFO:teuthology.orchestra.run.smithi104.stdout:-rw-r--r--. 1 ubuntu ubuntu 215336 Apr 26 04:32 /home/ubuntu/cephtest/cephadm 2024-04-26T04:32:07.686 DEBUG:teuthology.orchestra.run.smithi090:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T04:32:07.705 DEBUG:teuthology.orchestra.run.smithi104:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-26T04:32:07.732 INFO:tasks.cephadm:Pulling image quay.io/ceph/ceph:v18.2.0 on all hosts... 2024-04-26T04:32:07.733 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-04-26T04:32:07.748 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 pull 2024-04-26T04:32:09.206 INFO:teuthology.orchestra.run.smithi104.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-04-26T04:32:09.896 INFO:teuthology.orchestra.run.smithi090.stderr:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-04-26T04:32:29.937 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:32:29.937 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-04-26T04:32:29.937 INFO:teuthology.orchestra.run.smithi090.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-04-26T04:32:29.937 INFO:teuthology.orchestra.run.smithi090.stdout: "repo_digests": [ 2024-04-26T04:32:29.937 INFO:teuthology.orchestra.run.smithi090.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-04-26T04:32:29.937 INFO:teuthology.orchestra.run.smithi090.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-04-26T04:32:29.938 INFO:teuthology.orchestra.run.smithi090.stdout: ] 2024-04-26T04:32:29.938 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:32:33.866 INFO:teuthology.orchestra.run.smithi104.stdout:{ 2024-04-26T04:32:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "ceph_version": "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)", 2024-04-26T04:32:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "image_id": "dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946", 2024-04-26T04:32:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "repo_digests": [ 2024-04-26T04:32:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007", 2024-04-26T04:32:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: "quay.io/ceph/ceph@sha256:d58cf65589d0abf9d5261cd46fb62be3bfb29098febc78c0fcdf116a15274d27" 2024-04-26T04:32:33.867 INFO:teuthology.orchestra.run.smithi104.stdout: ] 2024-04-26T04:32:33.867 INFO:teuthology.orchestra.run.smithi104.stdout:} 2024-04-26T04:32:33.884 DEBUG:teuthology.orchestra.run.smithi090:> sudo mkdir -p /etc/ceph 2024-04-26T04:32:33.915 DEBUG:teuthology.orchestra.run.smithi104:> sudo mkdir -p /etc/ceph 2024-04-26T04:32:33.940 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 777 /etc/ceph 2024-04-26T04:32:33.982 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 777 /etc/ceph 2024-04-26T04:32:34.005 INFO:tasks.cephadm:Writing seed config... 2024-04-26T04:32:34.006 INFO:tasks.cephadm: override: [osd] osd_class_default_list = * 2024-04-26T04:32:34.006 INFO:tasks.cephadm: override: [osd] osd_class_load_list = * 2024-04-26T04:32:34.006 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-04-26T04:32:34.006 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-04-26T04:32:34.006 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-04-26T04:32:34.006 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-26T04:32:34.007 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [global] mon pg warn min per osd = 0 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-26T04:32:34.008 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-26T04:32:34.009 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-26T04:32:34.010 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:32:34.010 DEBUG:teuthology.orchestra.run.smithi090:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-26T04:32:34.039 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f018e902-0385-11ef-bc93-c7b262605968 mon pg warn min per osd = 0 [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = true bdev debug aio = true osd sloppy crc = true osd_class_default_list = * osd_class_load_list = * bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-26T04:32:34.039 DEBUG:teuthology.orchestra.run.smithi090:mon.smithi090> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service 2024-04-26T04:32:34.081 INFO:tasks.cephadm:Bootstrapping... 2024-04-26T04:32:34.081 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 -v bootstrap --fsid f018e902-0385-11ef-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-ip 172.21.15.90 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi090.stdout:-------------------------------------------------------------------------------- 2024-04-26T04:32:34.239 INFO:teuthology.orchestra.run.smithi090.stdout:cephadm ['--image', 'quay.io/ceph/ceph:v18.2.0', '-v', 'bootstrap', '--fsid', 'f018e902-0385-11ef-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-ip', '172.21.15.90', '--skip-admin-label'] 2024-04-26T04:32:34.264 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi090.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-26T04:32:34.265 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying podman|docker is present... 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying lvm2 is present... 2024-04-26T04:32:34.290 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying time synchronization is in place... 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T04:32:34.297 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T04:32:34.302 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T04:32:34.303 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T04:32:34.309 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-26T04:32:34.315 INFO:teuthology.orchestra.run.smithi090.stdout:Repeating the final host check... 2024-04-26T04:32:34.339 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-26T04:32:34.339 INFO:teuthology.orchestra.run.smithi090.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-26T04:32:34.339 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl is present 2024-04-26T04:32:34.339 INFO:teuthology.orchestra.run.smithi090.stdout:lvcreate is present 2024-04-26T04:32:34.345 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-26T04:32:34.345 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-26T04:32:34.350 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-26T04:32:34.351 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T04:32:34.357 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout enabled 2024-04-26T04:32:34.363 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout active 2024-04-26T04:32:34.363 INFO:teuthology.orchestra.run.smithi090.stdout:Unit chronyd.service is enabled and running 2024-04-26T04:32:34.363 INFO:teuthology.orchestra.run.smithi090.stdout:Host looks OK 2024-04-26T04:32:34.363 INFO:teuthology.orchestra.run.smithi090.stdout:Cluster fsid: f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:32:34.363 INFO:teuthology.orchestra.run.smithi090.stdout:Acquiring lock 139941505516544 on /run/cephadm/f018e902-0385-11ef-bc93-c7b262605968.lock 2024-04-26T04:32:34.363 INFO:teuthology.orchestra.run.smithi090.stdout:Lock 139941505516544 acquired on /run/cephadm/f018e902-0385-11ef-bc93-c7b262605968.lock 2024-04-26T04:32:34.363 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 3300 ... 2024-04-26T04:32:34.364 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying IP 172.21.15.90 port 6789 ... 2024-04-26T04:32:34.364 INFO:teuthology.orchestra.run.smithi090.stdout:Base mon IP(s) is [172.21.15.90:3300, 172.21.15.90:6789], mon addrv is [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-26T04:32:34.367 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.90 metric 100 2024-04-26T04:32:34.367 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.90 metric 100 2024-04-26T04:32:34.369 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-26T04:32:34.369 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-26T04:32:34.369 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-26T04:32:34.371 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-26T04:32:34.372 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-26T04:32:34.372 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T04:32:34.372 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-26T04:32:34.372 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout inet6 fe80::30a8:3f5b:7da9:df8e/64 scope link noprefixroute 2024-04-26T04:32:34.372 INFO:teuthology.orchestra.run.smithi090.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-26T04:32:34.372 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-26T04:32:34.373 INFO:teuthology.orchestra.run.smithi090.stdout:Mon IP `172.21.15.90` is in CIDR network `172.21.0.0/20` 2024-04-26T04:32:34.373 INFO:teuthology.orchestra.run.smithi090.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-26T04:32:34.373 INFO:teuthology.orchestra.run.smithi090.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-26T04:32:34.374 INFO:teuthology.orchestra.run.smithi090.stdout:Pulling container image quay.io/ceph/ceph:v18.2.0... 2024-04-26T04:32:35.214 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stdout dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-04-26T04:32:35.215 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Trying to pull quay.io/ceph/ceph:v18.2.0... 2024-04-26T04:32:35.215 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Getting image source signatures 2024-04-26T04:32:35.215 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:3bd20aeff60302f668275dc2005d10679ae56492967a3a5a54fd3dde85333aec 2024-04-26T04:32:35.215 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2024-04-26T04:32:35.215 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Copying config sha256:dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 2024-04-26T04:32:35.215 INFO:teuthology.orchestra.run.smithi090.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-26T04:32:45.457 INFO:teuthology.orchestra.run.smithi090.stdout:ceph: stdout ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-04-26T04:32:45.457 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph version: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable) 2024-04-26T04:32:45.458 INFO:teuthology.orchestra.run.smithi090.stdout:Extracting ceph user uid/gid from container image... 2024-04-26T04:32:49.761 INFO:teuthology.orchestra.run.smithi090.stdout:stat: stdout 167 167 2024-04-26T04:32:49.761 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial keys... 2024-04-26T04:32:50.628 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQByLitmyDSrAxAADRFFhA64SKvwtKnS6g45mQ== 2024-04-26T04:32:52.195 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQBzLitmoUFzJRAAX/sPgLJFqpOoChvU9s6fPA== 2024-04-26T04:32:53.149 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph-authtool: stdout AQB0LitmF7O1IBAAJqBtE42X8jcRO0G/cR5fnw== 2024-04-26T04:32:53.150 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial monmap... 2024-04-26T04:32:54.086 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T04:32:54.086 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2024-04-26T04:32:54.086 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:32:54.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T04:32:54.087 INFO:teuthology.orchestra.run.smithi090.stdout:monmaptool for smithi090 [v2:172.21.15.90:3300,v1:172.21.15.90:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-26T04:32:54.087 INFO:teuthology.orchestra.run.smithi090.stdout:setting min_mon_release = pacific 2024-04-26T04:32:54.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: set fsid to f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:32:54.087 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-26T04:32:54.087 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:32:54.087 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mon... 2024-04-26T04:32:55.691 INFO:teuthology.orchestra.run.smithi090.stdout:create mon.smithi090 on 2024-04-26T04:32:55.899 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-26T04:32:56.048 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-26T04:32:56.239 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f018e902-0385-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f018e902-0385-11ef-bc93-c7b262605968.target. 2024-04-26T04:32:56.239 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f018e902-0385-11ef-bc93-c7b262605968.target → /etc/systemd/system/ceph-f018e902-0385-11ef-bc93-c7b262605968.target. 2024-04-26T04:32:56.451 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090 2024-04-26T04:32:56.451 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service: Unit ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service not loaded. 2024-04-26T04:32:56.615 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f018e902-0385-11ef-bc93-c7b262605968.target.wants/ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service → /etc/systemd/system/ceph-f018e902-0385-11ef-bc93-c7b262605968@.service. 2024-04-26T04:32:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:56 smithi090 podman[26260]: 2024-04-26 04:32:56.758518642 +0000 UTC m=+0.025791997 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-04-26T04:32:57.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:57 smithi090 podman[26260]: 2024-04-26 04:32:57.017378299 +0000 UTC m=+0.284651713 container create 9c56a13eb766d8dcb1d209f5be275483a307d89ae77af221943533cc7fdb554c (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=HEAD) 2024-04-26T04:32:57.960 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:32:57.961 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T04:32:57.967 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:32:57.967 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T04:32:57.968 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T04:32:57.968 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T04:32:57.968 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon to start... 2024-04-26T04:32:57.968 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mon... 2024-04-26T04:32:58.081 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:57 smithi090 podman[26260]: 2024-04-26 04:32:57.879579651 +0000 UTC m=+1.146853000 container init 9c56a13eb766d8dcb1d209f5be275483a307d89ae77af221943533cc7fdb554c (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , RELEASE=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T04:32:58.081 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:57 smithi090 podman[26260]: 2024-04-26 04:32:57.883421885 +0000 UTC m=+1.150695227 container start 9c56a13eb766d8dcb1d209f5be275483a307d89ae77af221943533cc7fdb554c (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_BRANCH=HEAD) 2024-04-26T04:32:58.081 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:57 smithi090 ceph-mon[26278]: mkfs f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:32:58.081 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:57 smithi090 ceph-mon[26278]: mon.smithi090 is new leader, mons smithi090 in quorum (ranks 0) 2024-04-26T04:32:58.081 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:57 smithi090 bash[26260]: 9c56a13eb766d8dcb1d209f5be275483a307d89ae77af221943533cc7fdb554c 2024-04-26T04:32:58.081 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:57 smithi090 systemd[1]: Started Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:32:59.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout cluster: 2024-04-26T04:32:59.031 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout id: f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:32:59.031 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-26T04:32:59.031 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:32:59.031 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout services: 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum smithi090 (age 0.530254s) 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout data: 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout pgs: 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.380+0000 7fba6f568700 1 Processor -- start 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.380+0000 7fba6f568700 1 -- start start 2024-04-26T04:32:59.032 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.380+0000 7fba6f568700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.380+0000 7fba6f568700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba68106460 con 0x7fba68105b10 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba6d304700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba6d304700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53660/0 (socket says 172.21.15.90:53660) 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba6d304700 1 -- 172.21.15.90:0/2005073703 learned_addr learned my addr 172.21.15.90:0/2005073703 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba6d304700 1 -- 172.21.15.90:0/2005073703 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba681065a0 con 0x7fba68105b10 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba6d304700 1 --2- 172.21.15.90:0/2005073703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68105f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fba640098a0 tx=0x7fba64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d06126d1f2f614f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba5ffff700 1 -- 172.21.15.90:0/2005073703 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba64004180 con 0x7fba68105b10 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba5ffff700 1 -- 172.21.15.90:0/2005073703 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba640042e0 con 0x7fba68105b10 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.381+0000 7fba5ffff700 1 -- 172.21.15.90:0/2005073703 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba640036a0 con 0x7fba68105b10 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 -- 172.21.15.90:0/2005073703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 msgr2=0x7fba68105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 --2- 172.21.15.90:0/2005073703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68105f20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fba640098a0 tx=0x7fba64009bb0 comp rx=0 tx=0).stop 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 -- 172.21.15.90:0/2005073703 shutdown_connections 2024-04-26T04:32:59.033 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 --2- 172.21.15.90:0/2005073703 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68105f20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:32:59.034 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 -- 172.21.15.90:0/2005073703 >> 172.21.15.90:0/2005073703 conn(0x7fba68101180 msgr2=0x7fba68103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:32:59.034 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 -- 172.21.15.90:0/2005073703 shutdown_connections 2024-04-26T04:32:59.034 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 -- 172.21.15.90:0/2005073703 wait complete. 2024-04-26T04:32:59.034 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 Processor -- start 2024-04-26T04:32:59.034 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 -- start start 2024-04-26T04:32:59.034 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:32:59.034 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.382+0000 7fba6f568700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba68199550 con 0x7fba68105b10 2024-04-26T04:32:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba6d304700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:32:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba6d304700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53676/0 (socket says 172.21.15.90:53676) 2024-04-26T04:32:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba6d304700 1 -- 172.21.15.90:0/1721484854 learned_addr learned my addr 172.21.15.90:0/1721484854 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:32:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba6d304700 1 -- 172.21.15.90:0/1721484854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba64009550 con 0x7fba68105b10 2024-04-26T04:32:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba6d304700 1 --2- 172.21.15.90:0/1721484854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68199010 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fba68106840 tx=0x7fba64004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:32:59.037 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba5e7fc700 1 -- 172.21.15.90:0/1721484854 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba64003ce0 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba5e7fc700 1 -- 172.21.15.90:0/1721484854 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba64020070 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba68199750 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba5e7fc700 1 -- 172.21.15.90:0/1721484854 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fba64011630 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba68199b70 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba5e7fc700 1 -- 172.21.15.90:0/1721484854 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba64022030 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.383+0000 7fba5e7fc700 1 -- 172.21.15.90:0/1721484854 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba64025d40 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.384+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba681927b0 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.386+0000 7fba5e7fc700 1 -- 172.21.15.90:0/1721484854 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fba64025480 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.456+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fba6802cc30 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba5e7fc700 1 -- 172.21.15.90:0/1721484854 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+325 (secure 0 0 0) 0x7fba64011c20 con 0x7fba68105b10 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 msgr2=0x7fba68199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba6f568700 1 --2- 172.21.15.90:0/1721484854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68199010 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fba68106840 tx=0x7fba64004750 comp rx=0 tx=0).stop 2024-04-26T04:32:59.038 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 shutdown_connections 2024-04-26T04:32:59.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba6f568700 1 --2- 172.21.15.90:0/1721484854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fba68105b10 0x7fba68199010 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:32:59.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 >> 172.21.15.90:0/1721484854 conn(0x7fba68101180 msgr2=0x7fba6818d570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:32:59.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 shutdown_connections 2024-04-26T04:32:59.039 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:58.457+0000 7fba6f568700 1 -- 172.21.15.90:0/1721484854 wait complete. 2024-04-26T04:32:59.039 INFO:teuthology.orchestra.run.smithi090.stdout:mon is available 2024-04-26T04:32:59.039 INFO:teuthology.orchestra.run.smithi090.stdout:Assimilating anything we can from ceph.conf... 2024-04-26T04:32:59.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:58 smithi090 ceph-mon[26278]: mon.smithi090 is new leader, mons smithi090 in quorum (ranks 0) 2024-04-26T04:32:59.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:58 smithi090 ceph-mon[26278]: monmap e1: 1 mons at {smithi090=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} 2024-04-26T04:32:59.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:58 smithi090 ceph-mon[26278]: fsmap 2024-04-26T04:32:59.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:58 smithi090 ceph-mon[26278]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T04:32:59.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:58 smithi090 ceph-mon[26278]: mgrmap e1: no daemons active 2024-04-26T04:32:59.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:58 smithi090 ceph-mon[26278]: from='client.? 172.21.15.90:0/1721484854' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-26T04:33:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:59 smithi090 ceph-mon[26278]: from='client.? 172.21.15.90:0/2313226501' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T04:33:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:32:59 smithi090 ceph-mon[26278]: from='client.? 172.21.15.90:0/2313226501' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-26T04:33:00.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:00.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T04:33:00.401 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.90:3300,v1:172.21.15.90:6789] 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T04:33:00.402 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc099fc700 1 Processor -- start 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc099fc700 1 -- start start 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc099fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04108a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc099fc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04079d10 con 0x7fbc04106660 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc02ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04108a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc02ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04108a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53678/0 (socket says 172.21.15.90:53678) 2024-04-26T04:33:00.403 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc02ffd700 1 -- 172.21.15.90:0/4012376598 learned_addr learned my addr 172.21.15.90:0/4012376598 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc02ffd700 1 -- 172.21.15.90:0/4012376598 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc04079e50 con 0x7fbc04106660 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.420+0000 7fbc02ffd700 1 --2- 172.21.15.90:0/4012376598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04108a40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40098a0 tx=0x7fbbf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d39c645635bf64 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc01ffb700 1 -- 172.21.15.90:0/4012376598 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf4004180 con 0x7fbc04106660 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc01ffb700 1 -- 172.21.15.90:0/4012376598 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf40042e0 con 0x7fbc04106660 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc01ffb700 1 -- 172.21.15.90:0/4012376598 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf40036a0 con 0x7fbc04106660 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc099fc700 1 -- 172.21.15.90:0/4012376598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 msgr2=0x7fbc04108a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc099fc700 1 --2- 172.21.15.90:0/4012376598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04108a40 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fbbf40098a0 tx=0x7fbbf4009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc099fc700 1 -- 172.21.15.90:0/4012376598 shutdown_connections 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc099fc700 1 --2- 172.21.15.90:0/4012376598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04108a40 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc099fc700 1 -- 172.21.15.90:0/4012376598 >> 172.21.15.90:0/4012376598 conn(0x7fbc04100080 msgr2=0x7fbc04102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:00.404 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc099fc700 1 -- 172.21.15.90:0/4012376598 shutdown_connections 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.421+0000 7fbc099fc700 1 -- 172.21.15.90:0/4012376598 wait complete. 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc099fc700 1 Processor -- start 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc099fc700 1 -- start start 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc099fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc099fc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc04199550 con 0x7fbc04106660 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc02ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc02ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53684/0 (socket says 172.21.15.90:53684) 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc02ffd700 1 -- 172.21.15.90:0/2313226501 learned_addr learned my addr 172.21.15.90:0/2313226501 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.422+0000 7fbc02ffd700 1 -- 172.21.15.90:0/2313226501 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbf4009550 con 0x7fbc04106660 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc02ffd700 1 --2- 172.21.15.90:0/2313226501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04199010 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4004000 tx=0x7fbbf4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf4003ce0 con 0x7fbc04106660 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbf4020070 con 0x7fbc04106660 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc04199750 con 0x7fbc04106660 2024-04-26T04:33:00.405 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fbbf4011630 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc04199b70 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fbbf4022030 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.423+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fbbf4025d00 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.424+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc04192dd0 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.426+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fbbf4025480 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.493+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fbc0402ce60 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.497+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 7 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fbbf4011c20 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.497+0000 7fbc089fa700 1 -- 172.21.15.90:0/2313226501 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7fbbf401c500 con 0x7fbc04106660 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.498+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 msgr2=0x7fbc04199010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.498+0000 7fbc099fc700 1 --2- 172.21.15.90:0/2313226501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04199010 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4004000 tx=0x7fbbf4004750 comp rx=0 tx=0).stop 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.498+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 shutdown_connections 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.498+0000 7fbc099fc700 1 --2- 172.21.15.90:0/2313226501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc04106660 0x7fbc04199010 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:00.406 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.498+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 >> 172.21.15.90:0/2313226501 conn(0x7fbc04100080 msgr2=0x7fbc04101990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:00.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.498+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 shutdown_connections 2024-04-26T04:33:00.407 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:32:59.498+0000 7fbc099fc700 1 -- 172.21.15.90:0/2313226501 wait complete. 2024-04-26T04:33:00.407 INFO:teuthology.orchestra.run.smithi090.stdout:Generating new minimal ceph.conf... 2024-04-26T04:33:00.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:00 smithi090 ceph-mon[26278]: from='client.? 172.21.15.90:0/936689850' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.811+0000 7f416f8ff700 1 Processor -- start 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.812+0000 7f416f8ff700 1 -- start start 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.812+0000 7f416f8ff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.812+0000 7f416f8ff700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168106460 con 0x7f4168105b10 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.812+0000 7f416d69b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.812+0000 7f416d69b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53696/0 (socket says 172.21.15.90:53696) 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.812+0000 7f416d69b700 1 -- 172.21.15.90:0/240312549 learned_addr learned my addr 172.21.15.90:0/240312549 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:01.448 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.812+0000 7f416d69b700 1 -- 172.21.15.90:0/240312549 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41681065a0 con 0x7f4168105b10 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416d69b700 1 --2- 172.21.15.90:0/240312549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168105f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41640098a0 tx=0x7f4164009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4aa0ad8dcb7464c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f415ffff700 1 -- 172.21.15.90:0/240312549 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4164004180 con 0x7f4168105b10 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f415ffff700 1 -- 172.21.15.90:0/240312549 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41640042e0 con 0x7f4168105b10 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f415ffff700 1 -- 172.21.15.90:0/240312549 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4164011a50 con 0x7f4168105b10 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416f8ff700 1 -- 172.21.15.90:0/240312549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 msgr2=0x7f4168105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416f8ff700 1 --2- 172.21.15.90:0/240312549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168105f20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f41640098a0 tx=0x7f4164009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416f8ff700 1 -- 172.21.15.90:0/240312549 shutdown_connections 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416f8ff700 1 --2- 172.21.15.90:0/240312549 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168105f20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416f8ff700 1 -- 172.21.15.90:0/240312549 >> 172.21.15.90:0/240312549 conn(0x7f4168101180 msgr2=0x7f4168103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416f8ff700 1 -- 172.21.15.90:0/240312549 shutdown_connections 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.813+0000 7f416f8ff700 1 -- 172.21.15.90:0/240312549 wait complete. 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.814+0000 7f416f8ff700 1 Processor -- start 2024-04-26T04:33:01.449 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.814+0000 7f416f8ff700 1 -- start start 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.814+0000 7f416f8ff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.814+0000 7f416f8ff700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4168199d50 con 0x7f4168105b10 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.814+0000 7f416d69b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.814+0000 7f416d69b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53698/0 (socket says 172.21.15.90:53698) 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.814+0000 7f416d69b700 1 -- 172.21.15.90:0/936689850 learned_addr learned my addr 172.21.15.90:0/936689850 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f416d69b700 1 -- 172.21.15.90:0/936689850 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4164009550 con 0x7f4168105b10 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f416d69b700 1 --2- 172.21.15.90:0/936689850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168199810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4164000c00 tx=0x7f4164011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f415e7fc700 1 -- 172.21.15.90:0/936689850 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4164003b90 con 0x7f4168105b10 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4168199f50 con 0x7f4168105b10 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f415e7fc700 1 -- 172.21.15.90:0/936689850 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f41640248b0 con 0x7f4168105b10 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f415e7fc700 1 -- 172.21.15.90:0/936689850 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f416401a8e0 con 0x7f4168105b10 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f416819a370 con 0x7f4168105b10 2024-04-26T04:33:01.450 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f415e7fc700 1 -- 172.21.15.90:0/936689850 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f416402d3f0 con 0x7f4168105b10 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.815+0000 7f415e7fc700 1 -- 172.21.15.90:0/936689850 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f41640194d0 con 0x7f4168105b10 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.816+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4168062380 con 0x7f4168105b10 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.818+0000 7f415e7fc700 1 -- 172.21.15.90:0/936689850 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f4164024db0 con 0x7f4168105b10 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.885+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f4168107980 con 0x7f4168105b10 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f415e7fc700 1 -- 172.21.15.90:0/936689850 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f4164024ad0 con 0x7f4168105b10 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 msgr2=0x7f4168199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f416f8ff700 1 --2- 172.21.15.90:0/936689850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168199810 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4164000c00 tx=0x7f4164011ab0 comp rx=0 tx=0).stop 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 shutdown_connections 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f416f8ff700 1 --2- 172.21.15.90:0/936689850 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4168105b10 0x7f4168199810 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 >> 172.21.15.90:0/936689850 conn(0x7f4168101180 msgr2=0x7f4168101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 shutdown_connections 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:00.886+0000 7f416f8ff700 1 -- 172.21.15.90:0/936689850 wait complete. 2024-04-26T04:33:01.451 INFO:teuthology.orchestra.run.smithi090.stdout:Restarting the monitor... 2024-04-26T04:33:01.720 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:01 smithi090 systemd[1]: Stopping Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:33:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:01 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[26274]: 2024-04-26T04:33:01.717+0000 7f993b806700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi090 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:33:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:01 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[26274]: 2024-04-26T04:33:01.717+0000 7f993b806700 -1 mon.smithi090@0(leader) e1 *** Got Signal Terminated *** 2024-04-26T04:33:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:01 smithi090 podman[26523]: 2024-04-26 04:33:01.810665303 +0000 UTC m=+0.190926135 container died 9c56a13eb766d8dcb1d209f5be275483a307d89ae77af221943533cc7fdb554c (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True) 2024-04-26T04:33:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 podman[26523]: 2024-04-26 04:33:02.143962572 +0000 UTC m=+0.524223403 container cleanup 9c56a13eb766d8dcb1d209f5be275483a307d89ae77af221943533cc7fdb554c (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2) 2024-04-26T04:33:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 bash[26523]: ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090 2024-04-26T04:33:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 podman[26537]: 2024-04-26 04:33:02.320041555 +0000 UTC m=+0.507084770 container remove 9c56a13eb766d8dcb1d209f5be275483a307d89ae77af221943533cc7fdb554c (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, ceph=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD) 2024-04-26T04:33:02.878 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service: Deactivated successfully. 2024-04-26T04:33:02.878 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 systemd[1]: Stopped Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:33:02.879 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 systemd[1]: Starting Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:33:02.879 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 podman[26629]: 2024-04-26 04:33:02.7794166 +0000 UTC m=+0.114378250 container create cb4c73ccf4d5b04e1b37736e3510bcbcdade8f89a3bdaa1dbb0b6a018b81d457 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.build-date=20231212, RELEASE=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T04:33:02.879 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:02 smithi090 podman[26629]: 2024-04-26 04:33:02.683153268 +0000 UTC m=+0.018114927 image pull dc2bc1663786ad5862ef6efcff18243ab7dfeda0bcb911ddf757a0d20feca946 quay.io/ceph/ceph:v18.2.0 2024-04-26T04:33:03.129 INFO:teuthology.orchestra.run.smithi090.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-26T04:33:03.335 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 podman[26629]: 2024-04-26 04:33:03.050352015 +0000 UTC m=+0.385313670 container init cb4c73ccf4d5b04e1b37736e3510bcbcdade8f89a3bdaa1dbb0b6a018b81d457 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True) 2024-04-26T04:33:03.335 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 podman[26629]: 2024-04-26 04:33:03.053113162 +0000 UTC m=+0.388074814 container start cb4c73ccf4d5b04e1b37736e3510bcbcdade8f89a3bdaa1dbb0b6a018b81d457 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, maintainer=Guillaume Abrioux , io.buildah.version=1.29.1, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS) 2024-04-26T04:33:03.335 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T04:33:03.335 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable), process ceph-mon, pid 2 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: pidfile_write: ignore empty --pid-file 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: load: jerasure load: lrc 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: RocksDB version: 7.9.2 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Git sha 0 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Compile date 2023-08-03 19:21:13 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: DB SUMMARY 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: DB Session ID: QNYFLTCAO5CEHDBQL3E1 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: CURRENT file: CURRENT 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi090/store.db dir, Total Num: 1, files: 000008.sst 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi090/store.db: 000009.log size: 98258 ; 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.error_if_exists: 0 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.create_if_missing: 0 2024-04-26T04:33:03.336 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.paranoid_checks: 1 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.env: 0x55e56f504720 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.info_log: 0x55e57216d2c0 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.statistics: (nil) 2024-04-26T04:33:03.337 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.use_fsync: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_log_file_size: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.allow_fallocate: 1 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.use_direct_reads: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.db_log_dir: 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.wal_dir: 2024-04-26T04:33:03.339 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.write_buffer_manager: 0x55e5713de3c0 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T04:33:03.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.unordered_write: 0 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.row_cache: None 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.wal_filter: None 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.two_write_queues: 0 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T04:33:03.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.wal_compression: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.atomic_flush: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.log_readahead_size: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_background_jobs: 2 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_background_compactions: -1 2024-04-26T04:33:03.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_subcompactions: 1 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_open_files: -1 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T04:33:03.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_background_flushes: -1 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Compression algorithms supported: 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kZSTD supported: 0 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kXpressCompression supported: 0 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kZlibCompression supported: 1 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kSnappyCompression supported: 1 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kLZ4Compression supported: 1 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: kBZip2Compression supported: 0 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000010 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T04:33:03.344 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.merge_operator: 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_filter: None 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55e57216d3c0) 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_type: 0 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_index_type: 0 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_shortening: 1 2024-04-26T04:33:03.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: checksum: 4 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: no_block_cache: 0 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache: 0x55e57147f350 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_options: 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: capacity : 536870912 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_shard_bits : 4 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: strict_capacity_limit : 0 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_compressed: (nil) 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: persistent_cache: (nil) 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size: 4096 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size_deviation: 10 2024-04-26T04:33:03.346 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_restart_interval: 16 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_block_restart_interval: 1 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: metadata_block_size: 4096 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: partition_filters: 0 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: use_delta_encoding: 1 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: filter_policy: bloomfilter 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: whole_key_filtering: 1 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: verify_compression: 0 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: format_version: 5 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_index_compression: 1 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_align: 0 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_auto_readahead_size: 262144 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: prepopulate_block_cache: 0 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: initial_auto_readahead_size: 8192 2024-04-26T04:33:03.347 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression: NoCompression 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.num_levels: 7 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T04:33:03.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T04:33:03.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T04:33:03.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T04:33:03.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.table_properties_collectors: 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.inplace_update_support: 0 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.bloom_locality: 0 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.max_successive_merges: 0 2024-04-26T04:33:03.352 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.ttl: 2592000 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.enable_blob_files: false 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.min_blob_size: 0 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T04:33:03.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b4a63d83-9182-4f50-9e8e-6355b5cea0f5 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714105983082027, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714105983083010, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 93509, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 314, "table_properties": {"data_size": 91531, "index_size": 260, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13576, "raw_average_key_size": 49, "raw_value_size": 84140, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 273, "num_filter_entries": 273, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714105983, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b4a63d83-9182-4f50-9e8e-6355b5cea0f5", "db_session_id": "QNYFLTCAO5CEHDBQL3E1", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714105983083069, "job": 1, "event": "recovery_finished"} 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi090/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55e57151c000 2024-04-26T04:33:03.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: DB pointer 0x55e571508000 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** DB Stats ** 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T04:33:03.355 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: L0 2/0 93.19 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Sum 2/0 93.19 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T04:33:03.356 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative compaction: 0.00 GB write, 31.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval compaction: 0.00 GB write, 31.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache BinnedLRUCache@0x55e57147f350#2 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: starting mon.smithi090 rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi090 fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???) e1 preinit fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???).mds e1 new map 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???).mds e1 print_map 2024-04-26T04:33:03.357 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: e1 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: legacy client fscid: -1 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: No filesystems configured 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expand map: {default=false} 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta from 'false' to 'false' 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expanded map: {default=false} 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expand map: {default=info} 2024-04-26T04:33:03.358 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta from 'info' to 'info' 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expanded map: {default=info} 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expand map: {default=daemon} 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta from 'daemon' to 'daemon' 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expanded map: {default=daemon} 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expand map: {default=debug} 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta from 'debug' to 'debug' 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: expand_channel_meta expanded map: {default=debug} 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 bash[26629]: cb4c73ccf4d5b04e1b37736e3510bcbcdade8f89a3bdaa1dbb0b6a018b81d457 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 systemd[1]: Started Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mon.smithi090 is new leader, mons smithi090 in quorum (ranks 0) 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: monmap e1: 1 mons at {smithi090=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0]} removed_ranks: {} 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: fsmap 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: osdmap e1: 0 total, 0 up, 0 in 2024-04-26T04:33:03.359 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:03 smithi090 ceph-mon[26648]: mgrmap e1: no daemons active 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.530+0000 7fc3a54cb700 1 Processor -- start 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.530+0000 7fc3a54cb700 1 -- start start 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.530+0000 7fc3a54cb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0108a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.530+0000 7fc3a54cb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0079d30 con 0x7fc3a0106640 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0108a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0108a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53706/0 (socket says 172.21.15.90:53706) 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39effd700 1 -- 172.21.15.90:0/258781021 learned_addr learned my addr 172.21.15.90:0/258781021 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39effd700 1 -- 172.21.15.90:0/258781021 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3a0079e70 con 0x7fc3a0106640 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39effd700 1 --2- 172.21.15.90:0/258781021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0108a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3900098a0 tx=0x7fc390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22c38bfe1d60f93 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39e7fc700 1 -- 172.21.15.90:0/258781021 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc390004180 con 0x7fc3a0106640 2024-04-26T04:33:04.150 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39e7fc700 1 -- 172.21.15.90:0/258781021 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc3900042e0 con 0x7fc3a0106640 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.531+0000 7fc39e7fc700 1 -- 172.21.15.90:0/258781021 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc390011a50 con 0x7fc3a0106640 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.532+0000 7fc3a54cb700 1 -- 172.21.15.90:0/258781021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 msgr2=0x7fc3a0108a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.532+0000 7fc3a54cb700 1 --2- 172.21.15.90:0/258781021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0108a20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fc3900098a0 tx=0x7fc390009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.532+0000 7fc3a54cb700 1 -- 172.21.15.90:0/258781021 shutdown_connections 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.532+0000 7fc3a54cb700 1 --2- 172.21.15.90:0/258781021 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0108a20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.532+0000 7fc3a54cb700 1 -- 172.21.15.90:0/258781021 >> 172.21.15.90:0/258781021 conn(0x7fc3a0100080 msgr2=0x7fc3a0102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.532+0000 7fc3a54cb700 1 -- 172.21.15.90:0/258781021 shutdown_connections 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.532+0000 7fc3a54cb700 1 -- 172.21.15.90:0/258781021 wait complete. 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc3a54cb700 1 Processor -- start 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc3a54cb700 1 -- start start 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc3a54cb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc3a54cb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3a0199d60 con 0x7fc3a0106640 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc39effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:04.151 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc39effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53712/0 (socket says 172.21.15.90:53712) 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc39effd700 1 -- 172.21.15.90:0/4188058615 learned_addr learned my addr 172.21.15.90:0/4188058615 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.533+0000 7fc39effd700 1 -- 172.21.15.90:0/4188058615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc390009550 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc39effd700 1 --2- 172.21.15.90:0/4188058615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0199820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc390006af0 tx=0x7fc390011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc39cff9700 1 -- 172.21.15.90:0/4188058615 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc390003b90 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc39cff9700 1 -- 172.21.15.90:0/4188058615 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fc39001a8a0 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3a0199fc0 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc39cff9700 1 -- 172.21.15.90:0/4188058615 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc390019880 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3a019a440 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc39cff9700 1 -- 172.21.15.90:0/4188058615 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc39002d440 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.534+0000 7fc39cff9700 1 -- 172.21.15.90:0/4188058615 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc3900247c0 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.535+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a0109b20 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.537+0000 7fc39cff9700 1 -- 172.21.15.90:0/4188058615 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7fc39001aa10 con 0x7fc3a0106640 2024-04-26T04:33:04.152 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.604+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fc3a0109d10 con 0x7fc3a0106640 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.609+0000 7fc39cff9700 1 -- 172.21.15.90:0/4188058615 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fc39001abb0 con 0x7fc3a0106640 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.610+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 msgr2=0x7fc3a0199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.610+0000 7fc3a54cb700 1 --2- 172.21.15.90:0/4188058615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0199820 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fc390006af0 tx=0x7fc390011a90 comp rx=0 tx=0).stop 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.610+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 shutdown_connections 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.610+0000 7fc3a54cb700 1 --2- 172.21.15.90:0/4188058615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3a0106640 0x7fc3a0199820 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.610+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 >> 172.21.15.90:0/4188058615 conn(0x7fc3a0100080 msgr2=0x7fc3a0101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.610+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 shutdown_connections 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:03.610+0000 7fc3a54cb700 1 -- 172.21.15.90:0/4188058615 wait complete. 2024-04-26T04:33:04.153 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-26T04:33:04.176 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:33:04.176 INFO:teuthology.orchestra.run.smithi090.stdout:Creating mgr... 2024-04-26T04:33:04.176 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-26T04:33:04.177 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-26T04:33:04.177 INFO:teuthology.orchestra.run.smithi090.stdout:Verifying port 0.0.0.0:8443 ... 2024-04-26T04:33:04.427 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f018e902-0385-11ef-bc93-c7b262605968@mgr.smithi090.iwrvwa 2024-04-26T04:33:04.427 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f018e902-0385-11ef-bc93-c7b262605968@mgr.smithi090.iwrvwa.service: Unit ceph-f018e902-0385-11ef-bc93-c7b262605968@mgr.smithi090.iwrvwa.service not loaded. 2024-04-26T04:33:04.573 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f018e902-0385-11ef-bc93-c7b262605968.target.wants/ceph-f018e902-0385-11ef-bc93-c7b262605968@mgr.smithi090.iwrvwa.service → /etc/systemd/system/ceph-f018e902-0385-11ef-bc93-c7b262605968@.service. 2024-04-26T04:33:04.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:04 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/4188058615' entity='client.admin' 2024-04-26T04:33:05.081 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:33:05.081 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T04:33:05.088 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:33:05.088 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T04:33:05.088 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T04:33:05.088 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to enable service . firewalld.service is not available 2024-04-26T04:33:05.096 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:33:05.096 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T04:33:05.103 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:33:05.104 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T04:33:05.104 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T04:33:05.104 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[9283, 8765, 8443]>. firewalld.service is not available 2024-04-26T04:33:05.104 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr to start... 2024-04-26T04:33:05.104 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr... 2024-04-26T04:33:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:05 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2428025892' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "f018e902-0385-11ef-bc93-c7b262605968", 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T04:33:06.220 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T04:33:06.221 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T04:33:06.222 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T04:33:06.223 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T04:32:57.928966+0000", 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.535+0000 7f05d5e45700 1 Processor -- start 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.535+0000 7f05d5e45700 1 -- start start 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.536+0000 7f05d5e45700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d01075e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.536+0000 7f05d5e45700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0107b20 con 0x7f05d01071d0 2024-04-26T04:33:06.224 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.536+0000 7f05cf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d01075e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.536+0000 7f05cf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d01075e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53734/0 (socket says 172.21.15.90:53734) 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.536+0000 7f05cf7fe700 1 -- 172.21.15.90:0/2227318431 learned_addr learned my addr 172.21.15.90:0/2227318431 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.536+0000 7f05cf7fe700 1 -- 172.21.15.90:0/2227318431 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05d0107c60 con 0x7f05d01071d0 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.537+0000 7f05cf7fe700 1 --2- 172.21.15.90:0/2227318431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d01075e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05c00098a0 tx=0x7f05c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1a599dfb79f303a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.537+0000 7f05ce7fc700 1 -- 172.21.15.90:0/2227318431 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c0004180 con 0x7f05d01071d0 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.537+0000 7f05ce7fc700 1 -- 172.21.15.90:0/2227318431 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f05c00042e0 con 0x7f05d01071d0 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.537+0000 7f05ce7fc700 1 -- 172.21.15.90:0/2227318431 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c0011a50 con 0x7f05d01071d0 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 -- 172.21.15.90:0/2227318431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 msgr2=0x7f05d01075e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 --2- 172.21.15.90:0/2227318431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d01075e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f05c00098a0 tx=0x7f05c0009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 -- 172.21.15.90:0/2227318431 shutdown_connections 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 --2- 172.21.15.90:0/2227318431 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d01075e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 -- 172.21.15.90:0/2227318431 >> 172.21.15.90:0/2227318431 conn(0x7f05d00ff7e0 msgr2=0x7f05d0101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 -- 172.21.15.90:0/2227318431 shutdown_connections 2024-04-26T04:33:06.225 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 -- 172.21.15.90:0/2227318431 wait complete. 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.538+0000 7f05d5e45700 1 Processor -- start 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05d5e45700 1 -- start start 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05d5e45700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d0199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05d5e45700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05d0199db0 con 0x7f05d01071d0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05cf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d0199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05cf7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d0199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53748/0 (socket says 172.21.15.90:53748) 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05cf7fe700 1 -- 172.21.15.90:0/2428025892 learned_addr learned my addr 172.21.15.90:0/2428025892 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05cf7fe700 1 -- 172.21.15.90:0/2428025892 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05c0009550 con 0x7f05d01071d0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05cf7fe700 1 --2- 172.21.15.90:0/2428025892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d0199870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05c00119e0 tx=0x7f05c0011ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05ccff9700 1 -- 172.21.15.90:0/2428025892 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c0003b90 con 0x7f05d01071d0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05ccff9700 1 -- 172.21.15.90:0/2428025892 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f05c001a8a0 con 0x7f05d01071d0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05d0199fb0 con 0x7f05d01071d0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.539+0000 7f05ccff9700 1 -- 172.21.15.90:0/2428025892 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f05c0019930 con 0x7f05d01071d0 2024-04-26T04:33:06.226 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.540+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05d019a3d0 con 0x7f05d01071d0 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.540+0000 7f05ccff9700 1 -- 172.21.15.90:0/2428025892 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f05c002d440 con 0x7f05d01071d0 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.540+0000 7f05ccff9700 1 -- 172.21.15.90:0/2428025892 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f05c00248b0 con 0x7f05d01071d0 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.540+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05d0062380 con 0x7f05d01071d0 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.543+0000 7f05ccff9700 1 -- 172.21.15.90:0/2428025892 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f05c001aa10 con 0x7f05d01071d0 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.615+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f05d0104f80 con 0x7f05d01071d0 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.616+0000 7f05ccff9700 1 -- 172.21.15.90:0/2428025892 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f05c001abb0 con 0x7f05d01071d0 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.617+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 msgr2=0x7f05d0199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.617+0000 7f05d5e45700 1 --2- 172.21.15.90:0/2428025892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d0199870 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f05c00119e0 tx=0x7f05c0011ac0 comp rx=0 tx=0).stop 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.617+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 shutdown_connections 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.617+0000 7f05d5e45700 1 --2- 172.21.15.90:0/2428025892 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f05d01071d0 0x7f05d0199870 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.617+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 >> 172.21.15.90:0/2428025892 conn(0x7f05d00ff7e0 msgr2=0x7f05d0100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.618+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 shutdown_connections 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:05.618+0000 7f05d5e45700 1 -- 172.21.15.90:0/2428025892 wait complete. 2024-04-26T04:33:06.227 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (1/15)... 2024-04-26T04:33:09.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:09.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:09.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "f018e902-0385-11ef-bc93-c7b262605968", 2024-04-26T04:33:09.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T04:33:09.298 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:09.299 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-26T04:33:09.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T04:33:09.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:09.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T04:33:09.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:09.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T04:33:09.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T04:33:09.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T04:33:09.310 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T04:33:09.311 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T04:32:57.928966+0000", 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:09.312 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48c54ac700 1 Processor -- start 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48c54ac700 1 -- start start 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48c54ac700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c01089e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48c54ac700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c0079d70 con 0x7f48c0106600 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48beffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c01089e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48beffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c01089e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53004/0 (socket says 172.21.15.90:53004) 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48beffd700 1 -- 172.21.15.90:0/3584271888 learned_addr learned my addr 172.21.15.90:0/3584271888 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48beffd700 1 -- 172.21.15.90:0/3584271888 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48c0079eb0 con 0x7f48c0106600 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.630+0000 7f48beffd700 1 --2- 172.21.15.90:0/3584271888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c01089e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48b00098a0 tx=0x7f48b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76b789d3bc97154d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48bdffb700 1 -- 172.21.15.90:0/3584271888 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48b0004180 con 0x7f48c0106600 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48bdffb700 1 -- 172.21.15.90:0/3584271888 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b00042e0 con 0x7f48c0106600 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48bdffb700 1 -- 172.21.15.90:0/3584271888 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48b0011a50 con 0x7f48c0106600 2024-04-26T04:33:09.313 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48c54ac700 1 -- 172.21.15.90:0/3584271888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 msgr2=0x7f48c01089e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48c54ac700 1 --2- 172.21.15.90:0/3584271888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c01089e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f48b00098a0 tx=0x7f48b0009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48c54ac700 1 -- 172.21.15.90:0/3584271888 shutdown_connections 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48c54ac700 1 --2- 172.21.15.90:0/3584271888 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c01089e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48c54ac700 1 -- 172.21.15.90:0/3584271888 >> 172.21.15.90:0/3584271888 conn(0x7f48c0100080 msgr2=0x7f48c0102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48c54ac700 1 -- 172.21.15.90:0/3584271888 shutdown_connections 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.631+0000 7f48c54ac700 1 -- 172.21.15.90:0/3584271888 wait complete. 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.632+0000 7f48c54ac700 1 Processor -- start 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.632+0000 7f48c54ac700 1 -- start start 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.632+0000 7f48c54ac700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c0105f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.632+0000 7f48c54ac700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c01064a0 con 0x7f48c0106600 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48beffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c0105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48beffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c0105f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53012/0 (socket says 172.21.15.90:53012) 2024-04-26T04:33:09.314 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48beffd700 1 -- 172.21.15.90:0/2254899261 learned_addr learned my addr 172.21.15.90:0/2254899261 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48beffd700 1 -- 172.21.15.90:0/2254899261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b0009550 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48beffd700 1 --2- 172.21.15.90:0/2254899261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c0105f60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48a7fff700 1 -- 172.21.15.90:0/2254899261 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48b0003b90 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48a7fff700 1 -- 172.21.15.90:0/2254899261 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f48b00248b0 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c01045b0 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48a7fff700 1 -- 172.21.15.90:0/2254899261 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f48b001a8e0 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c01049d0 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48a7fff700 1 -- 172.21.15.90:0/2254899261 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f48b002d3f0 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.633+0000 7f48a7fff700 1 -- 172.21.15.90:0/2254899261 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f48b0019490 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.634+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c0109b70 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.636+0000 7f48a7fff700 1 -- 172.21.15.90:0/2254899261 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f48b0024a20 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.710+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f48c0109d90 con 0x7f48c0106600 2024-04-26T04:33:09.315 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.710+0000 7f48a7fff700 1 -- 172.21.15.90:0/2254899261 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f48b0024440 con 0x7f48c0106600 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.711+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 msgr2=0x7f48c0105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.711+0000 7f48c54ac700 1 --2- 172.21.15.90:0/2254899261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c0105f60 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f48b0000c00 tx=0x7f48b0011ab0 comp rx=0 tx=0).stop 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.711+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 shutdown_connections 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.711+0000 7f48c54ac700 1 --2- 172.21.15.90:0/2254899261 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c0106600 0x7f48c0105f60 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.711+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 >> 172.21.15.90:0/2254899261 conn(0x7f48c0100080 msgr2=0x7f48c0101730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.711+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 shutdown_connections 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:08.711+0000 7f48c54ac700 1 -- 172.21.15.90:0/2254899261 wait complete. 2024-04-26T04:33:09.316 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (2/15)... 2024-04-26T04:33:09.370 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:09 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2254899261' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T04:33:11.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: Activating manager daemon smithi090.iwrvwa 2024-04-26T04:33:11.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: mgrmap e2: smithi090.iwrvwa(active, starting, since 0.00305803s) 2024-04-26T04:33:11.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:11.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:11 smithi090 ceph-mon[26648]: from='mgr.14100 172.21.15.90:0/2701011264' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "f018e902-0385-11ef-bc93-c7b262605968", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T04:33:12.473 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:12.474 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T04:33:12.475 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T04:33:12.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T04:33:12.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T04:32:57.928966+0000", 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:12.485 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.735+0000 7f0ce95b0700 1 Processor -- start 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.735+0000 7f0ce95b0700 1 -- start start 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.735+0000 7f0ce95b0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.735+0000 7f0ce95b0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce407a830 con 0x7f0ce407bdf0 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.735+0000 7f0ce2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.735+0000 7f0ce2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53102/0 (socket says 172.21.15.90:53102) 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.735+0000 7f0ce2ffd700 1 -- 172.21.15.90:0/3945141553 learned_addr learned my addr 172.21.15.90:0/3945141553 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.736+0000 7f0ce2ffd700 1 -- 172.21.15.90:0/3945141553 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ce407a970 con 0x7f0ce407bdf0 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.736+0000 7f0ce2ffd700 1 --2- 172.21.15.90:0/3945141553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce407a2f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009e80 tx=0x7f0cdc00b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbaeaf9b245ea68 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.736+0000 7f0ce1ffb700 1 -- 172.21.15.90:0/3945141553 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cdc00e070 con 0x7f0ce407bdf0 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.736+0000 7f0ce1ffb700 1 -- 172.21.15.90:0/3945141553 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0cdc003e90 con 0x7f0ce407bdf0 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.736+0000 7f0ce1ffb700 1 -- 172.21.15.90:0/3945141553 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cdc004380 con 0x7f0ce407bdf0 2024-04-26T04:33:12.486 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 -- 172.21.15.90:0/3945141553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 msgr2=0x7f0ce407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 --2- 172.21.15.90:0/3945141553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce407a2f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc009e80 tx=0x7f0cdc00b330 comp rx=0 tx=0).stop 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 -- 172.21.15.90:0/3945141553 shutdown_connections 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 --2- 172.21.15.90:0/3945141553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce407a2f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 -- 172.21.15.90:0/3945141553 >> 172.21.15.90:0/3945141553 conn(0x7f0ce4101180 msgr2=0x7f0ce4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 -- 172.21.15.90:0/3945141553 shutdown_connections 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 -- 172.21.15.90:0/3945141553 wait complete. 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 Processor -- start 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.737+0000 7f0ce95b0700 1 -- start start 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce95b0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce419ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce95b0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ce419bf80 con 0x7f0ce407bdf0 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce419ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce419ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53118/0 (socket says 172.21.15.90:53118) 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce2ffd700 1 -- 172.21.15.90:0/1141012545 learned_addr learned my addr 172.21.15.90:0/1141012545 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce2ffd700 1 -- 172.21.15.90:0/1141012545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cdc009b30 con 0x7f0ce407bdf0 2024-04-26T04:33:12.487 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce2ffd700 1 --2- 172.21.15.90:0/1141012545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce419ba40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b7b0 tx=0x7f0cdc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ccbfff700 1 -- 172.21.15.90:0/1141012545 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cdc00e040 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ccbfff700 1 -- 172.21.15.90:0/1141012545 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f0cdc003c70 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ccbfff700 1 -- 172.21.15.90:0/1141012545 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0cdc01b730 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ce419c180 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.738+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ce419c600 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.739+0000 7f0ccbfff700 1 -- 172.21.15.90:0/1141012545 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 2) v1 ==== 44840+0+0 (secure 0 0 0) 0x7f0cdc019040 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.739+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ce40623f0 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.739+0000 7f0ccbfff700 1 -- 172.21.15.90:0/1141012545 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0cdc04baa0 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.741+0000 7f0ccbfff700 1 -- 172.21.15.90:0/1141012545 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72422 (secure 0 0 0) 0x7f0ce40623f0 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.812+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f0ce419efe0 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.812+0000 7f0ccbfff700 1 -- 172.21.15.90:0/1141012545 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f0cdc011980 con 0x7f0ce407bdf0 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.813+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 msgr2=0x7f0ce419ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.813+0000 7f0ce95b0700 1 --2- 172.21.15.90:0/1141012545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce419ba40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f0cdc00b7b0 tx=0x7f0cdc004430 comp rx=0 tx=0).stop 2024-04-26T04:33:12.488 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.813+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 shutdown_connections 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.813+0000 7f0ce95b0700 1 --2- 172.21.15.90:0/1141012545 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0ce407bdf0 0x7f0ce419ba40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.813+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 >> 172.21.15.90:0/1141012545 conn(0x7f0ce4101180 msgr2=0x7f0ce4101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.813+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 shutdown_connections 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:11.813+0000 7f0ce95b0700 1 -- 172.21.15.90:0/1141012545 wait complete. 2024-04-26T04:33:12.489 INFO:teuthology.orchestra.run.smithi090.stdout:mgr not available, waiting (3/15)... 2024-04-26T04:33:12.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:12 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1141012545' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T04:33:12.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:12 smithi090 ceph-mon[26648]: mgrmap e3: smithi090.iwrvwa(active, since 1.00615s) 2024-04-26T04:33:15.166 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:15 smithi090 ceph-mon[26648]: mgrmap e4: smithi090.iwrvwa(active, since 2s) 2024-04-26T04:33:15.723 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsid": "f018e902-0385-11ef-bc93-c7b262605968", 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "health": { 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 0 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-26T04:33:15.724 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "smithi090" 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-26T04:33:15.725 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-26T04:33:15.735 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-26T04:33:15.735 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-26T04:33:15.735 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-26T04:33:15.735 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-26T04:33:15.736 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "restful" 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ], 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-26T04:33:15.737 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "modified": "2024-04-26T04:32:57.928966+0000", 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout }, 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.891+0000 7f90625e9700 1 Processor -- start 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.891+0000 7f90625e9700 1 -- start start 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.891+0000 7f90625e9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.891+0000 7f90625e9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c106460 con 0x7f905c105b10 2024-04-26T04:33:15.738 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.891+0000 7f905bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.892+0000 7f905bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53120/0 (socket says 172.21.15.90:53120) 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.892+0000 7f905bfff700 1 -- 172.21.15.90:0/1538681301 learned_addr learned my addr 172.21.15.90:0/1538681301 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.892+0000 7f905bfff700 1 -- 172.21.15.90:0/1538681301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f905c1065a0 con 0x7f905c105b10 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.892+0000 7f905bfff700 1 --2- 172.21.15.90:0/1538681301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c105f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f904c0098a0 tx=0x7f904c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93406c1707f1663c server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.892+0000 7f905affd700 1 -- 172.21.15.90:0/1538681301 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904c004180 con 0x7f905c105b10 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.892+0000 7f905affd700 1 -- 172.21.15.90:0/1538681301 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f904c0042e0 con 0x7f905c105b10 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.892+0000 7f905affd700 1 -- 172.21.15.90:0/1538681301 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904c011a50 con 0x7f905c105b10 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 -- 172.21.15.90:0/1538681301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 msgr2=0x7f905c105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 --2- 172.21.15.90:0/1538681301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c105f20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f904c0098a0 tx=0x7f904c009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 -- 172.21.15.90:0/1538681301 shutdown_connections 2024-04-26T04:33:15.739 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 --2- 172.21.15.90:0/1538681301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c105f20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 -- 172.21.15.90:0/1538681301 >> 172.21.15.90:0/1538681301 conn(0x7f905c101180 msgr2=0x7f905c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 -- 172.21.15.90:0/1538681301 shutdown_connections 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 -- 172.21.15.90:0/1538681301 wait complete. 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 Processor -- start 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.893+0000 7f90625e9700 1 -- start start 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f90625e9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f90625e9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f905c199d50 con 0x7f905c105b10 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f905bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f905bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53126/0 (socket says 172.21.15.90:53126) 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f905bfff700 1 -- 172.21.15.90:0/4146279686 learned_addr learned my addr 172.21.15.90:0/4146279686 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f905bfff700 1 -- 172.21.15.90:0/4146279686 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c009550 con 0x7f905c105b10 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f905bfff700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c199810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c00 tx=0x7f904c011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:15.740 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f90597fa700 1 -- 172.21.15.90:0/4146279686 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904c003b90 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f90597fa700 1 -- 172.21.15.90:0/4146279686 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f904c0248b0 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f905c199f50 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f90597fa700 1 -- 172.21.15.90:0/4146279686 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f904c01a8e0 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.894+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f905c19a370 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.895+0000 7f90597fa700 1 -- 172.21.15.90:0/4146279686 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7f904c01aba0 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.895+0000 7f90597fa700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f90440383a0 0x7f904403a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.895+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f905c062380 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.896+0000 7f90597fa700 1 -- 172.21.15.90:0/4146279686 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f904c04c1e0 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.896+0000 7f905b7fe700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f90440383a0 0x7f904403a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.896+0000 7f905b7fe700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f90440383a0 0x7f904403a850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9050006fd0 tx=0x7f9050006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:14.901+0000 7f90597fa700 1 -- 172.21.15.90:0/4146279686 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f904c018350 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.165+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f905c107a70 con 0x7f905c105b10 2024-04-26T04:33:15.741 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.165+0000 7f90597fa700 1 -- 172.21.15.90:0/4146279686 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1246 (secure 0 0 0) 0x7f904c008040 con 0x7f905c105b10 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f90440383a0 msgr2=0x7f904403a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f90440383a0 0x7f904403a850 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9050006fd0 tx=0x7f9050006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 msgr2=0x7f905c199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c199810 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f904c000c00 tx=0x7f904c011ab0 comp rx=0 tx=0).stop 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 shutdown_connections 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f90440383a0 0x7f904403a850 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 --2- 172.21.15.90:0/4146279686 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f905c105b10 0x7f905c199810 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 >> 172.21.15.90:0/4146279686 conn(0x7f905c101180 msgr2=0x7f905c101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 shutdown_connections 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:15.168+0000 7f90625e9700 1 -- 172.21.15.90:0/4146279686 wait complete. 2024-04-26T04:33:15.742 INFO:teuthology.orchestra.run.smithi090.stdout:mgr is available 2024-04-26T04:33:16.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:16 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/4146279686' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-26T04:33:16.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:16.886 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [global] 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout fsid = f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-26T04:33:16.887 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-26T04:33:16.888 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout [osd] 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.111+0000 7ff3ab4e5700 1 Processor -- start 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.111+0000 7ff3ab4e5700 1 -- start start 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.111+0000 7ff3ab4e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:16.889 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.111+0000 7ff3ab4e5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a407a830 con 0x7ff3a407bdf0 2024-04-26T04:33:16.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.111+0000 7ff3a9281700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:16.890 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.111+0000 7ff3a9281700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53142/0 (socket says 172.21.15.90:53142) 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.111+0000 7ff3a9281700 1 -- 172.21.15.90:0/2269323098 learned_addr learned my addr 172.21.15.90:0/2269323098 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff3a9281700 1 -- 172.21.15.90:0/2269323098 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a407a970 con 0x7ff3a407bdf0 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff3a9281700 1 --2- 172.21.15.90:0/2269323098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a407a2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00098a0 tx=0x7ff3a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9e36fe9be063592 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff39bfff700 1 -- 172.21.15.90:0/2269323098 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3a0004180 con 0x7ff3a407bdf0 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff39bfff700 1 -- 172.21.15.90:0/2269323098 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3a00042e0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff39bfff700 1 -- 172.21.15.90:0/2269323098 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3a0011a50 con 0x7ff3a407bdf0 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2269323098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 msgr2=0x7ff3a407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff3ab4e5700 1 --2- 172.21.15.90:0/2269323098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a407a2f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ff3a00098a0 tx=0x7ff3a0009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2269323098 shutdown_connections 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff3ab4e5700 1 --2- 172.21.15.90:0/2269323098 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a407a2f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:16.891 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.112+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2269323098 >> 172.21.15.90:0/2269323098 conn(0x7ff3a4101180 msgr2=0x7ff3a4103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2269323098 shutdown_connections 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2269323098 wait complete. 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3ab4e5700 1 Processor -- start 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3ab4e5700 1 -- start start 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3ab4e5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a419ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3ab4e5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3a419bfb0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3a9281700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a419ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3a9281700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a419ba70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53152/0 (socket says 172.21.15.90:53152) 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.113+0000 7ff3a9281700 1 -- 172.21.15.90:0/2352115296 learned_addr learned my addr 172.21.15.90:0/2352115296 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.114+0000 7ff3a9281700 1 -- 172.21.15.90:0/2352115296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3a0009550 con 0x7ff3a407bdf0 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.114+0000 7ff3a9281700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a419ba70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000c00 tx=0x7ff3a0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.114+0000 7ff39a7fc700 1 -- 172.21.15.90:0/2352115296 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3a0003b90 con 0x7ff3a407bdf0 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.114+0000 7ff39a7fc700 1 -- 172.21.15.90:0/2352115296 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ff3a00248b0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.892 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.114+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a419c1b0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.114+0000 7ff39a7fc700 1 -- 172.21.15.90:0/2352115296 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff3a001a8e0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.114+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3a419c5d0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.115+0000 7ff39a7fc700 1 -- 172.21.15.90:0/2352115296 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7ff3a001aba0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.115+0000 7ff39a7fc700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ff3900383a0 0x7ff39003a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.115+0000 7ff39a7fc700 1 -- 172.21.15.90:0/2352115296 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff3a004cf80 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.115+0000 7ff3a8a80700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ff3900383a0 0x7ff39003a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.115+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3a4062380 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.115+0000 7ff3a8a80700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ff3900383a0 0x7ff39003a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff394006fd0 tx=0x7ff394006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.120+0000 7ff39a7fc700 1 -- 172.21.15.90:0/2352115296 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff3a0023790 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.308+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7ff3a419f380 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.308+0000 7ff39a7fc700 1 -- 172.21.15.90:0/2352115296 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7ff3a004cda0 con 0x7ff3a407bdf0 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ff3900383a0 msgr2=0x7ff39003a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ff3900383a0 0x7ff39003a850 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff394006fd0 tx=0x7ff394006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:16.893 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 msgr2=0x7ff3a419ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a419ba70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7ff3a0000c00 tx=0x7ff3a0011ab0 comp rx=0 tx=0).stop 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 shutdown_connections 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ff3900383a0 0x7ff39003a850 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 --2- 172.21.15.90:0/2352115296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff3a407bdf0 0x7ff3a419ba70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 >> 172.21.15.90:0/2352115296 conn(0x7ff3a4101180 msgr2=0x7ff3a4102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 shutdown_connections 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:16.311+0000 7ff3ab4e5700 1 -- 172.21.15.90:0/2352115296 wait complete. 2024-04-26T04:33:16.894 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling cephadm module... 2024-04-26T04:33:17.197 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:17 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2352115296' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-26T04:33:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:18 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3383783201' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.294+0000 7fb44a64e700 1 Processor -- start 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.294+0000 7fb44a64e700 1 -- start start 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.294+0000 7fb44a64e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44407a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.294+0000 7fb44a64e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44407a810 con 0x7fb44407bdd0 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.294+0000 7fb443fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44407a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.294+0000 7fb443fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44407a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53162/0 (socket says 172.21.15.90:53162) 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.294+0000 7fb443fff700 1 -- 172.21.15.90:0/1914073709 learned_addr learned my addr 172.21.15.90:0/1914073709 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.295+0000 7fb443fff700 1 -- 172.21.15.90:0/1914073709 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb44407a950 con 0x7fb44407bdd0 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.295+0000 7fb443fff700 1 --2- 172.21.15.90:0/1914073709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44407a2d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4340098a0 tx=0x7fb434009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=806cc15ec6659017 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:18.662 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.295+0000 7fb442ffd700 1 -- 172.21.15.90:0/1914073709 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb434004180 con 0x7fb44407bdd0 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.295+0000 7fb442ffd700 1 -- 172.21.15.90:0/1914073709 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb4340042e0 con 0x7fb44407bdd0 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.295+0000 7fb442ffd700 1 -- 172.21.15.90:0/1914073709 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb434011a50 con 0x7fb44407bdd0 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.295+0000 7fb44a64e700 1 -- 172.21.15.90:0/1914073709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 msgr2=0x7fb44407a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.295+0000 7fb44a64e700 1 --2- 172.21.15.90:0/1914073709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44407a2d0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fb4340098a0 tx=0x7fb434009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 -- 172.21.15.90:0/1914073709 shutdown_connections 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 --2- 172.21.15.90:0/1914073709 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44407a2d0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 -- 172.21.15.90:0/1914073709 >> 172.21.15.90:0/1914073709 conn(0x7fb444101180 msgr2=0x7fb444103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 -- 172.21.15.90:0/1914073709 shutdown_connections 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 -- 172.21.15.90:0/1914073709 wait complete. 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 Processor -- start 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 -- start start 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44419ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:18.663 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.296+0000 7fb44a64e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb44419bfd0 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb443fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44419ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb443fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44419ba90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53176/0 (socket says 172.21.15.90:53176) 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb443fff700 1 -- 172.21.15.90:0/3383783201 learned_addr learned my addr 172.21.15.90:0/3383783201 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb443fff700 1 -- 172.21.15.90:0/3383783201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb434009550 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb443fff700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44419ba90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb434006af0 tx=0x7fb4340047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb434003a00 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fb434003b60 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb44419c1d0 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fb434019710 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.297+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb44419c5f0 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.298+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 4) v1 ==== 45073+0+0 (secure 0 0 0) 0x7fb4340199f0 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.298+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb44410be00 con 0x7fb44407bdd0 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.298+0000 7fb4417fa700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7fb42c038350 0x7fb42c03a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:18.664 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.298+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb43404c5b0 con 0x7fb44407bdd0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.298+0000 7fb4437fe700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7fb42c038350 0x7fb42c03a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.299+0000 7fb4437fe700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7fb42c038350 0x7fb42c03a800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb438006fd0 tx=0x7fb438006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.303+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb434019ca0 con 0x7fb44407bdd0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:17.541+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fb44410bfc0 con 0x7fb44407bdd0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.096+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fb43400fda0 con 0x7fb44407bdd0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.097+0000 7fb4417fa700 1 -- 172.21.15.90:0/3383783201 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fb434018350 con 0x7fb44407bdd0 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.100+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7fb42c038350 msgr2=0x7fb42c03a800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.100+0000 7fb44a64e700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7fb42c038350 0x7fb42c03a800 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fb438006fd0 tx=0x7fb438006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.100+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 msgr2=0x7fb44419ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.100+0000 7fb44a64e700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44419ba90 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb434006af0 tx=0x7fb4340047c0 comp rx=0 tx=0).stop 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.100+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 shutdown_connections 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.100+0000 7fb44a64e700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7fb42c038350 0x7fb42c03a800 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.100+0000 7fb44a64e700 1 --2- 172.21.15.90:0/3383783201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb44407bdd0 0x7fb44419ba90 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:18.665 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.101+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 >> 172.21.15.90:0/3383783201 conn(0x7fb444101180 msgr2=0x7fb444102830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.101+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 shutdown_connections 2024-04-26T04:33:18.666 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:18.101+0000 7fb44a64e700 1 -- 172.21.15.90:0/3383783201 wait complete. 2024-04-26T04:33:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:19 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3383783201' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-26T04:33:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:19 smithi090 ceph-mon[26648]: mgrmap e5: smithi090.iwrvwa(active, since 6s) 2024-04-26T04:33:19.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:19.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-26T04:33:19.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T04:33:19.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "smithi090.iwrvwa", 2024-04-26T04:33:19.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T04:33:19.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:19.947 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.071+0000 7ffa39338700 1 Processor -- start 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.071+0000 7ffa39338700 1 -- start start 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa39338700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa34104e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa39338700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34105360 con 0x7ffa34104a10 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa34104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa34104e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53468/0 (socket says 172.21.15.90:53468) 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa32ffd700 1 -- 172.21.15.90:0/2986194942 learned_addr learned my addr 172.21.15.90:0/2986194942 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa32ffd700 1 -- 172.21.15.90:0/2986194942 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa341054a0 con 0x7ffa34104a10 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa32ffd700 1 --2- 172.21.15.90:0/2986194942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa34104e20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400bb40 tx=0x7ffa2400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7854840c880254c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa31ffb700 1 -- 172.21.15.90:0/2986194942 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa2400dc00 con 0x7ffa34104a10 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa31ffb700 1 -- 172.21.15.90:0/2986194942 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa240044a0 con 0x7ffa34104a10 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.072+0000 7ffa31ffb700 1 -- 172.21.15.90:0/2986194942 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa24010470 con 0x7ffa34104a10 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.073+0000 7ffa39338700 1 -- 172.21.15.90:0/2986194942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 msgr2=0x7ffa34104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.073+0000 7ffa39338700 1 --2- 172.21.15.90:0/2986194942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa34104e20 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400bb40 tx=0x7ffa2400be50 comp rx=0 tx=0).stop 2024-04-26T04:33:19.948 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.073+0000 7ffa39338700 1 -- 172.21.15.90:0/2986194942 shutdown_connections 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.073+0000 7ffa39338700 1 --2- 172.21.15.90:0/2986194942 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa34104e20 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.073+0000 7ffa39338700 1 -- 172.21.15.90:0/2986194942 >> 172.21.15.90:0/2986194942 conn(0x7ffa34100060 msgr2=0x7ffa34102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.073+0000 7ffa39338700 1 -- 172.21.15.90:0/2986194942 shutdown_connections 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.073+0000 7ffa39338700 1 -- 172.21.15.90:0/2986194942 wait complete. 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa39338700 1 Processor -- start 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa39338700 1 -- start start 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa39338700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa341975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa39338700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34197b30 con 0x7ffa34104a10 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa341975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa32ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa341975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53474/0 (socket says 172.21.15.90:53474) 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa32ffd700 1 -- 172.21.15.90:0/4242203672 learned_addr learned my addr 172.21.15.90:0/4242203672 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa32ffd700 1 -- 172.21.15.90:0/4242203672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa2400b7f0 con 0x7ffa34104a10 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.074+0000 7ffa32ffd700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa341975f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400dfa0 tx=0x7ffa24003fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:19.949 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.075+0000 7ffa1bfff700 1 -- 172.21.15.90:0/4242203672 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa2401e070 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.075+0000 7ffa1bfff700 1 -- 172.21.15.90:0/4242203672 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7ffa240104e0 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.075+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa34197d30 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.075+0000 7ffa1bfff700 1 -- 172.21.15.90:0/4242203672 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffa24022a30 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.075+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa34198150 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.075+0000 7ffa1bfff700 1 -- 172.21.15.90:0/4242203672 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ffa2402c440 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.076+0000 7ffa1bfff700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 0x7ffa1c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.076+0000 7ffa1bfff700 1 -- 172.21.15.90:0/4242203672 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ffa2404e280 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.076+0000 7ffa327fc700 1 -- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 msgr2=0x7ffa1c03a8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1230348805 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.076+0000 7ffa327fc700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 0x7ffa1c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.076+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa34191340 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.081+0000 7ffa1bfff700 1 -- 172.21.15.90:0/4242203672 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffa24027070 con 0x7ffa34104a10 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.277+0000 7ffa327fc700 1 -- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 msgr2=0x7ffa1c03a8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1230348805 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.277+0000 7ffa327fc700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 0x7ffa1c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:33:19.950 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.363+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7ffa341069d0 con 0x7ffa34104a10 2024-04-26T04:33:19.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.365+0000 7ffa1bfff700 1 -- 172.21.15.90:0/4242203672 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+103 (secure 0 0 0) 0x7ffa2402a950 con 0x7ffa34104a10 2024-04-26T04:33:19.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 msgr2=0x7ffa1c03a8a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:33:19.951 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 0x7ffa1c03a8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 msgr2=0x7ffa341975f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa341975f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400dfa0 tx=0x7ffa24003fa0 comp rx=0 tx=0).stop 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 shutdown_connections 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7ffa1c0383f0 0x7ffa1c03a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 --2- 172.21.15.90:0/4242203672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa34104a10 0x7ffa341975f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 >> 172.21.15.90:0/4242203672 conn(0x7ffa34100060 msgr2=0x7ffa3418e480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 shutdown_connections 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:19.369+0000 7ffa39338700 1 -- 172.21.15.90:0/4242203672 wait complete. 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-26T04:33:19.952 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 5... 2024-04-26T04:33:20.189 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:20 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/4242203672' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T04:33:24.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: Active manager daemon smithi090.iwrvwa restarted 2024-04-26T04:33:24.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: Activating manager daemon smithi090.iwrvwa 2024-04-26T04:33:24.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: osdmap e2: 0 total, 0 up, 0 in 2024-04-26T04:33:24.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: mgrmap e6: smithi090.iwrvwa(active, starting, since 0.00341572s) 2024-04-26T04:33:24.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:33:24.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:24 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:33:25.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:25 smithi090 ceph-mon[26648]: Found migration_current of "None". Setting to last migration. 2024-04-26T04:33:25.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:25 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:25.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:25 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:25.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:25 smithi090 ceph-mon[26648]: mgrmap e7: smithi090.iwrvwa(active, since 1.00537s) 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f081c13c700 1 Processor -- start 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f081c13c700 1 -- start start 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f081c13c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814104210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f081c13c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814104750 con 0x7f0814103e00 2024-04-26T04:33:26.300 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f0819ed8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f0819ed8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814104210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53482/0 (socket says 172.21.15.90:53482) 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f0819ed8700 1 -- 172.21.15.90:0/3505787141 learned_addr learned my addr 172.21.15.90:0/3505787141 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.374+0000 7f0819ed8700 1 -- 172.21.15.90:0/3505787141 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0814104890 con 0x7f0814103e00 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.375+0000 7f0819ed8700 1 --2- 172.21.15.90:0/3505787141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814104210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f08100098a0 tx=0x7f0810009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83ef33a10a92a26a server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.375+0000 7f0818ed6700 1 -- 172.21.15.90:0/3505787141 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0810004180 con 0x7f0814103e00 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.375+0000 7f0818ed6700 1 -- 172.21.15.90:0/3505787141 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08100042e0 con 0x7f0814103e00 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.375+0000 7f0818ed6700 1 -- 172.21.15.90:0/3505787141 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0810011a50 con 0x7f0814103e00 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 -- 172.21.15.90:0/3505787141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 msgr2=0x7f0814104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 --2- 172.21.15.90:0/3505787141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814104210 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f08100098a0 tx=0x7f0810009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 -- 172.21.15.90:0/3505787141 shutdown_connections 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 --2- 172.21.15.90:0/3505787141 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814104210 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 -- 172.21.15.90:0/3505787141 >> 172.21.15.90:0/3505787141 conn(0x7f08140ff7e0 msgr2=0x7f0814101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:26.301 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 -- 172.21.15.90:0/3505787141 shutdown_connections 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 -- 172.21.15.90:0/3505787141 wait complete. 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 Processor -- start 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.376+0000 7f081c13c700 1 -- start start 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f081c13c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814106790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f081c13c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814106cd0 con 0x7f0814103e00 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f0819ed8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f0819ed8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53484/0 (socket says 172.21.15.90:53484) 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f0819ed8700 1 -- 172.21.15.90:0/2480870859 learned_addr learned my addr 172.21.15.90:0/2480870859 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f0819ed8700 1 -- 172.21.15.90:0/2480870859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0810009550 con 0x7f0814103e00 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f0819ed8700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814106790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0810006af0 tx=0x7f0810011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0810003b90 con 0x7f0814103e00 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f081001a8a0 con 0x7f0814103e00 2024-04-26T04:33:26.302 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0814104de0 con 0x7f0814103e00 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0810019880 con 0x7f0814103e00 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.377+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0814105200 con 0x7f0814103e00 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.378+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 5) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0810019b40 con 0x7f0814103e00 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.379+0000 7f080affd700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.379+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 --> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f080003af70 con 0x7f08000383b0 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.379+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f081004c6f0 con 0x7f0814103e00 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.379+0000 7f08196d7700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 msgr2=0x7f080003a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1230348805 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.379+0000 7f08196d7700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.580+0000 7f08196d7700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 msgr2=0x7f080003a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1230348805 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.580+0000 7f08196d7700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.981+0000 7f08196d7700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 msgr2=0x7f080003a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1230348805 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:20.981+0000 7f08196d7700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:21.782+0000 7f08196d7700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 msgr2=0x7f080003a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1230348805 2024-04-26T04:33:26.303 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:21.782+0000 7f08196d7700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:23.383+0000 7f08196d7700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 msgr2=0x7f080003a860 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/1230348805 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:23.383+0000 7f08196d7700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:24.671+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 6) v1 ==== 44851+0+0 (secure 0 0 0) 0x7f081002dce0 con 0x7f0814103e00 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:24.671+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 msgr2=0x7f080003a860 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:24.671+0000 7f080affd700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.672+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7f081002d440 con 0x7f0814103e00 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.673+0000 7f080affd700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f080003bd20 0x7f080003e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.673+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f080003af70 con 0x7f080003bd20 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.673+0000 7f08196d7700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f080003bd20 0x7f080003e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.674+0000 7f08196d7700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f080003bd20 0x7f080003e100 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0804003a10 tx=0x7f0804009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.674+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f080003af70 con 0x7f080003bd20 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f0814106330 con 0x7f080003bd20 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f080affd700 1 -- 172.21.15.90:0/2480870859 <== mgr.14120 v2:172.21.15.90:6800/2176954768 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0814106330 con 0x7f080003bd20 2024-04-26T04:33:26.304 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f080003bd20 msgr2=0x7f080003e100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f080003bd20 0x7f080003e100 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f0804003a10 tx=0x7f0804009290 comp rx=0 tx=0).stop 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 msgr2=0x7f0814106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814106790 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0810006af0 tx=0x7f0810011a90 comp rx=0 tx=0).stop 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 shutdown_connections 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f080003bd20 0x7f080003e100 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0814103e00 0x7f0814106790 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 --2- 172.21.15.90:0/2480870859 >> [v2:172.21.15.90:6800/1230348805,v1:172.21.15.90:6801/1230348805] conn(0x7f08000383b0 0x7f080003a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 >> 172.21.15.90:0/2480870859 conn(0x7f08140ff7e0 msgr2=0x7f08141005e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 shutdown_connections 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:25.681+0000 7f081c13c700 1 -- 172.21.15.90:0/2480870859 wait complete. 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 5 is available 2024-04-26T04:33:26.305 INFO:teuthology.orchestra.run.smithi090.stdout:Setting orchestrator backend to cephadm... 2024-04-26T04:33:26.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:26 smithi090 ceph-mon[26648]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T04:33:26.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:26 smithi090 ceph-mon[26648]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T04:33:26.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:26 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:25] ENGINE Bus STARTING 2024-04-26T04:33:26.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:26 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:25] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T04:33:26.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:26 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:25] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T04:33:26.895 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:26 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:25] ENGINE Bus STARTED 2024-04-26T04:33:26.895 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:26 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:27.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.680+0000 7fdbb33f6700 1 Processor -- start 2024-04-26T04:33:27.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.680+0000 7fdbb33f6700 1 -- start start 2024-04-26T04:33:27.480 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.680+0000 7fdbb33f6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac079cf0 0x7fdbac10aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.680+0000 7fdbb33f6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac1064c0 con 0x7fdbac079cf0 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdbb1192700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac079cf0 0x7fdbac10aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdbb1192700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac079cf0 0x7fdbac10aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53550/0 (socket says 172.21.15.90:53550) 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdbb1192700 1 -- 172.21.15.90:0/3028334839 learned_addr learned my addr 172.21.15.90:0/3028334839 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdbb1192700 1 -- 172.21.15.90:0/3028334839 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac10b010 con 0x7fdbac079cf0 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdbb1192700 1 --2- 172.21.15.90:0/3028334839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac079cf0 0x7fdbac10aad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009e80 tx=0x7fdba800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d819e239ba3356 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdba3fff700 1 -- 172.21.15.90:0/3028334839 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdba800e070 con 0x7fdbac079cf0 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdba3fff700 1 -- 172.21.15.90:0/3028334839 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdba8003e90 con 0x7fdbac079cf0 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.681+0000 7fdba3fff700 1 -- 172.21.15.90:0/3028334839 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdba8004380 con 0x7fdbac079cf0 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3028334839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac079cf0 msgr2=0x7fdbac10aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 --2- 172.21.15.90:0/3028334839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac079cf0 0x7fdbac10aad0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009e80 tx=0x7fdba800b330 comp rx=0 tx=0).stop 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3028334839 shutdown_connections 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 --2- 172.21.15.90:0/3028334839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac079cf0 0x7fdbac10aad0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:27.481 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3028334839 >> 172.21.15.90:0/3028334839 conn(0x7fdbac100060 msgr2=0x7fdbac1024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3028334839 shutdown_connections 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3028334839 wait complete. 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.682+0000 7fdbb33f6700 1 Processor -- start 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb33f6700 1 -- start start 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb33f6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac19ba50 0x7fdbac19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb33f6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac19c3a0 con 0x7fdbac19ba50 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb1192700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac19ba50 0x7fdbac19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb1192700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac19ba50 0x7fdbac19be60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53560/0 (socket says 172.21.15.90:53560) 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb1192700 1 -- 172.21.15.90:0/3157280084 learned_addr learned my addr 172.21.15.90:0/3157280084 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb1192700 1 -- 172.21.15.90:0/3157280084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba8009b30 con 0x7fdbac19ba50 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb1192700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac19ba50 0x7fdbac19be60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009e50 tx=0x7fdba8004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdba800e040 con 0x7fdbac19ba50 2024-04-26T04:33:27.482 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fdba8003cf0 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac19c5a0 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdba801b770 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.683+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac19f200 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.684+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 7) v1 ==== 44978+0+0 (secure 0 0 0) 0x7fdba8019040 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.684+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac079cf0 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.685+0000 7fdba27fc700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fdb980382d0 0x7fdb9803a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.685+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fdba804ba80 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.685+0000 7fdbb0991700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fdb980382d0 0x7fdb9803a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.685+0000 7fdbb0991700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fdb980382d0 0x7fdb9803a780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006fd0 tx=0x7fdb9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.689+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdba8023ba0 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.726+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fdba801ad50 con 0x7fdbac19ba50 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.893+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbac1023c0 con 0x7fdb980382d0 2024-04-26T04:33:27.483 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.900+0000 7fdba27fc700 1 -- 172.21.15.90:0/3157280084 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fdbac1023c0 con 0x7fdb980382d0 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fdb980382d0 msgr2=0x7fdb9803a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fdb980382d0 0x7fdb9803a780 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c006fd0 tx=0x7fdb9c006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac19ba50 msgr2=0x7fdbac19be60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac19ba50 0x7fdbac19be60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fdba8009e50 tx=0x7fdba8004920 comp rx=0 tx=0).stop 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 shutdown_connections 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fdb980382d0 0x7fdb9803a780 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 --2- 172.21.15.90:0/3157280084 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdbac19ba50 0x7fdbac19be60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 >> 172.21.15.90:0/3157280084 conn(0x7fdbac100060 msgr2=0x7fdbac1017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 shutdown_connections 2024-04-26T04:33:27.484 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:26.902+0000 7fdbb33f6700 1 -- 172.21.15.90:0/3157280084 wait complete. 2024-04-26T04:33:28.116 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:27 smithi090 ceph-mon[26648]: mgrmap e8: smithi090.iwrvwa(active, since 2s) 2024-04-26T04:33:28.116 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:27 smithi090 ceph-mon[26648]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:28.116 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:27 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:28.116 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:27 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:28.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-26T04:33:28.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.916+0000 7f441f459700 1 Processor -- start 2024-04-26T04:33:28.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.916+0000 7f441f459700 1 -- start start 2024-04-26T04:33:28.699 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.916+0000 7f441f459700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f4418104e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.916+0000 7f441f459700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418105360 con 0x7f4418104a10 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f441d1f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f4418104e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f441d1f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f4418104e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48510/0 (socket says 172.21.15.90:48510) 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f441d1f5700 1 -- 172.21.15.90:0/4065636155 learned_addr learned my addr 172.21.15.90:0/4065636155 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f441d1f5700 1 -- 172.21.15.90:0/4065636155 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44181054a0 con 0x7f4418104a10 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f441d1f5700 1 --2- 172.21.15.90:0/4065636155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f4418104e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4414009e80 tx=0x7f441400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbd8eb884d3ec36d server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f440ffff700 1 -- 172.21.15.90:0/4065636155 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f441400e070 con 0x7f4418104a10 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f440ffff700 1 -- 172.21.15.90:0/4065636155 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4414003e90 con 0x7f4418104a10 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.917+0000 7f440ffff700 1 -- 172.21.15.90:0/4065636155 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4414004380 con 0x7f4418104a10 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 -- 172.21.15.90:0/4065636155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 msgr2=0x7f4418104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 --2- 172.21.15.90:0/4065636155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f4418104e20 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4414009e80 tx=0x7f441400b330 comp rx=0 tx=0).stop 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 -- 172.21.15.90:0/4065636155 shutdown_connections 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 --2- 172.21.15.90:0/4065636155 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f4418104e20 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:28.700 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 -- 172.21.15.90:0/4065636155 >> 172.21.15.90:0/4065636155 conn(0x7f4418100060 msgr2=0x7f4418102490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 -- 172.21.15.90:0/4065636155 shutdown_connections 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 -- 172.21.15.90:0/4065636155 wait complete. 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 Processor -- start 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.918+0000 7f441f459700 1 -- start start 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441f459700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f44181997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441f459700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4418199d20 con 0x7f4418104a10 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441d1f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f44181997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441d1f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f44181997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48522/0 (socket says 172.21.15.90:48522) 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441d1f5700 1 -- 172.21.15.90:0/3349505589 learned_addr learned my addr 172.21.15.90:0/3349505589 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441d1f5700 1 -- 172.21.15.90:0/3349505589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4414009b30 con 0x7f4418104a10 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441d1f5700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f44181997e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4414000f80 tx=0x7f4414004920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f440e7fc700 1 -- 172.21.15.90:0/3349505589 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f441400e040 con 0x7f4418104a10 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f440e7fc700 1 -- 172.21.15.90:0/3349505589 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f4414003cf0 con 0x7f4418104a10 2024-04-26T04:33:28.701 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4418199f20 con 0x7f4418104a10 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f440e7fc700 1 -- 172.21.15.90:0/3349505589 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f441401b930 con 0x7f4418104a10 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.919+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f441819a3a0 con 0x7f4418104a10 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.920+0000 7f440e7fc700 1 -- 172.21.15.90:0/3349505589 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f4414019040 con 0x7f4418104a10 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.920+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4418062380 con 0x7f4418104a10 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.920+0000 7f440e7fc700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4404038440 0x7f440403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.920+0000 7f440e7fc700 1 -- 172.21.15.90:0/3349505589 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f441404be80 con 0x7f4418104a10 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.920+0000 7f441c9f4700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4404038440 0x7f440403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.921+0000 7f441c9f4700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4404038440 0x7f440403a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4408006fd0 tx=0x7f4408006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:27.925+0000 7f440e7fc700 1 -- 172.21.15.90:0/3349505589 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4414011980 con 0x7f4418104a10 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.114+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f44181021c0 con 0x7f4404038440 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.115+0000 7f440e7fc700 1 -- 172.21.15.90:0/3349505589 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f44181021c0 con 0x7f4404038440 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.117+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4404038440 msgr2=0x7f440403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:28.702 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.117+0000 7f441f459700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4404038440 0x7f440403a8f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f4408006fd0 tx=0x7f4408006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.117+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 msgr2=0x7f44181997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.117+0000 7f441f459700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f44181997e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4414000f80 tx=0x7f4414004920 comp rx=0 tx=0).stop 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.117+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 shutdown_connections 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.117+0000 7f441f459700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4404038440 0x7f440403a8f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.117+0000 7f441f459700 1 --2- 172.21.15.90:0/3349505589 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4418104a10 0x7f44181997e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.118+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 >> 172.21.15.90:0/3349505589 conn(0x7f4418100060 msgr2=0x7f44180798d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.118+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 shutdown_connections 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:28.118+0000 7f441f459700 1 -- 172.21.15.90:0/3349505589 wait complete. 2024-04-26T04:33:28.703 INFO:teuthology.orchestra.run.smithi090.stdout:Generating ssh key... 2024-04-26T04:33:29.283 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:29 smithi090 ceph-mon[26648]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.084+0000 7fe815573700 1 Processor -- start 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.084+0000 7fe815573700 1 -- start start 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.084+0000 7fe815573700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe810074490 0x7fe81010aa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:30.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.084+0000 7fe815573700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe810106440 con 0x7fe810074490 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe810074490 0x7fe81010aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe810074490 0x7fe81010aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48530/0 (socket says 172.21.15.90:48530) 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80effd700 1 -- 172.21.15.90:0/3087798634 learned_addr learned my addr 172.21.15.90:0/3087798634 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80effd700 1 -- 172.21.15.90:0/3087798634 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe81010af90 con 0x7fe810074490 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80effd700 1 --2- 172.21.15.90:0/3087798634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe810074490 0x7fe81010aa50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe808009e80 tx=0x7fe80800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f298d32dd4416047 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80dffb700 1 -- 172.21.15.90:0/3087798634 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80800e070 con 0x7fe810074490 2024-04-26T04:33:30.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80dffb700 1 -- 172.21.15.90:0/3087798634 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe808003e90 con 0x7fe810074490 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.085+0000 7fe80dffb700 1 -- 172.21.15.90:0/3087798634 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe808004380 con 0x7fe810074490 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.086+0000 7fe815573700 1 -- 172.21.15.90:0/3087798634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe810074490 msgr2=0x7fe81010aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.086+0000 7fe815573700 1 --2- 172.21.15.90:0/3087798634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe810074490 0x7fe81010aa50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe808009e80 tx=0x7fe80800b330 comp rx=0 tx=0).stop 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.086+0000 7fe815573700 1 -- 172.21.15.90:0/3087798634 shutdown_connections 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.086+0000 7fe815573700 1 --2- 172.21.15.90:0/3087798634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe810074490 0x7fe81010aa50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.086+0000 7fe815573700 1 -- 172.21.15.90:0/3087798634 >> 172.21.15.90:0/3087798634 conn(0x7fe810100040 msgr2=0x7fe810102450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:30.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.086+0000 7fe815573700 1 -- 172.21.15.90:0/3087798634 shutdown_connections 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.086+0000 7fe815573700 1 -- 172.21.15.90:0/3087798634 wait complete. 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe815573700 1 Processor -- start 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe815573700 1 -- start start 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe815573700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe81019b930 0x7fe81019bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe815573700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe81019c280 con 0x7fe81019b930 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe80effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe81019b930 0x7fe81019bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe80effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe81019b930 0x7fe81019bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48536/0 (socket says 172.21.15.90:48536) 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe80effd700 1 -- 172.21.15.90:0/3065997257 learned_addr learned my addr 172.21.15.90:0/3065997257 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe80effd700 1 -- 172.21.15.90:0/3065997257 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe808009b30 con 0x7fe81019b930 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.087+0000 7fe80effd700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe81019b930 0x7fe81019bd40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe80800b7b0 tx=0x7fe808004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.088+0000 7fe7f7fff700 1 -- 172.21.15.90:0/3065997257 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80800e040 con 0x7fe81019b930 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.088+0000 7fe7f7fff700 1 -- 172.21.15.90:0/3065997257 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fe808003c70 con 0x7fe81019b930 2024-04-26T04:33:30.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.088+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe81019c480 con 0x7fe81019b930 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.088+0000 7fe7f7fff700 1 -- 172.21.15.90:0/3065997257 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe80801b730 con 0x7fe81019b930 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.088+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe81019f0e0 con 0x7fe81019b930 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.088+0000 7fe7f7fff700 1 -- 172.21.15.90:0/3065997257 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fe808019040 con 0x7fe81019b930 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.089+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe810062380 con 0x7fe81019b930 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.089+0000 7fe7f7fff700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fe7f80383b0 0x7fe7f803a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.089+0000 7fe7f7fff700 1 -- 172.21.15.90:0/3065997257 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe80804bdf0 con 0x7fe81019b930 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.089+0000 7fe80e7fc700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fe7f80383b0 0x7fe7f803a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.089+0000 7fe80e7fc700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fe7f80383b0 0x7fe7f803a860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe804006fd0 tx=0x7fe804006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.093+0000 7fe7f7fff700 1 -- 172.21.15.90:0/3065997257 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe808011980 con 0x7fe81019b930 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.281+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8101014c0 con 0x7fe7f80383b0 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.455+0000 7fe7f7fff700 1 -- 172.21.15.90:0/3065997257 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fe8101014c0 con 0x7fe7f80383b0 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.457+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fe7f80383b0 msgr2=0x7fe7f803a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:30.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.457+0000 7fe815573700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fe7f80383b0 0x7fe7f803a860 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe804006fd0 tx=0x7fe804006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe81019b930 msgr2=0x7fe81019bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe81019b930 0x7fe81019bd40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe80800b7b0 tx=0x7fe808004430 comp rx=0 tx=0).stop 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 shutdown_connections 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fe7f80383b0 0x7fe7f803a860 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 --2- 172.21.15.90:0/3065997257 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe81019b930 0x7fe81019bd40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 >> 172.21.15.90:0/3065997257 conn(0x7fe810100040 msgr2=0x7fe810100db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 shutdown_connections 2024-04-26T04:33:30.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:29.458+0000 7fe815573700 1 -- 172.21.15.90:0/3065997257 wait complete. 2024-04-26T04:33:30.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:30 smithi090 ceph-mon[26648]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:30.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:30 smithi090 ceph-mon[26648]: Generating ssh key... 2024-04-26T04:33:30.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:30 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:30.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:30 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:31.257 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYjdXsuliVep/WqNcKeShgMRDdYE+ztBQGzpUfzxZVYMrG/xa9kviHxa2FCfaPH/iEP3tMNj/Yi9j65oZti55qoU0JHWEC1xNTpplD5F+pxOQWpDU3E6xNZBdJOVT71Qqt1qEPgAPRzHCb0DVmni/kCqF0uJOdv8F+geeIXJ0IQ5DdN+hEvWtsALTERNvv/J7n//QOcmAxZ++hfPPSLMTpMuI1c31pkSwv0XLgEGJmMf/kxKqHWWdm8WdjjwsK0Gdwbfzpe8Mlz3BxOuCQoYxMkr16+WARMZaFfFHvXGXFlMkEp1Tbv/0gtbHTpUB9Is1wWlt8zta+Z4a1KYbBWrfO2ZdkiU8SCNXD3RB6jYCcBtAiZZYcJiZcWEVRrxvT33eYy+LRXYJxah0ndosDDDtuJuu/LcNjGhO+/jyyNgnNtqB8LImzlbWufA2fzgnBT5jfBZ4AwcOhDkZmAuAnEwfA/ovk/7Nxuj4Fjc5QdAOOreCAGMp9x4rGMJiDG1xDPZs= ceph-f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:33:31.257 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d30a2700 1 Processor -- start 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d30a2700 1 -- start start 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d30a2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d30a2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc106460 con 0x7f72cc105b10 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d0e3e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d0e3e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48540/0 (socket says 172.21.15.90:48540) 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d0e3e700 1 -- 172.21.15.90:0/1112933688 learned_addr learned my addr 172.21.15.90:0/1112933688 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d0e3e700 1 -- 172.21.15.90:0/1112933688 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72cc1065a0 con 0x7f72cc105b10 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72d0e3e700 1 --2- 172.21.15.90:0/1112933688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc105f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098a0 tx=0x7f72c0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1d833b8aaf9b428 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72cb7fe700 1 -- 172.21.15.90:0/1112933688 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72c0004180 con 0x7f72cc105b10 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72cb7fe700 1 -- 172.21.15.90:0/1112933688 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72c00042e0 con 0x7f72cc105b10 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.443+0000 7f72cb7fe700 1 -- 172.21.15.90:0/1112933688 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72c0011a50 con 0x7f72cc105b10 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.444+0000 7f72d30a2700 1 -- 172.21.15.90:0/1112933688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 msgr2=0x7f72cc105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:31.258 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.444+0000 7f72d30a2700 1 --2- 172.21.15.90:0/1112933688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc105f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f72c00098a0 tx=0x7f72c0009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.444+0000 7f72d30a2700 1 -- 172.21.15.90:0/1112933688 shutdown_connections 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.444+0000 7f72d30a2700 1 --2- 172.21.15.90:0/1112933688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc105f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.444+0000 7f72d30a2700 1 -- 172.21.15.90:0/1112933688 >> 172.21.15.90:0/1112933688 conn(0x7f72cc101180 msgr2=0x7f72cc103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.444+0000 7f72d30a2700 1 -- 172.21.15.90:0/1112933688 shutdown_connections 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.444+0000 7f72d30a2700 1 -- 172.21.15.90:0/1112933688 wait complete. 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d30a2700 1 Processor -- start 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d30a2700 1 -- start start 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d30a2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc199810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d30a2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72cc199d50 con 0x7f72cc105b10 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d0e3e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc199810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d0e3e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc199810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48546/0 (socket says 172.21.15.90:48546) 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d0e3e700 1 -- 172.21.15.90:0/3006308447 learned_addr learned my addr 172.21.15.90:0/3006308447 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d0e3e700 1 -- 172.21.15.90:0/3006308447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c0009550 con 0x7f72cc105b10 2024-04-26T04:33:31.259 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d0e3e700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc199810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72c0000c00 tx=0x7f72c0011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72c9ffb700 1 -- 172.21.15.90:0/3006308447 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72c0003b90 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72c9ffb700 1 -- 172.21.15.90:0/3006308447 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f72c00248b0 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72cc199f50 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.445+0000 7f72c9ffb700 1 -- 172.21.15.90:0/3006308447 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f72c001a8e0 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.446+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72cc19a370 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.446+0000 7f72c9ffb700 1 -- 172.21.15.90:0/3006308447 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f72c0024440 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.446+0000 7f72c9ffb700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f72bc038440 0x7f72bc03a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.447+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72cc062380 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.447+0000 7f72c9ffb700 1 -- 172.21.15.90:0/3006308447 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f72c004c400 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.447+0000 7f72cbfff700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f72bc038440 0x7f72bc03a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.447+0000 7f72cbfff700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f72bc038440 0x7f72bc03a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f72b8006fd0 tx=0x7f72b8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.452+0000 7f72c9ffb700 1 -- 172.21.15.90:0/3006308447 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f72c00189a0 con 0x7f72cc105b10 2024-04-26T04:33:31.260 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.642+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f72cc102610 con 0x7f72bc038440 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.643+0000 7f72c9ffb700 1 -- 172.21.15.90:0/3006308447 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f72cc102610 con 0x7f72bc038440 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f72bc038440 msgr2=0x7f72bc03a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f72bc038440 0x7f72bc03a8f0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f72b8006fd0 tx=0x7f72b8006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 msgr2=0x7f72cc199810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc199810 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f72c0000c00 tx=0x7f72c0011ab0 comp rx=0 tx=0).stop 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 shutdown_connections 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f72bc038440 0x7f72bc03a8f0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 --2- 172.21.15.90:0/3006308447 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f72cc105b10 0x7f72cc199810 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 >> 172.21.15.90:0/3006308447 conn(0x7f72cc101180 msgr2=0x7f72cc101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 shutdown_connections 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:30.645+0000 7f72d30a2700 1 -- 172.21.15.90:0/3006308447 wait complete. 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:Adding key to root@localhost authorized_keys... 2024-04-26T04:33:31.261 INFO:teuthology.orchestra.run.smithi090.stdout:Adding host smithi090... 2024-04-26T04:33:31.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:31 smithi090 ceph-mon[26648]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:32.581 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:32 smithi090 ceph-mon[26648]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:34.460 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:34 smithi090 ceph-mon[26648]: Deploying cephadm binary to smithi090 2024-04-26T04:33:34.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Added host 'smithi090' with addr '172.21.15.90' 2024-04-26T04:33:34.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.645+0000 7f755f692700 1 Processor -- start 2024-04-26T04:33:34.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.645+0000 7f755f692700 1 -- start start 2024-04-26T04:33:34.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.645+0000 7f755f692700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f7558108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:34.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.645+0000 7f755f692700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75580744d0 con 0x7f7558106650 2024-04-26T04:33:34.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.645+0000 7f755d42e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f7558108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:34.711 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.645+0000 7f755d42e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f7558108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48554/0 (socket says 172.21.15.90:48554) 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.645+0000 7f755d42e700 1 -- 172.21.15.90:0/2827020780 learned_addr learned my addr 172.21.15.90:0/2827020780 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.646+0000 7f755d42e700 1 -- 172.21.15.90:0/2827020780 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7558074610 con 0x7f7558106650 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.646+0000 7f755d42e700 1 --2- 172.21.15.90:0/2827020780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f7558108a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75480098a0 tx=0x7f7548009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e6ea38aa9d638e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.646+0000 7f754ffff700 1 -- 172.21.15.90:0/2827020780 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7548004180 con 0x7f7558106650 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.646+0000 7f754ffff700 1 -- 172.21.15.90:0/2827020780 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75480042e0 con 0x7f7558106650 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.646+0000 7f754ffff700 1 -- 172.21.15.90:0/2827020780 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7548011a50 con 0x7f7558106650 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 -- 172.21.15.90:0/2827020780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 msgr2=0x7f7558108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:34.712 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 --2- 172.21.15.90:0/2827020780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f7558108a30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f75480098a0 tx=0x7f7548009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 -- 172.21.15.90:0/2827020780 shutdown_connections 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 --2- 172.21.15.90:0/2827020780 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f7558108a30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 -- 172.21.15.90:0/2827020780 >> 172.21.15.90:0/2827020780 conn(0x7f7558100070 msgr2=0x7f7558102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 -- 172.21.15.90:0/2827020780 shutdown_connections 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 -- 172.21.15.90:0/2827020780 wait complete. 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 Processor -- start 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 -- start start 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f75581997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.647+0000 7f755f692700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7558199d20 con 0x7f7558106650 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f755d42e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f75581997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f755d42e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f75581997e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48566/0 (socket says 172.21.15.90:48566) 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f755d42e700 1 -- 172.21.15.90:0/1729278207 learned_addr learned my addr 172.21.15.90:0/1729278207 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:34.713 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f755d42e700 1 -- 172.21.15.90:0/1729278207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7548009550 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f755d42e700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f75581997e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f7548011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f754e7fc700 1 -- 172.21.15.90:0/1729278207 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7548003b90 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f754e7fc700 1 -- 172.21.15.90:0/1729278207 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f75480248b0 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7558199f80 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f754e7fc700 1 -- 172.21.15.90:0/1729278207 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f754801a8e0 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.648+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f755819a400 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.649+0000 7f754e7fc700 1 -- 172.21.15.90:0/1729278207 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f754801ab50 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.649+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7558109b30 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.649+0000 7f754e7fc700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f7544038440 0x7f754403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.649+0000 7f754e7fc700 1 -- 172.21.15.90:0/1729278207 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f754804c360 con 0x7f7558106650 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.649+0000 7f755cc2d700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f7544038440 0x7f754403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:34.714 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.650+0000 7f755cc2d700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f7544038440 0x7f754403a8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7554006fd0 tx=0x7f7554006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.654+0000 7f754e7fc700 1 -- 172.21.15.90:0/1729278207 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f75480233a0 con 0x7f7558106650 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:31.872+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi090", "addr": "172.21.15.90", "target": ["mon-mgr", ""]}) v1 -- 0x7f7558102370 con 0x7f7544038440 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.142+0000 7f754e7fc700 1 -- 172.21.15.90:0/1729278207 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f7558102370 con 0x7f7544038440 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f7544038440 msgr2=0x7f754403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f7544038440 0x7f754403a8f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7554006fd0 tx=0x7f7554006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 msgr2=0x7f75581997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f75581997e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7548000c00 tx=0x7f7548011ab0 comp rx=0 tx=0).stop 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 shutdown_connections 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f7544038440 0x7f754403a8f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 --2- 172.21.15.90:0/1729278207 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7558106650 0x7f75581997e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 >> 172.21.15.90:0/1729278207 conn(0x7f7558100070 msgr2=0x7f7558101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 shutdown_connections 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:34.145+0000 7f755f692700 1 -- 172.21.15.90:0/1729278207 wait complete. 2024-04-26T04:33:34.715 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying mon service with default placement... 2024-04-26T04:33:35.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:35 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:35.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:35 smithi090 ceph-mon[26648]: Added host smithi090 2024-04-26T04:33:35.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:35 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:36.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-26T04:33:36.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.104+0000 7f08e3122700 1 Processor -- start 2024-04-26T04:33:36.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.104+0000 7f08e3122700 1 -- start start 2024-04-26T04:33:36.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.104+0000 7f08e3122700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc104210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:36.021 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.104+0000 7f08e3122700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc104750 con 0x7f08dc103e00 2024-04-26T04:33:36.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.104+0000 7f08e0ebe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:36.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.104+0000 7f08e0ebe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc104210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48570/0 (socket says 172.21.15.90:48570) 2024-04-26T04:33:36.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.104+0000 7f08e0ebe700 1 -- 172.21.15.90:0/647353604 learned_addr learned my addr 172.21.15.90:0/647353604 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:36.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.105+0000 7f08e0ebe700 1 -- 172.21.15.90:0/647353604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08dc104890 con 0x7f08dc103e00 2024-04-26T04:33:36.022 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.105+0000 7f08e0ebe700 1 --2- 172.21.15.90:0/647353604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc104210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08d00098a0 tx=0x7f08d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=195456287a77fcd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:36.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.105+0000 7f08db7fe700 1 -- 172.21.15.90:0/647353604 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d0004180 con 0x7f08dc103e00 2024-04-26T04:33:36.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.105+0000 7f08db7fe700 1 -- 172.21.15.90:0/647353604 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08d00042e0 con 0x7f08dc103e00 2024-04-26T04:33:36.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.105+0000 7f08db7fe700 1 -- 172.21.15.90:0/647353604 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d0011a50 con 0x7f08dc103e00 2024-04-26T04:33:36.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.105+0000 7f08e3122700 1 -- 172.21.15.90:0/647353604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 msgr2=0x7f08dc104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:36.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.105+0000 7f08e3122700 1 --2- 172.21.15.90:0/647353604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc104210 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f08d00098a0 tx=0x7f08d0009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:36.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 -- 172.21.15.90:0/647353604 shutdown_connections 2024-04-26T04:33:36.023 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 --2- 172.21.15.90:0/647353604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc104210 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 -- 172.21.15.90:0/647353604 >> 172.21.15.90:0/647353604 conn(0x7f08dc0ff7e0 msgr2=0x7f08dc101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 -- 172.21.15.90:0/647353604 shutdown_connections 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 -- 172.21.15.90:0/647353604 wait complete. 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 Processor -- start 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 -- start start 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc106790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e3122700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08dc106cd0 con 0x7f08dc103e00 2024-04-26T04:33:36.024 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e0ebe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:36.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e0ebe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48582/0 (socket says 172.21.15.90:48582) 2024-04-26T04:33:36.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.106+0000 7f08e0ebe700 1 -- 172.21.15.90:0/3807810762 learned_addr learned my addr 172.21.15.90:0/3807810762 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:36.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.107+0000 7f08e0ebe700 1 -- 172.21.15.90:0/3807810762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08d0009550 con 0x7f08dc103e00 2024-04-26T04:33:36.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.107+0000 7f08e0ebe700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc106790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f08d0006af0 tx=0x7f08d0011a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:36.025 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.107+0000 7f08d9ffb700 1 -- 172.21.15.90:0/3807810762 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d0003b90 con 0x7f08dc103e00 2024-04-26T04:33:36.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.107+0000 7f08d9ffb700 1 -- 172.21.15.90:0/3807810762 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7f08d001a8a0 con 0x7f08dc103e00 2024-04-26T04:33:36.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.107+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08dc104de0 con 0x7f08dc103e00 2024-04-26T04:33:36.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.107+0000 7f08d9ffb700 1 -- 172.21.15.90:0/3807810762 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f08d0019880 con 0x7f08dc103e00 2024-04-26T04:33:36.026 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.107+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08dc105200 con 0x7f08dc103e00 2024-04-26T04:33:36.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.108+0000 7f08d9ffb700 1 -- 172.21.15.90:0/3807810762 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f08d001aa10 con 0x7f08dc103e00 2024-04-26T04:33:36.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.108+0000 7f08d9ffb700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f08cc038400 0x7f08cc03a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:36.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.108+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08dc1079c0 con 0x7f08dc103e00 2024-04-26T04:33:36.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.108+0000 7f08d9ffb700 1 -- 172.21.15.90:0/3807810762 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f08d004c690 con 0x7f08dc103e00 2024-04-26T04:33:36.027 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.108+0000 7f08dbfff700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f08cc038400 0x7f08cc03a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:36.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.109+0000 7f08dbfff700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f08cc038400 0x7f08cc03a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006fd0 tx=0x7f08c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:36.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.113+0000 7f08d9ffb700 1 -- 172.21.15.90:0/3807810762 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f08d001a430 con 0x7f08dc103e00 2024-04-26T04:33:36.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.330+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}) v1 -- 0x7f08dc101ad0 con 0x7f08cc038400 2024-04-26T04:33:36.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.334+0000 7f08d9ffb700 1 -- 172.21.15.90:0/3807810762 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f08dc101ad0 con 0x7f08cc038400 2024-04-26T04:33:36.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.336+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f08cc038400 msgr2=0x7f08cc03a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:36.028 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.336+0000 7f08e3122700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f08cc038400 0x7f08cc03a8b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f08c8006fd0 tx=0x7f08c8006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:36.029 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.336+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 msgr2=0x7f08dc106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:36.029 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.336+0000 7f08e3122700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc106790 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f08d0006af0 tx=0x7f08d0011a90 comp rx=0 tx=0).stop 2024-04-26T04:33:36.029 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.336+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 shutdown_connections 2024-04-26T04:33:36.030 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.336+0000 7f08e3122700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f08cc038400 0x7f08cc03a8b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:36.030 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.337+0000 7f08e3122700 1 --2- 172.21.15.90:0/3807810762 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08dc103e00 0x7f08dc106790 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:36.030 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.337+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 >> 172.21.15.90:0/3807810762 conn(0x7f08dc0ff7e0 msgr2=0x7f08dc100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:36.030 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.337+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 shutdown_connections 2024-04-26T04:33:36.030 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:35.337+0000 7f08e3122700 1 -- 172.21.15.90:0/3807810762 wait complete. 2024-04-26T04:33:36.030 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying mgr service with default placement... 2024-04-26T04:33:36.653 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:36 smithi090 ceph-mon[26648]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:36.653 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:36 smithi090 ceph-mon[26648]: Saving service mon spec with placement count:5 2024-04-26T04:33:36.653 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:36 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:37.946 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:37 smithi090 ceph-mon[26648]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:37.946 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:37 smithi090 ceph-mon[26648]: Saving service mgr spec with placement count:2 2024-04-26T04:33:37.946 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:37 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:37.946 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:37 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:37.956 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc4d999700 1 Processor -- start 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc4d999700 1 -- start start 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc4d999700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48104210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc4d999700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc48104750 con 0x7fcc48103e00 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48104210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48588/0 (socket says 172.21.15.90:48588) 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc46ffd700 1 -- 172.21.15.90:0/60767076 learned_addr learned my addr 172.21.15.90:0/60767076 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc46ffd700 1 -- 172.21.15.90:0/60767076 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc48104890 con 0x7fcc48103e00 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc46ffd700 1 --2- 172.21.15.90:0/60767076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48104210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc380098a0 tx=0x7fcc38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1248901eeab9da5e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc45ffb700 1 -- 172.21.15.90:0/60767076 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc38004180 con 0x7fcc48103e00 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.438+0000 7fcc45ffb700 1 -- 172.21.15.90:0/60767076 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc380042e0 con 0x7fcc48103e00 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc45ffb700 1 -- 172.21.15.90:0/60767076 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc38011a50 con 0x7fcc48103e00 2024-04-26T04:33:37.957 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc4d999700 1 -- 172.21.15.90:0/60767076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 msgr2=0x7fcc48104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc4d999700 1 --2- 172.21.15.90:0/60767076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48104210 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fcc380098a0 tx=0x7fcc38009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc4d999700 1 -- 172.21.15.90:0/60767076 shutdown_connections 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc4d999700 1 --2- 172.21.15.90:0/60767076 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48104210 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc4d999700 1 -- 172.21.15.90:0/60767076 >> 172.21.15.90:0/60767076 conn(0x7fcc480ff7e0 msgr2=0x7fcc48101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc4d999700 1 -- 172.21.15.90:0/60767076 shutdown_connections 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.439+0000 7fcc4d999700 1 -- 172.21.15.90:0/60767076 wait complete. 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc4d999700 1 Processor -- start 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc4d999700 1 -- start start 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc4d999700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc4d999700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc48199da0 con 0x7fcc48103e00 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc46ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48199860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48592/0 (socket says 172.21.15.90:48592) 2024-04-26T04:33:37.958 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc46ffd700 1 -- 172.21.15.90:0/301938043 learned_addr learned my addr 172.21.15.90:0/301938043 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc46ffd700 1 -- 172.21.15.90:0/301938043 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc38009550 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.440+0000 7fcc46ffd700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48199860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc38000c00 tx=0x7fcc38011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.441+0000 7fcc4c997700 1 -- 172.21.15.90:0/301938043 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc38003b90 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.441+0000 7fcc4c997700 1 -- 172.21.15.90:0/301938043 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1037+0+0 (secure 0 0 0) 0x7fcc380248b0 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.441+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc48199fa0 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.441+0000 7fcc4c997700 1 -- 172.21.15.90:0/301938043 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fcc3801a8e0 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.441+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc4819a3c0 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.442+0000 7fcc4c997700 1 -- 172.21.15.90:0/301938043 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7fcc3801ab50 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.442+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc48062380 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.442+0000 7fcc4c997700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fcc30038440 0x7fcc3003a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.442+0000 7fcc467fc700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fcc30038440 0x7fcc3003a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.442+0000 7fcc4c997700 1 -- 172.21.15.90:0/301938043 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fcc3804c4c0 con 0x7fcc48103e00 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.442+0000 7fcc467fc700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fcc30038440 0x7fcc3003a8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:37.959 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.446+0000 7fcc4c997700 1 -- 172.21.15.90:0/301938043 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcc380233a0 con 0x7fcc48103e00 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.652+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fcc48101ad0 con 0x7fcc30038440 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.656+0000 7fcc4c997700 1 -- 172.21.15.90:0/301938043 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcc48101ad0 con 0x7fcc30038440 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.658+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fcc30038440 msgr2=0x7fcc3003a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.658+0000 7fcc4d999700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fcc30038440 0x7fcc3003a8f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fcc3c006fd0 tx=0x7fcc3c006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.658+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 msgr2=0x7fcc48199860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.658+0000 7fcc4d999700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48199860 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc38000c00 tx=0x7fcc38011ab0 comp rx=0 tx=0).stop 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.659+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 shutdown_connections 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.659+0000 7fcc4d999700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7fcc30038440 0x7fcc3003a8f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.659+0000 7fcc4d999700 1 --2- 172.21.15.90:0/301938043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcc48103e00 0x7fcc48199860 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.659+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 >> 172.21.15.90:0/301938043 conn(0x7fcc480ff7e0 msgr2=0x7fcc48100eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.659+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 shutdown_connections 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:36.659+0000 7fcc4d999700 1 -- 172.21.15.90:0/301938043 wait complete. 2024-04-26T04:33:37.960 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying crash service with default placement... 2024-04-26T04:33:39.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:39 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:39.348 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:39 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:39.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled crash update... 2024-04-26T04:33:39.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.381+0000 7ff54bd15700 1 Processor -- start 2024-04-26T04:33:39.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.381+0000 7ff54bd15700 1 -- start start 2024-04-26T04:33:39.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.381+0000 7ff54bd15700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff544106400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:39.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.381+0000 7ff54bd15700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff544106940 con 0x7ff544105ff0 2024-04-26T04:33:39.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.381+0000 7ff549ab1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff544106400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:39.356 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.381+0000 7ff549ab1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff544106400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49002/0 (socket says 172.21.15.90:49002) 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.381+0000 7ff549ab1700 1 -- 172.21.15.90:0/981897107 learned_addr learned my addr 172.21.15.90:0/981897107 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.382+0000 7ff549ab1700 1 -- 172.21.15.90:0/981897107 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544106a80 con 0x7ff544105ff0 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.382+0000 7ff549ab1700 1 --2- 172.21.15.90:0/981897107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff544106400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5400098a0 tx=0x7ff540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f53bac0e6c6272b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.382+0000 7ff548aaf700 1 -- 172.21.15.90:0/981897107 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff540004180 con 0x7ff544105ff0 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.382+0000 7ff548aaf700 1 -- 172.21.15.90:0/981897107 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff5400042e0 con 0x7ff544105ff0 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.382+0000 7ff548aaf700 1 -- 172.21.15.90:0/981897107 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff540011af0 con 0x7ff544105ff0 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.383+0000 7ff54bd15700 1 -- 172.21.15.90:0/981897107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 msgr2=0x7ff544106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.383+0000 7ff54bd15700 1 --2- 172.21.15.90:0/981897107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff544106400 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff5400098a0 tx=0x7ff540009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.383+0000 7ff54bd15700 1 -- 172.21.15.90:0/981897107 shutdown_connections 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.383+0000 7ff54bd15700 1 --2- 172.21.15.90:0/981897107 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff544106400 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.383+0000 7ff54bd15700 1 -- 172.21.15.90:0/981897107 >> 172.21.15.90:0/981897107 conn(0x7ff5440ff7e0 msgr2=0x7ff544101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.383+0000 7ff54bd15700 1 -- 172.21.15.90:0/981897107 shutdown_connections 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.383+0000 7ff54bd15700 1 -- 172.21.15.90:0/981897107 wait complete. 2024-04-26T04:33:39.357 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.384+0000 7ff54bd15700 1 Processor -- start 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.384+0000 7ff54bd15700 1 -- start start 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.384+0000 7ff54bd15700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff5441046a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.384+0000 7ff54bd15700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54419c610 con 0x7ff544105ff0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.384+0000 7ff549ab1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff5441046a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.384+0000 7ff549ab1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff5441046a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49014/0 (socket says 172.21.15.90:49014) 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.384+0000 7ff549ab1700 1 -- 172.21.15.90:0/828704004 learned_addr learned my addr 172.21.15.90:0/828704004 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.385+0000 7ff549ab1700 1 -- 172.21.15.90:0/828704004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff540009550 con 0x7ff544105ff0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.385+0000 7ff549ab1700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff5441046a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff540011ad0 tx=0x7ff540011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.385+0000 7ff53affd700 1 -- 172.21.15.90:0/828704004 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff540003ab0 con 0x7ff544105ff0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.385+0000 7ff53affd700 1 -- 172.21.15.90:0/828704004 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff540003c10 con 0x7ff544105ff0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.385+0000 7ff53affd700 1 -- 172.21.15.90:0/828704004 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff5400198e0 con 0x7ff544105ff0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.385+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54419c750 con 0x7ff544105ff0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.385+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54419cab0 con 0x7ff544105ff0 2024-04-26T04:33:39.358 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.386+0000 7ff53affd700 1 -- 172.21.15.90:0/828704004 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff540019ba0 con 0x7ff544105ff0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.386+0000 7ff53affd700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff5300383f0 0x7ff53003a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.386+0000 7ff53affd700 1 -- 172.21.15.90:0/828704004 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff54004c1e0 con 0x7ff544105ff0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.386+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff544062380 con 0x7ff544105ff0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.386+0000 7ff5492b0700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff5300383f0 0x7ff53003a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.387+0000 7ff5492b0700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff5300383f0 0x7ff53003a8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff534006fd0 tx=0x7ff534006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.392+0000 7ff53affd700 1 -- 172.21.15.90:0/828704004 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff54002b300 con 0x7ff544105ff0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.614+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}) v1 -- 0x7ff544108eb0 con 0x7ff5300383f0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.618+0000 7ff53affd700 1 -- 172.21.15.90:0/828704004 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7ff544108eb0 con 0x7ff5300383f0 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff5300383f0 msgr2=0x7ff53003a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff5300383f0 0x7ff53003a8a0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff534006fd0 tx=0x7ff534006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 msgr2=0x7ff5441046a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff5441046a0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff540011ad0 tx=0x7ff540011bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:39.359 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 shutdown_connections 2024-04-26T04:33:39.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff5300383f0 0x7ff53003a8a0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:39.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 --2- 172.21.15.90:0/828704004 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff544105ff0 0x7ff5441046a0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:39.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 >> 172.21.15.90:0/828704004 conn(0x7ff5440ff7e0 msgr2=0x7ff544100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:39.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 shutdown_connections 2024-04-26T04:33:39.360 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:38.620+0000 7ff54bd15700 1 -- 172.21.15.90:0/828704004 wait complete. 2024-04-26T04:33:39.360 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying ceph-exporter service with default placement... 2024-04-26T04:33:40.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:40 smithi090 ceph-mon[26648]: from='client.14146 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "crash", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:40.058 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:40 smithi090 ceph-mon[26648]: Saving service crash spec with placement * 2024-04-26T04:33:40.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:40 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled ceph-exporter update... 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b718e7700 1 Processor -- start 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b718e7700 1 -- start start 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b718e7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b718e7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c07a810 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b6affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b6affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49030/0 (socket says 172.21.15.90:49030) 2024-04-26T04:33:40.985 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b6affd700 1 -- 172.21.15.90:0/2382383908 learned_addr learned my addr 172.21.15.90:0/2382383908 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.142+0000 7f0b6affd700 1 -- 172.21.15.90:0/2382383908 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b6c07a950 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b6affd700 1 --2- 172.21.15.90:0/2382383908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c07a2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098a0 tx=0x7f0b60009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87030d1e169a7ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b69ffb700 1 -- 172.21.15.90:0/2382383908 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b60004180 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b69ffb700 1 -- 172.21.15.90:0/2382383908 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0b600042e0 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b69ffb700 1 -- 172.21.15.90:0/2382383908 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b60011af0 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b718e7700 1 -- 172.21.15.90:0/2382383908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 msgr2=0x7f0b6c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b718e7700 1 --2- 172.21.15.90:0/2382383908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c07a2d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0b600098a0 tx=0x7f0b60009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:40.986 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b718e7700 1 -- 172.21.15.90:0/2382383908 shutdown_connections 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b718e7700 1 --2- 172.21.15.90:0/2382383908 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c07a2d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b718e7700 1 -- 172.21.15.90:0/2382383908 >> 172.21.15.90:0/2382383908 conn(0x7f0b6c101180 msgr2=0x7f0b6c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b718e7700 1 -- 172.21.15.90:0/2382383908 shutdown_connections 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.143+0000 7f0b718e7700 1 -- 172.21.15.90:0/2382383908 wait complete. 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.144+0000 7f0b718e7700 1 Processor -- start 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.144+0000 7f0b718e7700 1 -- start start 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.144+0000 7f0b718e7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.144+0000 7f0b718e7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c19c020 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.144+0000 7f0b6affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.144+0000 7f0b6affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c19bae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49036/0 (socket says 172.21.15.90:49036) 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.144+0000 7f0b6affd700 1 -- 172.21.15.90:0/4000698611 learned_addr learned my addr 172.21.15.90:0/4000698611 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.145+0000 7f0b6affd700 1 -- 172.21.15.90:0/4000698611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b60009550 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.145+0000 7f0b6affd700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c19bae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b60000c00 tx=0x7f0b60011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:40.987 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.145+0000 7f0b708e5700 1 -- 172.21.15.90:0/4000698611 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b60003a70 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.145+0000 7f0b708e5700 1 -- 172.21.15.90:0/4000698611 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0b60003bd0 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.145+0000 7f0b708e5700 1 -- 172.21.15.90:0/4000698611 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f0b600238c0 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.145+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c19c220 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.145+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c19c640 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.146+0000 7f0b708e5700 1 -- 172.21.15.90:0/4000698611 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f0b60023b80 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.146+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b6c062380 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.146+0000 7f0b708e5700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f0b540383f0 0x7f0b5403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.146+0000 7f0b708e5700 1 -- 172.21.15.90:0/4000698611 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0b6004c4c0 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.146+0000 7f0b6a7fc700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f0b540383f0 0x7f0b5403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.147+0000 7f0b6a7fc700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f0b540383f0 0x7f0b5403a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006fd0 tx=0x7f0b5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:40.988 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.151+0000 7f0b708e5700 1 -- 172.21.15.90:0/4000698611 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0b600109d0 con 0x7f0b6c07bdd0 2024-04-26T04:33:40.989 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.371+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b6c103460 con 0x7f0b540383f0 2024-04-26T04:33:40.989 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.378+0000 7f0b708e5700 1 -- 172.21.15.90:0/4000698611 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f0b6c103460 con 0x7f0b540383f0 2024-04-26T04:33:40.989 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f0b540383f0 msgr2=0x7f0b5403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:40.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f0b540383f0 0x7f0b5403a8a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c006fd0 tx=0x7f0b5c006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:40.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 msgr2=0x7f0b6c19bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:40.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c19bae0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0b60000c00 tx=0x7f0b60011b50 comp rx=0 tx=0).stop 2024-04-26T04:33:40.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 shutdown_connections 2024-04-26T04:33:40.990 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f0b540383f0 0x7f0b5403a8a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:40.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 --2- 172.21.15.90:0/4000698611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c07bdd0 0x7f0b6c19bae0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:40.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.380+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 >> 172.21.15.90:0/4000698611 conn(0x7f0b6c101180 msgr2=0x7f0b6c102840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:40.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.381+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 shutdown_connections 2024-04-26T04:33:40.991 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:40.381+0000 7f0b718e7700 1 -- 172.21.15.90:0/4000698611 wait complete. 2024-04-26T04:33:40.991 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying prometheus service with default placement... 2024-04-26T04:33:41.630 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:41 smithi090 ceph-mon[26648]: from='client.14148 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "ceph-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:41.631 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:41 smithi090 ceph-mon[26648]: Saving service ceph-exporter spec with placement * 2024-04-26T04:33:41.631 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:41 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:41.631 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:41 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:42.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:42 smithi090 ceph-mon[26648]: from='client.14150 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:42.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:42 smithi090 ceph-mon[26648]: Saving service prometheus spec with placement count:1 2024-04-26T04:33:42.921 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:42 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:42.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled prometheus update... 2024-04-26T04:33:42.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.400+0000 7f54a2680700 1 Processor -- start 2024-04-26T04:33:42.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.400+0000 7f54a2680700 1 -- start start 2024-04-26T04:33:42.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f54a2680700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:42.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f54a2680700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c106460 con 0x7f549c105b10 2024-04-26T04:33:42.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f549bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:42.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f549bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49042/0 (socket says 172.21.15.90:49042) 2024-04-26T04:33:42.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f549bfff700 1 -- 172.21.15.90:0/3436606337 learned_addr learned my addr 172.21.15.90:0/3436606337 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:42.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f549bfff700 1 -- 172.21.15.90:0/3436606337 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c1065a0 con 0x7f549c105b10 2024-04-26T04:33:42.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f549bfff700 1 --2- 172.21.15.90:0/3436606337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c105f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54900098a0 tx=0x7f5490009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f68b6f009e41005 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.401+0000 7f549affd700 1 -- 172.21.15.90:0/3436606337 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5490004180 con 0x7f549c105b10 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.402+0000 7f549affd700 1 -- 172.21.15.90:0/3436606337 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f54900042e0 con 0x7f549c105b10 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.402+0000 7f549affd700 1 -- 172.21.15.90:0/3436606337 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5490011af0 con 0x7f549c105b10 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.402+0000 7f54a2680700 1 -- 172.21.15.90:0/3436606337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 msgr2=0x7f549c105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.402+0000 7f54a2680700 1 --2- 172.21.15.90:0/3436606337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c105f20 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f54900098a0 tx=0x7f5490009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.402+0000 7f54a2680700 1 -- 172.21.15.90:0/3436606337 shutdown_connections 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.402+0000 7f54a2680700 1 --2- 172.21.15.90:0/3436606337 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c105f20 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:42.934 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.402+0000 7f54a2680700 1 -- 172.21.15.90:0/3436606337 >> 172.21.15.90:0/3436606337 conn(0x7f549c101180 msgr2=0x7f549c103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.403+0000 7f54a2680700 1 -- 172.21.15.90:0/3436606337 shutdown_connections 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.403+0000 7f54a2680700 1 -- 172.21.15.90:0/3436606337 wait complete. 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.403+0000 7f54a2680700 1 Processor -- start 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.403+0000 7f54a2680700 1 -- start start 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f54a2680700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f54a2680700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c199dd0 con 0x7f549c105b10 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f549bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:42.935 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f549bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49046/0 (socket says 172.21.15.90:49046) 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f549bfff700 1 -- 172.21.15.90:0/2704888739 learned_addr learned my addr 172.21.15.90:0/2704888739 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f549bfff700 1 -- 172.21.15.90:0/2704888739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5490009550 con 0x7f549c105b10 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f549bfff700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c199890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5490011ad0 tx=0x7f5490011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f54997fa700 1 -- 172.21.15.90:0/2704888739 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5490003ab0 con 0x7f549c105b10 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f54997fa700 1 -- 172.21.15.90:0/2704888739 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5490003c10 con 0x7f549c105b10 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c199fd0 con 0x7f549c105b10 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f54997fa700 1 -- 172.21.15.90:0/2704888739 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f54900198e0 con 0x7f549c105b10 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.404+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c19a3f0 con 0x7f549c105b10 2024-04-26T04:33:42.936 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.405+0000 7f54997fa700 1 -- 172.21.15.90:0/2704888739 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f5490019ba0 con 0x7f549c105b10 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.405+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c062380 con 0x7f549c105b10 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.406+0000 7f54997fa700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f54840383f0 0x7f548403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.406+0000 7f54997fa700 1 -- 172.21.15.90:0/2704888739 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f549004c190 con 0x7f549c105b10 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.406+0000 7f549b7fe700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f54840383f0 0x7f548403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.406+0000 7f549b7fe700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f54840383f0 0x7f548403a8a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f548c006fd0 tx=0x7f548c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.412+0000 7f54997fa700 1 -- 172.21.15.90:0/2704888739 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f549002b300 con 0x7f549c105b10 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.623+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "target": ["mon-mgr", ""]}) v1 -- 0x7f549c102670 con 0x7f54840383f0 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.627+0000 7f54997fa700 1 -- 172.21.15.90:0/2704888739 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f549c102670 con 0x7f54840383f0 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f54840383f0 msgr2=0x7f548403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f54840383f0 0x7f548403a8a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f548c006fd0 tx=0x7f548c006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 msgr2=0x7f549c199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c199890 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f5490011ad0 tx=0x7f5490011bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:42.937 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 shutdown_connections 2024-04-26T04:33:42.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f54840383f0 0x7f548403a8a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:42.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 --2- 172.21.15.90:0/2704888739 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f549c105b10 0x7f549c199890 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:42.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 >> 172.21.15.90:0/2704888739 conn(0x7f549c101180 msgr2=0x7f549c101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:42.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 shutdown_connections 2024-04-26T04:33:42.938 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:41.630+0000 7f54a2680700 1 -- 172.21.15.90:0/2704888739 wait complete. 2024-04-26T04:33:42.938 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying grafana service with default placement... 2024-04-26T04:33:44.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled grafana update... 2024-04-26T04:33:44.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6b01492700 1 Processor -- start 2024-04-26T04:33:44.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6b01492700 1 -- start start 2024-04-26T04:33:44.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6b01492700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1086c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:44.098 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6b01492700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc108c00 con 0x7f6afc074470 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1086c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1086c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49048/0 (socket says 172.21.15.90:49048) 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6afaffd700 1 -- 172.21.15.90:0/2086155265 learned_addr learned my addr 172.21.15.90:0/2086155265 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6afaffd700 1 -- 172.21.15.90:0/2086155265 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6afc108d40 con 0x7f6afc074470 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6afaffd700 1 --2- 172.21.15.90:0/2086155265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1086c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0098a0 tx=0x7f6aec009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de32a316242edb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.320+0000 7f6af9ffb700 1 -- 172.21.15.90:0/2086155265 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6aec004180 con 0x7f6afc074470 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6af9ffb700 1 -- 172.21.15.90:0/2086155265 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6aec0042e0 con 0x7f6afc074470 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6af9ffb700 1 -- 172.21.15.90:0/2086155265 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6aec011af0 con 0x7f6afc074470 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6b01492700 1 -- 172.21.15.90:0/2086155265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 msgr2=0x7f6afc1086c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6b01492700 1 --2- 172.21.15.90:0/2086155265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1086c0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6aec0098a0 tx=0x7f6aec009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6b01492700 1 -- 172.21.15.90:0/2086155265 shutdown_connections 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6b01492700 1 --2- 172.21.15.90:0/2086155265 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1086c0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:44.099 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6b01492700 1 -- 172.21.15.90:0/2086155265 >> 172.21.15.90:0/2086155265 conn(0x7f6afc100070 msgr2=0x7f6afc102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6b01492700 1 -- 172.21.15.90:0/2086155265 shutdown_connections 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.321+0000 7f6b01492700 1 -- 172.21.15.90:0/2086155265 wait complete. 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6b01492700 1 Processor -- start 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6b01492700 1 -- start start 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6b01492700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6b01492700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6afc199df0 con 0x7f6afc074470 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6afaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49060/0 (socket says 172.21.15.90:49060) 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6afaffd700 1 -- 172.21.15.90:0/1868209144 learned_addr learned my addr 172.21.15.90:0/1868209144 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.322+0000 7f6afaffd700 1 -- 172.21.15.90:0/1868209144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6aec009550 con 0x7f6afc074470 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.323+0000 7f6afaffd700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1998b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6aec011aa0 tx=0x7f6aec011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.323+0000 7f6ae3fff700 1 -- 172.21.15.90:0/1868209144 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6aec0039f0 con 0x7f6afc074470 2024-04-26T04:33:44.100 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.323+0000 7f6ae3fff700 1 -- 172.21.15.90:0/1868209144 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6aec003b50 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.323+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6afc199ff0 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.323+0000 7f6ae3fff700 1 -- 172.21.15.90:0/1868209144 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6aec019850 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.323+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6afc19a410 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.324+0000 7f6ae3fff700 1 -- 172.21.15.90:0/1868209144 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f6aec019ac0 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.324+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6afc062380 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.324+0000 7f6ae3fff700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f6ae40383b0 0x7f6ae403a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.324+0000 7f6ae3fff700 1 -- 172.21.15.90:0/1868209144 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f6aec04c5f0 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.324+0000 7f6afa7fc700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f6ae40383b0 0x7f6ae403a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.324+0000 7f6afa7fc700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f6ae40383b0 0x7f6ae403a860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6af0006fd0 tx=0x7f6af0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.329+0000 7f6ae3fff700 1 -- 172.21.15.90:0/1868209144 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6aec010380 con 0x7f6afc074470 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.530+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}) v1 -- 0x7f6afc102320 con 0x7f6ae40383b0 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.535+0000 7f6ae3fff700 1 -- 172.21.15.90:0/1868209144 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6afc102320 con 0x7f6ae40383b0 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f6ae40383b0 msgr2=0x7f6ae403a860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:44.101 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f6ae40383b0 0x7f6ae403a860 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6af0006fd0 tx=0x7f6af0006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 msgr2=0x7f6afc1998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1998b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6aec011aa0 tx=0x7f6aec011b80 comp rx=0 tx=0).stop 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 shutdown_connections 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f6ae40383b0 0x7f6ae403a860 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 --2- 172.21.15.90:0/1868209144 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6afc074470 0x7f6afc1998b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 >> 172.21.15.90:0/1868209144 conn(0x7f6afc100070 msgr2=0x7f6afc101700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 shutdown_connections 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:43.537+0000 7f6b01492700 1 -- 172.21.15.90:0/1868209144 wait complete. 2024-04-26T04:33:44.102 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying node-exporter service with default placement... 2024-04-26T04:33:44.709 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:44 smithi090 ceph-mon[26648]: from='client.14152 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:44.709 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:44 smithi090 ceph-mon[26648]: Saving service grafana spec with placement count:1 2024-04-26T04:33:44.709 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:44 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled node-exporter update... 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.496+0000 7ff0c8735700 1 Processor -- start 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.496+0000 7ff0c8735700 1 -- start start 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.497+0000 7ff0c8735700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0105f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.497+0000 7ff0c8735700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0106460 con 0x7ff0c0105b10 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.497+0000 7ff0c64d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0105f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.497+0000 7ff0c64d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0105f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49062/0 (socket says 172.21.15.90:49062) 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.497+0000 7ff0c64d1700 1 -- 172.21.15.90:0/1913900253 learned_addr learned my addr 172.21.15.90:0/1913900253 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.497+0000 7ff0c64d1700 1 -- 172.21.15.90:0/1913900253 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c01065a0 con 0x7ff0c0105b10 2024-04-26T04:33:45.287 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.497+0000 7ff0c64d1700 1 --2- 172.21.15.90:0/1913900253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0105f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098a0 tx=0x7ff0b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4edfc5f8cefefc7b server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c54cf700 1 -- 172.21.15.90:0/1913900253 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b4004180 con 0x7ff0c0105b10 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c54cf700 1 -- 172.21.15.90:0/1913900253 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0b40042e0 con 0x7ff0c0105b10 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c54cf700 1 -- 172.21.15.90:0/1913900253 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b4011af0 con 0x7ff0c0105b10 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c8735700 1 -- 172.21.15.90:0/1913900253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 msgr2=0x7ff0c0105f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c8735700 1 --2- 172.21.15.90:0/1913900253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0105f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0b40098a0 tx=0x7ff0b4009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c8735700 1 -- 172.21.15.90:0/1913900253 shutdown_connections 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c8735700 1 --2- 172.21.15.90:0/1913900253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0105f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c8735700 1 -- 172.21.15.90:0/1913900253 >> 172.21.15.90:0/1913900253 conn(0x7ff0c0101180 msgr2=0x7ff0c0103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:45.288 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c8735700 1 -- 172.21.15.90:0/1913900253 shutdown_connections 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.498+0000 7ff0c8735700 1 -- 172.21.15.90:0/1913900253 wait complete. 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c8735700 1 Processor -- start 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c8735700 1 -- start start 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c8735700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c8735700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c0199d90 con 0x7ff0c0105b10 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c64d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c64d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49064/0 (socket says 172.21.15.90:49064) 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c64d1700 1 -- 172.21.15.90:0/1730893219 learned_addr learned my addr 172.21.15.90:0/1730893219 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c64d1700 1 -- 172.21.15.90:0/1730893219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b4009550 con 0x7ff0c0105b10 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.499+0000 7ff0c64d1700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0199850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4011ad0 tx=0x7ff0b4011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:45.289 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.500+0000 7ff0b37fe700 1 -- 172.21.15.90:0/1730893219 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b4003ab0 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.500+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c0199f90 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.500+0000 7ff0b37fe700 1 -- 172.21.15.90:0/1730893219 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff0b4003c10 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.500+0000 7ff0b37fe700 1 -- 172.21.15.90:0/1730893219 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff0b40198e0 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.500+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c019a3b0 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.500+0000 7ff0b37fe700 1 -- 172.21.15.90:0/1730893219 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7ff0b4019ba0 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.501+0000 7ff0b37fe700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff0ac0383f0 0x7ff0ac03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.501+0000 7ff0b37fe700 1 -- 172.21.15.90:0/1730893219 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7ff0b404c1d0 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.501+0000 7ff0c5cd0700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff0ac0383f0 0x7ff0ac03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.501+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c0062380 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.501+0000 7ff0c5cd0700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff0ac0383f0 0x7ff0ac03a8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc006fd0 tx=0x7ff0bc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.506+0000 7ff0b37fe700 1 -- 172.21.15.90:0/1730893219 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff0b402b300 con 0x7ff0c0105b10 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.708+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0c0102610 con 0x7ff0ac0383f0 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.712+0000 7ff0b37fe700 1 -- 172.21.15.90:0/1730893219 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7ff0c0102610 con 0x7ff0ac0383f0 2024-04-26T04:33:45.290 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff0ac0383f0 msgr2=0x7ff0ac03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff0ac0383f0 0x7ff0ac03a8a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc006fd0 tx=0x7ff0bc006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 msgr2=0x7ff0c0199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0199850 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff0b4011ad0 tx=0x7ff0b4011bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 shutdown_connections 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7ff0ac0383f0 0x7ff0ac03a8a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 --2- 172.21.15.90:0/1730893219 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0c0105b10 0x7ff0c0199850 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 >> 172.21.15.90:0/1730893219 conn(0x7ff0c0101180 msgr2=0x7ff0c0101f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 shutdown_connections 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:44.715+0000 7ff0c8735700 1 -- 172.21.15.90:0/1730893219 wait complete. 2024-04-26T04:33:45.291 INFO:teuthology.orchestra.run.smithi090.stdout:Deploying alertmanager service with default placement... 2024-04-26T04:33:45.917 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:45 smithi090 ceph-mon[26648]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:33:45.917 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:45 smithi090 ceph-mon[26648]: from='client.14154 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:45.917 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:45 smithi090 ceph-mon[26648]: Saving service node-exporter spec with placement * 2024-04-26T04:33:45.917 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:45 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:46.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Scheduled alertmanager update... 2024-04-26T04:33:46.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.687+0000 7f5b7eadc700 1 Processor -- start 2024-04-26T04:33:46.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7eadc700 1 -- start start 2024-04-26T04:33:46.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7eadc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b78105320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:46.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7eadc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b78105860 con 0x7f5b78104f10 2024-04-26T04:33:46.546 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7c878700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b78105320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7c878700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b78105320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49074/0 (socket says 172.21.15.90:49074) 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7c878700 1 -- 172.21.15.90:0/2505809854 learned_addr learned my addr 172.21.15.90:0/2505809854 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7c878700 1 -- 172.21.15.90:0/2505809854 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b781059a0 con 0x7f5b78104f10 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b7c878700 1 --2- 172.21.15.90:0/2505809854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b78105320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0098a0 tx=0x7f5b6c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c35e24a3a838cd server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.688+0000 7f5b777fe700 1 -- 172.21.15.90:0/2505809854 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b6c004180 con 0x7f5b78104f10 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b777fe700 1 -- 172.21.15.90:0/2505809854 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c0042e0 con 0x7f5b78104f10 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b777fe700 1 -- 172.21.15.90:0/2505809854 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b6c011af0 con 0x7f5b78104f10 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b7eadc700 1 -- 172.21.15.90:0/2505809854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 msgr2=0x7f5b78105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b7eadc700 1 --2- 172.21.15.90:0/2505809854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b78105320 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c0098a0 tx=0x7f5b6c009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:46.547 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b7eadc700 1 -- 172.21.15.90:0/2505809854 shutdown_connections 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b7eadc700 1 --2- 172.21.15.90:0/2505809854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b78105320 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b7eadc700 1 -- 172.21.15.90:0/2505809854 >> 172.21.15.90:0/2505809854 conn(0x7f5b780ff7e0 msgr2=0x7f5b78101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b7eadc700 1 -- 172.21.15.90:0/2505809854 shutdown_connections 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.689+0000 7f5b7eadc700 1 -- 172.21.15.90:0/2505809854 wait complete. 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7eadc700 1 Processor -- start 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7eadc700 1 -- start start 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7eadc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b781987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7eadc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b78198cf0 con 0x7f5b78104f10 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7c878700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b781987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7c878700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b781987b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49086/0 (socket says 172.21.15.90:49086) 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7c878700 1 -- 172.21.15.90:0/3248465572 learned_addr learned my addr 172.21.15.90:0/3248465572 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7c878700 1 -- 172.21.15.90:0/3248465572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b6c009550 con 0x7f5b78104f10 2024-04-26T04:33:46.548 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.690+0000 7f5b7c878700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b781987b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c006ba0 tx=0x7f5b6c011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.691+0000 7f5b75ffb700 1 -- 172.21.15.90:0/3248465572 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b6c003b00 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.691+0000 7f5b75ffb700 1 -- 172.21.15.90:0/3248465572 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5b6c01a8e0 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.691+0000 7f5b75ffb700 1 -- 172.21.15.90:0/3248465572 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f5b6c019990 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.691+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b78198ef0 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.691+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b78199310 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.692+0000 7f5b75ffb700 1 -- 172.21.15.90:0/3248465572 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f5b6c019c50 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.692+0000 7f5b75ffb700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f5b680383a0 0x7f5b6803a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.692+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b78062380 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.692+0000 7f5b75ffb700 1 -- 172.21.15.90:0/3248465572 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5b6c04c5b0 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.692+0000 7f5b77fff700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f5b680383a0 0x7f5b6803a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.692+0000 7f5b77fff700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f5b680383a0 0x7f5b6803a850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5b64006fd0 tx=0x7f5b64006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.697+0000 7f5b75ffb700 1 -- 172.21.15.90:0/3248465572 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5b6c018530 con 0x7f5b78104f10 2024-04-26T04:33:46.549 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.898+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b78101ab0 con 0x7f5b680383a0 2024-04-26T04:33:46.550 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.903+0000 7f5b75ffb700 1 -- 172.21.15.90:0/3248465572 <== mgr.14120 v2:172.21.15.90:6800/2176954768 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f5b78101ab0 con 0x7f5b680383a0 2024-04-26T04:33:46.550 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f5b680383a0 msgr2=0x7f5b6803a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:46.550 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f5b680383a0 0x7f5b6803a850 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f5b64006fd0 tx=0x7f5b64006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:46.550 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 msgr2=0x7f5b781987b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:46.550 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b781987b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f5b6c006ba0 tx=0x7f5b6c011b50 comp rx=0 tx=0).stop 2024-04-26T04:33:46.550 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 shutdown_connections 2024-04-26T04:33:46.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f5b680383a0 0x7f5b6803a850 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:46.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 --2- 172.21.15.90:0/3248465572 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b78104f10 0x7f5b781987b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:46.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 >> 172.21.15.90:0/3248465572 conn(0x7f5b780ff7e0 msgr2=0x7f5b78100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:46.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 shutdown_connections 2024-04-26T04:33:46.551 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:45.905+0000 7f5b7eadc700 1 -- 172.21.15.90:0/3248465572 wait complete. 2024-04-26T04:33:47.164 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:46 smithi090 ceph-mon[26648]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:33:47.164 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:46 smithi090 ceph-mon[26648]: Saving service alertmanager spec with placement count:1 2024-04-26T04:33:47.164 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:46 smithi090 ceph-mon[26648]: from='mgr.14120 172.21.15.90:0/3203305665' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:33:47.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.964+0000 7f8bc45f1700 1 Processor -- start 2024-04-26T04:33:47.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.964+0000 7f8bc45f1700 1 -- start start 2024-04-26T04:33:47.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.965+0000 7f8bc45f1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc1089f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:47.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.965+0000 7f8bc45f1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc074510 con 0x7f8bbc106610 2024-04-26T04:33:47.929 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.965+0000 7f8bc238d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc1089f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.965+0000 7f8bc238d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc1089f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49100/0 (socket says 172.21.15.90:49100) 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.965+0000 7f8bc238d700 1 -- 172.21.15.90:0/116237142 learned_addr learned my addr 172.21.15.90:0/116237142 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.965+0000 7f8bc238d700 1 -- 172.21.15.90:0/116237142 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bbc074650 con 0x7f8bbc106610 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.965+0000 7f8bc238d700 1 --2- 172.21.15.90:0/116237142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc1089f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80098a0 tx=0x7f8bb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e6e192282b14285 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc138b700 1 -- 172.21.15.90:0/116237142 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb8004180 con 0x7f8bbc106610 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc138b700 1 -- 172.21.15.90:0/116237142 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bb80042e0 con 0x7f8bbc106610 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc138b700 1 -- 172.21.15.90:0/116237142 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb8011af0 con 0x7f8bbc106610 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc45f1700 1 -- 172.21.15.90:0/116237142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 msgr2=0x7f8bbc1089f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc45f1700 1 --2- 172.21.15.90:0/116237142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc1089f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bb80098a0 tx=0x7f8bb8009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc45f1700 1 -- 172.21.15.90:0/116237142 shutdown_connections 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc45f1700 1 --2- 172.21.15.90:0/116237142 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc1089f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc45f1700 1 -- 172.21.15.90:0/116237142 >> 172.21.15.90:0/116237142 conn(0x7f8bbc100070 msgr2=0x7f8bbc102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:47.930 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc45f1700 1 -- 172.21.15.90:0/116237142 shutdown_connections 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.966+0000 7f8bc45f1700 1 -- 172.21.15.90:0/116237142 wait complete. 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc45f1700 1 Processor -- start 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc45f1700 1 -- start start 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc45f1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc105f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc45f1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bbc1064b0 con 0x7f8bbc106610 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc238d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc105f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc238d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc105f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49112/0 (socket says 172.21.15.90:49112) 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc238d700 1 -- 172.21.15.90:0/3480489428 learned_addr learned my addr 172.21.15.90:0/3480489428 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc238d700 1 -- 172.21.15.90:0/3480489428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb8009550 con 0x7f8bbc106610 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.967+0000 7f8bc238d700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc105f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8006ba0 tx=0x7f8bb8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.968+0000 7f8bb37fe700 1 -- 172.21.15.90:0/3480489428 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb8003b00 con 0x7f8bbc106610 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.968+0000 7f8bb37fe700 1 -- 172.21.15.90:0/3480489428 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8bb801a8e0 con 0x7f8bbc106610 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.968+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bbc1045c0 con 0x7f8bbc106610 2024-04-26T04:33:47.931 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.968+0000 7f8bb37fe700 1 -- 172.21.15.90:0/3480489428 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f8bb8019990 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.968+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bbc1049e0 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.969+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bbc109bf0 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.969+0000 7f8bb37fe700 1 -- 172.21.15.90:0/3480489428 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f8bb8019c50 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.969+0000 7f8bb37fe700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f8ba80383a0 0x7f8ba803a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.969+0000 7f8bb37fe700 1 -- 172.21.15.90:0/3480489428 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8bb804c7a0 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.969+0000 7f8bc1b8c700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f8ba80383a0 0x7f8ba803a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.969+0000 7f8bc1b8c700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f8ba80383a0 0x7f8ba803a850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006fd0 tx=0x7f8bac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:46.973+0000 7f8bb37fe700 1 -- 172.21.15.90:0/3480489428 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8bb8018b40 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.162+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f8bbc1054a0 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.166+0000 7f8bb37fe700 1 -- 172.21.15.90:0/3480489428 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8bb8018b40 con 0x7f8bbc106610 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.168+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f8ba80383a0 msgr2=0x7f8ba803a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.168+0000 7f8bc45f1700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f8ba80383a0 0x7f8ba803a850 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8bac006fd0 tx=0x7f8bac006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.168+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 msgr2=0x7f8bbc105f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:47.932 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.168+0000 7f8bc45f1700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc105f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f8bb8006ba0 tx=0x7f8bb8011b50 comp rx=0 tx=0).stop 2024-04-26T04:33:47.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.169+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 shutdown_connections 2024-04-26T04:33:47.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.169+0000 7f8bc45f1700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f8ba80383a0 0x7f8ba803a850 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:47.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.169+0000 7f8bc45f1700 1 --2- 172.21.15.90:0/3480489428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8bbc106610 0x7f8bbc105f70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:47.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.169+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 >> 172.21.15.90:0/3480489428 conn(0x7f8bbc100070 msgr2=0x7f8bbc101780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:47.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.169+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 shutdown_connections 2024-04-26T04:33:47.933 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:47.169+0000 7f8bc45f1700 1 -- 172.21.15.90:0/3480489428 wait complete. 2024-04-26T04:33:48.161 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:47 smithi090 ceph-mon[26648]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:33:48.161 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:47 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3480489428' entity='client.admin' 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.344+0000 7f63c003c700 1 Processor -- start 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.344+0000 7f63c003c700 1 -- start start 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.344+0000 7f63c003c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8104210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.344+0000 7f63c003c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b8104750 con 0x7f63b8103e00 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.344+0000 7f63bddd8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8104210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.344+0000 7f63bddd8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8104210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54050/0 (socket says 172.21.15.90:54050) 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.344+0000 7f63bddd8700 1 -- 172.21.15.90:0/570486429 learned_addr learned my addr 172.21.15.90:0/570486429 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63bddd8700 1 -- 172.21.15.90:0/570486429 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b8104890 con 0x7f63b8103e00 2024-04-26T04:33:49.139 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63bddd8700 1 --2- 172.21.15.90:0/570486429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8104210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63b40098a0 tx=0x7f63b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8993c446e2296a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63bcdd6700 1 -- 172.21.15.90:0/570486429 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b4004180 con 0x7f63b8103e00 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63bcdd6700 1 -- 172.21.15.90:0/570486429 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f63b40042e0 con 0x7f63b8103e00 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63bcdd6700 1 -- 172.21.15.90:0/570486429 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b4011af0 con 0x7f63b8103e00 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63c003c700 1 -- 172.21.15.90:0/570486429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 msgr2=0x7f63b8104210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63c003c700 1 --2- 172.21.15.90:0/570486429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8104210 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63b40098a0 tx=0x7f63b4009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63c003c700 1 -- 172.21.15.90:0/570486429 shutdown_connections 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63c003c700 1 --2- 172.21.15.90:0/570486429 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8104210 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.345+0000 7f63c003c700 1 -- 172.21.15.90:0/570486429 >> 172.21.15.90:0/570486429 conn(0x7f63b80ff7e0 msgr2=0x7f63b8101c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63c003c700 1 -- 172.21.15.90:0/570486429 shutdown_connections 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63c003c700 1 -- 172.21.15.90:0/570486429 wait complete. 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63c003c700 1 Processor -- start 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63c003c700 1 -- start start 2024-04-26T04:33:49.140 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63c003c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8106790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63c003c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b8106cd0 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63bddd8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8106790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63bddd8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8106790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54064/0 (socket says 172.21.15.90:54064) 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.346+0000 7f63bddd8700 1 -- 172.21.15.90:0/2793279866 learned_addr learned my addr 172.21.15.90:0/2793279866 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.347+0000 7f63bddd8700 1 -- 172.21.15.90:0/2793279866 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b4009550 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.347+0000 7f63bddd8700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8106790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006ba0 tx=0x7f63b4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.347+0000 7f63aeffd700 1 -- 172.21.15.90:0/2793279866 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b4003b00 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.347+0000 7f63aeffd700 1 -- 172.21.15.90:0/2793279866 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f63b401a8e0 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.347+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b8104de0 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.347+0000 7f63aeffd700 1 -- 172.21.15.90:0/2793279866 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f63b4019990 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.347+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b8105200 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.348+0000 7f63aeffd700 1 -- 172.21.15.90:0/2793279866 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f63b4019c00 con 0x7f63b8103e00 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.348+0000 7f63aeffd700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f63a40383f0 0x7f63a403a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:49.141 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.348+0000 7f63aeffd700 1 -- 172.21.15.90:0/2793279866 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f63b404c550 con 0x7f63b8103e00 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.348+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b8062380 con 0x7f63b8103e00 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.348+0000 7f63bd5d7700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f63a40383f0 0x7f63a403a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.348+0000 7f63bd5d7700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f63a40383f0 0x7f63a403a8a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006fd0 tx=0x7f63a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.353+0000 7f63aeffd700 1 -- 172.21.15.90:0/2793279866 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f63b402d440 con 0x7f63b8103e00 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.540+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f63b8105c40 con 0x7f63b8103e00 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.543+0000 7f63aeffd700 1 -- 172.21.15.90:0/2793279866 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f63b4049020 con 0x7f63b8103e00 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f63a40383f0 msgr2=0x7f63a403a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f63a40383f0 0x7f63a403a8a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f63a8006fd0 tx=0x7f63a8006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 msgr2=0x7f63b8106790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8106790 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f63b4006ba0 tx=0x7f63b4011b50 comp rx=0 tx=0).stop 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 shutdown_connections 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f63a40383f0 0x7f63a403a8a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:49.142 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 --2- 172.21.15.90:0/2793279866 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b8103e00 0x7f63b8106790 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:49.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.545+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 >> 172.21.15.90:0/2793279866 conn(0x7f63b80ff7e0 msgr2=0x7f63b8100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:49.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.546+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 shutdown_connections 2024-04-26T04:33:49.143 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:48.546+0000 7f63c003c700 1 -- 172.21.15.90:0/2793279866 wait complete. 2024-04-26T04:33:49.143 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling the dashboard module... 2024-04-26T04:33:49.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:49 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2793279866' entity='client.admin' 2024-04-26T04:33:49.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:49 smithi090 ceph-mon[26648]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:33:50.870 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:50 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1048135858' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-26T04:33:51.181 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.519+0000 7f66ffb0a700 1 Processor -- start 2024-04-26T04:33:51.181 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.519+0000 7f66ffb0a700 1 -- start start 2024-04-26T04:33:51.181 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.519+0000 7f66ffb0a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f8079cf0 0x7f66f810aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:51.181 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.519+0000 7f66ffb0a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f81064c0 con 0x7f66f8079cf0 2024-04-26T04:33:51.181 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fd8a6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f8079cf0 0x7f66f810aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fd8a6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f8079cf0 0x7f66f810aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54078/0 (socket says 172.21.15.90:54078) 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fd8a6700 1 -- 172.21.15.90:0/2888683868 learned_addr learned my addr 172.21.15.90:0/2888683868 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fd8a6700 1 -- 172.21.15.90:0/2888683868 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f810b010 con 0x7f66f8079cf0 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fd8a6700 1 --2- 172.21.15.90:0/2888683868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f8079cf0 0x7f66f810aad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66e80098a0 tx=0x7f66e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7480af9b87111532 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fc8a4700 1 -- 172.21.15.90:0/2888683868 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66e8004180 con 0x7f66f8079cf0 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fc8a4700 1 -- 172.21.15.90:0/2888683868 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66e80042e0 con 0x7f66f8079cf0 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.520+0000 7f66fc8a4700 1 -- 172.21.15.90:0/2888683868 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66e8011af0 con 0x7f66f8079cf0 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.521+0000 7f66ffb0a700 1 -- 172.21.15.90:0/2888683868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f8079cf0 msgr2=0x7f66f810aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.521+0000 7f66ffb0a700 1 --2- 172.21.15.90:0/2888683868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f8079cf0 0x7f66f810aad0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f66e80098a0 tx=0x7f66e8009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.521+0000 7f66ffb0a700 1 -- 172.21.15.90:0/2888683868 shutdown_connections 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.521+0000 7f66ffb0a700 1 --2- 172.21.15.90:0/2888683868 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f8079cf0 0x7f66f810aad0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.521+0000 7f66ffb0a700 1 -- 172.21.15.90:0/2888683868 >> 172.21.15.90:0/2888683868 conn(0x7f66f8100060 msgr2=0x7f66f81024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:51.182 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.521+0000 7f66ffb0a700 1 -- 172.21.15.90:0/2888683868 shutdown_connections 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.521+0000 7f66ffb0a700 1 -- 172.21.15.90:0/2888683868 wait complete. 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66ffb0a700 1 Processor -- start 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66ffb0a700 1 -- start start 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66ffb0a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f819baf0 0x7f66f819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66ffb0a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f819c440 con 0x7f66f819baf0 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66fd8a6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f819baf0 0x7f66f819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66fd8a6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f819baf0 0x7f66f819bf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54094/0 (socket says 172.21.15.90:54094) 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66fd8a6700 1 -- 172.21.15.90:0/1048135858 learned_addr learned my addr 172.21.15.90:0/1048135858 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66fd8a6700 1 -- 172.21.15.90:0/1048135858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009550 con 0x7f66f819baf0 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66fd8a6700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f819baf0 0x7f66f819bf00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66e8011670 tx=0x7f66e8011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66e80039f0 con 0x7f66f819baf0 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66e8003b50 con 0x7f66f819baf0 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f819c640 con 0x7f66f819baf0 2024-04-26T04:33:51.183 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66e80247c0 con 0x7f66f819baf0 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.522+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f819f2a0 con 0x7f66f819baf0 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.523+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 8) v1 ==== 45084+0+0 (secure 0 0 0) 0x7f66e801ac00 con 0x7f66f819baf0 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.523+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f8195b60 con 0x7f66f819baf0 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.523+0000 7f66eeffd700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66e4038440 0x7f66e403a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.524+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f66e804c3b0 con 0x7f66f819baf0 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.524+0000 7f66fd0a5700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66e4038440 0x7f66e403a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.524+0000 7f66fd0a5700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66e4038440 0x7f66e403a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006fd0 tx=0x7f66f4006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:51.184 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.528+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f66e801a430 con 0x7f66f819baf0 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:49.773+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f66f8079cf0 con 0x7f66f819baf0 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.549+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f66e802d770 con 0x7f66f819baf0 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.551+0000 7f66eeffd700 1 -- 172.21.15.90:0/1048135858 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f66e801a430 con 0x7f66f819baf0 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66e4038440 msgr2=0x7f66e403a8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66e4038440 0x7f66e403a8f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66f4006fd0 tx=0x7f66f4006e40 comp rx=0 tx=0).stop 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f819baf0 msgr2=0x7f66f819bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f819baf0 0x7f66f819bf00 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f66e8011670 tx=0x7f66e8011b50 comp rx=0 tx=0).stop 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 shutdown_connections 2024-04-26T04:33:51.185 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66e4038440 0x7f66e403a8f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:51.186 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 --2- 172.21.15.90:0/1048135858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66f819baf0 0x7f66f819bf00 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:51.186 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.553+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 >> 172.21.15.90:0/1048135858 conn(0x7f66f8100060 msgr2=0x7f66f8101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:51.186 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.554+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 shutdown_connections 2024-04-26T04:33:51.186 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:50.554+0000 7f66ffb0a700 1 -- 172.21.15.90:0/1048135858 wait complete. 2024-04-26T04:33:51.825 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:51 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1048135858' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-26T04:33:51.825 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:51 smithi090 ceph-mon[26648]: mgrmap e9: smithi090.iwrvwa(active, since 25s) 2024-04-26T04:33:52.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:52.507 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "active_name": "smithi090.iwrvwa", 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f6708c35700 1 Processor -- start 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f6708c35700 1 -- start start 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f6708c35700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f6704104e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f6708c35700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6704105350 con 0x7f6704104a00 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f670259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f6704104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f670259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f6704104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54130/0 (socket says 172.21.15.90:54130) 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f670259c700 1 -- 172.21.15.90:0/804522153 learned_addr learned my addr 172.21.15.90:0/804522153 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.618+0000 7f670259c700 1 -- 172.21.15.90:0/804522153 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6704105490 con 0x7f6704104a00 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.619+0000 7f670259c700 1 --2- 172.21.15.90:0/804522153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f6704104e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66f40098a0 tx=0x7f66f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807745822b0f353 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:52.508 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.619+0000 7f670159a700 1 -- 172.21.15.90:0/804522153 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66f4004180 con 0x7f6704104a00 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.619+0000 7f670159a700 1 -- 172.21.15.90:0/804522153 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66f40042e0 con 0x7f6704104a00 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.619+0000 7f670159a700 1 -- 172.21.15.90:0/804522153 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66f4011af0 con 0x7f6704104a00 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.619+0000 7f6708c35700 1 -- 172.21.15.90:0/804522153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 msgr2=0x7f6704104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.619+0000 7f6708c35700 1 --2- 172.21.15.90:0/804522153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f6704104e10 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f66f40098a0 tx=0x7f66f4009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.620+0000 7f6708c35700 1 -- 172.21.15.90:0/804522153 shutdown_connections 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.620+0000 7f6708c35700 1 --2- 172.21.15.90:0/804522153 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f6704104e10 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.620+0000 7f6708c35700 1 -- 172.21.15.90:0/804522153 >> 172.21.15.90:0/804522153 conn(0x7f6704100070 msgr2=0x7f6704102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.620+0000 7f6708c35700 1 -- 172.21.15.90:0/804522153 shutdown_connections 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.620+0000 7f6708c35700 1 -- 172.21.15.90:0/804522153 wait complete. 2024-04-26T04:33:52.509 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f6708c35700 1 Processor -- start 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f6708c35700 1 -- start start 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f6708c35700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f67040742e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f6708c35700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f670419c5b0 con 0x7f6704104a00 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f670259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f67040742e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f670259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f67040742e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54136/0 (socket says 172.21.15.90:54136) 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f670259c700 1 -- 172.21.15.90:0/1830902058 learned_addr learned my addr 172.21.15.90:0/1830902058 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f670259c700 1 -- 172.21.15.90:0/1830902058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4009550 con 0x7f6704104a00 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f670259c700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f67040742e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f66f4011ad0 tx=0x7f66f4011bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.621+0000 7f66f37fe700 1 -- 172.21.15.90:0/1830902058 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66f4003ab0 con 0x7f6704104a00 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.622+0000 7f66f37fe700 1 -- 172.21.15.90:0/1830902058 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f66f4003c10 con 0x7f6704104a00 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.622+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f670419c6f0 con 0x7f6704104a00 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.622+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f670419cb10 con 0x7f6704104a00 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.622+0000 7f66f37fe700 1 -- 172.21.15.90:0/1830902058 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f66f40198e0 con 0x7f6704104a00 2024-04-26T04:33:52.510 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.622+0000 7f66f37fe700 1 -- 172.21.15.90:0/1830902058 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f66f4019b50 con 0x7f6704104a00 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.622+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6704062380 con 0x7f6704104a00 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.623+0000 7f66f37fe700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 0x7f66ec03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.623+0000 7f66f37fe700 1 -- 172.21.15.90:0/1830902058 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f66f404c1b0 con 0x7f6704104a00 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.623+0000 7f6701d9b700 1 -- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 msgr2=0x7f66ec03a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2176954768 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.623+0000 7f6701d9b700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 0x7f66ec03a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.627+0000 7f66f37fe700 1 -- 172.21.15.90:0/1830902058 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f66f402b300 con 0x7f6704104a00 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.824+0000 7f6701d9b700 1 -- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 msgr2=0x7f66ec03a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2176954768 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.824+0000 7f6701d9b700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 0x7f66ec03a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.907+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f670419d180 con 0x7f6704104a00 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.908+0000 7f66f37fe700 1 -- 172.21.15.90:0/1830902058 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+103 (secure 0 0 0) 0x7f66f4018350 con 0x7f6704104a00 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 msgr2=0x7f66ec03a940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 0x7f66ec03a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 msgr2=0x7f67040742e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:52.511 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f67040742e0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f66f4011ad0 tx=0x7f66f4011bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 shutdown_connections 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f66ec038490 0x7f66ec03a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 --2- 172.21.15.90:0/1830902058 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6704104a00 0x7f67040742e0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 >> 172.21.15.90:0/1830902058 conn(0x7f6704100070 msgr2=0x7f6704100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 shutdown_connections 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:51.910+0000 7f6708c35700 1 -- 172.21.15.90:0/1830902058 wait complete. 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for the mgr to restart... 2024-04-26T04:33:52.512 INFO:teuthology.orchestra.run.smithi090.stdout:Waiting for mgr epoch 9... 2024-04-26T04:33:52.836 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:52 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1830902058' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: Active manager daemon smithi090.iwrvwa restarted 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: Activating manager daemon smithi090.iwrvwa 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: osdmap e3: 0 total, 0 up, 0 in 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: mgrmap e10: smithi090.iwrvwa(active, starting, since 0.00391704s) 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:33:57.154 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:33:57.155 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:33:57.155 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:33:57.155 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:57.155 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:33:57.155 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:33:57.155 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:33:58.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:58 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:57] ENGINE Bus STARTING 2024-04-26T04:33:58.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:58 smithi090 ceph-mon[26648]: mgrmap e11: smithi090.iwrvwa(active, since 1.00567s) 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout { 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout } 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.935+0000 7f4174e37700 1 Processor -- start 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.935+0000 7f4174e37700 1 -- start start 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.936+0000 7f4174e37700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170108a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.936+0000 7f4174e37700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41700744d0 con 0x7f4170106650 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.936+0000 7f416e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170108a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.936+0000 7f416e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170108a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54152/0 (socket says 172.21.15.90:54152) 2024-04-26T04:33:58.789 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.936+0000 7f416e59c700 1 -- 172.21.15.90:0/2929621561 learned_addr learned my addr 172.21.15.90:0/2929621561 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.936+0000 7f416e59c700 1 -- 172.21.15.90:0/2929621561 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4170074610 con 0x7f4170106650 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.936+0000 7f416e59c700 1 --2- 172.21.15.90:0/2929621561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170108a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f4160009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b9ebe827761a045 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f416d59a700 1 -- 172.21.15.90:0/2929621561 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4160004180 con 0x7f4170106650 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f416d59a700 1 -- 172.21.15.90:0/2929621561 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f41600042e0 con 0x7f4170106650 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f416d59a700 1 -- 172.21.15.90:0/2929621561 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4160011af0 con 0x7f4170106650 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f4174e37700 1 -- 172.21.15.90:0/2929621561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 msgr2=0x7f4170108a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f4174e37700 1 --2- 172.21.15.90:0/2929621561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170108a30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f41600098a0 tx=0x7f4160009bb0 comp rx=0 tx=0).stop 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f4174e37700 1 -- 172.21.15.90:0/2929621561 shutdown_connections 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f4174e37700 1 --2- 172.21.15.90:0/2929621561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170108a30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f4174e37700 1 -- 172.21.15.90:0/2929621561 >> 172.21.15.90:0/2929621561 conn(0x7f4170100070 msgr2=0x7f4170102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f4174e37700 1 -- 172.21.15.90:0/2929621561 shutdown_connections 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.937+0000 7f4174e37700 1 -- 172.21.15.90:0/2929621561 wait complete. 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.938+0000 7f4174e37700 1 Processor -- start 2024-04-26T04:33:58.790 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.938+0000 7f4174e37700 1 -- start start 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.938+0000 7f4174e37700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170105fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.938+0000 7f4174e37700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41701064f0 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.938+0000 7f416e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170105fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.938+0000 7f416e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170105fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54158/0 (socket says 172.21.15.90:54158) 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.938+0000 7f416e59c700 1 -- 172.21.15.90:0/2571101966 learned_addr learned my addr 172.21.15.90:0/2571101966 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.939+0000 7f416e59c700 1 -- 172.21.15.90:0/2571101966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4160009550 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.939+0000 7f416e59c700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170105fb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.939+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4160003a70 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.939+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4160003bd0 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.939+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4170104660 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.939+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f41600238c0 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.939+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4170104ae0 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.940+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 9) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f4160023b30 con 0x7f4170106650 2024-04-26T04:33:58.791 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.940+0000 7f415f7fe700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.940+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 --> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f415803b050 con 0x7f4158038490 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.941+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f416004c340 con 0x7f4170106650 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.940+0000 7f416dd9b700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 msgr2=0x7f415803a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2176954768 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:52.941+0000 7f416dd9b700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:53.142+0000 7f416dd9b700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 msgr2=0x7f415803a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2176954768 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:53.142+0000 7f416dd9b700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:53.543+0000 7f416dd9b700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 msgr2=0x7f415803a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2176954768 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:53.543+0000 7f416dd9b700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:54.344+0000 7f416dd9b700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 msgr2=0x7f415803a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2176954768 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:54.344+0000 7f416dd9b700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:55.945+0000 7f416dd9b700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 msgr2=0x7f415803a940 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2176954768 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:55.945+0000 7f416dd9b700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-26T04:33:58.792 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:57.090+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mgrmap(e 10) v1 ==== 44864+0+0 (secure 0 0 0) 0x7f416001ad00 con 0x7f4170106650 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:57.090+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 msgr2=0x7f415803a940 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:57.090+0000 7f415f7fe700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.091+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7f416004dd90 con 0x7f4170106650 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.091+0000 7f415f7fe700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f415803bef0 0x7f415803e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.091+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f415803b050 con 0x7f415803bef0 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.092+0000 7f416dd9b700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f415803bef0 0x7f415803e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.092+0000 7f416dd9b700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f415803bef0 0x7f415803e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4164003a10 tx=0x7f4164009270 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.093+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mgr.14164 v2:172.21.15.90:6800/2647874706 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7f415803b050 con 0x7f415803bef0 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.099+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f4170105b50 con 0x7f415803bef0 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.099+0000 7f415f7fe700 1 -- 172.21.15.90:0/2571101966 <== mgr.14164 v2:172.21.15.90:6800/2647874706 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f4170105b50 con 0x7f415803bef0 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f415803bef0 msgr2=0x7f415803e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f415803bef0 0x7f415803e2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f4164003a10 tx=0x7f4164009270 comp rx=0 tx=0).stop 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 msgr2=0x7f4170105fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:33:58.793 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170105fb0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4160000c00 tx=0x7f4160011b50 comp rx=0 tx=0).stop 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 shutdown_connections 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f415803bef0 0x7f415803e2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4170106650 0x7f4170105fb0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 --2- 172.21.15.90:0/2571101966 >> [v2:172.21.15.90:6800/2176954768,v1:172.21.15.90:6801/2176954768] conn(0x7f4158038490 0x7f415803a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 >> 172.21.15.90:0/2571101966 conn(0x7f4170100070 msgr2=0x7f4170101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 shutdown_connections 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:58.100+0000 7f4174e37700 1 -- 172.21.15.90:0/2571101966 wait complete. 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:mgr epoch 9 is available 2024-04-26T04:33:58.794 INFO:teuthology.orchestra.run.smithi090.stdout:Generating a dashboard self-signed certificate... 2024-04-26T04:33:59.434 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:59 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:57] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T04:33:59.434 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:59 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:57] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T04:33:59.434 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:59 smithi090 ceph-mon[26648]: [26/Apr/2024:04:33:57] ENGINE Bus STARTED 2024-04-26T04:33:59.434 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:59 smithi090 ceph-mon[26648]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-26T04:33:59.435 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:59 smithi090 ceph-mon[26648]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-26T04:33:59.435 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:33:59 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:00.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-26T04:34:00.121 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f71b0700 1 Processor -- start 2024-04-26T04:34:00.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f71b0700 1 -- start start 2024-04-26T04:34:00.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f71b0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01086e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:00.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f71b0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0108c20 con 0x7fc1f0074450 2024-04-26T04:34:00.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f4f4c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01086e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:00.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f4f4c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01086e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48788/0 (socket says 172.21.15.90:48788) 2024-04-26T04:34:00.122 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f4f4c700 1 -- 172.21.15.90:0/2862658329 learned_addr learned my addr 172.21.15.90:0/2862658329 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.220+0000 7fc1f4f4c700 1 -- 172.21.15.90:0/2862658329 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0108d60 con 0x7fc1f0074450 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.221+0000 7fc1f4f4c700 1 --2- 172.21.15.90:0/2862658329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01086e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098a0 tx=0x7fc1e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=630954ed67298156 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.221+0000 7fc1ef7fe700 1 -- 172.21.15.90:0/2862658329 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1e4004180 con 0x7fc1f0074450 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.221+0000 7fc1ef7fe700 1 -- 172.21.15.90:0/2862658329 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc1e40042e0 con 0x7fc1f0074450 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.221+0000 7fc1ef7fe700 1 -- 172.21.15.90:0/2862658329 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1e4011af0 con 0x7fc1f0074450 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 -- 172.21.15.90:0/2862658329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 msgr2=0x7fc1f01086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 --2- 172.21.15.90:0/2862658329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01086e0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098a0 tx=0x7fc1e4009bb0 comp rx=0 tx=0).stop 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 -- 172.21.15.90:0/2862658329 shutdown_connections 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 --2- 172.21.15.90:0/2862658329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01086e0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 -- 172.21.15.90:0/2862658329 >> 172.21.15.90:0/2862658329 conn(0x7fc1f0100070 msgr2=0x7fc1f0102480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 -- 172.21.15.90:0/2862658329 shutdown_connections 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 -- 172.21.15.90:0/2862658329 wait complete. 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.222+0000 7fc1f71b0700 1 Processor -- start 2024-04-26T04:34:00.123 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f71b0700 1 -- start start 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f71b0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f71b0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0199df0 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f4f4c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f4f4c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48802/0 (socket says 172.21.15.90:48802) 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f4f4c700 1 -- 172.21.15.90:0/3188387225 learned_addr learned my addr 172.21.15.90:0/3188387225 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f4f4c700 1 -- 172.21.15.90:0/3188387225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e4009550 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1f4f4c700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01998b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4011aa0 tx=0x7fc1e4011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.223+0000 7fc1edffb700 1 -- 172.21.15.90:0/3188387225 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1e40039f0 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.224+0000 7fc1edffb700 1 -- 172.21.15.90:0/3188387225 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc1e4003b50 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.224+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f0199ff0 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.224+0000 7fc1edffb700 1 -- 172.21.15.90:0/3188387225 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fc1e4019850 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.224+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f019a410 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.225+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f0062380 con 0x7fc1f0074450 2024-04-26T04:34:00.124 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.225+0000 7fc1edffb700 1 -- 172.21.15.90:0/3188387225 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 11) v1 ==== 44991+0+0 (secure 0 0 0) 0x7fc1e40199b0 con 0x7fc1f0074450 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.225+0000 7fc1edffb700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fc1e0038330 0x7fc1e003a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.225+0000 7fc1edffb700 1 -- 172.21.15.90:0/3188387225 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc1e404c330 con 0x7fc1f0074450 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.225+0000 7fc1effff700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fc1e0038330 0x7fc1e003a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.226+0000 7fc1effff700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fc1e0038330 0x7fc1e003a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc006fd0 tx=0x7fc1dc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.230+0000 7fc1edffb700 1 -- 172.21.15.90:0/3188387225 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc1e4010380 con 0x7fc1f0074450 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.432+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc1f0102370 con 0x7fc1e0038330 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.509+0000 7fc1edffb700 1 -- 172.21.15.90:0/3188387225 <== mgr.14164 v2:172.21.15.90:6800/2647874706 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc1f0102370 con 0x7fc1e0038330 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fc1e0038330 msgr2=0x7fc1e003a7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fc1e0038330 0x7fc1e003a7e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc006fd0 tx=0x7fc1dc006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 msgr2=0x7fc1f01998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01998b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4011aa0 tx=0x7fc1e4011b80 comp rx=0 tx=0).stop 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 shutdown_connections 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fc1e0038330 0x7fc1e003a7e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:00.125 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 --2- 172.21.15.90:0/3188387225 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc1f0074450 0x7fc1f01998b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:00.126 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 >> 172.21.15.90:0/3188387225 conn(0x7fc1f0100070 msgr2=0x7fc1f0101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:00.126 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 shutdown_connections 2024-04-26T04:34:00.126 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:33:59.511+0000 7fc1f71b0700 1 -- 172.21.15.90:0/3188387225 wait complete. 2024-04-26T04:34:00.126 INFO:teuthology.orchestra.run.smithi090.stdout:Creating initial admin user... 2024-04-26T04:34:00.736 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:00 smithi090 ceph-mon[26648]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:34:00.736 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:00 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:00.736 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:00 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:00.737 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:00 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:00.737 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:00 smithi090 ceph-mon[26648]: mgrmap e12: smithi090.iwrvwa(active, since 2s) 2024-04-26T04:34:01.973 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$OnJHbOQTOOnf95Yuw0ocG.vYtnh5ZEf0AsTJQzhSAg5bVtKm/WNsG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1714106040, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-26T04:34:01.973 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.502+0000 7fe555cad700 1 Processor -- start 2024-04-26T04:34:01.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.502+0000 7fe555cad700 1 -- start start 2024-04-26T04:34:01.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.502+0000 7fe555cad700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:01.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.502+0000 7fe555cad700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55007a850 con 0x7fe55007be10 2024-04-26T04:34:01.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.502+0000 7fe54f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:01.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.503+0000 7fe54f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48806/0 (socket says 172.21.15.90:48806) 2024-04-26T04:34:01.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.503+0000 7fe54f7fe700 1 -- 172.21.15.90:0/3256224994 learned_addr learned my addr 172.21.15.90:0/3256224994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:01.974 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.503+0000 7fe54f7fe700 1 -- 172.21.15.90:0/3256224994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe55007a990 con 0x7fe55007be10 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.503+0000 7fe54f7fe700 1 --2- 172.21.15.90:0/3256224994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55007a310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5400098a0 tx=0x7fe540009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9401f178c7231b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.503+0000 7fe54e7fc700 1 -- 172.21.15.90:0/3256224994 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe540004180 con 0x7fe55007be10 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.503+0000 7fe54e7fc700 1 -- 172.21.15.90:0/3256224994 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe5400042e0 con 0x7fe55007be10 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.503+0000 7fe54e7fc700 1 -- 172.21.15.90:0/3256224994 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe540011af0 con 0x7fe55007be10 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.504+0000 7fe555cad700 1 -- 172.21.15.90:0/3256224994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 msgr2=0x7fe55007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.504+0000 7fe555cad700 1 --2- 172.21.15.90:0/3256224994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55007a310 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe5400098a0 tx=0x7fe540009bb0 comp rx=0 tx=0).stop 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.504+0000 7fe555cad700 1 -- 172.21.15.90:0/3256224994 shutdown_connections 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.504+0000 7fe555cad700 1 --2- 172.21.15.90:0/3256224994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55007a310 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.504+0000 7fe555cad700 1 -- 172.21.15.90:0/3256224994 >> 172.21.15.90:0/3256224994 conn(0x7fe550101180 msgr2=0x7fe550103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.504+0000 7fe555cad700 1 -- 172.21.15.90:0/3256224994 shutdown_connections 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.504+0000 7fe555cad700 1 -- 172.21.15.90:0/3256224994 wait complete. 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe555cad700 1 Processor -- start 2024-04-26T04:34:01.975 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe555cad700 1 -- start start 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe555cad700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55019bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe555cad700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55019c010 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe54f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55019bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe54f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55019bad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48810/0 (socket says 172.21.15.90:48810) 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe54f7fe700 1 -- 172.21.15.90:0/3095925994 learned_addr learned my addr 172.21.15.90:0/3095925994 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.505+0000 7fe54f7fe700 1 -- 172.21.15.90:0/3095925994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe540009550 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.506+0000 7fe54f7fe700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55019bad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe540000c00 tx=0x7fe540011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.508+0000 7fe54cff9700 1 -- 172.21.15.90:0/3095925994 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe540003a70 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.508+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe55019c210 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.508+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55019c630 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.508+0000 7fe54cff9700 1 -- 172.21.15.90:0/3095925994 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fe540003bd0 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.509+0000 7fe54cff9700 1 -- 172.21.15.90:0/3095925994 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fe5400238c0 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.509+0000 7fe54cff9700 1 -- 172.21.15.90:0/3095925994 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fe540023b30 con 0x7fe55007be10 2024-04-26T04:34:01.976 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.509+0000 7fe54cff9700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fe538038490 0x7fe53803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.510+0000 7fe54cff9700 1 -- 172.21.15.90:0/3095925994 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe54004d410 con 0x7fe55007be10 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.510+0000 7fe54effd700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fe538038490 0x7fe53803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.510+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe550062380 con 0x7fe55007be10 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.510+0000 7fe54effd700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fe538038490 0x7fe53803a940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe544006fd0 tx=0x7fe544006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.515+0000 7fe54cff9700 1 -- 172.21.15.90:0/3095925994 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fe54002b300 con 0x7fe55007be10 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.729+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fe55019f3e0 con 0x7fe538038490 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.953+0000 7fe54cff9700 1 -- 172.21.15.90:0/3095925994 <== mgr.14164 v2:172.21.15.90:6800/2647874706 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fe55019f3e0 con 0x7fe538038490 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fe538038490 msgr2=0x7fe53803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fe538038490 0x7fe53803a940 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe544006fd0 tx=0x7fe544006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 msgr2=0x7fe55019bad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:01.977 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55019bad0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe540000c00 tx=0x7fe540011b50 comp rx=0 tx=0).stop 2024-04-26T04:34:01.978 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 shutdown_connections 2024-04-26T04:34:01.978 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fe538038490 0x7fe53803a940 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:01.978 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 --2- 172.21.15.90:0/3095925994 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe55007be10 0x7fe55019bad0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:01.978 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.955+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 >> 172.21.15.90:0/3095925994 conn(0x7fe550101180 msgr2=0x7fe5501027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:01.978 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.956+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 shutdown_connections 2024-04-26T04:34:01.978 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:00.956+0000 7fe555cad700 1 -- 172.21.15.90:0/3095925994 wait complete. 2024-04-26T04:34:01.978 INFO:teuthology.orchestra.run.smithi090.stdout:Fetching dashboard port number... 2024-04-26T04:34:02.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:01 smithi090 ceph-mon[26648]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:34:02.206 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:01 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:03.207 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:02 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3472975773' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stdout 8443 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.426+0000 7fcce7194700 1 Processor -- start 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.426+0000 7fcce7194700 1 -- start start 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.426+0000 7fcce7194700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0108a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.426+0000 7fcce7194700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce0079d30 con 0x7fcce0106680 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.426+0000 7fcce4f30700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0108a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.427+0000 7fcce4f30700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0108a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48824/0 (socket says 172.21.15.90:48824) 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.427+0000 7fcce4f30700 1 -- 172.21.15.90:0/331497299 learned_addr learned my addr 172.21.15.90:0/331497299 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.427+0000 7fcce4f30700 1 -- 172.21.15.90:0/331497299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce0079e70 con 0x7fcce0106680 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.427+0000 7fcce4f30700 1 --2- 172.21.15.90:0/331497299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0108a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098a0 tx=0x7fccd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0b3cb407516d76 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:03.215 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.427+0000 7fccdf7fe700 1 -- 172.21.15.90:0/331497299 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccd4004180 con 0x7fcce0106680 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.427+0000 7fccdf7fe700 1 -- 172.21.15.90:0/331497299 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fccd40042e0 con 0x7fcce0106680 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.427+0000 7fccdf7fe700 1 -- 172.21.15.90:0/331497299 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccd4011af0 con 0x7fcce0106680 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 -- 172.21.15.90:0/331497299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 msgr2=0x7fcce0108a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 --2- 172.21.15.90:0/331497299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0108a60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fccd40098a0 tx=0x7fccd4009bb0 comp rx=0 tx=0).stop 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 -- 172.21.15.90:0/331497299 shutdown_connections 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 --2- 172.21.15.90:0/331497299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0108a60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 -- 172.21.15.90:0/331497299 >> 172.21.15.90:0/331497299 conn(0x7fcce0100060 msgr2=0x7fcce01024b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 -- 172.21.15.90:0/331497299 shutdown_connections 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 -- 172.21.15.90:0/331497299 wait complete. 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 Processor -- start 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.428+0000 7fcce7194700 1 -- start start 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce7194700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0105fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce7194700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce0106520 con 0x7fcce0106680 2024-04-26T04:34:03.216 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce4f30700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0105fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce4f30700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0105fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48830/0 (socket says 172.21.15.90:48830) 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce4f30700 1 -- 172.21.15.90:0/3472975773 learned_addr learned my addr 172.21.15.90:0/3472975773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce4f30700 1 -- 172.21.15.90:0/3472975773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd4009550 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce4f30700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0105fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011670 tx=0x7fccd4011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fccddffb700 1 -- 172.21.15.90:0/3472975773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccd40039f0 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce0104690 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fccddffb700 1 -- 172.21.15.90:0/3472975773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fccd4003b50 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.429+0000 7fccddffb700 1 -- 172.21.15.90:0/3472975773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fccd40247c0 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.430+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce0104b10 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.430+0000 7fccddffb700 1 -- 172.21.15.90:0/3472975773 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7fccd402d3f0 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.431+0000 7fccddffb700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fccd0038490 0x7fccd003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.431+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce0109ba0 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.431+0000 7fccddffb700 1 -- 172.21.15.90:0/3472975773 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fccd404c4a0 con 0x7fcce0106680 2024-04-26T04:34:03.217 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.431+0000 7fccdffff700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fccd0038490 0x7fccd003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.431+0000 7fccdffff700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fccd0038490 0x7fccd003a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcccc006fd0 tx=0x7fcccc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.435+0000 7fccddffb700 1 -- 172.21.15.90:0/3472975773 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fccd4010970 con 0x7fcce0106680 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.632+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fcce0105170 con 0x7fcce0106680 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.632+0000 7fccddffb700 1 -- 172.21.15.90:0/3472975773 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fccd4020220 con 0x7fcce0106680 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.635+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fccd0038490 msgr2=0x7fccd003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.635+0000 7fcce7194700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fccd0038490 0x7fccd003a940 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcccc006fd0 tx=0x7fcccc006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.635+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 msgr2=0x7fcce0105fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.635+0000 7fcce7194700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0105fe0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fccd4011670 tx=0x7fccd4011b50 comp rx=0 tx=0).stop 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.635+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 shutdown_connections 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.636+0000 7fcce7194700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fccd0038490 0x7fccd003a940 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.636+0000 7fcce7194700 1 --2- 172.21.15.90:0/3472975773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcce0106680 0x7fcce0105fe0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.636+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 >> 172.21.15.90:0/3472975773 conn(0x7fcce0100060 msgr2=0x7fcce01017a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:03.218 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.636+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 shutdown_connections 2024-04-26T04:34:03.219 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:02.636+0000 7fcce7194700 1 -- 172.21.15.90:0/3472975773 wait complete. 2024-04-26T04:34:03.221 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-26T04:34:03.221 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout disabled 2024-04-26T04:34:03.228 INFO:teuthology.orchestra.run.smithi090.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-26T04:34:03.228 INFO:teuthology.orchestra.run.smithi090.stdout:systemctl: stdout inactive 2024-04-26T04:34:03.228 INFO:teuthology.orchestra.run.smithi090.stdout:firewalld.service is not enabled 2024-04-26T04:34:03.228 INFO:teuthology.orchestra.run.smithi090.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-26T04:34:03.230 INFO:teuthology.orchestra.run.smithi090.stdout:Ceph Dashboard is now available at: 2024-04-26T04:34:03.230 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:03.230 INFO:teuthology.orchestra.run.smithi090.stdout: URL: https://smithi090:8443/ 2024-04-26T04:34:03.230 INFO:teuthology.orchestra.run.smithi090.stdout: User: admin 2024-04-26T04:34:03.230 INFO:teuthology.orchestra.run.smithi090.stdout: Password: o5ihp251zd 2024-04-26T04:34:03.230 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:03.231 INFO:teuthology.orchestra.run.smithi090.stdout:Saving cluster configuration to /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config directory 2024-04-26T04:34:03.231 INFO:teuthology.orchestra.run.smithi090.stdout:Enabling autotune for osd_memory_target 2024-04-26T04:34:04.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f99275700 1 Processor -- start 2024-04-26T04:34:04.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f99275700 1 -- start start 2024-04-26T04:34:04.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f99275700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9407a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:04.491 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f99275700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9407a830 con 0x7f7f9407bdf0 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9407a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9407a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48832/0 (socket says 172.21.15.90:48832) 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f92ffd700 1 -- 172.21.15.90:0/3987777491 learned_addr learned my addr 172.21.15.90:0/3987777491 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.669+0000 7f7f92ffd700 1 -- 172.21.15.90:0/3987777491 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f9407a970 con 0x7f7f9407bdf0 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f92ffd700 1 --2- 172.21.15.90:0/3987777491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9407a2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f840098a0 tx=0x7f7f84009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59cf10f4ebc6b0e server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f91ffb700 1 -- 172.21.15.90:0/3987777491 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f84004180 con 0x7f7f9407bdf0 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f91ffb700 1 -- 172.21.15.90:0/3987777491 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f840042e0 con 0x7f7f9407bdf0 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f91ffb700 1 -- 172.21.15.90:0/3987777491 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f84011af0 con 0x7f7f9407bdf0 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f99275700 1 -- 172.21.15.90:0/3987777491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 msgr2=0x7f7f9407a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f99275700 1 --2- 172.21.15.90:0/3987777491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9407a2f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f7f840098a0 tx=0x7f7f84009bb0 comp rx=0 tx=0).stop 2024-04-26T04:34:04.492 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f99275700 1 -- 172.21.15.90:0/3987777491 shutdown_connections 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f99275700 1 --2- 172.21.15.90:0/3987777491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9407a2f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f99275700 1 -- 172.21.15.90:0/3987777491 >> 172.21.15.90:0/3987777491 conn(0x7f7f94101180 msgr2=0x7f7f94103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f99275700 1 -- 172.21.15.90:0/3987777491 shutdown_connections 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.670+0000 7f7f99275700 1 -- 172.21.15.90:0/3987777491 wait complete. 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.671+0000 7f7f99275700 1 Processor -- start 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.671+0000 7f7f99275700 1 -- start start 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.671+0000 7f7f99275700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9419bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.671+0000 7f7f99275700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f9419c050 con 0x7f7f9407bdf0 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.671+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9419bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.671+0000 7f7f92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9419bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48840/0 (socket says 172.21.15.90:48840) 2024-04-26T04:34:04.493 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.671+0000 7f7f92ffd700 1 -- 172.21.15.90:0/3473549814 learned_addr learned my addr 172.21.15.90:0/3473549814 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:04.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.672+0000 7f7f92ffd700 1 -- 172.21.15.90:0/3473549814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f84009550 con 0x7f7f9407bdf0 2024-04-26T04:34:04.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.672+0000 7f7f92ffd700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9419bb10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f84006ba0 tx=0x7f7f84011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:04.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.672+0000 7f7f7bfff700 1 -- 172.21.15.90:0/3473549814 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f84003b00 con 0x7f7f9407bdf0 2024-04-26T04:34:04.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.672+0000 7f7f7bfff700 1 -- 172.21.15.90:0/3473549814 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7f84003c60 con 0x7f7f9407bdf0 2024-04-26T04:34:04.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.672+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f9419c250 con 0x7f7f9407bdf0 2024-04-26T04:34:04.494 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.672+0000 7f7f7bfff700 1 -- 172.21.15.90:0/3473549814 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7f84019820 con 0x7f7f9407bdf0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.672+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f9419c670 con 0x7f7f9407bdf0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.673+0000 7f7f7bfff700 1 -- 172.21.15.90:0/3473549814 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 12) v1 ==== 45097+0+0 (secure 0 0 0) 0x7f7f84019ae0 con 0x7f7f9407bdf0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.673+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f94062380 con 0x7f7f9407bdf0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.673+0000 7f7f7bfff700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7f7c0383f0 0x7f7f7c03a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.673+0000 7f7f7bfff700 1 -- 172.21.15.90:0/3473549814 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7f8404c620 con 0x7f7f9407bdf0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.673+0000 7f7f927fc700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7f7c0383f0 0x7f7f7c03a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.673+0000 7f7f927fc700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7f7c0383f0 0x7f7f7c03a8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f88006fd0 tx=0x7f7f88006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.678+0000 7f7f7bfff700 1 -- 172.21.15.90:0/3473549814 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7f8402b750 con 0x7f7f9407bdf0 2024-04-26T04:34:04.495 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.872+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f7f9419f560 con 0x7f7f9407bdf0 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.872+0000 7f7f7bfff700 1 -- 172.21.15.90:0/3473549814 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f7f84018350 con 0x7f7f9407bdf0 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7f7c0383f0 msgr2=0x7f7f7c03a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7f7c0383f0 0x7f7f7c03a8a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f7f88006fd0 tx=0x7f7f88006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 msgr2=0x7f7f9419bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9419bb10 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7f84006ba0 tx=0x7f7f84011b50 comp rx=0 tx=0).stop 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 shutdown_connections 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7f7c0383f0 0x7f7f7c03a8a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 --2- 172.21.15.90:0/3473549814 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7f9407bdf0 0x7f7f9419bb10 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 >> 172.21.15.90:0/3473549814 conn(0x7f7f94101180 msgr2=0x7f7f941027f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 shutdown_connections 2024-04-26T04:34:04.496 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:03.874+0000 7f7f99275700 1 -- 172.21.15.90:0/3473549814 wait complete. 2024-04-26T04:34:05.177 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:04 smithi090 ceph-mon[26648]: mgrmap e13: smithi090.iwrvwa(active, since 6s) 2024-04-26T04:34:05.728 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.867+0000 7f6a856a9700 1 Processor -- start 2024-04-26T04:34:05.728 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.867+0000 7f6a856a9700 1 -- start start 2024-04-26T04:34:05.728 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.867+0000 7f6a856a9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8007a310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:05.728 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.867+0000 7f6a856a9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8007a850 con 0x7f6a8007be10 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.867+0000 7f6a7effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8007a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.867+0000 7f6a7effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8007a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48844/0 (socket says 172.21.15.90:48844) 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.867+0000 7f6a7effd700 1 -- 172.21.15.90:0/583326226 learned_addr learned my addr 172.21.15.90:0/583326226 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a7effd700 1 -- 172.21.15.90:0/583326226 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a8007a990 con 0x7f6a8007be10 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a7effd700 1 --2- 172.21.15.90:0/583326226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8007a310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098a0 tx=0x7f6a70009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3549b5a81f6ec63 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a7dffb700 1 -- 172.21.15.90:0/583326226 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a70004180 con 0x7f6a8007be10 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a7dffb700 1 -- 172.21.15.90:0/583326226 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a700042e0 con 0x7f6a8007be10 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a7dffb700 1 -- 172.21.15.90:0/583326226 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a70011af0 con 0x7f6a8007be10 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a856a9700 1 -- 172.21.15.90:0/583326226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 msgr2=0x7f6a8007a310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a856a9700 1 --2- 172.21.15.90:0/583326226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8007a310 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6a700098a0 tx=0x7f6a70009bb0 comp rx=0 tx=0).stop 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a856a9700 1 -- 172.21.15.90:0/583326226 shutdown_connections 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a856a9700 1 --2- 172.21.15.90:0/583326226 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8007a310 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:05.729 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.868+0000 7f6a856a9700 1 -- 172.21.15.90:0/583326226 >> 172.21.15.90:0/583326226 conn(0x7f6a80101180 msgr2=0x7f6a80103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.869+0000 7f6a856a9700 1 -- 172.21.15.90:0/583326226 shutdown_connections 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.869+0000 7f6a856a9700 1 -- 172.21.15.90:0/583326226 wait complete. 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.869+0000 7f6a856a9700 1 Processor -- start 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.869+0000 7f6a856a9700 1 -- start start 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.869+0000 7f6a856a9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.869+0000 7f6a856a9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a8019c050 con 0x7f6a8007be10 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a7effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a7effd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8019bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48856/0 (socket says 172.21.15.90:48856) 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a7effd700 1 -- 172.21.15.90:0/2450482573 learned_addr learned my addr 172.21.15.90:0/2450482573 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a7effd700 1 -- 172.21.15.90:0/2450482573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a70009550 con 0x7f6a8007be10 2024-04-26T04:34:05.730 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a7effd700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8019bb10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a70000c00 tx=0x7f6a70011b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a67fff700 1 -- 172.21.15.90:0/2450482573 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a70003a70 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a8019c250 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a67fff700 1 -- 172.21.15.90:0/2450482573 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6a70003bd0 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a67fff700 1 -- 172.21.15.90:0/2450482573 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f6a700238c0 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.870+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a8019c670 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.871+0000 7f6a67fff700 1 -- 172.21.15.90:0/2450482573 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f6a70023b80 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.871+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a80062380 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.871+0000 7f6a67fff700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f6a680384e0 0x7f6a6803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.871+0000 7f6a67fff700 1 -- 172.21.15.90:0/2450482573 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6a7004c600 con 0x7f6a8007be10 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.871+0000 7f6a7e7fc700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f6a680384e0 0x7f6a6803a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.872+0000 7f6a7e7fc700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f6a680384e0 0x7f6a6803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a74006fd0 tx=0x7f6a74006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:05.731 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:04.876+0000 7f6a67fff700 1 -- 172.21.15.90:0/2450482573 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6a70010d10 con 0x7f6a8007be10 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.176+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f6a8019f560 con 0x7f6a8007be10 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.181+0000 7f6a67fff700 1 -- 172.21.15.90:0/2450482573 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v34)=0 set mgr/dashboard/cluster/status v34) v1 ==== 153+0+0 (secure 0 0 0) 0x7f6a70010b30 con 0x7f6a8007be10 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f6a680384e0 msgr2=0x7f6a6803a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f6a680384e0 0x7f6a6803a990 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f6a74006fd0 tx=0x7f6a74006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 msgr2=0x7f6a8019bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8019bb10 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6a70000c00 tx=0x7f6a70011b50 comp rx=0 tx=0).stop 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 shutdown_connections 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f6a680384e0 0x7f6a6803a990 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 --2- 172.21.15.90:0/2450482573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a8007be10 0x7f6a8019bb10 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 >> 172.21.15.90:0/2450482573 conn(0x7f6a80101180 msgr2=0x7f6a80102820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 shutdown_connections 2024-04-26T04:34:05.732 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr 2024-04-26T04:34:05.183+0000 7f6a856a9700 1 -- 172.21.15.90:0/2450482573 wait complete. 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout:Or, if you are only running a single cluster on this host: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: ceph telemetry on 2024-04-26T04:34:05.733 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.734 INFO:teuthology.orchestra.run.smithi090.stdout:For more information see: 2024-04-26T04:34:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-26T04:34:05.734 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:05.734 INFO:teuthology.orchestra.run.smithi090.stdout:Bootstrap complete. 2024-04-26T04:34:05.755 INFO:tasks.cephadm:Fetching config... 2024-04-26T04:34:05.755 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:34:05.755 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-26T04:34:05.771 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-26T04:34:05.771 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:34:05.771 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-26T04:34:05.827 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-26T04:34:05.828 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:34:05.828 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/keyring of=/dev/stdout 2024-04-26T04:34:05.892 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-26T04:34:05.893 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:34:05.893 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-26T04:34:05.948 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-26T04:34:05.948 DEBUG:teuthology.orchestra.run.smithi090:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYjdXsuliVep/WqNcKeShgMRDdYE+ztBQGzpUfzxZVYMrG/xa9kviHxa2FCfaPH/iEP3tMNj/Yi9j65oZti55qoU0JHWEC1xNTpplD5F+pxOQWpDU3E6xNZBdJOVT71Qqt1qEPgAPRzHCb0DVmni/kCqF0uJOdv8F+geeIXJ0IQ5DdN+hEvWtsALTERNvv/J7n//QOcmAxZ++hfPPSLMTpMuI1c31pkSwv0XLgEGJmMf/kxKqHWWdm8WdjjwsK0Gdwbfzpe8Mlz3BxOuCQoYxMkr16+WARMZaFfFHvXGXFlMkEp1Tbv/0gtbHTpUB9Is1wWlt8zta+Z4a1KYbBWrfO2ZdkiU8SCNXD3RB6jYCcBtAiZZYcJiZcWEVRrxvT33eYy+LRXYJxah0ndosDDDtuJuu/LcNjGhO+/jyyNgnNtqB8LImzlbWufA2fzgnBT5jfBZ4AwcOhDkZmAuAnEwfA/ovk/7Nxuj4Fjc5QdAOOreCAGMp9x4rGMJiDG1xDPZs= ceph-f018e902-0385-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T04:34:06.020 INFO:teuthology.orchestra.run.smithi090.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYjdXsuliVep/WqNcKeShgMRDdYE+ztBQGzpUfzxZVYMrG/xa9kviHxa2FCfaPH/iEP3tMNj/Yi9j65oZti55qoU0JHWEC1xNTpplD5F+pxOQWpDU3E6xNZBdJOVT71Qqt1qEPgAPRzHCb0DVmni/kCqF0uJOdv8F+geeIXJ0IQ5DdN+hEvWtsALTERNvv/J7n//QOcmAxZ++hfPPSLMTpMuI1c31pkSwv0XLgEGJmMf/kxKqHWWdm8WdjjwsK0Gdwbfzpe8Mlz3BxOuCQoYxMkr16+WARMZaFfFHvXGXFlMkEp1Tbv/0gtbHTpUB9Is1wWlt8zta+Z4a1KYbBWrfO2ZdkiU8SCNXD3RB6jYCcBtAiZZYcJiZcWEVRrxvT33eYy+LRXYJxah0ndosDDDtuJuu/LcNjGhO+/jyyNgnNtqB8LImzlbWufA2fzgnBT5jfBZ4AwcOhDkZmAuAnEwfA/ovk/7Nxuj4Fjc5QdAOOreCAGMp9x4rGMJiDG1xDPZs= ceph-f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:34:06.031 DEBUG:teuthology.orchestra.run.smithi104:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYjdXsuliVep/WqNcKeShgMRDdYE+ztBQGzpUfzxZVYMrG/xa9kviHxa2FCfaPH/iEP3tMNj/Yi9j65oZti55qoU0JHWEC1xNTpplD5F+pxOQWpDU3E6xNZBdJOVT71Qqt1qEPgAPRzHCb0DVmni/kCqF0uJOdv8F+geeIXJ0IQ5DdN+hEvWtsALTERNvv/J7n//QOcmAxZ++hfPPSLMTpMuI1c31pkSwv0XLgEGJmMf/kxKqHWWdm8WdjjwsK0Gdwbfzpe8Mlz3BxOuCQoYxMkr16+WARMZaFfFHvXGXFlMkEp1Tbv/0gtbHTpUB9Is1wWlt8zta+Z4a1KYbBWrfO2ZdkiU8SCNXD3RB6jYCcBtAiZZYcJiZcWEVRrxvT33eYy+LRXYJxah0ndosDDDtuJuu/LcNjGhO+/jyyNgnNtqB8LImzlbWufA2fzgnBT5jfBZ4AwcOhDkZmAuAnEwfA/ovk/7Nxuj4Fjc5QdAOOreCAGMp9x4rGMJiDG1xDPZs= ceph-f018e902-0385-11ef-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-26T04:34:06.070 INFO:teuthology.orchestra.run.smithi104.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDYjdXsuliVep/WqNcKeShgMRDdYE+ztBQGzpUfzxZVYMrG/xa9kviHxa2FCfaPH/iEP3tMNj/Yi9j65oZti55qoU0JHWEC1xNTpplD5F+pxOQWpDU3E6xNZBdJOVT71Qqt1qEPgAPRzHCb0DVmni/kCqF0uJOdv8F+geeIXJ0IQ5DdN+hEvWtsALTERNvv/J7n//QOcmAxZ++hfPPSLMTpMuI1c31pkSwv0XLgEGJmMf/kxKqHWWdm8WdjjwsK0Gdwbfzpe8Mlz3BxOuCQoYxMkr16+WARMZaFfFHvXGXFlMkEp1Tbv/0gtbHTpUB9Is1wWlt8zta+Z4a1KYbBWrfO2ZdkiU8SCNXD3RB6jYCcBtAiZZYcJiZcWEVRrxvT33eYy+LRXYJxah0ndosDDDtuJuu/LcNjGhO+/jyyNgnNtqB8LImzlbWufA2fzgnBT5jfBZ4AwcOhDkZmAuAnEwfA/ovk/7Nxuj4Fjc5QdAOOreCAGMp9x4rGMJiDG1xDPZs= ceph-f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:34:06.082 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-26T04:34:06.281 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:34:06.414 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:06 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2450482573' entity='client.admin' 2024-04-26T04:34:08.246 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:08.246 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:08.246 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:34:08.246 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:08.247 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:34:08.247 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-26T04:34:08.247 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:34:08.247 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:08 smithi090 ceph-mon[26648]: Deploying daemon ceph-exporter.smithi090 on smithi090 2024-04-26T04:34:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.442+0000 7ffb34375700 1 -- 172.21.15.90:0/2121365428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 msgr2=0x7ffb2c100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.442+0000 7ffb34375700 1 --2- 172.21.15.90:0/2121365428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c100230 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c009910 tx=0x7ffb1c009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:08.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 -- 172.21.15.90:0/2121365428 shutdown_connections 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 --2- 172.21.15.90:0/2121365428 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c100230 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 -- 172.21.15.90:0/2121365428 >> 172.21.15.90:0/2121365428 conn(0x7ffb2c0fb3d0 msgr2=0x7ffb2c0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 -- 172.21.15.90:0/2121365428 shutdown_connections 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 -- 172.21.15.90:0/2121365428 wait complete. 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 Processor -- start 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 -- start start 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb34375700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c075030 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb32111700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb32111700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52776/0 (socket says 172.21.15.90:52776) 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.443+0000 7ffb32111700 1 -- 172.21.15.90:0/967502801 learned_addr learned my addr 172.21.15.90:0/967502801 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.444+0000 7ffb32111700 1 -- 172.21.15.90:0/967502801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb1c0095f0 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.444+0000 7ffb32111700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c074af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c005b40 tx=0x7ffb1c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.444+0000 7ffb237fe700 1 -- 172.21.15.90:0/967502801 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb1c01d070 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.444+0000 7ffb237fe700 1 -- 172.21.15.90:0/967502801 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb1c00be10 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.444+0000 7ffb237fe700 1 -- 172.21.15.90:0/967502801 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffb1c00f460 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.444+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c073140 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.444+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c0735e0 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.445+0000 7ffb237fe700 1 -- 172.21.15.90:0/967502801 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ffb1c0227e0 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.445+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c062380 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.445+0000 7ffb237fe700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffb18038490 0x7ffb1803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.445+0000 7ffb237fe700 1 -- 172.21.15.90:0/967502801 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffb1c04c270 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.445+0000 7ffb31910700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffb18038490 0x7ffb1803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:08.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.446+0000 7ffb31910700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffb18038490 0x7ffb1803a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffb28006fd0 tx=0x7ffb28006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:08.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.450+0000 7ffb237fe700 1 -- 172.21.15.90:0/967502801 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffb1c02b590 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.642+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7ffb2c18e8b0 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.646+0000 7ffb237fe700 1 -- 172.21.15.90:0/967502801 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7ffb1c027070 con 0x7ffb2c0ffe20 2024-04-26T04:34:08.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffb18038490 msgr2=0x7ffb1803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:08.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffb18038490 0x7ffb1803a940 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ffb28006fd0 tx=0x7ffb28006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:08.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 msgr2=0x7ffb2c074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:08.649 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c074af0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c005b40 tx=0x7ffb1c004740 comp rx=0 tx=0).stop 2024-04-26T04:34:08.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 shutdown_connections 2024-04-26T04:34:08.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffb18038490 0x7ffb1803a940 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:08.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 --2- 172.21.15.90:0/967502801 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb2c0ffe20 0x7ffb2c074af0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:08.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 >> 172.21.15.90:0/967502801 conn(0x7ffb2c0fb3d0 msgr2=0x7ffb2c0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:08.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 shutdown_connections 2024-04-26T04:34:08.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:08.648+0000 7ffb34375700 1 -- 172.21.15.90:0/967502801 wait complete. 2024-04-26T04:34:09.229 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-26T04:34:09.230 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-26T04:34:09.509 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:34:09.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:09 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/967502801' entity='client.admin' 2024-04-26T04:34:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.125+0000 7fd75d4b8700 1 -- 172.21.15.90:0/1814276851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 msgr2=0x7fd758102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.125+0000 7fd75d4b8700 1 --2- 172.21.15.90:0/1814276851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758102c90 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd748009910 tx=0x7fd748009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 -- 172.21.15.90:0/1814276851 shutdown_connections 2024-04-26T04:34:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 --2- 172.21.15.90:0/1814276851 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758102c90 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:12.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 -- 172.21.15.90:0/1814276851 >> 172.21.15.90:0/1814276851 conn(0x7fd7580fa2b0 msgr2=0x7fd7580fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 -- 172.21.15.90:0/1814276851 shutdown_connections 2024-04-26T04:34:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 -- 172.21.15.90:0/1814276851 wait complete. 2024-04-26T04:34:12.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 Processor -- start 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 -- start start 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758194f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.126+0000 7fd75d4b8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd758195490 con 0x7fd7581008b0 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd756ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758194f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd756ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758194f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52790/0 (socket says 172.21.15.90:52790) 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd756ffd700 1 -- 172.21.15.90:0/2232598414 learned_addr learned my addr 172.21.15.90:0/2232598414 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd756ffd700 1 -- 172.21.15.90:0/2232598414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480095f0 con 0x7fd7581008b0 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd756ffd700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758194f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd748005b40 tx=0x7fd748004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd754ff9700 1 -- 172.21.15.90:0/2232598414 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd74801d070 con 0x7fd7581008b0 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd758195690 con 0x7fd7581008b0 2024-04-26T04:34:12.130 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd754ff9700 1 -- 172.21.15.90:0/2232598414 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd74800be10 con 0x7fd7581008b0 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd758195b30 con 0x7fd7581008b0 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.127+0000 7fd754ff9700 1 -- 172.21.15.90:0/2232598414 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd74800f460 con 0x7fd7581008b0 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.128+0000 7fd754ff9700 1 -- 172.21.15.90:0/2232598414 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fd74800f6d0 con 0x7fd7581008b0 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.128+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75818eb00 con 0x7fd7581008b0 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.128+0000 7fd754ff9700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd744038490 0x7fd74403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.128+0000 7fd754ff9700 1 -- 172.21.15.90:0/2232598414 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd74804c600 con 0x7fd7581008b0 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.128+0000 7fd74e7fc700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd744038490 0x7fd74403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:12.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.129+0000 7fd74e7fc700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd744038490 0x7fd74403a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:12.134 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.133+0000 7fd754ff9700 1 -- 172.21.15.90:0/2232598414 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd74802a360 con 0x7fd7581008b0 2024-04-26T04:34:12.331 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.330+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fd758061220 con 0x7fd744038490 2024-04-26T04:34:12.334 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.333+0000 7fd754ff9700 1 -- 172.21.15.90:0/2232598414 <== mgr.14164 v2:172.21.15.90:6800/2647874706 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fd758061220 con 0x7fd744038490 2024-04-26T04:34:12.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd744038490 msgr2=0x7fd74403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:12.337 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd744038490 0x7fd74403a940 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd740006fd0 tx=0x7fd740006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 msgr2=0x7fd758194f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758194f50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fd748005b40 tx=0x7fd748004740 comp rx=0 tx=0).stop 2024-04-26T04:34:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 shutdown_connections 2024-04-26T04:34:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd744038490 0x7fd74403a940 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 --2- 172.21.15.90:0/2232598414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7581008b0 0x7fd758194f50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:12.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.335+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 >> 172.21.15.90:0/2232598414 conn(0x7fd7580fa2b0 msgr2=0x7fd7580faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.336+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 shutdown_connections 2024-04-26T04:34:12.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:12.336+0000 7fd75d4b8700 1 -- 172.21.15.90:0/2232598414 wait complete. 2024-04-26T04:34:13.223 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi104 2024-04-26T04:34:13.223 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:34:13.223 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.conf 2024-04-26T04:34:13.242 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:34:13.243 DEBUG:teuthology.orchestra.run.smithi104:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:34:13.300 INFO:tasks.cephadm:Adding host smithi104 to orchestrator... 2024-04-26T04:34:13.301 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch host add smithi104 2024-04-26T04:34:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:13 smithi090 ceph-mon[26648]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:34:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:13 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:13.816 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:34:15.024 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:14 smithi090 ceph-mon[26648]: Deploying daemon crash.smithi090 on smithi090 2024-04-26T04:34:15.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.234+0000 7f49fe490700 1 -- 172.21.15.90:0/3700128647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f80ff560 msgr2=0x7f49f8101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:15.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.234+0000 7f49fe490700 1 --2- 172.21.15.90:0/3700128647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f80ff560 0x7f49f8101940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009910 tx=0x7f49e8009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:15.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.234+0000 7f49fe490700 1 -- 172.21.15.90:0/3700128647 shutdown_connections 2024-04-26T04:34:15.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.234+0000 7f49fe490700 1 --2- 172.21.15.90:0/3700128647 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f80ff560 0x7f49f8101940 secure :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f49e8009910 tx=0x7f49e8009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:15.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.234+0000 7f49fe490700 1 -- 172.21.15.90:0/3700128647 >> 172.21.15.90:0/3700128647 conn(0x7f49f80f9600 msgr2=0x7f49f80fba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:15.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.234+0000 7f49fe490700 1 -- 172.21.15.90:0/3700128647 shutdown_connections 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.234+0000 7f49fe490700 1 -- 172.21.15.90:0/3700128647 wait complete. 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.235+0000 7f49fe490700 1 Processor -- start 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.235+0000 7f49fe490700 1 -- start start 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.235+0000 7f49fe490700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f8195200 0x7f49f8195610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.235+0000 7f49fe490700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49f8195b50 con 0x7f49f8195200 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.235+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f8195200 0x7f49f8195610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.235+0000 7f49f7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f8195200 0x7f49f8195610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52832/0 (socket says 172.21.15.90:52832) 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.235+0000 7f49f7fff700 1 -- 172.21.15.90:0/1934953327 learned_addr learned my addr 172.21.15.90:0/1934953327 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.236+0000 7f49f7fff700 1 -- 172.21.15.90:0/1934953327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49e80095f0 con 0x7f49f8195200 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.236+0000 7f49f7fff700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f8195200 0x7f49f8195610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005950 tx=0x7f49e8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:15.238 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.236+0000 7f49f57fa700 1 -- 172.21.15.90:0/1934953327 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49e801d070 con 0x7f49f8195200 2024-04-26T04:34:15.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.236+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49f8195d50 con 0x7f49f8195200 2024-04-26T04:34:15.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.236+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49f81989b0 con 0x7f49f8195200 2024-04-26T04:34:15.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.236+0000 7f49f57fa700 1 -- 172.21.15.90:0/1934953327 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f49e800be10 con 0x7f49f8195200 2024-04-26T04:34:15.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.236+0000 7f49f57fa700 1 -- 172.21.15.90:0/1934953327 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f49e8022df0 con 0x7f49f8195200 2024-04-26T04:34:15.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.237+0000 7f49f57fa700 1 -- 172.21.15.90:0/1934953327 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f49e800f460 con 0x7f49f8195200 2024-04-26T04:34:15.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.237+0000 7f49f57fa700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f49e00384e0 0x7f49e003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:15.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.237+0000 7f49f57fa700 1 -- 172.21.15.90:0/1934953327 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f49e804d1b0 con 0x7f49f8195200 2024-04-26T04:34:15.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.237+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49f8104b80 con 0x7f49f8195200 2024-04-26T04:34:15.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.238+0000 7f49f77fe700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f49e00384e0 0x7f49e003a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:15.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.238+0000 7f49f77fe700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f49e00384e0 0x7f49e003a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:15.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.242+0000 7f49f57fa700 1 -- 172.21.15.90:0/1934953327 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f49e8027070 con 0x7f49f8195200 2024-04-26T04:34:15.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:15.446+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}) v1 -- 0x7f49f8199020 con 0x7f49e00384e0 2024-04-26T04:34:15.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:15 smithi090 ceph-mon[26648]: from='client.14191 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:34:16.934 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:16 smithi090 ceph-mon[26648]: Deploying cephadm binary to smithi104 2024-04-26T04:34:18.070 INFO:teuthology.orchestra.run.smithi090.stdout:Added host 'smithi104' with addr '172.21.15.104' 2024-04-26T04:34:18.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.065+0000 7f49f57fa700 1 -- 172.21.15.90:0/1934953327 <== mgr.14164 v2:172.21.15.90:6800/2647874706 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f49f8199020 con 0x7f49e00384e0 2024-04-26T04:34:18.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f49e00384e0 msgr2=0x7f49e003a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:18.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f49e00384e0 0x7f49e003a990 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f49ec006fd0 tx=0x7f49ec006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:18.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f8195200 msgr2=0x7f49f8195610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:18.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f8195200 0x7f49f8195610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f49e8005950 tx=0x7f49e8009e60 comp rx=0 tx=0).stop 2024-04-26T04:34:18.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 shutdown_connections 2024-04-26T04:34:18.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f49e00384e0 0x7f49e003a990 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:18.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 --2- 172.21.15.90:0/1934953327 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49f8195200 0x7f49f8195610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:18.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.070+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 >> 172.21.15.90:0/1934953327 conn(0x7f49f80f9600 msgr2=0x7f49f80fa350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:18.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.071+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 shutdown_connections 2024-04-26T04:34:18.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:18.073+0000 7f49fe490700 1 -- 172.21.15.90:0/1934953327 wait complete. 2024-04-26T04:34:18.401 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: Deploying daemon node-exporter.smithi090 on smithi090 2024-04-26T04:34:18.402 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:18 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:18.697 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-26T04:34:18.923 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:34:19.374 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:19 smithi090 ceph-mon[26648]: Added host smithi104 2024-04-26T04:34:20.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:20 smithi090 ceph-mon[26648]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:21.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 -- 172.21.15.90:0/2668878785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 msgr2=0x7f3b180ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 --2- 172.21.15.90:0/2668878785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b180ff130 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c009910 tx=0x7f3b0c009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 -- 172.21.15.90:0/2668878785 shutdown_connections 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 --2- 172.21.15.90:0/2668878785 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b180ff130 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 -- 172.21.15.90:0/2668878785 >> 172.21.15.90:0/2668878785 conn(0x7f3b180fa2b0 msgr2=0x7f3b180fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 -- 172.21.15.90:0/2668878785 shutdown_connections 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 -- 172.21.15.90:0/2668878785 wait complete. 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.461+0000 7f3b1d181700 1 Processor -- start 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b1d181700 1 -- start start 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b1d181700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b18194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b1d181700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b181954f0 con 0x7f3b180fed20 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b16d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b18194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b16d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b18194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52604/0 (socket says 172.21.15.90:52604) 2024-04-26T04:34:21.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b16d9d700 1 -- 172.21.15.90:0/4187972281 learned_addr learned my addr 172.21.15.90:0/4187972281 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b16d9d700 1 -- 172.21.15.90:0/4187972281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b0c0095f0 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b16d9d700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b18194fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c005b40 tx=0x7f3b0c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3afffff700 1 -- 172.21.15.90:0/4187972281 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b0c01d070 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b181956f0 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3afffff700 1 -- 172.21.15.90:0/4187972281 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3b0c00be10 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3afffff700 1 -- 172.21.15.90:0/4187972281 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f3b0c00f460 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.462+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b18195b90 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.463+0000 7f3afffff700 1 -- 172.21.15.90:0/4187972281 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f3b0c00f6d0 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.463+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b1818ea10 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.463+0000 7f3afffff700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3b00038490 0x7f3b0003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.463+0000 7f3afffff700 1 -- 172.21.15.90:0/4187972281 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f3b0c04c3e0 con 0x7f3b180fed20 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.464+0000 7f3b1659c700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3b00038490 0x7f3b0003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.464+0000 7f3b1659c700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3b00038490 0x7f3b0003a940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006fd0 tx=0x7f3b08006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:21.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.468+0000 7f3afffff700 1 -- 172.21.15.90:0/4187972281 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3b0c02a360 con 0x7f3b180fed20 2024-04-26T04:34:21.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.672+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b18061190 con 0x7f3b00038490 2024-04-26T04:34:21.674 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.673+0000 7f3afffff700 1 -- 172.21.15.90:0/4187972281 <== mgr.14164 v2:172.21.15.90:6800/2647874706 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+160 (secure 0 0 0) 0x7f3b18061190 con 0x7f3b00038490 2024-04-26T04:34:21.675 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:34:21.675 INFO:teuthology.orchestra.run.smithi090.stdout:[{"addr": "172.21.15.90", "hostname": "smithi090", "labels": [], "status": ""}, {"addr": "172.21.15.104", "hostname": "smithi104", "labels": [], "status": ""}] 2024-04-26T04:34:21.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3b00038490 msgr2=0x7f3b0003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:21.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3b00038490 0x7f3b0003a940 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f3b08006fd0 tx=0x7f3b08006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:21.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 msgr2=0x7f3b18194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:21.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b18194fb0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3b0c005b40 tx=0x7f3b0c004740 comp rx=0 tx=0).stop 2024-04-26T04:34:21.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 shutdown_connections 2024-04-26T04:34:21.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3b00038490 0x7f3b0003a940 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:21.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 --2- 172.21.15.90:0/4187972281 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3b180fed20 0x7f3b18194fb0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:21.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 >> 172.21.15.90:0/4187972281 conn(0x7f3b180fa2b0 msgr2=0x7f3b180690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:21.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 shutdown_connections 2024-04-26T04:34:21.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:21.676+0000 7f3b1d181700 1 -- 172.21.15.90:0/4187972281 wait complete. 2024-04-26T04:34:22.439 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:22 smithi090 ceph-mon[26648]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:22.746 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-26T04:34:22.747 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-26T04:34:22.953 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:34:23.429 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:23 smithi090 ceph-mon[26648]: from='client.14193 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:34:23.429 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:23 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:23.430 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:23 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:23.430 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:23 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:23.430 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:23 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:23.430 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:23 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:23.430 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:23 smithi090 ceph-mon[26648]: Deploying daemon alertmanager.smithi090 on smithi090 2024-04-26T04:34:24.716 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:24 smithi090 ceph-mon[26648]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 -- 172.21.15.90:0/818367128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 msgr2=0x7f35a8100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:25.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 --2- 172.21.15.90:0/818367128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8100230 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f35a4009910 tx=0x7f35a4009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 -- 172.21.15.90:0/818367128 shutdown_connections 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 --2- 172.21.15.90:0/818367128 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8100230 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 -- 172.21.15.90:0/818367128 >> 172.21.15.90:0/818367128 conn(0x7f35a80fb3d0 msgr2=0x7f35a80fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 -- 172.21.15.90:0/818367128 shutdown_connections 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 -- 172.21.15.90:0/818367128 wait complete. 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.245+0000 7f35b0143700 1 Processor -- start 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35b0143700 1 -- start start 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35b0143700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35b0143700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35a8075030 con 0x7f35a80ffe20 2024-04-26T04:34:25.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35adedf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35adedf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52616/0 (socket says 172.21.15.90:52616) 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35adedf700 1 -- 172.21.15.90:0/3968097722 learned_addr learned my addr 172.21.15.90:0/3968097722 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35adedf700 1 -- 172.21.15.90:0/3968097722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35a40095f0 con 0x7f35a80ffe20 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35adedf700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8074af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005b40 tx=0x7f35a4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f359effd700 1 -- 172.21.15.90:0/3968097722 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35a401d070 con 0x7f35a80ffe20 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f359effd700 1 -- 172.21.15.90:0/3968097722 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f35a400be10 con 0x7f35a80ffe20 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f359effd700 1 -- 172.21.15.90:0/3968097722 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f35a400f460 con 0x7f35a80ffe20 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35a8073140 con 0x7f35a80ffe20 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.246+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35a80735e0 con 0x7f35a80ffe20 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.247+0000 7f359effd700 1 -- 172.21.15.90:0/3968097722 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f35a40227e0 con 0x7f35a80ffe20 2024-04-26T04:34:25.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.247+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35a8062380 con 0x7f35a80ffe20 2024-04-26T04:34:25.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.247+0000 7f359effd700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f35940384e0 0x7f359403a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:25.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.247+0000 7f359effd700 1 -- 172.21.15.90:0/3968097722 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f35a404c270 con 0x7f35a80ffe20 2024-04-26T04:34:25.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.248+0000 7f35ad6de700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f35940384e0 0x7f359403a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:25.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.248+0000 7f35ad6de700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f35940384e0 0x7f359403a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3598006fd0 tx=0x7f3598006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:25.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.252+0000 7f359effd700 1 -- 172.21.15.90:0/3968097722 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f35a402b590 con 0x7f35a80ffe20 2024-04-26T04:34:25.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:25.449+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f35a818e460 con 0x7f35a80ffe20 2024-04-26T04:34:26.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.444+0000 7f359effd700 1 -- 172.21.15.90:0/3968097722 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f35a4027070 con 0x7f35a80ffe20 2024-04-26T04:34:26.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f35940384e0 msgr2=0x7f359403a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:26.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f35940384e0 0x7f359403a990 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3598006fd0 tx=0x7f3598006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:26.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 msgr2=0x7f35a8074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:26.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8074af0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f35a4005b40 tx=0x7f35a4004740 comp rx=0 tx=0).stop 2024-04-26T04:34:26.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 shutdown_connections 2024-04-26T04:34:26.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f35940384e0 0x7f359403a990 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:26.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 --2- 172.21.15.90:0/3968097722 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35a80ffe20 0x7f35a8074af0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:26.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 >> 172.21.15.90:0/3968097722 conn(0x7f35a80fb3d0 msgr2=0x7f35a80fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:26.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 shutdown_connections 2024-04-26T04:34:26.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:34:26.446+0000 7f35b0143700 1 -- 172.21.15.90:0/3968097722 wait complete. 2024-04-26T04:34:26.448 INFO:teuthology.orchestra.run.smithi090.stderr:adjusted tunables profile to default 2024-04-26T04:34:26.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:26 smithi090 ceph-mon[26648]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:26.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:26 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3968097722' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-26T04:34:27.487 INFO:tasks.cephadm:Adding mon.smithi090 on smithi090 2024-04-26T04:34:27.488 INFO:tasks.cephadm:Adding mon.smithi104 on smithi104 2024-04-26T04:34:27.488 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch apply mon '2;smithi090:172.21.15.90=smithi090;smithi104:172.21.15.104=smithi104' 2024-04-26T04:34:27.687 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:27.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:27 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3968097722' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-26T04:34:27.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:27 smithi090 ceph-mon[26648]: osdmap e4: 0 total, 0 up, 0 in 2024-04-26T04:34:27.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:27 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:27.733 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:28.640 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:28 smithi090 ceph-mon[26648]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:29.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.436+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1415186386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 msgr2=0x7fd544100230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:29.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.436+0000 7fd54ba8e700 1 --2- 172.21.15.104:0/1415186386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544100230 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fd534009910 tx=0x7fd534009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:29.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.436+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1415186386 shutdown_connections 2024-04-26T04:34:29.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.436+0000 7fd54ba8e700 1 --2- 172.21.15.104:0/1415186386 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544100230 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:29.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.436+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1415186386 >> 172.21.15.104:0/1415186386 conn(0x7fd5440fb3d0 msgr2=0x7fd5440fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:29.438 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.436+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1415186386 shutdown_connections 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.436+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1415186386 wait complete. 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54ba8e700 1 Processor -- start 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54ba8e700 1 -- start start 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54ba8e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54ba8e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd544075030 con 0x7fd5440ffe20 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54982a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54982a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:44570/0 (socket says 172.21.15.104:44570) 2024-04-26T04:34:29.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54982a700 1 -- 172.21.15.104:0/1772693694 learned_addr learned my addr 172.21.15.104:0/1772693694 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:29.440 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54982a700 1 -- 172.21.15.104:0/1772693694 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5340095f0 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.437+0000 7fd54982a700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544074af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd534005b40 tx=0x7fd534004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.438+0000 7fd53affd700 1 -- 172.21.15.104:0/1772693694 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd53401d070 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.438+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd544073140 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.438+0000 7fd53affd700 1 -- 172.21.15.104:0/1772693694 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd53400be10 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.438+0000 7fd53affd700 1 -- 172.21.15.104:0/1772693694 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd53400f460 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.438+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5440735e0 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.438+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd544062380 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.439+0000 7fd53affd700 1 -- 172.21.15.104:0/1772693694 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fd5340227e0 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.439+0000 7fd53affd700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd530038490 0x7fd53003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.439+0000 7fd53affd700 1 -- 172.21.15.104:0/1772693694 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd53404c2a0 con 0x7fd5440ffe20 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.439+0000 7fd549029700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd530038490 0x7fd53003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:29.441 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.439+0000 7fd549029700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd530038490 0x7fd53003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd540006fd0 tx=0x7fd540006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:29.445 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.444+0000 7fd53affd700 1 -- 172.21.15.104:0/1772693694 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd53402b590 con 0x7fd5440ffe20 2024-04-26T04:34:29.644 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.642+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 --> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi090:172.21.15.90=smithi090;smithi104:172.21.15.104=smithi104", "target": ["mon-mgr", ""]}) v1 -- 0x7fd544073820 con 0x7fd530038490 2024-04-26T04:34:29.651 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.650+0000 7fd53affd700 1 -- 172.21.15.104:0/1772693694 <== mgr.14164 v2:172.21.15.90:6800/2647874706 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fd544073820 con 0x7fd530038490 2024-04-26T04:34:29.651 INFO:teuthology.orchestra.run.smithi104.stdout:Scheduled mon update... 2024-04-26T04:34:29.653 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd530038490 msgr2=0x7fd53003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:29.653 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd530038490 0x7fd53003a940 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd540006fd0 tx=0x7fd540006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 msgr2=0x7fd544074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544074af0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fd534005b40 tx=0x7fd534004740 comp rx=0 tx=0).stop 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 shutdown_connections 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd530038490 0x7fd53003a940 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 --2- 172.21.15.104:0/1772693694 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd5440ffe20 0x7fd544074af0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 >> 172.21.15.104:0/1772693694 conn(0x7fd5440fb3d0 msgr2=0x7fd5440fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 shutdown_connections 2024-04-26T04:34:29.654 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:29.652+0000 7fd54ba8e700 1 -- 172.21.15.104:0/1772693694 wait complete. 2024-04-26T04:34:30.195 DEBUG:teuthology.orchestra.run.smithi104:mon.smithi104> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi104.service 2024-04-26T04:34:30.198 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:30.198 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:30.420 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:30.466 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:30.526 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:30 smithi090 ceph-mon[26648]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:30.527 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:30 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:31.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:31 smithi090 ceph-mon[26648]: from='client.14197 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "2;smithi090:172.21.15.90=smithi090;smithi104:172.21.15.104=smithi104", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:34:31.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:31 smithi090 ceph-mon[26648]: Saving service mon spec with placement smithi090:172.21.15.90=smithi090;smithi104:172.21.15.104=smithi104;count:2 2024-04-26T04:34:31.934 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 -- 172.21.15.104:0/155553559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 msgr2=0x7f35980ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 --2- 172.21.15.104:0/155553559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980ff140 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3588009910 tx=0x7f3588009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 -- 172.21.15.104:0/155553559 shutdown_connections 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 --2- 172.21.15.104:0/155553559 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980ff140 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 -- 172.21.15.104:0/155553559 >> 172.21.15.104:0/155553559 conn(0x7f35980fa2c0 msgr2=0x7f35980fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 -- 172.21.15.104:0/155553559 shutdown_connections 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 -- 172.21.15.104:0/155553559 wait complete. 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 Processor -- start 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 -- start start 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980693a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.933+0000 7f359fef4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3598197cc0 con 0x7f35980fed30 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f359dc90700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f359dc90700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980693a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:44592/0 (socket says 172.21.15.104:44592) 2024-04-26T04:34:31.935 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f359dc90700 1 -- 172.21.15.104:0/1048626145 learned_addr learned my addr 172.21.15.104:0/1048626145 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f359dc90700 1 -- 172.21.15.104:0/1048626145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35880095f0 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f359dc90700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980693a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3588005950 tx=0x7f3588009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f358effd700 1 -- 172.21.15.104:0/1048626145 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f358801d070 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3598197e00 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f358effd700 1 -- 172.21.15.104:0/1048626145 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f358800be10 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f358effd700 1 -- 172.21.15.104:0/1048626145 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f358800f460 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.934+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35981982a0 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.935+0000 7f358effd700 1 -- 172.21.15.104:0/1048626145 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f358800b960 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.935+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359818da10 con 0x7f35980fed30 2024-04-26T04:34:31.937 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.935+0000 7f358effd700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3584038490 0x7f358403a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.935+0000 7f358effd700 1 -- 172.21.15.104:0/1048626145 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f358804c430 con 0x7f35980fed30 2024-04-26T04:34:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.936+0000 7f359d48f700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3584038490 0x7f358403a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:31.938 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.936+0000 7f359d48f700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3584038490 0x7f358403a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3594006fd0 tx=0x7f3594006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:31.943 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:31.942+0000 7f358effd700 1 -- 172.21.15.104:0/1048626145 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3588013250 con 0x7f35980fed30 2024-04-26T04:34:32.208 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.206+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f359802cc30 con 0x7f35980fed30 2024-04-26T04:34:32.208 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.206+0000 7f358effd700 1 -- 172.21.15.104:0/1048626145 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3588027030 con 0x7f35980fed30 2024-04-26T04:34:32.208 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:32.208 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3584038490 msgr2=0x7f358403a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3584038490 0x7f358403a940 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3594006fd0 tx=0x7f3594006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 msgr2=0x7f35980693a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980693a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3588005950 tx=0x7f3588009e60 comp rx=0 tx=0).stop 2024-04-26T04:34:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 shutdown_connections 2024-04-26T04:34:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f3584038490 0x7f358403a940 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:32.210 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 --2- 172.21.15.104:0/1048626145 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f35980fed30 0x7f35980693a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:32.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 >> 172.21.15.104:0/1048626145 conn(0x7f35980fa2c0 msgr2=0x7f35980faf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:32.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 shutdown_connections 2024-04-26T04:34:32.211 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:32.208+0000 7f359fef4700 1 -- 172.21.15.104:0/1048626145 wait complete. 2024-04-26T04:34:32.211 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: Regenerating cephadm self-signed grafana TLS certificates 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-26T04:34:32.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-04-26T04:34:32.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: Deploying daemon grafana.smithi090 on smithi090 2024-04-26T04:34:32.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:32.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:32 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/1048626145' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:33.797 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:33.797 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:33.997 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:34.043 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:34.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:34 smithi090 ceph-mon[26648]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:35.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 -- 172.21.15.104:0/2700216546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 msgr2=0x7ff7240fe8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:35.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 --2- 172.21.15.104:0/2700216546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff7240fe8b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff714009910 tx=0x7ff714009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:35.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 -- 172.21.15.104:0/2700216546 shutdown_connections 2024-04-26T04:34:35.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 --2- 172.21.15.104:0/2700216546 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff7240fe8b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:35.347 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 -- 172.21.15.104:0/2700216546 >> 172.21.15.104:0/2700216546 conn(0x7ff7240f9a50 msgr2=0x7ff7240fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 -- 172.21.15.104:0/2700216546 shutdown_connections 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 -- 172.21.15.104:0/2700216546 wait complete. 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.343+0000 7ff729f56700 1 Processor -- start 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff729f56700 1 -- start start 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff729f56700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff724193ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff729f56700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff724194410 con 0x7ff7240fe4a0 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff7237fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff724193ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff7237fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff724193ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:44602/0 (socket says 172.21.15.104:44602) 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff7237fe700 1 -- 172.21.15.104:0/61735524 learned_addr learned my addr 172.21.15.104:0/61735524 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:35.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff7237fe700 1 -- 172.21.15.104:0/61735524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7140095f0 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff7237fe700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff724193ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff714005b40 tx=0x7ff714004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff720ff9700 1 -- 172.21.15.104:0/61735524 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff71401f070 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.344+0000 7ff720ff9700 1 -- 172.21.15.104:0/61735524 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff71400be10 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.345+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff724194610 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.345+0000 7ff720ff9700 1 -- 172.21.15.104:0/61735524 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ff714024620 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.345+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff724194ab0 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.346+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7241042d0 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.346+0000 7ff720ff9700 1 -- 172.21.15.104:0/61735524 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ff7140199c0 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.346+0000 7ff720ff9700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ff70c038490 0x7ff70c03a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.346+0000 7ff720ff9700 1 -- 172.21.15.104:0/61735524 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff71404e560 con 0x7ff7240fe4a0 2024-04-26T04:34:35.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.346+0000 7ff722ffd700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ff70c038490 0x7ff70c03a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:35.350 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.346+0000 7ff722ffd700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ff70c038490 0x7ff70c03a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff718006fd0 tx=0x7ff718006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:35.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.351+0000 7ff720ff9700 1 -- 172.21.15.104:0/61735524 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff71402c540 con 0x7ff7240fe4a0 2024-04-26T04:34:35.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.616+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ff7241044c0 con 0x7ff7240fe4a0 2024-04-26T04:34:35.618 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.617+0000 7ff720ff9700 1 -- 172.21.15.104:0/61735524 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ff714029020 con 0x7ff7240fe4a0 2024-04-26T04:34:35.618 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:35.619 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:35.620 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ff70c038490 msgr2=0x7ff70c03a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:35.620 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ff70c038490 0x7ff70c03a940 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ff718006fd0 tx=0x7ff718006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:35.620 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 msgr2=0x7ff724193ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:35.620 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff724193ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff714005b40 tx=0x7ff714004740 comp rx=0 tx=0).stop 2024-04-26T04:34:35.620 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 shutdown_connections 2024-04-26T04:34:35.620 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ff70c038490 0x7ff70c03a940 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:35.621 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 --2- 172.21.15.104:0/61735524 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7240fe4a0 0x7ff724193ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:35.621 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 >> 172.21.15.104:0/61735524 conn(0x7ff7240f9a50 msgr2=0x7ff7240faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:35.621 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 shutdown_connections 2024-04-26T04:34:35.621 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:35.619+0000 7ff729f56700 1 -- 172.21.15.104:0/61735524 wait complete. 2024-04-26T04:34:35.621 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:36 smithi090 ceph-mon[26648]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:36 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/61735524' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:37.223 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:37.223 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:37.430 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:37.476 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:38.711 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:38 smithi090 ceph-mon[26648]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:38.904 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.902+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/2061985072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 msgr2=0x7fd7c8100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.902+0000 7fd7ce6bb700 1 --2- 172.21.15.104:0/2061985072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8100240 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8009910 tx=0x7fd7b8009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.902+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/2061985072 shutdown_connections 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.902+0000 7fd7ce6bb700 1 --2- 172.21.15.104:0/2061985072 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8100240 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.902+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/2061985072 >> 172.21.15.104:0/2061985072 conn(0x7fd7c80fb3c0 msgr2=0x7fd7c80fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.902+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/2061985072 shutdown_connections 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.902+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/2061985072 wait complete. 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.903+0000 7fd7ce6bb700 1 Processor -- start 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.903+0000 7fd7ce6bb700 1 -- start start 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.903+0000 7fd7ce6bb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.903+0000 7fd7ce6bb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7c8075030 con 0x7fd7c80ffe30 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:59296/0 (socket says 172.21.15.104:59296) 2024-04-26T04:34:38.905 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c7fff700 1 -- 172.21.15.104:0/335601993 learned_addr learned my addr 172.21.15.104:0/335601993 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:38.907 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c7fff700 1 -- 172.21.15.104:0/335601993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7b80095f0 con 0x7fd7c80ffe30 2024-04-26T04:34:38.907 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c7fff700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8074af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8005b40 tx=0x7fd7b8004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:38.907 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c57fa700 1 -- 172.21.15.104:0/335601993 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7b801d070 con 0x7fd7c80ffe30 2024-04-26T04:34:38.907 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c57fa700 1 -- 172.21.15.104:0/335601993 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd7b800be10 con 0x7fd7c80ffe30 2024-04-26T04:34:38.907 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c8073140 con 0x7fd7c80ffe30 2024-04-26T04:34:38.907 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7c57fa700 1 -- 172.21.15.104:0/335601993 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fd7b800f460 con 0x7fd7c80ffe30 2024-04-26T04:34:38.907 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.904+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7c80735e0 con 0x7fd7c80ffe30 2024-04-26T04:34:38.908 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.905+0000 7fd7c57fa700 1 -- 172.21.15.104:0/335601993 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7fd7b80227e0 con 0x7fd7c80ffe30 2024-04-26T04:34:38.908 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.905+0000 7fd7c57fa700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd7b0038490 0x7fd7b003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:38.908 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.905+0000 7fd7c57fa700 1 -- 172.21.15.104:0/335601993 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd7b804c2a0 con 0x7fd7c80ffe30 2024-04-26T04:34:38.908 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.905+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7c8073ff0 con 0x7fd7c80ffe30 2024-04-26T04:34:38.908 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.906+0000 7fd7c77fe700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd7b0038490 0x7fd7b003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:38.908 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.906+0000 7fd7c77fe700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd7b0038490 0x7fd7b003a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc006fd0 tx=0x7fd7bc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:38.916 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:38.915+0000 7fd7c57fa700 1 -- 172.21.15.104:0/335601993 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd7b8027020 con 0x7fd7c80ffe30 2024-04-26T04:34:39.179 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.177+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fd7c818e930 con 0x7fd7c80ffe30 2024-04-26T04:34:39.179 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.178+0000 7fd7c57fa700 1 -- 172.21.15.104:0/335601993 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fd7b802b590 con 0x7fd7c80ffe30 2024-04-26T04:34:39.179 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:39.179 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:39.181 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd7b0038490 msgr2=0x7fd7b003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:39.181 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd7b0038490 0x7fd7b003a940 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7bc006fd0 tx=0x7fd7bc006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:39.181 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 msgr2=0x7fd7c8074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:39.181 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8074af0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8005b40 tx=0x7fd7b8004740 comp rx=0 tx=0).stop 2024-04-26T04:34:39.182 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 shutdown_connections 2024-04-26T04:34:39.182 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7fd7b0038490 0x7fd7b003a940 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:39.182 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 --2- 172.21.15.104:0/335601993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7c80ffe30 0x7fd7c8074af0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:39.182 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 >> 172.21.15.104:0/335601993 conn(0x7fd7c80fb3c0 msgr2=0x7fd7c80fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:39.182 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 shutdown_connections 2024-04-26T04:34:39.182 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:39.180+0000 7fd7ce6bb700 1 -- 172.21.15.104:0/335601993 wait complete. 2024-04-26T04:34:39.182 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:39.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:39 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/335601993' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:40.670 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:40.671 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:40.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:40 smithi090 ceph-mon[26648]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:40.871 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:40.917 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:42.382 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.378+0000 7f2173ee1700 1 -- 172.21.15.104:0/1984139289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 msgr2=0x7f216c100280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.378+0000 7f2173ee1700 1 --2- 172.21.15.104:0/1984139289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c100280 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f215c009910 tx=0x7f215c009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 -- 172.21.15.104:0/1984139289 shutdown_connections 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 --2- 172.21.15.104:0/1984139289 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c100280 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 -- 172.21.15.104:0/1984139289 >> 172.21.15.104:0/1984139289 conn(0x7f216c0fb470 msgr2=0x7f216c0fd8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 -- 172.21.15.104:0/1984139289 shutdown_connections 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 -- 172.21.15.104:0/1984139289 wait complete. 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 Processor -- start 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 -- start start 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c194fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:42.383 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.379+0000 7f2173ee1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c1954e0 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2171c7d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c194fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2171c7d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c194fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:59302/0 (socket says 172.21.15.104:59302) 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2171c7d700 1 -- 172.21.15.104:0/405049615 learned_addr learned my addr 172.21.15.104:0/405049615 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2171c7d700 1 -- 172.21.15.104:0/405049615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f215c0095f0 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2171c7d700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c194fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f215c005b40 tx=0x7f215c004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2162ffd700 1 -- 172.21.15.104:0/405049615 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f215c01d070 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c1956e0 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2162ffd700 1 -- 172.21.15.104:0/405049615 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f215c00be10 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2162ffd700 1 -- 172.21.15.104:0/405049615 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f215c00f460 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.380+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c195b80 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.381+0000 7f2162ffd700 1 -- 172.21.15.104:0/405049615 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f215c0227e0 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.381+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c18ea70 con 0x7f216c0ffe70 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.381+0000 7f2162ffd700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f2158038490 0x7f215803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:42.384 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.381+0000 7f2162ffd700 1 -- 172.21.15.104:0/405049615 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f215c04c4a0 con 0x7f216c0ffe70 2024-04-26T04:34:42.385 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.381+0000 7f217147c700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f2158038490 0x7f215803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:42.385 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.382+0000 7f217147c700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f2158038490 0x7f215803a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2168006fd0 tx=0x7f2168006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:42.387 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.386+0000 7f2162ffd700 1 -- 172.21.15.104:0/405049615 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f215c02bba0 con 0x7f216c0ffe70 2024-04-26T04:34:42.572 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:42 smithi090 ceph-mon[26648]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:42.648 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.647+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f216c02cc30 con 0x7f216c0ffe70 2024-04-26T04:34:42.650 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.648+0000 7f2162ffd700 1 -- 172.21.15.104:0/405049615 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f215c027030 con 0x7f216c0ffe70 2024-04-26T04:34:42.650 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:42.650 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:42.651 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f2158038490 msgr2=0x7f215803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:42.651 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f2158038490 0x7f215803a940 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f2168006fd0 tx=0x7f2168006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 msgr2=0x7f216c194fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c194fa0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f215c005b40 tx=0x7f215c004740 comp rx=0 tx=0).stop 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 shutdown_connections 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f2158038490 0x7f215803a940 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 --2- 172.21.15.104:0/405049615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f216c0ffe70 0x7f216c194fa0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 >> 172.21.15.104:0/405049615 conn(0x7f216c0fb470 msgr2=0x7f216c0fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 shutdown_connections 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:42.650+0000 7f2173ee1700 1 -- 172.21.15.104:0/405049615 wait complete. 2024-04-26T04:34:42.652 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:43.718 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:43 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/405049615' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:44.281 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:44.281 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:44.480 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:44.526 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:44 smithi090 ceph-mon[26648]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:45.924 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 -- 172.21.15.104:0/1715574091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 msgr2=0x7f02dc0fe4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 --2- 172.21.15.104:0/1715574091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc0fe4f0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f02d8009910 tx=0x7f02d8009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 -- 172.21.15.104:0/1715574091 shutdown_connections 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 --2- 172.21.15.104:0/1715574091 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc0fe4f0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 -- 172.21.15.104:0/1715574091 >> 172.21.15.104:0/1715574091 conn(0x7f02dc0f9a50 msgr2=0x7f02dc0fbe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 -- 172.21.15.104:0/1715574091 shutdown_connections 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 -- 172.21.15.104:0/1715574091 wait complete. 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 Processor -- start 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.922+0000 7f02e450d700 1 -- start start 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e450d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc19b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e450d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02dc19bac0 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e22a9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc19b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e22a9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc19b580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:59310/0 (socket says 172.21.15.104:59310) 2024-04-26T04:34:45.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e22a9700 1 -- 172.21.15.104:0/3104029604 learned_addr learned my addr 172.21.15.104:0/3104029604 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:45.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e22a9700 1 -- 172.21.15.104:0/3104029604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02d80095f0 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e22a9700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc19b580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f02d80092e0 tx=0x7f02d8009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:45.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02d37fe700 1 -- 172.21.15.104:0/3104029604 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02d801d070 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02dc19bcc0 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02d37fe700 1 -- 172.21.15.104:0/3104029604 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f02d800be10 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02d37fe700 1 -- 172.21.15.104:0/3104029604 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f02d800f460 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.923+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02dc19c160 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.927 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.924+0000 7f02d37fe700 1 -- 172.21.15.104:0/3104029604 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f02d800b960 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.927 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.924+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02dc1951a0 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.927 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.925+0000 7f02d37fe700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f02c8038490 0x7f02c803a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:45.927 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.925+0000 7f02d37fe700 1 -- 172.21.15.104:0/3104029604 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f02d804c4b0 con 0x7f02dc0fe0e0 2024-04-26T04:34:45.927 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.925+0000 7f02e1aa8700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f02c8038490 0x7f02c803a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:45.927 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.925+0000 7f02e1aa8700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f02c8038490 0x7f02c803a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02cc006fd0 tx=0x7f02cc006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:45.930 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:45.929+0000 7f02d37fe700 1 -- 172.21.15.104:0/3104029604 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f02d8005c00 con 0x7f02dc0fe0e0 2024-04-26T04:34:46.194 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.193+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f02dc02cc30 con 0x7f02dc0fe0e0 2024-04-26T04:34:46.195 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.193+0000 7f02d37fe700 1 -- 172.21.15.104:0/3104029604 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f02d8027070 con 0x7f02dc0fe0e0 2024-04-26T04:34:46.195 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:46.195 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:46.196 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f02c8038490 msgr2=0x7f02c803a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f02c8038490 0x7f02c803a940 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02cc006fd0 tx=0x7f02cc006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 msgr2=0x7f02dc19b580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc19b580 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f02d80092e0 tx=0x7f02d8009e60 comp rx=0 tx=0).stop 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 shutdown_connections 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f02c8038490 0x7f02c803a940 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 --2- 172.21.15.104:0/3104029604 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f02dc0fe0e0 0x7f02dc19b580 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 >> 172.21.15.104:0/3104029604 conn(0x7f02dc0f9a50 msgr2=0x7f02dc101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 shutdown_connections 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:46.195+0000 7f02e450d700 1 -- 172.21.15.104:0/3104029604 wait complete. 2024-04-26T04:34:46.197 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:46.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:46.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:46.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:46.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:46 smithi090 ceph-mon[26648]: Deploying daemon prometheus.smithi090 on smithi090 2024-04-26T04:34:47.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:47 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/3104029604' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:47.751 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:47.751 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:47.962 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:48.007 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:48 smithi090 ceph-mon[26648]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:48 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:49.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.526+0000 7f2608640700 1 -- 172.21.15.104:0/58411652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 msgr2=0x7f2600100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:49.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.526+0000 7f2608640700 1 --2- 172.21.15.104:0/58411652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f2600100700 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f25fc009910 tx=0x7f25fc009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:49.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.526+0000 7f2608640700 1 -- 172.21.15.104:0/58411652 shutdown_connections 2024-04-26T04:34:49.528 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.526+0000 7f2608640700 1 --2- 172.21.15.104:0/58411652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f2600100700 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:49.529 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.526+0000 7f2608640700 1 -- 172.21.15.104:0/58411652 >> 172.21.15.104:0/58411652 conn(0x7f26000f9af0 msgr2=0x7f26000fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:49.529 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.526+0000 7f2608640700 1 -- 172.21.15.104:0/58411652 shutdown_connections 2024-04-26T04:34:49.529 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.526+0000 7f2608640700 1 -- 172.21.15.104:0/58411652 wait complete. 2024-04-26T04:34:49.529 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.527+0000 7f2608640700 1 Processor -- start 2024-04-26T04:34:49.529 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.527+0000 7f2608640700 1 -- start start 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.527+0000 7f2608640700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f26000fe9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.527+0000 7f2608640700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2600197d20 con 0x7f26001002f0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.527+0000 7f26063dc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f26000fe9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.527+0000 7f26063dc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f26000fe9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:47600/0 (socket says 172.21.15.104:47600) 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.527+0000 7f26063dc700 1 -- 172.21.15.104:0/2902061193 learned_addr learned my addr 172.21.15.104:0/2902061193 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.528+0000 7f26063dc700 1 -- 172.21.15.104:0/2902061193 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25fc0095f0 con 0x7f26001002f0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.528+0000 7f26063dc700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f26000fe9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005950 tx=0x7f25fc009e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.528+0000 7f25f77fe700 1 -- 172.21.15.104:0/2902061193 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25fc01d070 con 0x7f26001002f0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.528+0000 7f25f77fe700 1 -- 172.21.15.104:0/2902061193 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f25fc00be10 con 0x7f26001002f0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.528+0000 7f25f77fe700 1 -- 172.21.15.104:0/2902061193 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f25fc00f460 con 0x7f26001002f0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.528+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2600197e60 con 0x7f26001002f0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.528+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26001982a0 con 0x7f26001002f0 2024-04-26T04:34:49.531 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.529+0000 7f25f77fe700 1 -- 172.21.15.104:0/2902061193 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7f25fc00b960 con 0x7f26001002f0 2024-04-26T04:34:49.532 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.529+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f260018d970 con 0x7f26001002f0 2024-04-26T04:34:49.532 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.529+0000 7f25f77fe700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f25ec0384e0 0x7f25ec03a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:49.532 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.529+0000 7f25f77fe700 1 -- 172.21.15.104:0/2902061193 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f25fc04c4a0 con 0x7f26001002f0 2024-04-26T04:34:49.532 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.529+0000 7f2605bdb700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f25ec0384e0 0x7f25ec03a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:49.532 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.530+0000 7f2605bdb700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f25ec0384e0 0x7f25ec03a990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006fd0 tx=0x7f25f0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:49.535 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.534+0000 7f25f77fe700 1 -- 172.21.15.104:0/2902061193 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f25fc02bba0 con 0x7f26001002f0 2024-04-26T04:34:49.800 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.799+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f260002cc30 con 0x7f26001002f0 2024-04-26T04:34:49.801 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.799+0000 7f25f77fe700 1 -- 172.21.15.104:0/2902061193 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f25fc027030 con 0x7f26001002f0 2024-04-26T04:34:49.801 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:49.801 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:49.803 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f25ec0384e0 msgr2=0x7f25ec03a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:49.803 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f25ec0384e0 0x7f25ec03a990 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f25f0006fd0 tx=0x7f25f0006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:49.803 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 msgr2=0x7f26000fe9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:49.803 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f26000fe9a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f25fc005950 tx=0x7f25fc009e60 comp rx=0 tx=0).stop 2024-04-26T04:34:49.803 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 shutdown_connections 2024-04-26T04:34:49.803 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f25ec0384e0 0x7f25ec03a990 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:49.803 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 --2- 172.21.15.104:0/2902061193 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26001002f0 0x7f26000fe9a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:49.804 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.801+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 >> 172.21.15.104:0/2902061193 conn(0x7f26000f9af0 msgr2=0x7f26000fa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:49.804 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.802+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 shutdown_connections 2024-04-26T04:34:49.804 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:49.802+0000 7f2608640700 1 -- 172.21.15.104:0/2902061193 wait complete. 2024-04-26T04:34:49.804 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:50.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:50 smithi090 ceph-mon[26648]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:50.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:50 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/2902061193' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:51.371 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:51.371 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:51.572 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:51.617 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:52.434 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:52 smithi090 ceph-mon[26648]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.114+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3753650388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 msgr2=0x7ffbe40ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.114+0000 7ffbec1d0700 1 --2- 172.21.15.104:0/3753650388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe40ff130 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4009910 tx=0x7ffbd4009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.114+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3753650388 shutdown_connections 2024-04-26T04:34:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.114+0000 7ffbec1d0700 1 --2- 172.21.15.104:0/3753650388 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe40ff130 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.114+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3753650388 >> 172.21.15.104:0/3753650388 conn(0x7ffbe40fa2b0 msgr2=0x7ffbe40fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:53.117 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.114+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3753650388 shutdown_connections 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.114+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3753650388 wait complete. 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.115+0000 7ffbec1d0700 1 Processor -- start 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.115+0000 7ffbec1d0700 1 -- start start 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.115+0000 7ffbec1d0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe4193ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.115+0000 7ffbec1d0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe41943e0 con 0x7ffbe40fed20 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.115+0000 7ffbe9f6c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe4193ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.115+0000 7ffbe9f6c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe4193ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:47610/0 (socket says 172.21.15.104:47610) 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.115+0000 7ffbe9f6c700 1 -- 172.21.15.104:0/3217299134 learned_addr learned my addr 172.21.15.104:0/3217299134 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.116+0000 7ffbe9f6c700 1 -- 172.21.15.104:0/3217299134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd40095f0 con 0x7ffbe40fed20 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.116+0000 7ffbe9f6c700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe4193ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005b40 tx=0x7ffbd4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.116+0000 7ffbdaffd700 1 -- 172.21.15.104:0/3217299134 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbd401d070 con 0x7ffbe40fed20 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.116+0000 7ffbdaffd700 1 -- 172.21.15.104:0/3217299134 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffbd400be10 con 0x7ffbe40fed20 2024-04-26T04:34:53.118 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.116+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe41945e0 con 0x7ffbe40fed20 2024-04-26T04:34:53.119 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.116+0000 7ffbdaffd700 1 -- 172.21.15.104:0/3217299134 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7ffbd400f460 con 0x7ffbe40fed20 2024-04-26T04:34:53.119 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.116+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe4194a80 con 0x7ffbe40fed20 2024-04-26T04:34:53.119 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.117+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe418da00 con 0x7ffbe40fed20 2024-04-26T04:34:53.119 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.117+0000 7ffbdaffd700 1 -- 172.21.15.104:0/3217299134 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 13) v1 ==== 45140+0+0 (secure 0 0 0) 0x7ffbd40227e0 con 0x7ffbe40fed20 2024-04-26T04:34:53.120 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.117+0000 7ffbdaffd700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffbd0038490 0x7ffbd003a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:53.120 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.117+0000 7ffbdaffd700 1 -- 172.21.15.104:0/3217299134 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffbd404c4a0 con 0x7ffbe40fed20 2024-04-26T04:34:53.120 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.117+0000 7ffbe976b700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffbd0038490 0x7ffbd003a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:53.120 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.118+0000 7ffbe976b700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffbd0038490 0x7ffbd003a940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006fd0 tx=0x7ffbe0006e40 comp rx=0 tx=0).ready entity=mgr.14164 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:53.123 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.122+0000 7ffbdaffd700 1 -- 172.21.15.104:0/3217299134 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffbd402bba0 con 0x7ffbe40fed20 2024-04-26T04:34:53.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.391+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7ffbe402cc30 con 0x7ffbe40fed20 2024-04-26T04:34:53.392 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.391+0000 7ffbdaffd700 1 -- 172.21.15.104:0/3217299134 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7ffbd4027030 con 0x7ffbe40fed20 2024-04-26T04:34:53.393 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:53.393 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffbd0038490 msgr2=0x7ffbd003a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffbd0038490 0x7ffbd003a940 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7ffbe0006fd0 tx=0x7ffbe0006e40 comp rx=0 tx=0).stop 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 msgr2=0x7ffbe4193ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe4193ea0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ffbd4005b40 tx=0x7ffbd4004740 comp rx=0 tx=0).stop 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 shutdown_connections 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7ffbd0038490 0x7ffbd003a940 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 --2- 172.21.15.104:0/3217299134 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffbe40fed20 0x7ffbe4193ea0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 >> 172.21.15.104:0/3217299134 conn(0x7ffbe40fa2b0 msgr2=0x7ffbe40faf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:53.395 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 shutdown_connections 2024-04-26T04:34:53.396 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:53.394+0000 7ffbec1d0700 1 -- 172.21.15.104:0/3217299134 wait complete. 2024-04-26T04:34:53.396 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:54.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:54 smithi090 ceph-mon[26648]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:54.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:54 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/3217299134' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:54.963 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:54.964 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:55.164 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:55.210 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:56.776 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.774+0000 7f7af69d7700 1 -- 172.21.15.104:0/3254480953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 msgr2=0x7f7af00ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:56.776 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.774+0000 7f7af69d7700 1 --2- 172.21.15.104:0/3254480953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00ff140 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0009910 tx=0x7f7ae0009c20 comp rx=0 tx=0).stop 2024-04-26T04:34:56.776 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.774+0000 7f7af69d7700 1 -- 172.21.15.104:0/3254480953 shutdown_connections 2024-04-26T04:34:56.776 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.774+0000 7f7af69d7700 1 --2- 172.21.15.104:0/3254480953 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00ff140 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.774+0000 7f7af69d7700 1 -- 172.21.15.104:0/3254480953 >> 172.21.15.104:0/3254480953 conn(0x7f7af00fa2c0 msgr2=0x7f7af00fc710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.774+0000 7f7af69d7700 1 -- 172.21.15.104:0/3254480953 shutdown_connections 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.774+0000 7f7af69d7700 1 -- 172.21.15.104:0/3254480953 wait complete. 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7af69d7700 1 Processor -- start 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7af69d7700 1 -- start start 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7af69d7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00fc270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7af69d7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7af00fc7b0 con 0x7f7af00fed30 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00fc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7aeffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00fc270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:41218/0 (socket says 172.21.15.104:41218) 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7aeffff700 1 -- 172.21.15.104:0/3607545183 learned_addr learned my addr 172.21.15.104:0/3607545183 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:34:56.777 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.775+0000 7f7aeffff700 1 -- 172.21.15.104:0/3607545183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ae00095f0 con 0x7f7af00fed30 2024-04-26T04:34:56.778 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.776+0000 7f7aeffff700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00fc270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0005b40 tx=0x7f7ae0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:34:56.778 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.776+0000 7f7aed7fa700 1 -- 172.21.15.104:0/3607545183 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae001d070 con 0x7f7af00fed30 2024-04-26T04:34:56.778 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.776+0000 7f7aed7fa700 1 -- 172.21.15.104:0/3607545183 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7ae000be10 con 0x7f7af00fed30 2024-04-26T04:34:56.778 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.776+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af00fa8c0 con 0x7f7af00fed30 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.776+0000 7f7aed7fa700 1 -- 172.21.15.104:0/3607545183 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7ae000f460 con 0x7f7af00fed30 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.776+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7af00fad60 con 0x7f7af00fed30 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.777+0000 7f7aed7fa700 1 -- 172.21.15.104:0/3607545183 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f7ae000b960 con 0x7f7af00fed30 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.777+0000 7f7aed7fa700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 0x7f7ad803a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.777+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7af0062380 con 0x7f7af00fed30 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.777+0000 7f7aed7fa700 1 -- 172.21.15.104:0/3607545183 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7ae004c530 con 0x7f7af00fed30 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.777+0000 7f7aef7fe700 1 -- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 msgr2=0x7f7ad803a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2647874706 2024-04-26T04:34:56.779 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.777+0000 7f7aef7fe700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 0x7f7ad803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:34:56.782 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.782+0000 7f7aed7fa700 1 -- 172.21.15.104:0/3607545183 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7ae002a360 con 0x7f7af00fed30 2024-04-26T04:34:56.957 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:56 smithi090 ceph-mon[26648]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:34:56.957 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:56 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:56.957 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:56 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:56.957 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:56 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:34:56.957 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:56 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-04-26T04:34:56.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.978+0000 7f7aef7fe700 1 -- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 msgr2=0x7f7ad803a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2647874706 2024-04-26T04:34:56.979 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:56.978+0000 7f7aef7fe700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 0x7f7ad803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:34:57.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.048+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7af018e400 con 0x7f7af00fed30 2024-04-26T04:34:57.049 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.048+0000 7f7aed7fa700 1 -- 172.21.15.104:0/3607545183 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7ae0027070 con 0x7f7af00fed30 2024-04-26T04:34:57.049 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:34:57.050 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:34:57.051 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 msgr2=0x7f7ad803a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 0x7f7ad803a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 msgr2=0x7f7af00fc270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00fc270 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f7ae0005b40 tx=0x7f7ae0004740 comp rx=0 tx=0).stop 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 shutdown_connections 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f7ad80384e0 0x7f7ad803a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 --2- 172.21.15.104:0/3607545183 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7af00fed30 0x7f7af00fc270 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.050+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 >> 172.21.15.104:0/3607545183 conn(0x7f7af00fa2c0 msgr2=0x7f7af0069070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.051+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 shutdown_connections 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:34:57.051+0000 7f7af69d7700 1 -- 172.21.15.104:0/3607545183 wait complete. 2024-04-26T04:34:57.052 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:34:57.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:57 smithi090 ceph-mon[26648]: from='mgr.14164 172.21.15.90:0/540844794' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-04-26T04:34:57.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:57 smithi090 ceph-mon[26648]: mgrmap e14: smithi090.iwrvwa(active, since 59s) 2024-04-26T04:34:57.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:34:57 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/3607545183' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:34:58.633 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:34:58.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:34:58.832 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:34:58.877 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:35:00.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.402+0000 7f602d911700 1 -- 172.21.15.104:0/888316040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 msgr2=0x7f60280ff130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:00.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.402+0000 7f602d911700 1 --2- 172.21.15.104:0/888316040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f60280ff130 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f6018009910 tx=0x7f6018009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:00.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.402+0000 7f602d911700 1 -- 172.21.15.104:0/888316040 shutdown_connections 2024-04-26T04:35:00.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.402+0000 7f602d911700 1 --2- 172.21.15.104:0/888316040 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f60280ff130 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:00.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.402+0000 7f602d911700 1 -- 172.21.15.104:0/888316040 >> 172.21.15.104:0/888316040 conn(0x7f60280fa2b0 msgr2=0x7f60280fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:00.405 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.402+0000 7f602d911700 1 -- 172.21.15.104:0/888316040 shutdown_connections 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.402+0000 7f602d911700 1 -- 172.21.15.104:0/888316040 wait complete. 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f602d911700 1 Processor -- start 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f602d911700 1 -- start start 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f602d911700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f6028194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f602d911700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60281954f0 con 0x7f60280fed20 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f6028194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f6026ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f6028194fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:41228/0 (socket says 172.21.15.104:41228) 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f6026ffd700 1 -- 172.21.15.104:0/127051163 learned_addr learned my addr 172.21.15.104:0/127051163 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.403+0000 7f6026ffd700 1 -- 172.21.15.104:0/127051163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60180095f0 con 0x7f60280fed20 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.404+0000 7f6026ffd700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f6028194fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6018005b40 tx=0x7f6018004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.404+0000 7f602c90f700 1 -- 172.21.15.104:0/127051163 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f601801d070 con 0x7f60280fed20 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.404+0000 7f602c90f700 1 -- 172.21.15.104:0/127051163 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f601800be10 con 0x7f60280fed20 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.404+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60281956f0 con 0x7f60280fed20 2024-04-26T04:35:00.406 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.404+0000 7f602c90f700 1 -- 172.21.15.104:0/127051163 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f601800f460 con 0x7f60280fed20 2024-04-26T04:35:00.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.404+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6028195b90 con 0x7f60280fed20 2024-04-26T04:35:00.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.405+0000 7f602c90f700 1 -- 172.21.15.104:0/127051163 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 14) v1 ==== 45154+0+0 (secure 0 0 0) 0x7f601800f6d0 con 0x7f60280fed20 2024-04-26T04:35:00.407 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.405+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f602818ea10 con 0x7f60280fed20 2024-04-26T04:35:00.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.405+0000 7f602c90f700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 0x7f601003a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:00.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.405+0000 7f602c90f700 1 -- 172.21.15.104:0/127051163 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f601804c6d0 con 0x7f60280fed20 2024-04-26T04:35:00.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.405+0000 7f60267fc700 1 -- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 msgr2=0x7f601003a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2647874706 2024-04-26T04:35:00.408 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.405+0000 7f60267fc700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 0x7f601003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:35:00.411 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.410+0000 7f602c90f700 1 -- 172.21.15.104:0/127051163 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f601802a360 con 0x7f60280fed20 2024-04-26T04:35:00.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.606+0000 7f60267fc700 1 -- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 msgr2=0x7f601003a990 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.90:6800/2647874706 2024-04-26T04:35:00.607 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.606+0000 7f60267fc700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 0x7f601003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-26T04:35:00.674 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.673+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f602802cc30 con 0x7f60280fed20 2024-04-26T04:35:00.675 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.673+0000 7f602c90f700 1 -- 172.21.15.104:0/127051163 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f6018027030 con 0x7f60280fed20 2024-04-26T04:35:00.675 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:00.675 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 msgr2=0x7f601003a990 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 0x7f601003a990 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 msgr2=0x7f6028194fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f6028194fb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f6018005b40 tx=0x7f6018004740 comp rx=0 tx=0).stop 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 shutdown_connections 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:6800/2647874706,v1:172.21.15.90:6801/2647874706] conn(0x7f60100384e0 0x7f601003a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 --2- 172.21.15.104:0/127051163 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f60280fed20 0x7f6028194fb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:00.677 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 >> 172.21.15.104:0/127051163 conn(0x7f60280fa2b0 msgr2=0x7f60280690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:00.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 shutdown_connections 2024-04-26T04:35:00.678 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:00.676+0000 7f602d911700 1 -- 172.21.15.104:0/127051163 wait complete. 2024-04-26T04:35:00.678 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:35:01.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:00 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/127051163' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:02.412 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:35:02.413 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:35:02.612 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:35:02.663 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:35:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.249+0000 7f306f2cd700 1 -- 172.21.15.104:0/2405197187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 msgr2=0x7f3068100240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.249+0000 7f306f2cd700 1 --2- 172.21.15.104:0/2405197187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068100240 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f3064009910 tx=0x7f3064009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.249+0000 7f306f2cd700 1 -- 172.21.15.104:0/2405197187 shutdown_connections 2024-04-26T04:35:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.249+0000 7f306f2cd700 1 --2- 172.21.15.104:0/2405197187 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068100240 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.249+0000 7f306f2cd700 1 -- 172.21.15.104:0/2405197187 >> 172.21.15.104:0/2405197187 conn(0x7f30680fb3c0 msgr2=0x7f30680fd810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.249+0000 7f306f2cd700 1 -- 172.21.15.104:0/2405197187 shutdown_connections 2024-04-26T04:35:04.251 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.249+0000 7f306f2cd700 1 -- 172.21.15.104:0/2405197187 wait complete. 2024-04-26T04:35:04.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.250+0000 7f306f2cd700 1 Processor -- start 2024-04-26T04:35:04.252 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.250+0000 7f306f2cd700 1 -- start start 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.250+0000 7f306f2cd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.250+0000 7f306f2cd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3068075030 con 0x7f30680ffe30 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.250+0000 7f306d069700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.250+0000 7f306d069700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:41246/0 (socket says 172.21.15.104:41246) 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.250+0000 7f306d069700 1 -- 172.21.15.104:0/4015084391 learned_addr learned my addr 172.21.15.104:0/4015084391 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.251+0000 7f306d069700 1 -- 172.21.15.104:0/4015084391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30640095f0 con 0x7f30680ffe30 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.251+0000 7f306d069700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068074af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3064005b40 tx=0x7f3064004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.251+0000 7f305e7fc700 1 -- 172.21.15.104:0/4015084391 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f306401d070 con 0x7f30680ffe30 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.251+0000 7f305e7fc700 1 -- 172.21.15.104:0/4015084391 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f306400be10 con 0x7f30680ffe30 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.251+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3068073140 con 0x7f30680ffe30 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.251+0000 7f305e7fc700 1 -- 172.21.15.104:0/4015084391 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f306400f460 con 0x7f30680ffe30 2024-04-26T04:35:04.253 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.251+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30680735e0 con 0x7f30680ffe30 2024-04-26T04:35:04.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.252+0000 7f305e7fc700 1 -- 172.21.15.104:0/4015084391 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 16) v1 ==== 45005+0+0 (secure 0 0 0) 0x7f306400f5c0 con 0x7f30680ffe30 2024-04-26T04:35:04.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.252+0000 7f305e7fc700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3054038380 0x7f305403a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:04.254 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.252+0000 7f305e7fc700 1 -- 172.21.15.104:0/4015084391 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f306404c130 con 0x7f30680ffe30 2024-04-26T04:35:04.255 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.252+0000 7f306c868700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3054038380 0x7f305403a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:04.255 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.252+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3068073ff0 con 0x7f30680ffe30 2024-04-26T04:35:04.255 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.253+0000 7f306c868700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3054038380 0x7f305403a830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3058006fd0 tx=0x7f3058006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:04.258 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.257+0000 7f305e7fc700 1 -- 172.21.15.104:0/4015084391 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3064027030 con 0x7f30680ffe30 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: Active manager daemon smithi090.iwrvwa restarted 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: Activating manager daemon smithi090.iwrvwa 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: osdmap e5: 0 total, 0 up, 0 in 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: mgrmap e15: smithi090.iwrvwa(active, starting, since 0.00799478s) 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:35:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:35:04.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:35:04.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:04.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:35:04.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:35:04.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:35:04.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:04.545 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.544+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f3068062380 con 0x7f30680ffe30 2024-04-26T04:35:04.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.545+0000 7f305e7fc700 1 -- 172.21.15.104:0/4015084391 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f3064020960 con 0x7f30680ffe30 2024-04-26T04:35:04.546 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:04.546 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:35:04.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3054038380 msgr2=0x7f305403a830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:04.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3054038380 0x7f305403a830 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f3058006fd0 tx=0x7f3058006e40 comp rx=0 tx=0).stop 2024-04-26T04:35:04.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 msgr2=0x7f3068074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:04.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068074af0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3064005b40 tx=0x7f3064004740 comp rx=0 tx=0).stop 2024-04-26T04:35:04.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 shutdown_connections 2024-04-26T04:35:04.549 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3054038380 0x7f305403a830 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:04.549 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 --2- 172.21.15.104:0/4015084391 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f30680ffe30 0x7f3068074af0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:04.549 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 >> 172.21.15.104:0/4015084391 conn(0x7f30680fb3c0 msgr2=0x7f30680fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:04.549 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 shutdown_connections 2024-04-26T04:35:04.549 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:04.547+0000 7f306f2cd700 1 -- 172.21.15.104:0/4015084391 wait complete. 2024-04-26T04:35:04.549 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: [26/Apr/2024:04:35:03] ENGINE Bus STARTING 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: [26/Apr/2024:04:35:04] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: [26/Apr/2024:04:35:04] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: [26/Apr/2024:04:35:04] ENGINE Bus STARTED 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: mgrmap e16: smithi090.iwrvwa(active, since 1.00875s) 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/4015084391' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:05.545 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:05 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:06.073 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:35:06.073 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:35:06.952 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:35:07.010 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /etc/ceph/ceph.conf 2024-04-26T04:35:07.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:07 smithi090 ceph-mon[26648]: mgrmap e17: smithi090.iwrvwa(active, since 2s) 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 -- 172.21.15.104:0/2723743939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 msgr2=0x7f15cc0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 --2- 172.21.15.104:0/2723743939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc0731e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f15bc009910 tx=0x7f15bc009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 -- 172.21.15.104:0/2723743939 shutdown_connections 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 --2- 172.21.15.104:0/2723743939 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc0731e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 -- 172.21.15.104:0/2723743939 >> 172.21.15.104:0/2723743939 conn(0x7f15cc0fb3d0 msgr2=0x7f15cc0fd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 -- 172.21.15.104:0/2723743939 shutdown_connections 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 -- 172.21.15.104:0/2723743939 wait complete. 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 Processor -- start 2024-04-26T04:35:08.703 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.700+0000 7f15d26a1700 1 -- start start 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15d26a1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc19b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15d26a1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15cc19ba90 con 0x7f15cc074d80 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc19b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15cbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc19b550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:45872/0 (socket says 172.21.15.104:45872) 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15cbfff700 1 -- 172.21.15.104:0/4163391730 learned_addr learned my addr 172.21.15.104:0/4163391730 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15cbfff700 1 -- 172.21.15.104:0/4163391730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15bc0095f0 con 0x7f15cc074d80 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15cbfff700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc19b550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005b40 tx=0x7f15bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.701+0000 7f15c97fa700 1 -- 172.21.15.104:0/4163391730 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f15bc01d070 con 0x7f15cc074d80 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.702+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15cc19bc90 con 0x7f15cc074d80 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.702+0000 7f15c97fa700 1 -- 172.21.15.104:0/4163391730 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f15bc00be10 con 0x7f15cc074d80 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.702+0000 7f15c97fa700 1 -- 172.21.15.104:0/4163391730 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f15bc00f460 con 0x7f15cc074d80 2024-04-26T04:35:08.704 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.702+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15cc19c130 con 0x7f15cc074d80 2024-04-26T04:35:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.702+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15cc195100 con 0x7f15cc074d80 2024-04-26T04:35:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.702+0000 7f15c97fa700 1 -- 172.21.15.104:0/4163391730 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f15bc00b960 con 0x7f15cc074d80 2024-04-26T04:35:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.703+0000 7f15c97fa700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f15b4038580 0x7f15b403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.703+0000 7f15c97fa700 1 -- 172.21.15.104:0/4163391730 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f15bc04c5f0 con 0x7f15cc074d80 2024-04-26T04:35:08.705 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.703+0000 7f15cb7fe700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f15b4038580 0x7f15b403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:08.706 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.703+0000 7f15cb7fe700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f15b4038580 0x7f15b403aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:08.710 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.709+0000 7f15c97fa700 1 -- 172.21.15.104:0/4163391730 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f15bc02a360 con 0x7f15cc074d80 2024-04-26T04:35:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi090", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:35:08.982 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.980+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f15cc02d050 con 0x7f15cc074d80 2024-04-26T04:35:08.982 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.981+0000 7f15c97fa700 1 -- 172.21.15.104:0/4163391730 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f15bc027030 con 0x7f15cc074d80 2024-04-26T04:35:08.983 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:08.983 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.983+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f15b4038580 msgr2=0x7f15b403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.983+0000 7f15d26a1700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f15b4038580 0x7f15b403aa30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f15c0006fd0 tx=0x7f15c0006e40 comp rx=0 tx=0).stop 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.983+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 msgr2=0x7f15cc19b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.983+0000 7f15d26a1700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc19b550 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f15bc005b40 tx=0x7f15bc004740 comp rx=0 tx=0).stop 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.984+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 shutdown_connections 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.984+0000 7f15d26a1700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f15b4038580 0x7f15b403aa30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.984+0000 7f15d26a1700 1 --2- 172.21.15.104:0/4163391730 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f15cc074d80 0x7f15cc19b550 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.984+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 >> 172.21.15.104:0/4163391730 conn(0x7f15cc0fb3d0 msgr2=0x7f15cc0fc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.984+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 shutdown_connections 2024-04-26T04:35:08.985 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:08.984+0000 7f15d26a1700 1 -- 172.21.15.104:0/4163391730 wait complete. 2024-04-26T04:35:08.986 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:35:09.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/4163391730' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:09.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:09.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:09.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:09.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:09.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd/host:smithi104", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:35:09.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:09.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:09 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:35:10.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:10.786 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:35:10.787 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:35:11.409 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:11 smithi090 ceph-mon[26648]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:35:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:11 smithi090 ceph-mon[26648]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:35:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:11 smithi090 ceph-mon[26648]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:35:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:35:11.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-26T04:35:11.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:11.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:11 smithi090 ceph-mon[26648]: Deploying daemon ceph-exporter.smithi104 on smithi104 2024-04-26T04:35:13.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 -- 172.21.15.104:0/1603304993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 msgr2=0x7fddac0ff380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:13.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 --2- 172.21.15.104:0/1603304993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac0ff380 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fdda0009910 tx=0x7fdda0009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:13.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 -- 172.21.15.104:0/1603304993 shutdown_connections 2024-04-26T04:35:13.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 --2- 172.21.15.104:0/1603304993 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac0ff380 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:13.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 -- 172.21.15.104:0/1603304993 >> 172.21.15.104:0/1603304993 conn(0x7fddac0fa890 msgr2=0x7fddac0fcca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:13.467 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 -- 172.21.15.104:0/1603304993 shutdown_connections 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 -- 172.21.15.104:0/1603304993 wait complete. 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 Processor -- start 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.464+0000 7fddb3005700 1 -- start start 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.465+0000 7fddb3005700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac1961b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.465+0000 7fddb3005700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddac1966f0 con 0x7fddac0fefb0 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.465+0000 7fddb0da1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac1961b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.465+0000 7fddb0da1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac1961b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:45886/0 (socket says 172.21.15.104:45886) 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.465+0000 7fddb0da1700 1 -- 172.21.15.104:0/2987162611 learned_addr learned my addr 172.21.15.104:0/2987162611 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.465+0000 7fddb0da1700 1 -- 172.21.15.104:0/2987162611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdda00095f0 con 0x7fddac0fefb0 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.465+0000 7fddb0da1700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac1961b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005b40 tx=0x7fdda0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.466+0000 7fdda9ffb700 1 -- 172.21.15.104:0/2987162611 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdda001d070 con 0x7fddac0fefb0 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.466+0000 7fdda9ffb700 1 -- 172.21.15.104:0/2987162611 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdda00044d0 con 0x7fddac0fefb0 2024-04-26T04:35:13.468 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.466+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddac1968f0 con 0x7fddac0fefb0 2024-04-26T04:35:13.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.466+0000 7fdda9ffb700 1 -- 172.21.15.104:0/2987162611 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7fdda0003930 con 0x7fddac0fefb0 2024-04-26T04:35:13.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.466+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddac196cb0 con 0x7fddac0fefb0 2024-04-26T04:35:13.469 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.467+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddac107230 con 0x7fddac0fefb0 2024-04-26T04:35:13.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.467+0000 7fdda9ffb700 1 -- 172.21.15.104:0/2987162611 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7fdda000b960 con 0x7fddac0fefb0 2024-04-26T04:35:13.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.467+0000 7fdda9ffb700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdd9c038580 0x7fdd9c03aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:13.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.467+0000 7fdda9ffb700 1 -- 172.21.15.104:0/2987162611 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7fdda004c500 con 0x7fddac0fefb0 2024-04-26T04:35:13.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.467+0000 7fddabfff700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdd9c038580 0x7fdd9c03aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:13.470 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.468+0000 7fddabfff700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdd9c038580 0x7fdd9c03aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006fd0 tx=0x7fdd98006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:13.473 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.472+0000 7fdda9ffb700 1 -- 172.21.15.104:0/2987162611 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdda002a930 con 0x7fddac0fefb0 2024-04-26T04:35:13.755 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.754+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fddac068e10 con 0x7fddac0fefb0 2024-04-26T04:35:13.756 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.755+0000 7fdda9ffb700 1 -- 172.21.15.104:0/2987162611 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7fdda002a360 con 0x7fddac0fefb0 2024-04-26T04:35:13.758 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:13.758 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:35:13.758 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdd9c038580 msgr2=0x7fdd9c03aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdd9c038580 0x7fdd9c03aa30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fdd98006fd0 tx=0x7fdd98006e40 comp rx=0 tx=0).stop 2024-04-26T04:35:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 msgr2=0x7fddac1961b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac1961b0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fdda0005b40 tx=0x7fdda0004c80 comp rx=0 tx=0).stop 2024-04-26T04:35:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 shutdown_connections 2024-04-26T04:35:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdd9c038580 0x7fdd9c03aa30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 --2- 172.21.15.104:0/2987162611 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fddac0fefb0 0x7fddac1961b0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:13.759 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 >> 172.21.15.104:0/2987162611 conn(0x7fddac0fa890 msgr2=0x7fddac103190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:13.762 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 shutdown_connections 2024-04-26T04:35:13.763 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:13.757+0000 7fddb3005700 1 -- 172.21.15.104:0/2987162611 wait complete. 2024-04-26T04:35:13.763 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:35:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:14 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/2987162611' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:15.333 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:35:15.333 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:35:15.527 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:15.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:15.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:15.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:15.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:15.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:35:15.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]': finished 2024-04-26T04:35:15.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:15.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:15 smithi090 ceph-mon[26648]: Deploying daemon crash.smithi104 on smithi104 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.635+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/826034482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 msgr2=0x7f4c780ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.635+0000 7f4c7d9ed700 1 --2- 172.21.15.104:0/826034482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c780ff060 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f4c68009910 tx=0x7f4c68009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.635+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/826034482 shutdown_connections 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.635+0000 7f4c7d9ed700 1 --2- 172.21.15.104:0/826034482 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c780ff060 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.635+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/826034482 >> 172.21.15.104:0/826034482 conn(0x7f4c78075750 msgr2=0x7f4c78075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.635+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/826034482 shutdown_connections 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.635+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/826034482 wait complete. 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.636+0000 7f4c7d9ed700 1 Processor -- start 2024-04-26T04:35:17.638 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.636+0000 7f4c7d9ed700 1 -- start start 2024-04-26T04:35:17.639 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.636+0000 7f4c7d9ed700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c78195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:17.639 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.636+0000 7f4c7d9ed700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c78195880 con 0x7f4c780fec90 2024-04-26T04:35:17.639 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c76ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c78195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:17.639 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c76ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c78195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:44690/0 (socket says 172.21.15.104:44690) 2024-04-26T04:35:17.639 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c76ffd700 1 -- 172.21.15.104:0/3553354982 learned_addr learned my addr 172.21.15.104:0/3553354982 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:17.639 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c76ffd700 1 -- 172.21.15.104:0/3553354982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c680095f0 con 0x7f4c780fec90 2024-04-26T04:35:17.640 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c76ffd700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c78195340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005b40 tx=0x7f4c68004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:17.640 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c7c9eb700 1 -- 172.21.15.104:0/3553354982 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c6801d070 con 0x7f4c780fec90 2024-04-26T04:35:17.640 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c78195a80 con 0x7f4c780fec90 2024-04-26T04:35:17.640 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c7c9eb700 1 -- 172.21.15.104:0/3553354982 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4c680044d0 con 0x7f4c780fec90 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c7c9eb700 1 -- 172.21.15.104:0/3553354982 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f4c68003930 con 0x7f4c780fec90 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.637+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c78195e40 con 0x7f4c780fec90 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.638+0000 7f4c7c9eb700 1 -- 172.21.15.104:0/3553354982 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f4c6800b960 con 0x7f4c780fec90 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.638+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c78062380 con 0x7f4c780fec90 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.638+0000 7f4c7c9eb700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4c64038580 0x7f4c6403aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.638+0000 7f4c7c9eb700 1 -- 172.21.15.104:0/3553354982 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f4c6804c4b0 con 0x7f4c780fec90 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.639+0000 7f4c6e7fc700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4c64038580 0x7f4c6403aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:17.641 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.639+0000 7f4c6e7fc700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4c64038580 0x7f4c6403aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006fd0 tx=0x7f4c60006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:17.644 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.643+0000 7f4c7c9eb700 1 -- 172.21.15.104:0/3553354982 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4c6802a540 con 0x7f4c780fec90 2024-04-26T04:35:17.922 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.921+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f4c78106fd0 con 0x7f4c780fec90 2024-04-26T04:35:17.923 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.922+0000 7f4c7c9eb700 1 -- 172.21.15.104:0/3553354982 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f4c6802a360 con 0x7f4c780fec90 2024-04-26T04:35:17.923 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:17.923 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:35:17.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4c64038580 msgr2=0x7f4c6403aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:17.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4c64038580 0x7f4c6403aa30 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f4c60006fd0 tx=0x7f4c60006e40 comp rx=0 tx=0).stop 2024-04-26T04:35:17.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 msgr2=0x7f4c78195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:17.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c78195340 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4c68005b40 tx=0x7f4c68004c80 comp rx=0 tx=0).stop 2024-04-26T04:35:17.925 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 shutdown_connections 2024-04-26T04:35:17.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4c64038580 0x7f4c6403aa30 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:17.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 --2- 172.21.15.104:0/3553354982 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c780fec90 0x7f4c78195340 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:17.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 >> 172.21.15.104:0/3553354982 conn(0x7f4c78075750 msgr2=0x7f4c781045f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:17.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 shutdown_connections 2024-04-26T04:35:17.926 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:17.924+0000 7f4c7d9ed700 1 -- 172.21.15.104:0/3553354982 wait complete. 2024-04-26T04:35:17.926 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:35:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: Deploying daemon node-exporter.smithi104 on smithi104 2024-04-26T04:35:19.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/3553354982' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:19.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:19.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:35:19.503 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:35:19.504 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:35:19.707 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:22.345 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.344+0000 7f94c490d700 1 -- 172.21.15.104:0/1058532543 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 msgr2=0x7f94c01013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:22.345 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.344+0000 7f94c490d700 1 --2- 172.21.15.104:0/1058532543 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01013c0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f94b0009910 tx=0x7f94b0009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:22.346 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.344+0000 7f94c490d700 1 -- 172.21.15.104:0/1058532543 shutdown_connections 2024-04-26T04:35:22.346 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.344+0000 7f94c490d700 1 --2- 172.21.15.104:0/1058532543 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01013c0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:22.346 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.344+0000 7f94c490d700 1 -- 172.21.15.104:0/1058532543 >> 172.21.15.104:0/1058532543 conn(0x7f94c00fa850 msgr2=0x7f94c00fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:22.346 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.344+0000 7f94c490d700 1 -- 172.21.15.104:0/1058532543 shutdown_connections 2024-04-26T04:35:22.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.345+0000 7f94c490d700 1 -- 172.21.15.104:0/1058532543 wait complete. 2024-04-26T04:35:22.348 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.346+0000 7f94c490d700 1 Processor -- start 2024-04-26T04:35:22.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.346+0000 7f94c490d700 1 -- start start 2024-04-26T04:35:22.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.346+0000 7f94c490d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:22.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.346+0000 7f94c490d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c01958f0 con 0x7f94c0100ff0 2024-04-26T04:35:22.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.346+0000 7f94be59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:22.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.346+0000 7f94be59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:44728/0 (socket says 172.21.15.104:44728) 2024-04-26T04:35:22.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94be59c700 1 -- 172.21.15.104:0/1875532837 learned_addr learned my addr 172.21.15.104:0/1875532837 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:22.349 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94be59c700 1 -- 172.21.15.104:0/1875532837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94b00095f0 con 0x7f94c0100ff0 2024-04-26T04:35:22.350 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94be59c700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01953b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005b40 tx=0x7f94b0004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:22.350 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94af7fe700 1 -- 172.21.15.104:0/1875532837 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94b001d070 con 0x7f94c0100ff0 2024-04-26T04:35:22.350 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c0195af0 con 0x7f94c0100ff0 2024-04-26T04:35:22.350 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94af7fe700 1 -- 172.21.15.104:0/1875532837 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f94b00044d0 con 0x7f94c0100ff0 2024-04-26T04:35:22.350 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94af7fe700 1 -- 172.21.15.104:0/1875532837 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f94b0003930 con 0x7f94c0100ff0 2024-04-26T04:35:22.350 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.347+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c0195eb0 con 0x7f94c0100ff0 2024-04-26T04:35:22.351 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.348+0000 7f94af7fe700 1 -- 172.21.15.104:0/1875532837 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f94b0003bf0 con 0x7f94c0100ff0 2024-04-26T04:35:22.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.348+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c0062380 con 0x7f94c0100ff0 2024-04-26T04:35:22.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.348+0000 7f94af7fe700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f94a8038580 0x7f94a803aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:22.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.348+0000 7f94af7fe700 1 -- 172.21.15.104:0/1875532837 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f94b004c4b0 con 0x7f94c0100ff0 2024-04-26T04:35:22.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.349+0000 7f94bdd9b700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f94a8038580 0x7f94a803aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:22.352 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.349+0000 7f94bdd9b700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f94a8038580 0x7f94a803aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:22.354 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.353+0000 7f94af7fe700 1 -- 172.21.15.104:0/1875532837 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f94b002a950 con 0x7f94c0100ff0 2024-04-26T04:35:22.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.658+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f94c0106fb0 con 0x7f94c0100ff0 2024-04-26T04:35:22.660 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.659+0000 7f94af7fe700 1 -- 172.21.15.104:0/1875532837 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f94b002a360 con 0x7f94c0100ff0 2024-04-26T04:35:22.660 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:22.660 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:35:22.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f94a8038580 msgr2=0x7f94a803aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:22.662 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f94a8038580 0x7f94a803aa30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f94b4006fd0 tx=0x7f94b4006e40 comp rx=0 tx=0).stop 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 msgr2=0x7f94c01953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01953b0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f94b0005b40 tx=0x7f94b0004c80 comp rx=0 tx=0).stop 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 shutdown_connections 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f94a8038580 0x7f94a803aa30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 --2- 172.21.15.104:0/1875532837 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f94c0100ff0 0x7f94c01953b0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 >> 172.21.15.104:0/1875532837 conn(0x7f94c00fa850 msgr2=0x7f94c0104640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 shutdown_connections 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:22.661+0000 7f94c490d700 1 -- 172.21.15.104:0/1875532837 wait complete. 2024-04-26T04:35:22.663 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:35:23.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:23.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:23.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:23.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:23.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:35:23.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-26T04:35:23.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:35:23.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:23.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: Deploying daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:35:23.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:23 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/1875532837' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:24.700 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:35:24.700 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:35:24.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:24 smithi090 ceph-mon[26648]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:24.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:24 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:25.084 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:35:26.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:26.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: Deploying daemon mon.smithi104 on smithi104 2024-04-26T04:35:26.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:26 smithi090 ceph-mon[26648]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.543+0000 7f7b7e867700 1 -- 172.21.15.104:0/4126922437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 msgr2=0x7f7b780ff250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.543+0000 7f7b7e867700 1 --2- 172.21.15.104:0/4126922437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b780ff250 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7b68009910 tx=0x7f7b68009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:26.545 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.543+0000 7f7b7e867700 1 -- 172.21.15.104:0/4126922437 shutdown_connections 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.543+0000 7f7b7e867700 1 --2- 172.21.15.104:0/4126922437 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b780ff250 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.543+0000 7f7b7e867700 1 -- 172.21.15.104:0/4126922437 >> 172.21.15.104:0/4126922437 conn(0x7f7b78075550 msgr2=0x7f7b78075950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.543+0000 7f7b7e867700 1 -- 172.21.15.104:0/4126922437 shutdown_connections 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.543+0000 7f7b7e867700 1 -- 172.21.15.104:0/4126922437 wait complete. 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.544+0000 7f7b7e867700 1 Processor -- start 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.544+0000 7f7b7e867700 1 -- start start 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.544+0000 7f7b7e867700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b78100060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.544+0000 7f7b7e867700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b78101fe0 con 0x7f7b780fee80 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.544+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b78100060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.544+0000 7f7b77fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b78100060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.104:59076/0 (socket says 172.21.15.104:59076) 2024-04-26T04:35:26.546 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.544+0000 7f7b77fff700 1 -- 172.21.15.104:0/3041869248 learned_addr learned my addr 172.21.15.104:0/3041869248 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.545+0000 7f7b77fff700 1 -- 172.21.15.104:0/3041869248 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b680095f0 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.545+0000 7f7b77fff700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b78100060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b68005b40 tx=0x7f7b68004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.545+0000 7f7b757fa700 1 -- 172.21.15.104:0/3041869248 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b6801d070 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.545+0000 7f7b757fa700 1 -- 172.21.15.104:0/3041869248 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7b680044d0 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.545+0000 7f7b757fa700 1 -- 172.21.15.104:0/3041869248 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 229+0+0 (secure 0 0 0) 0x7f7b68003930 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.545+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b781005a0 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.545+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b78100a20 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.546+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b78107900 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.546+0000 7f7b757fa700 1 -- 172.21.15.104:0/3041869248 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 17) v1 ==== 45197+0+0 (secure 0 0 0) 0x7f7b68003bf0 con 0x7f7b780fee80 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.547+0000 7f7b757fa700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b60038580 0x7f7b6003aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:26.548 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.547+0000 7f7b757fa700 1 -- 172.21.15.104:0/3041869248 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f7b6804c400 con 0x7f7b780fee80 2024-04-26T04:35:26.549 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.547+0000 7f7b777fe700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b60038580 0x7f7b6003aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:26.549 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.547+0000 7f7b777fe700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b60038580 0x7f7b6003aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c006fd0 tx=0x7f7b6c006e40 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:26.553 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.552+0000 7f7b757fa700 1 -- 172.21.15.104:0/3041869248 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7b6802a540 con 0x7f7b780fee80 2024-04-26T04:35:26.833 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.832+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f7b78100cc0 con 0x7f7b780fee80 2024-04-26T04:35:26.834 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.833+0000 7f7b757fa700 1 -- 172.21.15.104:0/3041869248 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+744 (secure 0 0 0) 0x7f7b6802a360 con 0x7f7b780fee80 2024-04-26T04:35:26.834 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:26.834 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":1,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:32:53.473219Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-26T04:35:26.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b60038580 msgr2=0x7f7b6003aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:26.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b60038580 0x7f7b6003aa30 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f7b6c006fd0 tx=0x7f7b6c006e40 comp rx=0 tx=0).stop 2024-04-26T04:35:26.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 msgr2=0x7f7b78100060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:26.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b78100060 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7b68005b40 tx=0x7f7b68004c80 comp rx=0 tx=0).stop 2024-04-26T04:35:26.836 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 shutdown_connections 2024-04-26T04:35:26.837 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b60038580 0x7f7b6003aa30 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:26.837 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 --2- 172.21.15.104:0/3041869248 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b780fee80 0x7f7b78100060 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:26.837 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 >> 172.21.15.104:0/3041869248 conn(0x7f7b78075550 msgr2=0x7f7b78104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:26.837 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 shutdown_connections 2024-04-26T04:35:26.837 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:26.835+0000 7f7b7e867700 1 -- 172.21.15.104:0/3041869248 wait complete. 2024-04-26T04:35:26.837 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 1 2024-04-26T04:35:27.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:27 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/3041869248' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:28.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:28 smithi090 ceph-mon[26648]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:28.970 INFO:tasks.cephadm:Waiting for 2 mons in monmap... 2024-04-26T04:35:28.970 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-26T04:35:29.194 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:35:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:29 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:30.623 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:30 smithi090 ceph-mon[26648]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: mon.smithi090 calling monitor election 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/4228609861' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:35.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: mon.smithi104 calling monitor election 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: mon.smithi090 is new leader, mons smithi090,smithi104 in quorum (ranks 0,1) 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: monmap e2: 2 mons at {smithi090=[v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0],smithi104=[v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0]} removed_ranks: {} 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: fsmap 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: osdmap e5: 0 total, 0 up, 0 in 2024-04-26T04:35:35.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: mgrmap e17: smithi090.iwrvwa(active, since 32s) 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: Standby manager daemon smithi104.wwoewi started 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/4228609861' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: overall HEALTH_OK 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/4228609861' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/4228609861' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:35.724 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:35 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:35.821 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.819+0000 7f17d09bf700 1 -- 172.21.15.104:0/699572948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17b40057d0 msgr2=0x7f17b4007c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:35.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.819+0000 7f17d09bf700 1 --2- 172.21.15.104:0/699572948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17b40057d0 0x7f17b4007c50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f17cc0fee70 tx=0x7f17bc00fe60 comp rx=0 tx=0).stop 2024-04-26T04:35:35.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.819+0000 7f17d09bf700 1 -- 172.21.15.104:0/699572948 shutdown_connections 2024-04-26T04:35:35.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.819+0000 7f17d09bf700 1 --2- 172.21.15.104:0/699572948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17b40057d0 0x7f17b4007c50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:35.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.819+0000 7f17d09bf700 1 -- 172.21.15.104:0/699572948 >> 172.21.15.104:0/699572948 conn(0x7f17cc074c10 msgr2=0x7f17cc075010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:35.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.819+0000 7f17d09bf700 1 -- 172.21.15.104:0/699572948 shutdown_connections 2024-04-26T04:35:35.822 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.819+0000 7f17d09bf700 1 -- 172.21.15.104:0/699572948 wait complete. 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17d09bf700 1 Processor -- start 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17d09bf700 1 -- start start 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17d09bf700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 0x7f17cc197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17d09bf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17cc197c50 0x7f17cc19b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17d09bf700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc198260 con 0x7f17cc197c50 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17d09bf700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17cc1983a0 con 0x7f17b40057d0 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17ca59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 0x7f17cc197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:35.825 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17ca59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 0x7f17cc197710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:49950/0 (socket says 172.21.15.104:49950) 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17ca59c700 1 -- 172.21.15.104:0/2087399904 learned_addr learned my addr 172.21.15.104:0/2087399904 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17ca59c700 1 -- 172.21.15.104:0/2087399904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 msgr2=0x7f17cc197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17ca59c700 1 -- 172.21.15.104:0/2087399904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 msgr2=0x7f17cc197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.820+0000 7f17ca59c700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 0x7f17cc197710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17c9d9b700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17cc197c50 0x7f17cc19b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17ca59c700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 0x7f17cc197710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17c9d9b700 1 -- 172.21.15.104:0/2087399904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 msgr2=0x7f17cc197710 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17c9d9b700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 0x7f17cc197710 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17c9d9b700 1 -- 172.21.15.104:0/2087399904 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17bc00f8b0 con 0x7f17cc197c50 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17c9d9b700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17cc197c50 0x7f17cc19b010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b4d0 tx=0x7f17c000b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:35.826 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17bb7fe700 1 -- 172.21.15.104:0/2087399904 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17c0011800 con 0x7f17cc197c50 2024-04-26T04:35:35.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17bb7fe700 1 -- 172.21.15.104:0/2087399904 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f17c0011e10 con 0x7f17cc197c50 2024-04-26T04:35:35.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17cc19b610 con 0x7f17cc197c50 2024-04-26T04:35:35.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17bb7fe700 1 -- 172.21.15.104:0/2087399904 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f17c000f2e0 con 0x7f17cc197c50 2024-04-26T04:35:35.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.821+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17cc19bb60 con 0x7f17cc197c50 2024-04-26T04:35:35.827 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.822+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17cc062380 con 0x7f17cc197c50 2024-04-26T04:35:35.828 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.823+0000 7f17bb7fe700 1 -- 172.21.15.104:0/2087399904 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f17c0010380 con 0x7f17cc197c50 2024-04-26T04:35:35.828 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.824+0000 7f17bb7fe700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f17ac06c750 0x7f17ac06ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:35.828 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.824+0000 7f17bb7fe700 1 -- 172.21.15.104:0/2087399904 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f17c008a3a0 con 0x7f17cc197c50 2024-04-26T04:35:35.828 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.824+0000 7f17ca59c700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f17ac06c750 0x7f17ac06ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:35.828 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.824+0000 7f17ca59c700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f17ac06c750 0x7f17ac06ec00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001eb0 tx=0x7f17bc005c20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:35.831 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:35.830+0000 7f17bb7fe700 1 -- 172.21.15.104:0/2087399904 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f17c0055b00 con 0x7f17cc197c50 2024-04-26T04:35:36.124 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.122+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f17cc105d40 con 0x7f17cc197c50 2024-04-26T04:35:36.125 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.123+0000 7f17bb7fe700 1 -- 172.21.15.104:0/2087399904 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1022 (secure 0 0 0) 0x7f17c0015410 con 0x7f17cc197c50 2024-04-26T04:35:36.125 INFO:teuthology.orchestra.run.smithi104.stdout: 2024-04-26T04:35:36.125 INFO:teuthology.orchestra.run.smithi104.stdout:{"epoch":2,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","modified":"2024-04-26T04:35:30.367475Z","created":"2024-04-26T04:32:53.473219Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"smithi090","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:3300","nonce":0},{"type":"v1","addr":"172.21.15.90:6789","nonce":0}]},"addr":"172.21.15.90:6789/0","public_addr":"172.21.15.90:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"smithi104","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:3300","nonce":0},{"type":"v1","addr":"172.21.15.104:6789","nonce":0}]},"addr":"172.21.15.104:6789/0","public_addr":"172.21.15.104:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f17ac06c750 msgr2=0x7f17ac06ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f17ac06c750 0x7f17ac06ec00 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f17bc001eb0 tx=0x7f17bc005c20 comp rx=0 tx=0).stop 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17cc197c50 msgr2=0x7f17cc19b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17cc197c50 0x7f17cc19b010 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f17c000b4d0 tx=0x7f17c000b890 comp rx=0 tx=0).stop 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 shutdown_connections 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f17ac06c750 0x7f17ac06ec00 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f17b40057d0 0x7f17cc197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:36.130 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 --2- 172.21.15.104:0/2087399904 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f17cc197c50 0x7f17cc19b010 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:36.131 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 >> 172.21.15.104:0/2087399904 conn(0x7f17cc074c10 msgr2=0x7f17cc100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:36.131 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 shutdown_connections 2024-04-26T04:35:36.131 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:35:36.125+0000 7f17d09bf700 1 -- 172.21.15.104:0/2087399904 wait complete. 2024-04-26T04:35:36.131 INFO:teuthology.orchestra.run.smithi104.stderr:dumped monmap epoch 2 2024-04-26T04:35:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:36 smithi090 ceph-mon[26648]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:36 smithi090 ceph-mon[26648]: mgrmap e18: smithi090.iwrvwa(active, since 32s), standbys: smithi104.wwoewi 2024-04-26T04:35:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:35:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:35:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:36 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/2087399904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:36.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:36 smithi104 ceph-mon[32369]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:36 smithi104 ceph-mon[32369]: mgrmap e18: smithi090.iwrvwa(active, since 32s), standbys: smithi104.wwoewi 2024-04-26T04:35:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:35:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:35:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:36 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/2087399904' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-26T04:35:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:35:36.866 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-26T04:35:36.866 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-26T04:35:37.080 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:35:38.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:38 smithi090 ceph-mon[26648]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:38.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:38 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:38.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:38 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:38.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:38 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:38.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:38 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:35:38.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:38 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.740+0000 7f3a14aaa700 1 -- 172.21.15.90:0/2505504515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 msgr2=0x7f3a100ff060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.740+0000 7f3a14aaa700 1 --2- 172.21.15.90:0/2505504515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a100ff060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3a00009910 tx=0x7f3a00009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.740+0000 7f3a14aaa700 1 -- 172.21.15.90:0/2505504515 shutdown_connections 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.740+0000 7f3a14aaa700 1 --2- 172.21.15.90:0/2505504515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a100ff060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.740+0000 7f3a14aaa700 1 -- 172.21.15.90:0/2505504515 >> 172.21.15.90:0/2505504515 conn(0x7f3a10075750 msgr2=0x7f3a10075b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.740+0000 7f3a14aaa700 1 -- 172.21.15.90:0/2505504515 shutdown_connections 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.740+0000 7f3a14aaa700 1 -- 172.21.15.90:0/2505504515 wait complete. 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a14aaa700 1 Processor -- start 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a14aaa700 1 -- start start 2024-04-26T04:35:38.744 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a14aaa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a10195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a14aaa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a101959b0 0x7f3a10199e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a14aaa700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10195fc0 con 0x7f3a100fec90 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a14aaa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a10196130 con 0x7f3a101959b0 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a0e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a10195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a0e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a10195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43792/0 (socket says 172.21.15.90:43792) 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.741+0000 7f3a0e59c700 1 -- 172.21.15.90:0/61092071 learned_addr learned my addr 172.21.15.90:0/61092071 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f3a0e59c700 1 -- 172.21.15.90:0/61092071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a101959b0 msgr2=0x7f3a10199e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f3a0e59c700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a101959b0 0x7f3a10199e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f3a0e59c700 1 -- 172.21.15.90:0/61092071 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a000095f0 con 0x7f3a100fec90 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f3a0e59c700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a10195470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000bcf0 tx=0x7f3a0000bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f39ff7fe700 1 -- 172.21.15.90:0/61092071 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a0001d070 con 0x7f3a100fec90 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f39ff7fe700 1 -- 172.21.15.90:0/61092071 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3a00005140 con 0x7f3a100fec90 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1019a3c0 con 0x7f3a100fec90 2024-04-26T04:35:38.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f39ff7fe700 1 -- 172.21.15.90:0/61092071 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a00017440 con 0x7f3a100fec90 2024-04-26T04:35:38.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.742+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1019a990 con 0x7f3a100fec90 2024-04-26T04:35:38.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.743+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a10107550 con 0x7f3a100fec90 2024-04-26T04:35:38.746 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.743+0000 7f39ff7fe700 1 -- 172.21.15.90:0/61092071 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f3a0000f6f0 con 0x7f3a100fec90 2024-04-26T04:35:38.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.744+0000 7f39ff7fe700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f39f806c760 0x7f39f806ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:38.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.744+0000 7f39ff7fe700 1 -- 172.21.15.90:0/61092071 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f3a0008bae0 con 0x7f3a100fec90 2024-04-26T04:35:38.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.744+0000 7f3a0dd9b700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f39f806c760 0x7f39f806ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:38.747 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.744+0000 7f3a0dd9b700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f39f806c760 0x7f39f806ec10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a040098e0 tx=0x7f3a04006dd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:38.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.748+0000 7f39ff7fe700 1 -- 172.21.15.90:0/61092071 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3a000572a0 con 0x7f3a100fec90 2024-04-26T04:35:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:38 smithi104 ceph-mon[32369]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:38 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:38 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:38 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:38.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:38 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:35:38.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:38 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:38.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.949+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f3a10196890 con 0x7f3a100fec90 2024-04-26T04:35:38.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.949+0000 7f39ff7fe700 1 -- 172.21.15.90:0/61092071 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v10) v1 ==== 76+0+225 (secure 0 0 0) 0x7f3a00026090 con 0x7f3a100fec90 2024-04-26T04:35:38.951 INFO:teuthology.orchestra.run.smithi090.stdout:# minimal ceph.conf for f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:35:38.951 INFO:teuthology.orchestra.run.smithi090.stdout:[global] 2024-04-26T04:35:38.951 INFO:teuthology.orchestra.run.smithi090.stdout: fsid = f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:35:38.951 INFO:teuthology.orchestra.run.smithi090.stdout: mon_host = [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] 2024-04-26T04:35:38.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f39f806c760 msgr2=0x7f39f806ec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:38.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f39f806c760 0x7f39f806ec10 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f3a040098e0 tx=0x7f3a04006dd0 comp rx=0 tx=0).stop 2024-04-26T04:35:38.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 msgr2=0x7f3a10195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:38.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a10195470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f3a0000bcf0 tx=0x7f3a0000bdd0 comp rx=0 tx=0).stop 2024-04-26T04:35:38.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 shutdown_connections 2024-04-26T04:35:38.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f39f806c760 0x7f39f806ec10 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:38.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a100fec90 0x7f3a10195470 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:38.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 --2- 172.21.15.90:0/61092071 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a101959b0 0x7f3a10199e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:38.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 >> 172.21.15.90:0/61092071 conn(0x7f3a10075750 msgr2=0x7f3a10104690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:38.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 shutdown_connections 2024-04-26T04:35:38.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:38.952+0000 7f3a14aaa700 1 -- 172.21.15.90:0/61092071 wait complete. 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.780 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: Reconfiguring mon.smithi090 (unknown last config time)... 2024-04-26T04:35:39.781 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:35:39.781 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:35:39.781 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:39.781 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-26T04:35:39.781 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:39 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/61092071' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:39.805 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-26T04:35:39.806 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:35:39.806 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T04:35:39.833 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:35:39.833 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:35:39.898 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:35:39.898 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.conf 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:39.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: Reconfiguring mon.smithi090 (unknown last config time)... 2024-04-26T04:35:39.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:35:39.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:35:39.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:39.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-26T04:35:39.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:39 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/61092071' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:39.929 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:35:39.929 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:35:39.996 INFO:tasks.cephadm:Deploying OSDs... 2024-04-26T04:35:39.997 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:35:39.997 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:35:40.015 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:35:40.016 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_1 2024-04-26T04:35:40.070 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:35:40.070 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:40.070 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-04-26T04:35:40.070 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:40.070 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:40.071 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 04:34:54.941692981 +0000 2024-04-26T04:35:40.071 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 04:29:40.132516882 +0000 2024-04-26T04:35:40.071 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 04:29:40.132516882 +0000 2024-04-26T04:35:40.071 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 04:29:40.132516882 +0000 2024-04-26T04:35:40.071 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:35:40.134 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T04:35:40.134 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T04:35:40.134 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000189451 s, 2.7 MB/s 2024-04-26T04:35:40.135 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:35:40.191 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_2 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 04:34:54.942692965 +0000 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 04:29:40.475511359 +0000 2024-04-26T04:35:40.246 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 04:29:40.475511359 +0000 2024-04-26T04:35:40.247 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 04:29:40.475511359 +0000 2024-04-26T04:35:40.247 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:35:40.308 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T04:35:40.308 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T04:35:40.308 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000217959 s, 2.3 MB/s 2024-04-26T04:35:40.309 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:35:40.364 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_3 2024-04-26T04:35:40.418 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:35:40.418 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:40.418 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T04:35:40.418 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:40.419 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:40.419 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 04:34:54.942692965 +0000 2024-04-26T04:35:40.419 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 04:29:40.817505853 +0000 2024-04-26T04:35:40.419 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 04:29:40.817505853 +0000 2024-04-26T04:35:40.419 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 04:29:40.817505853 +0000 2024-04-26T04:35:40.419 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:35:40.480 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T04:35:40.481 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T04:35:40.481 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000187689 s, 2.7 MB/s 2024-04-26T04:35:40.482 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:35:40.537 DEBUG:teuthology.orchestra.run.smithi090:> stat /dev/vg_nvme/lv_4 2024-04-26T04:35:40.591 INFO:teuthology.orchestra.run.smithi090.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 2024-04-26 04:34:54.942692965 +0000 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 04:29:41.139500669 +0000 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 04:29:41.139500669 +0000 2024-04-26T04:35:40.592 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 04:29:41.139500669 +0000 2024-04-26T04:35:40.592 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:35:40.654 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records in 2024-04-26T04:35:40.654 INFO:teuthology.orchestra.run.smithi090.stderr:1+0 records out 2024-04-26T04:35:40.654 INFO:teuthology.orchestra.run.smithi090.stderr:512 bytes copied, 0.000210902 s, 2.4 MB/s 2024-04-26T04:35:40.655 DEBUG:teuthology.orchestra.run.smithi090:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:35:40.711 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:35:40.711 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/scratch_devs of=/dev/stdout 2024-04-26T04:35:40.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:40 smithi090 ceph-mon[26648]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:40.729 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-26T04:35:40.730 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_1 2024-04-26T04:35:40.788 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-26 04:35:29.332982637 +0000 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-26 04:29:40.135302086 +0000 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-26 04:29:40.135302086 +0000 2024-04-26T04:35:40.789 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-26 04:29:40.135302086 +0000 2024-04-26T04:35:40.789 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-26T04:35:40.852 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:40 smithi104 ceph-mon[32369]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:40.854 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-26T04:35:40.855 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-26T04:35:40.855 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000221081 s, 2.3 MB/s 2024-04-26T04:35:40.857 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-26T04:35:40.916 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_2 2024-04-26T04:35:40.972 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-26T04:35:40.972 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:40.973 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-26T04:35:40.973 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:40.973 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:40.973 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-26 04:35:29.333982627 +0000 2024-04-26T04:35:40.973 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-26 04:29:40.479298398 +0000 2024-04-26T04:35:40.973 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-26 04:29:40.479298398 +0000 2024-04-26T04:35:40.973 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-26 04:29:40.479298398 +0000 2024-04-26T04:35:40.973 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-26T04:35:41.037 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-26T04:35:41.037 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-26T04:35:41.037 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000206235 s, 2.5 MB/s 2024-04-26T04:35:41.038 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-26T04:35:41.095 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_3 2024-04-26T04:35:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-26T04:35:41.150 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:41.150 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-26T04:35:41.150 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:41.150 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:41.150 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-26 04:35:29.333982627 +0000 2024-04-26T04:35:41.150 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-26 04:29:40.824294695 +0000 2024-04-26T04:35:41.151 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-26 04:29:40.824294695 +0000 2024-04-26T04:35:41.151 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-26 04:29:40.824294695 +0000 2024-04-26T04:35:41.151 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-26T04:35:41.213 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-26T04:35:41.213 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-26T04:35:41.213 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.00021443 s, 2.4 MB/s 2024-04-26T04:35:41.214 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-26T04:35:41.270 DEBUG:teuthology.orchestra.run.smithi104:> stat /dev/vg_nvme/lv_4 2024-04-26T04:35:41.325 INFO:teuthology.orchestra.run.smithi104.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-26T04:35:41.325 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-26T04:35:41.325 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-26T04:35:41.325 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:35:41.325 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:device_t:s0 2024-04-26T04:35:41.325 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 2024-04-26 04:35:29.333982627 +0000 2024-04-26T04:35:41.326 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-26 04:29:41.164291046 +0000 2024-04-26T04:35:41.326 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-26 04:29:41.164291046 +0000 2024-04-26T04:35:41.326 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-26 04:29:41.164291046 +0000 2024-04-26T04:35:41.326 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-26T04:35:41.388 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records in 2024-04-26T04:35:41.388 INFO:teuthology.orchestra.run.smithi104.stderr:1+0 records out 2024-04-26T04:35:41.389 INFO:teuthology.orchestra.run.smithi104.stderr:512 bytes copied, 0.000208053 s, 2.5 MB/s 2024-04-26T04:35:41.390 DEBUG:teuthology.orchestra.run.smithi104:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-26T04:35:41.445 INFO:tasks.cephadm:Deploying osd.0 on smithi090 with /dev/vg_nvme/lv_4... 2024-04-26T04:35:41.446 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: Reconfiguring mgr.smithi090.iwrvwa (unknown last config time)... 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: Reconfiguring daemon mgr.smithi090.iwrvwa on smithi090 2024-04-26T04:35:42.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:41 smithi090 ceph-mon[26648]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:42.164 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:35:42.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:42.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:42.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: Reconfiguring mgr.smithi090.iwrvwa (unknown last config time)... 2024-04-26T04:35:42.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:35:42.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:35:42.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:42.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: Reconfiguring daemon mgr.smithi090.iwrvwa on smithi090 2024-04-26T04:35:42.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:41 smithi104 ceph-mon[32369]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:43.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:43.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:43.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:43 smithi090 ceph-mon[26648]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-26T04:35:43.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:35:43.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:43.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:43 smithi090 ceph-mon[26648]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-26T04:35:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:43 smithi104 ceph-mon[32369]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-26T04:35:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:35:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:43.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:43 smithi104 ceph-mon[32369]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-26T04:35:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:44 smithi090 ceph-mon[26648]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:44 smithi104 ceph-mon[32369]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:45.341 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T04:35:45.388 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T04:35:45.388 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-26T04:35:45.388 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-26T04:35:45.388 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0119857 s, 875 MB/s 2024-04-26T04:35:45.388 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-26T04:35:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:45 smithi090 ceph-mon[26648]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-26T04:35:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:35:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:45 smithi090 ceph-mon[26648]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-26T04:35:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:45 smithi090 ceph-mon[26648]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:45 smithi104 ceph-mon[32369]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-26T04:35:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:35:46.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:46.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:45 smithi104 ceph-mon[32369]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-26T04:35:46.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:45 smithi104 ceph-mon[32369]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:46.793 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_4 2024-04-26T04:35:47.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:47 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:47.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:47 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:47.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:47 smithi090 ceph-mon[26648]: Reconfiguring alertmanager.smithi090 (dependencies changed)... 2024-04-26T04:35:47.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:47 smithi090 ceph-mon[26648]: Reconfiguring daemon alertmanager.smithi090 on smithi090 2024-04-26T04:35:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:47 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:47 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:47 smithi104 ceph-mon[32369]: Reconfiguring alertmanager.smithi090 (dependencies changed)... 2024-04-26T04:35:47.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:47 smithi104 ceph-mon[32369]: Reconfiguring daemon alertmanager.smithi090 on smithi090 2024-04-26T04:35:48.662 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:48 smithi090 ceph-mon[26648]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:48.663 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:35:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:48 smithi104 ceph-mon[32369]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:35:49.437 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:35:50.663 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:50 smithi090 ceph-mon[26648]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:50 smithi104 ceph-mon[32369]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 -- 172.21.15.90:0/1344094002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 msgr2=0x7f4994100860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 --2- 172.21.15.90:0/1344094002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994100860 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f4984009910 tx=0x7f4984009c20 comp rx=0 tx=0).stop 2024-04-26T04:35:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 -- 172.21.15.90:0/1344094002 shutdown_connections 2024-04-26T04:35:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 --2- 172.21.15.90:0/1344094002 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994100860 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 --2- 172.21.15.90:0/1344094002 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49940ff150 0x7f49940ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 -- 172.21.15.90:0/1344094002 >> 172.21.15.90:0/1344094002 conn(0x7f49940fa7a0 msgr2=0x7f49940fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:35:50.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 -- 172.21.15.90:0/1344094002 shutdown_connections 2024-04-26T04:35:50.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.916+0000 7f499a47a700 1 -- 172.21.15.90:0/1344094002 wait complete. 2024-04-26T04:35:50.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f499a47a700 1 Processor -- start 2024-04-26T04:35:50.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f499a47a700 1 -- start start 2024-04-26T04:35:50.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f499a47a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49940ff150 0x7f4994195e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f499a47a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f499a47a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49941969a0 con 0x7f49941003f0 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f499a47a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4994196ae0 con 0x7f49940ff150 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f49937fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f49937fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994196380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56174/0 (socket says 172.21.15.90:56174) 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.917+0000 7f49937fe700 1 -- 172.21.15.90:0/2378427918 learned_addr learned my addr 172.21.15.90:0/2378427918 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f4993fff700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49940ff150 0x7f4994195e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:50.921 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f49937fe700 1 -- 172.21.15.90:0/2378427918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49940ff150 msgr2=0x7f4994195e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f49937fe700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49940ff150 0x7f4994195e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f49937fe700 1 -- 172.21.15.90:0/2378427918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49840095f0 con 0x7f49941003f0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f4993fff700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49940ff150 0x7f4994195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f49937fe700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994196380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f498400b5c0 tx=0x7f498400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f49917fa700 1 -- 172.21.15.90:0/2378427918 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f498401d070 con 0x7f49941003f0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f49917fa700 1 -- 172.21.15.90:0/2378427918 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4984022470 con 0x7f49941003f0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f49917fa700 1 -- 172.21.15.90:0/2378427918 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f498400f690 con 0x7f49941003f0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499419b530 con 0x7f49941003f0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.918+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499419ba20 con 0x7f49941003f0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.919+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4994068ab0 con 0x7f49941003f0 2024-04-26T04:35:50.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.920+0000 7f49917fa700 1 -- 172.21.15.90:0/2378427918 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f4984004b70 con 0x7f49941003f0 2024-04-26T04:35:50.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.920+0000 7f49917fa700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f498006c610 0x7f498006eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:35:50.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.920+0000 7f49917fa700 1 -- 172.21.15.90:0/2378427918 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1370+0+0 (secure 0 0 0) 0x7f498408be00 con 0x7f49941003f0 2024-04-26T04:35:50.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.920+0000 7f4993fff700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f498006c610 0x7f498006eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:35:50.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.920+0000 7f4993fff700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f498006c610 0x7f498006eac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f497c007900 tx=0x7f497c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:35:50.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:50.925+0000 7f49917fa700 1 -- 172.21.15.90:0/2378427918 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f498405b120 con 0x7f49941003f0 2024-04-26T04:35:51.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:35:51.134+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f49940694e0 con 0x7f498006c610 2024-04-26T04:35:51.663 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:51 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:35:51.663 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:51 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:35:51.663 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:51 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:51 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:35:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:51 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:35:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:51 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:35:52.670 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:52 smithi090 ceph-mon[26648]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:35:52.670 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:52 smithi090 ceph-mon[26648]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:52 smithi104 ceph-mon[32369]: from='client.14250 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:35:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:52 smithi104 ceph-mon[32369]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:54.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:54 smithi090 ceph-mon[26648]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:54.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:54 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:54.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:54 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:54.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:54 smithi090 ceph-mon[26648]: Reconfiguring grafana.smithi090 (dependencies changed)... 2024-04-26T04:35:54.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:54 smithi090 ceph-mon[26648]: Reconfiguring daemon grafana.smithi090 on smithi090 2024-04-26T04:35:55.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:54 smithi104 ceph-mon[32369]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:55.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:54 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:55.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:54 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:35:55.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:54 smithi104 ceph-mon[32369]: Reconfiguring grafana.smithi090 (dependencies changed)... 2024-04-26T04:35:55.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:54 smithi104 ceph-mon[32369]: Reconfiguring daemon grafana.smithi090 on smithi090 2024-04-26T04:35:55.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:55 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/305367148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57491e88-0d1c-41ac-a1f5-3662e5c0a467"}]: dispatch 2024-04-26T04:35:55.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:55 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/305367148' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57491e88-0d1c-41ac-a1f5-3662e5c0a467"}]': finished 2024-04-26T04:35:55.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:55 smithi090 ceph-mon[26648]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T04:35:55.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:35:55.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:55 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/522324953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:35:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:55 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/305367148' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "57491e88-0d1c-41ac-a1f5-3662e5c0a467"}]: dispatch 2024-04-26T04:35:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:55 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/305367148' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "57491e88-0d1c-41ac-a1f5-3662e5c0a467"}]': finished 2024-04-26T04:35:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:55 smithi104 ceph-mon[32369]: osdmap e6: 1 total, 0 up, 1 in 2024-04-26T04:35:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:35:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:55 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/522324953' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:35:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:56 smithi090 ceph-mon[26648]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:57.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:56 smithi104 ceph-mon[32369]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:58.861 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:35:58 smithi104 ceph-mon[32369]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:35:58.963 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:35:58 smithi090 ceph-mon[26648]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:00.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:00 smithi090 ceph-mon[26648]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:00 smithi104 ceph-mon[32369]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:02.923 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:02 smithi090 ceph-mon[26648]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:03.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:02 smithi104 ceph-mon[32369]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:04.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:04 smithi090 ceph-mon[26648]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:04.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:04.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:04.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:04 smithi104 ceph-mon[32369]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:04.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:05.432 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:05 smithi090 ceph-mon[26648]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-26T04:36:05.432 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:05 smithi090 ceph-mon[26648]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-26T04:36:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:05 smithi104 ceph-mon[32369]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-26T04:36:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:05 smithi104 ceph-mon[32369]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-26T04:36:06.412 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:06 smithi104 ceph-mon[32369]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:06.413 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:06 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:36:06.413 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:06 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:06.413 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:06 smithi104 ceph-mon[32369]: Deploying daemon osd.0 on smithi090 2024-04-26T04:36:06.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:06 smithi090 ceph-mon[26648]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:06.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:06 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:36:06.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:06 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:06.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:06 smithi090 ceph-mon[26648]: Deploying daemon osd.0 on smithi090 2024-04-26T04:36:08.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:08 smithi090 ceph-mon[26648]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:08.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:08 smithi104 ceph-mon[32369]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:10.491 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:10 smithi104 ceph-mon[32369]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:10.491 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:10 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:10.491 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:10 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:10.491 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:10 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:36:10.491 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:10 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:10.592 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:10 smithi090 ceph-mon[26648]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:10.593 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:10.593 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:10.593 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:36:10.593 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:11 smithi090 ceph-mon[26648]: Reconfiguring ceph-exporter.smithi104 (monmap changed)... 2024-04-26T04:36:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:11 smithi090 ceph-mon[26648]: Reconfiguring daemon ceph-exporter.smithi104 on smithi104 2024-04-26T04:36:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:36:11.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:11 smithi104 ceph-mon[32369]: Reconfiguring ceph-exporter.smithi104 (monmap changed)... 2024-04-26T04:36:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:11 smithi104 ceph-mon[32369]: Reconfiguring daemon ceph-exporter.smithi104 on smithi104 2024-04-26T04:36:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:36:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:12 smithi090 ceph-mon[26648]: Reconfiguring crash.smithi104 (monmap changed)... 2024-04-26T04:36:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:12 smithi090 ceph-mon[26648]: Reconfiguring daemon crash.smithi104 on smithi104 2024-04-26T04:36:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:12 smithi090 ceph-mon[26648]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:12.561 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:12 smithi104 ceph-mon[32369]: Reconfiguring crash.smithi104 (monmap changed)... 2024-04-26T04:36:12.561 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:12 smithi104 ceph-mon[32369]: Reconfiguring daemon crash.smithi104 on smithi104 2024-04-26T04:36:12.561 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:12 smithi104 ceph-mon[32369]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:13.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:13 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:13.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:13 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:13.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:13 smithi090 ceph-mon[26648]: Reconfiguring mgr.smithi104.wwoewi (monmap changed)... 2024-04-26T04:36:13.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:13 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:36:13.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:13 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:36:13.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:13 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:13.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:13 smithi090 ceph-mon[26648]: Reconfiguring daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:36:13.880 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:13 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:13.880 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:13 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:13.881 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:13 smithi104 ceph-mon[32369]: Reconfiguring mgr.smithi104.wwoewi (monmap changed)... 2024-04-26T04:36:13.881 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:13 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:36:13.881 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:13 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:36:13.881 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:13 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:13.881 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:13 smithi104 ceph-mon[32369]: Reconfiguring daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:36:14.911 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:14 smithi090 ceph-mon[26648]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:14.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:14.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:14.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:36:14.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:36:14.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:14 smithi104 ceph-mon[32369]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:36:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:36:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:15.904 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: Reconfiguring mon.smithi104 (monmap changed)... 2024-04-26T04:36:15.904 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: Reconfiguring daemon mon.smithi104 on smithi104 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090:9093"}]: dispatch 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090:3000"}]: dispatch 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:36:15.905 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:36:15.906 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.906 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:15.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: Reconfiguring mon.smithi104 (monmap changed)... 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: Reconfiguring daemon mon.smithi104 on smithi104 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090:9093"}]: dispatch 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090:3000"}]: dispatch 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:36:15.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:36:15.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:15.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:16.689 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:16 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:36:16.689 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:16 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090:9093"}]: dispatch 2024-04-26T04:36:16.689 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:16 smithi090 ceph-mon[26648]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:16.689 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:16 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:36:16.689 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:16 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090:3000"}]: dispatch 2024-04-26T04:36:16.689 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:16 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:36:16.689 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:16 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:36:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:16 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:36:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:16 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi090:9093"}]: dispatch 2024-04-26T04:36:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:16 smithi104 ceph-mon[32369]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:16 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:36:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:16 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi090:3000"}]: dispatch 2024-04-26T04:36:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:16 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:36:17.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:16 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:36:17.861 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 0 on host 'smithi090' 2024-04-26T04:36:17.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.859+0000 7f49917fa700 1 -- 172.21.15.90:0/2378427918 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f49940694e0 con 0x7f498006c610 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f498006c610 msgr2=0x7f498006eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f498006c610 0x7f498006eac0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f497c007900 tx=0x7f497c008040 comp rx=0 tx=0).stop 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 msgr2=0x7f4994196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994196380 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f498400b5c0 tx=0x7f498400be20 comp rx=0 tx=0).stop 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 shutdown_connections 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f498006c610 0x7f498006eac0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f49940ff150 0x7f4994195e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 --2- 172.21.15.90:0/2378427918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f49941003f0 0x7f4994196380 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 >> 172.21.15.90:0/2378427918 conn(0x7f49940fa7a0 msgr2=0x7f4994068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 shutdown_connections 2024-04-26T04:36:17.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:17.861+0000 7f499a47a700 1 -- 172.21.15.90:0/2378427918 wait complete. 2024-04-26T04:36:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:18 smithi090 ceph-mon[26648]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:18 smithi104 ceph-mon[32369]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:18.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:18.796 DEBUG:teuthology.orchestra.run.smithi090:osd.0> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.0.service 2024-04-26T04:36:18.799 INFO:tasks.cephadm:Deploying osd.1 on smithi090 with /dev/vg_nvme/lv_3... 2024-04-26T04:36:18.799 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T04:36:19.072 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:36:19.126 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:36:18 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[41569]: 2024-04-26T04:36:18.847+0000 7fdf64054640 -1 osd.0 0 log_to_monitors true 2024-04-26T04:36:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:19 smithi104 ceph-mon[32369]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:36:19.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:19.446 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:19 smithi090 ceph-mon[26648]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:36:20.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:36:20 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[41569]: 2024-04-26T04:36:20.129+0000 7fdf58eca700 -1 osd.0 0 waiting for initial osdmap 2024-04-26T04:36:20.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:36:20 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[41569]: 2024-04-26T04:36:20.133+0000 7fdf554c0700 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:36:20.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:20 smithi090 ceph-mon[26648]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:36:20.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:20 smithi090 ceph-mon[26648]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T04:36:20.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:20 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:20.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:20 smithi090 ceph-mon[26648]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:36:20.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:20 smithi090 ceph-mon[26648]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:20 smithi104 ceph-mon[32369]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:36:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:20 smithi104 ceph-mon[32369]: osdmap e7: 1 total, 0 up, 1 in 2024-04-26T04:36:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:20 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:20 smithi104 ceph-mon[32369]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:36:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:20 smithi104 ceph-mon[32369]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:21.424 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:21 smithi090 ceph-mon[26648]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T04:36:21.424 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:21 smithi090 ceph-mon[26648]: osdmap e8: 1 total, 0 up, 1 in 2024-04-26T04:36:21.424 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:21.424 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:21.424 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:21.495 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T04:36:21.495 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T04:36:21.495 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-26T04:36:21.495 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-26T04:36:21.495 INFO:teuthology.orchestra.run.smithi090.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0124931 s, 839 MB/s 2024-04-26T04:36:21.496 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-26T04:36:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:21 smithi104 ceph-mon[32369]: from='osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T04:36:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:21 smithi104 ceph-mon[32369]: osdmap e8: 1 total, 0 up, 1 in 2024-04-26T04:36:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:21.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:22.116 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_3 2024-04-26T04:36:22.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:22 smithi090 ceph-mon[26648]: purged_snaps scrub starts 2024-04-26T04:36:22.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:22 smithi090 ceph-mon[26648]: purged_snaps scrub ok 2024-04-26T04:36:22.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:22 smithi090 ceph-mon[26648]: osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] boot 2024-04-26T04:36:22.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:22 smithi090 ceph-mon[26648]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T04:36:22.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:22.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:22 smithi090 ceph-mon[26648]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:22 smithi104 ceph-mon[32369]: purged_snaps scrub starts 2024-04-26T04:36:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:22 smithi104 ceph-mon[32369]: purged_snaps scrub ok 2024-04-26T04:36:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:22 smithi104 ceph-mon[32369]: osd.0 [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] boot 2024-04-26T04:36:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:22 smithi104 ceph-mon[32369]: osdmap e9: 1 total, 1 up, 1 in 2024-04-26T04:36:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:36:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:22 smithi104 ceph-mon[32369]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-26T04:36:22.553 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:36:23.432 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:23 smithi090 ceph-mon[26648]: osdmap e10: 1 total, 1 up, 1 in 2024-04-26T04:36:23.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:23 smithi104 ceph-mon[32369]: osdmap e10: 1 total, 1 up, 1 in 2024-04-26T04:36:24.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:24 smithi090 ceph-mon[26648]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:24.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:24 smithi104 ceph-mon[32369]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:24.807 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.803+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/1096813049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 msgr2=0x7f8b680ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.803+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/1096813049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b680ff550 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f8b58009910 tx=0x7f8b58009c20 comp rx=0 tx=0).stop 2024-04-26T04:36:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.803+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/1096813049 shutdown_connections 2024-04-26T04:36:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.803+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/1096813049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b68100340 0x7f8b68100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.803+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/1096813049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b680ff550 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.803+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/1096813049 >> 172.21.15.90:0/1096813049 conn(0x7f8b680fa6d0 msgr2=0x7f8b680fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:36:24.808 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.803+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/1096813049 shutdown_connections 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/1096813049 wait complete. 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b6dd4f700 1 Processor -- start 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b6dd4f700 1 -- start start 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b6dd4f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b68069950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b6dd4f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b68100340 0x7f8b6819a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b6dd4f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819ae80 con 0x7f8b680ff140 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b6dd4f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6819afc0 con 0x7f8b68100340 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b68069950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b677fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b68069950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45628/0 (socket says 172.21.15.90:45628) 2024-04-26T04:36:24.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b677fe700 1 -- 172.21.15.90:0/2121981232 learned_addr learned my addr 172.21.15.90:0/2121981232 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.804+0000 7f8b5ffff700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b68100340 0x7f8b6819a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b677fe700 1 -- 172.21.15.90:0/2121981232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b68100340 msgr2=0x7f8b6819a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b677fe700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b68100340 0x7f8b6819a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b677fe700 1 -- 172.21.15.90:0/2121981232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b580095f0 con 0x7f8b680ff140 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b5ffff700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b68100340 0x7f8b6819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b677fe700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b68069950 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800bda0 tx=0x7f8b5800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b657fa700 1 -- 172.21.15.90:0/2121981232 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5801d070 con 0x7f8b680ff140 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b657fa700 1 -- 172.21.15.90:0/2121981232 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8b58004c50 con 0x7f8b680ff140 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6819b240 con 0x7f8b680ff140 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b657fa700 1 -- 172.21.15.90:0/2121981232 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8b5800f690 con 0x7f8b680ff140 2024-04-26T04:36:24.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.805+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6819b730 con 0x7f8b680ff140 2024-04-26T04:36:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.806+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b68107d10 con 0x7f8b680ff140 2024-04-26T04:36:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.806+0000 7f8b657fa700 1 -- 172.21.15.90:0/2121981232 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f8b58022a20 con 0x7f8b680ff140 2024-04-26T04:36:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.807+0000 7f8b657fa700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8b5406c600 0x7f8b5406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:36:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.807+0000 7f8b657fa700 1 -- 172.21.15.90:0/2121981232 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1982+0+0 (secure 0 0 0) 0x7f8b5808c1d0 con 0x7f8b680ff140 2024-04-26T04:36:24.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.807+0000 7f8b5ffff700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8b5406c600 0x7f8b5406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:36:24.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.807+0000 7f8b5ffff700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8b5406c600 0x7f8b5406eab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b68069ed0 tx=0x7f8b50008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:36:24.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:24.811+0000 7f8b657fa700 1 -- 172.21.15.90:0/2121981232 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8b5805b1c0 con 0x7f8b680ff140 2024-04-26T04:36:25.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:25.013+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8b6819be30 con 0x7f8b5406c600 2024-04-26T04:36:25.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:25 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:36:25.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:25 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:36:25.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:25 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:25.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:25 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:36:25.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:25 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:36:25.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:25 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:26 smithi090 ceph-mon[26648]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:36:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:26.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:26 smithi090 ceph-mon[26648]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:26.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:26 smithi104 ceph-mon[32369]: from='client.14268 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:36:26.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:26.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:26.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:26 smithi104 ceph-mon[32369]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:28.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:28 smithi090 ceph-mon[26648]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:28.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:28 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1714782356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe237f1b-a589-452c-8e5e-98a9a31593eb"}]: dispatch 2024-04-26T04:36:28.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:28 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1714782356' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe237f1b-a589-452c-8e5e-98a9a31593eb"}]': finished 2024-04-26T04:36:28.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:28 smithi090 ceph-mon[26648]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T04:36:28.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:28 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:28.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:28 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2162582441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:36:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:28 smithi104 ceph-mon[32369]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:28 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1714782356' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "fe237f1b-a589-452c-8e5e-98a9a31593eb"}]: dispatch 2024-04-26T04:36:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:28 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1714782356' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "fe237f1b-a589-452c-8e5e-98a9a31593eb"}]': finished 2024-04-26T04:36:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:28 smithi104 ceph-mon[32369]: osdmap e11: 2 total, 1 up, 2 in 2024-04-26T04:36:28.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:28 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:28.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:28 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2162582441' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:36:30.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:30 smithi090 ceph-mon[26648]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:30.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:30 smithi104 ceph-mon[32369]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:32.464 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:32 smithi090 ceph-mon[26648]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:32.768 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:32 smithi104 ceph-mon[32369]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:34.193 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:36:34.194 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:34.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:36:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:34.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:36:34.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:35.468 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:35 smithi104 ceph-mon[32369]: Detected new or changed devices on smithi090 2024-04-26T04:36:35.469 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:35 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi090 to 11840M 2024-04-26T04:36:35.469 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:35 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:35.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:35 smithi090 ceph-mon[26648]: Detected new or changed devices on smithi090 2024-04-26T04:36:35.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:35 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi090 to 11840M 2024-04-26T04:36:35.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:36.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:36 smithi090 ceph-mon[26648]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:36.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:36.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:36 smithi104 ceph-mon[32369]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:37 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:36:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:37 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:37 smithi090 ceph-mon[26648]: Deploying daemon osd.1 on smithi090 2024-04-26T04:36:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:37 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:36:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:37 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:37 smithi104 ceph-mon[32369]: Deploying daemon osd.1 on smithi090 2024-04-26T04:36:38.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:38 smithi090 ceph-mon[26648]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:38 smithi104 ceph-mon[32369]: pgmap v46: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:40.364 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:40 smithi090 ceph-mon[26648]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:40.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:40 smithi104 ceph-mon[32369]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:42.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:42.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:42.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:42.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:36:42.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:42.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:41 smithi090 ceph-mon[26648]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:36:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:41 smithi104 ceph-mon[32369]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:44.896 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:44 smithi090 ceph-mon[26648]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:44.896 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:44.896 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:44.897 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:45.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:44 smithi104 ceph-mon[32369]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:45.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:45.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:46.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:46 smithi090 ceph-mon[26648]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:46.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:46 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:46.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:46 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:46 smithi104 ceph-mon[32369]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:46 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:46.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:46 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:47.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.574+0000 7f8b657fa700 1 -- 172.21.15.90:0/2121981232 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8b6819be30 con 0x7f8b5406c600 2024-04-26T04:36:47.576 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 1 on host 'smithi090' 2024-04-26T04:36:47.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8b5406c600 msgr2=0x7f8b5406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:47.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8b5406c600 0x7f8b5406eab0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f8b68069ed0 tx=0x7f8b50008040 comp rx=0 tx=0).stop 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 msgr2=0x7f8b68069950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b68069950 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800bda0 tx=0x7f8b5800be80 comp rx=0 tx=0).stop 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 shutdown_connections 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8b5406c600 0x7f8b5406eab0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8b680ff140 0x7f8b68069950 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 --2- 172.21.15.90:0/2121981232 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8b68100340 0x7f8b6819a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 >> 172.21.15.90:0/2121981232 conn(0x7f8b680fa6d0 msgr2=0x7f8b68103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 shutdown_connections 2024-04-26T04:36:47.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:47.577+0000 7f8b6dd4f700 1 -- 172.21.15.90:0/2121981232 wait complete. 2024-04-26T04:36:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:48 smithi090 ceph-mon[26648]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:48 smithi090 ceph-mon[26648]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:36:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:48.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:48 smithi104 ceph-mon[32369]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:48 smithi104 ceph-mon[32369]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:36:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:36:48.813 DEBUG:teuthology.orchestra.run.smithi090:osd.1> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.1.service 2024-04-26T04:36:48.815 INFO:tasks.cephadm:Deploying osd.2 on smithi090 with /dev/vg_nvme/lv_2... 2024-04-26T04:36:48.816 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T04:36:49.578 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:36:49.699 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:49 smithi090 ceph-mon[26648]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:36:49.699 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:49 smithi090 ceph-mon[26648]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T04:36:49.699 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:49 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:49.699 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:49 smithi090 ceph-mon[26648]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:36:49.700 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:36:49 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[47139]: 2024-04-26T04:36:49.418+0000 7f4c2292c700 -1 osd.1 0 waiting for initial osdmap 2024-04-26T04:36:49.700 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:36:49 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[47139]: 2024-04-26T04:36:49.421+0000 7f4c1bf1c700 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:36:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:49 smithi104 ceph-mon[32369]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:36:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:49 smithi104 ceph-mon[32369]: osdmap e12: 2 total, 1 up, 2 in 2024-04-26T04:36:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:49 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:49 smithi104 ceph-mon[32369]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:36:50.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:50 smithi090 ceph-mon[26648]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:50.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:50 smithi090 ceph-mon[26648]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T04:36:50.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:50 smithi090 ceph-mon[26648]: osdmap e13: 2 total, 1 up, 2 in 2024-04-26T04:36:50.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:50.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:50 smithi104 ceph-mon[32369]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-26T04:36:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:50 smithi104 ceph-mon[32369]: from='osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T04:36:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:50 smithi104 ceph-mon[32369]: osdmap e13: 2 total, 1 up, 2 in 2024-04-26T04:36:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:51.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:51 smithi090 ceph-mon[26648]: purged_snaps scrub starts 2024-04-26T04:36:51.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:51 smithi090 ceph-mon[26648]: purged_snaps scrub ok 2024-04-26T04:36:51.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:51 smithi090 ceph-mon[26648]: osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] boot 2024-04-26T04:36:51.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:51 smithi090 ceph-mon[26648]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T04:36:51.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:51 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:51.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:51 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:51.486 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:51 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:51 smithi104 ceph-mon[32369]: purged_snaps scrub starts 2024-04-26T04:36:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:51 smithi104 ceph-mon[32369]: purged_snaps scrub ok 2024-04-26T04:36:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:51 smithi104 ceph-mon[32369]: osd.1 [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] boot 2024-04-26T04:36:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:51 smithi104 ceph-mon[32369]: osdmap e14: 2 total, 2 up, 2 in 2024-04-26T04:36:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:51 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:36:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:51 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:51.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:51 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:52.424 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T04:36:52.424 INFO:teuthology.orchestra.run.smithi090.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T04:36:52.424 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10+0 records in 2024-04-26T04:36:52.424 INFO:teuthology.orchestra.run.smithi090.stderr:10+0 records out 2024-04-26T04:36:52.424 INFO:teuthology.orchestra.run.smithi090.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.012488 s, 840 MB/s 2024-04-26T04:36:52.424 INFO:teuthology.orchestra.run.smithi090.stderr:--> Zapping successful for: 2024-04-26T04:36:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:52 smithi090 ceph-mon[26648]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:52 smithi090 ceph-mon[26648]: osdmap e15: 2 total, 2 up, 2 in 2024-04-26T04:36:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:52 smithi104 ceph-mon[32369]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:52 smithi104 ceph-mon[32369]: osdmap e15: 2 total, 2 up, 2 in 2024-04-26T04:36:54.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:54 smithi090 ceph-mon[26648]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:54 smithi104 ceph-mon[32369]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:54.640 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi090:vg_nvme/lv_2 2024-04-26T04:36:54.864 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:36:55.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:55.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:55.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:36:55.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:36:55.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi090 to 5920M 2024-04-26T04:36:55.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:55.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:55.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:36:55.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:55.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:36:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:36:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi090 to 5920M 2024-04-26T04:36:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:36:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:36:56.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 -- 172.21.15.90:0/3361443642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80ffb60 msgr2=0x7f90b80fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:56.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 --2- 172.21.15.90:0/3361443642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80ffb60 0x7f90b80fffd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f90a8009910 tx=0x7f90a8009c20 comp rx=0 tx=0).stop 2024-04-26T04:36:56.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 -- 172.21.15.90:0/3361443642 shutdown_connections 2024-04-26T04:36:56.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 --2- 172.21.15.90:0/3361443642 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80ffb60 0x7f90b80fffd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:56.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 --2- 172.21.15.90:0/3361443642 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b80fe8c0 0x7f90b80fecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 -- 172.21.15.90:0/3361443642 >> 172.21.15.90:0/3361443642 conn(0x7f90b80f9f10 msgr2=0x7f90b80fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:36:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 -- 172.21.15.90:0/3361443642 shutdown_connections 2024-04-26T04:36:56.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 -- 172.21.15.90:0/3361443642 wait complete. 2024-04-26T04:36:56.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 Processor -- start 2024-04-26T04:36:56.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.399+0000 7f90bf983700 1 -- start start 2024-04-26T04:36:56.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bf983700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80fe8c0 0x7f90b8197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:36:56.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bf983700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b80ffb60 0x7f90b81981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bf983700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b81987f0 con 0x7f90b80fe8c0 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bf983700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b8198960 con 0x7f90b80ffb60 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bd71f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80fe8c0 0x7f90b8197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bd71f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80fe8c0 0x7f90b8197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51090/0 (socket says 172.21.15.90:51090) 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bd71f700 1 -- 172.21.15.90:0/1380070239 learned_addr learned my addr 172.21.15.90:0/1380070239 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bcf1e700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b80ffb60 0x7f90b81981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bd71f700 1 -- 172.21.15.90:0/1380070239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b80ffb60 msgr2=0x7f90b81981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bd71f700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b80ffb60 0x7f90b81981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bd71f700 1 -- 172.21.15.90:0/1380070239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a80095f0 con 0x7f90b80fe8c0 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bd71f700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80fe8c0 0x7f90b8197cb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f90b400dd40 tx=0x7f90b400a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90ae7fc700 1 -- 172.21.15.90:0/1380070239 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90b400ad10 con 0x7f90b80fe8c0 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b819d3f0 con 0x7f90b80fe8c0 2024-04-26T04:36:56.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90ae7fc700 1 -- 172.21.15.90:0/1380070239 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f90b400ae70 con 0x7f90b80fe8c0 2024-04-26T04:36:56.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.400+0000 7f90ae7fc700 1 -- 172.21.15.90:0/1380070239 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f90b40168d0 con 0x7f90b80fe8c0 2024-04-26T04:36:56.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.401+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b819d940 con 0x7f90b80fe8c0 2024-04-26T04:36:56.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.402+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b8106b00 con 0x7f90b80fe8c0 2024-04-26T04:36:56.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.402+0000 7f90ae7fc700 1 -- 172.21.15.90:0/1380070239 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f90b400e450 con 0x7f90b80fe8c0 2024-04-26T04:36:56.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.402+0000 7f90ae7fc700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f90a406c600 0x7f90a406eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:36:56.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.402+0000 7f90ae7fc700 1 -- 172.21.15.90:0/1380070239 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(15..15 src has 1..15) v4 ==== 2414+0+0 (secure 0 0 0) 0x7f90b408aa80 con 0x7f90b80fe8c0 2024-04-26T04:36:56.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.403+0000 7f90bcf1e700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f90a406c600 0x7f90a406eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:36:56.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.403+0000 7f90bcf1e700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f90a406c600 0x7f90a406eab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90b81991c0 tx=0x7f90a800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:36:56.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.407+0000 7f90ae7fc700 1 -- 172.21.15.90:0/1380070239 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f90b4050720 con 0x7f90b80fe8c0 2024-04-26T04:36:56.625 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:36:56.623+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f90b81044b0 con 0x7f90a406c600 2024-04-26T04:36:56.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:56 smithi090 ceph-mon[26648]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:56 smithi104 ceph-mon[32369]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:57.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:57 smithi090 ceph-mon[26648]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:36:57.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:57 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:36:57.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:57 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:36:57.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:57 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:57.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:57 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:57.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:57 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:57 smithi104 ceph-mon[32369]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi090:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:36:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:57 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:36:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:57 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:36:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:57 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:36:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:57 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:57 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:36:58.937 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:58 smithi104 ceph-mon[32369]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:58.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:58 smithi090 ceph-mon[26648]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:36:59.852 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:59 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2095040937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2"}]: dispatch 2024-04-26T04:36:59.852 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:59 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2095040937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2"}]': finished 2024-04-26T04:36:59.852 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:59 smithi090 ceph-mon[26648]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T04:36:59.852 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:36:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:59 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2095040937' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2"}]: dispatch 2024-04-26T04:37:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:59 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2095040937' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2"}]': finished 2024-04-26T04:37:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:59 smithi104 ceph-mon[32369]: osdmap e16: 3 total, 2 up, 3 in 2024-04-26T04:37:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:36:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:00.951 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:00 smithi090 ceph-mon[26648]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:00.951 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:00 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1463232261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:37:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:00 smithi104 ceph-mon[32369]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:00 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1463232261' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:37:02.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:02 smithi090 ceph-mon[26648]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:02.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:02 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:02.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:02 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:02.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:02 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:02.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:02 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:02.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:02 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:03.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:02 smithi104 ceph-mon[32369]: pgmap v63: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:03.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:02 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:03.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:02 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:03.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:02 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:03.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:02 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:03.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:02 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:04.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:04 smithi104 ceph-mon[32369]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:04.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:04 smithi090 ceph-mon[26648]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:06.653 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:06 smithi090 ceph-mon[26648]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:06.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:06 smithi104 ceph-mon[32369]: pgmap v65: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:07.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:07 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:37:07.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:07 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:07.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:07 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:37:07.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:07 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:08.628 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:08 smithi090 ceph-mon[26648]: Deploying daemon osd.2 on smithi090 2024-04-26T04:37:08.628 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:08 smithi090 ceph-mon[26648]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:08.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:08 smithi104 ceph-mon[32369]: Deploying daemon osd.2 on smithi090 2024-04-26T04:37:08.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:08 smithi104 ceph-mon[32369]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:10.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:10 smithi090 ceph-mon[26648]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:10.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:10 smithi104 ceph-mon[32369]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:12.657 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:12 smithi090 ceph-mon[26648]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:12.657 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:12 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:12.657 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:12 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:12.657 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:12 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:12 smithi104 ceph-mon[32369]: pgmap v68: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:12 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:12 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:12 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:14.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:14 smithi104 ceph-mon[32369]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:14.392 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:14 smithi090 ceph-mon[26648]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:15.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.275+0000 7f90ae7fc700 1 -- 172.21.15.90:0/1380070239 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f90b81044b0 con 0x7f90a406c600 2024-04-26T04:37:15.277 INFO:teuthology.orchestra.run.smithi090.stdout:Created osd(s) 2 on host 'smithi090' 2024-04-26T04:37:15.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f90a406c600 msgr2=0x7f90a406eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:15.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f90a406c600 0x7f90a406eab0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f90b81991c0 tx=0x7f90a800b540 comp rx=0 tx=0).stop 2024-04-26T04:37:15.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80fe8c0 msgr2=0x7f90b8197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:15.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80fe8c0 0x7f90b8197cb0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f90b400dd40 tx=0x7f90b400a530 comp rx=0 tx=0).stop 2024-04-26T04:37:15.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 shutdown_connections 2024-04-26T04:37:15.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f90a406c600 0x7f90a406eab0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:15.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f90b80fe8c0 0x7f90b8197cb0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:15.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 --2- 172.21.15.90:0/1380070239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f90b80ffb60 0x7f90b81981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:15.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 >> 172.21.15.90:0/1380070239 conn(0x7f90b80f9f10 msgr2=0x7f90b8102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:37:15.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 shutdown_connections 2024-04-26T04:37:15.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:37:15.278+0000 7f90bf983700 1 -- 172.21.15.90:0/1380070239 wait complete. 2024-04-26T04:37:15.897 DEBUG:teuthology.orchestra.run.smithi090:osd.2> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.2.service 2024-04-26T04:37:15.899 INFO:tasks.cephadm:Deploying osd.3 on smithi104 with /dev/vg_nvme/lv_4... 2024-04-26T04:37:15.900 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-26T04:37:16.110 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:37:16.162 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:16 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[52647]: 2024-04-26T04:37:16.062+0000 7f36a31e2640 -1 osd.2 0 log_to_monitors true 2024-04-26T04:37:16.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:16 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:16.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:16 smithi090 ceph-mon[26648]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:16.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:16 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:16.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:16 smithi090 ceph-mon[26648]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:37:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:16 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:16 smithi104 ceph-mon[32369]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:16 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:16 smithi104 ceph-mon[32369]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:37:17.380 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:17 smithi104 ceph-mon[32369]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:37:17.381 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:17 smithi104 ceph-mon[32369]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T04:37:17.381 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:17 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:17.381 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:17 smithi104 ceph-mon[32369]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:37:17.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[52647]: 2024-04-26T04:37:17.267+0000 7f369985b700 -1 osd.2 0 waiting for initial osdmap 2024-04-26T04:37:17.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[52647]: 2024-04-26T04:37:17.271+0000 7f3690646700 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:37:17.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:17 smithi090 ceph-mon[26648]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:37:17.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:17 smithi090 ceph-mon[26648]: osdmap e17: 3 total, 2 up, 3 in 2024-04-26T04:37:17.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:17 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:17.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:17 smithi090 ceph-mon[26648]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:37:17.863 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-26T04:37:17.863 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-26T04:37:17.863 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-26T04:37:17.863 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-26T04:37:17.864 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132477 s, 792 MB/s 2024-04-26T04:37:17.864 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-26T04:37:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:18 smithi090 ceph-mon[26648]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:18 smithi090 ceph-mon[26648]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T04:37:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:18 smithi090 ceph-mon[26648]: osdmap e18: 3 total, 2 up, 3 in 2024-04-26T04:37:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:18.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:18 smithi104 ceph-mon[32369]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:18.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:18 smithi104 ceph-mon[32369]: from='osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]': finished 2024-04-26T04:37:18.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:18 smithi104 ceph-mon[32369]: osdmap e18: 3 total, 2 up, 3 in 2024-04-26T04:37:18.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:18.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:18.519 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_4 2024-04-26T04:37:18.721 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:37:19.403 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: purged_snaps scrub starts 2024-04-26T04:37:19.403 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: purged_snaps scrub ok 2024-04-26T04:37:19.403 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:19.403 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] boot 2024-04-26T04:37:19.403 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T04:37:19.404 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:19.404 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:19.404 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:19.404 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: purged_snaps scrub starts 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: purged_snaps scrub ok 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: osd.2 [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] boot 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: osdmap e19: 3 total, 3 up, 3 in 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:19.457 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:20.218 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/1245376643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 msgr2=0x7f6f08101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:20.218 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/1245376643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08101890 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f6efc009910 tx=0x7f6efc009c20 comp rx=0 tx=0).stop 2024-04-26T04:37:20.218 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/1245376643 shutdown_connections 2024-04-26T04:37:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/1245376643 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08101890 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/1245376643 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f08100240 0x7f6f08100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/1245376643 >> 172.21.15.104:0/1245376643 conn(0x7f6f080fb7f0 msgr2=0x7f6f080fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:37:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/1245376643 shutdown_connections 2024-04-26T04:37:20.219 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.217+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/1245376643 wait complete. 2024-04-26T04:37:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f0cfe0700 1 Processor -- start 2024-04-26T04:37:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f0cfe0700 1 -- start start 2024-04-26T04:37:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f0cfe0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f08100240 0x7f6f08074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:37:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f0cfe0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:37:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f0cfe0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f08073680 con 0x7f6f08100240 2024-04-26T04:37:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f0cfe0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f080737c0 con 0x7f6f08101440 2024-04-26T04:37:20.221 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f05d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f05d9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:44922/0 (socket says 172.21.15.104:44922) 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f05d9b700 1 -- 172.21.15.104:0/4078888891 learned_addr learned my addr 172.21.15.104:0/4078888891 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f0659c700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f08100240 0x7f6f08074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f05d9b700 1 -- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f08100240 msgr2=0x7f6f08074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f05d9b700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f08100240 0x7f6f08074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.218+0000 7f6f05d9b700 1 -- 172.21.15.104:0/4078888891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6efc0095f0 con 0x7f6f08101440 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.219+0000 7f6f05d9b700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08073140 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000c00 tx=0x7f6efc00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.219+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6efc01d070 con 0x7f6f08101440 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.219+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6efc005dc0 con 0x7f6f08101440 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.219+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f08073a40 con 0x7f6f08101440 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.219+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f08073f30 con 0x7f6f08101440 2024-04-26T04:37:20.222 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.219+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6efc0226a0 con 0x7f6f08101440 2024-04-26T04:37:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.220+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f08108d90 con 0x7f6f08101440 2024-04-26T04:37:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.220+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 18) v1 ==== 89858+0+0 (secure 0 0 0) 0x7f6efc022800 con 0x7f6f08101440 2024-04-26T04:37:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.221+0000 7f6ef77fe700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6ef006c600 0x7f6ef006eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:37:20.223 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.221+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 3221+0+0 (secure 0 0 0) 0x7f6efc08c2d0 con 0x7f6f08101440 2024-04-26T04:37:20.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.221+0000 7f6f0659c700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6ef006c600 0x7f6ef006eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:37:20.224 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.221+0000 7f6f0659c700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6ef006c600 0x7f6ef006eab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f081012a0 tx=0x7f6ef8009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:37:20.226 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.225+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6efc05aff0 con 0x7f6f08101440 2024-04-26T04:37:20.398 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:20 smithi090 ceph-mon[26648]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:20.398 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:20 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T04:37:20.399 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:20 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T04:37:20.399 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:20 smithi090 ceph-mon[26648]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T04:37:20.399 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:20 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T04:37:20.424 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:20.423+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f6f08105d90 con 0x7f6ef006c600 2024-04-26T04:37:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:20 smithi104 ceph-mon[32369]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-04-26T04:37:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:20 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-26T04:37:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:20 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-26T04:37:20.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:20 smithi104 ceph-mon[32369]: osdmap e20: 3 total, 3 up, 3 in 2024-04-26T04:37:20.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:20 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-26T04:37:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:21 smithi104 ceph-mon[32369]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:37:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:37:21.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:37:21.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:21.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T04:37:21.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:21 smithi104 ceph-mon[32369]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T04:37:21.516 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54910]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:37:21.516 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54910]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:21.516 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54910]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:21.516 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54910]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:21.516 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:21 smithi090 ceph-mon[26648]: from='client.24129 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:37:21.516 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:37:21.516 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:37:21.516 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:21.516 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-26T04:37:21.516 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:21 smithi090 ceph-mon[26648]: osdmap e21: 3 total, 3 up, 3 in 2024-04-26T04:37:21.775 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54913]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:37:21.775 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54913]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:21.775 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54913]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:21.775 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54913]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:21.786 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:21.785+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6efc0038f0 con 0x7f6f08101440 2024-04-26T04:37:22.058 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54949]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:37:22.058 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54949]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:22.058 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54949]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:22.058 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:21 smithi090 sudo[54949]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:22.321 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55018]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:37:22.321 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55018]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:22.321 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55018]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:22.321 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55018]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:22.583 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55109]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:37:22.583 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55109]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:22.583 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55109]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:22.583 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55109]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: Detected new or changed devices on smithi090 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi090 to 3946M 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:22.584 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:22.585 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:22.585 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T04:37:22.585 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: mgrmap e19: smithi090.iwrvwa(active, since 2m), standbys: smithi104.wwoewi 2024-04-26T04:37:22.585 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:22.837 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55131]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:37:22.837 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55131]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:22.837 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55131]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:22.838 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55131]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:23.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 79 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: Detected new or changed devices on smithi090 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi090 to 3946M 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:23.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: osdmap e22: 3 total, 3 up, 3 in 2024-04-26T04:37:23.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: mgrmap e19: smithi090.iwrvwa(active, since 2m), standbys: smithi104.wwoewi 2024-04-26T04:37:23.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:22 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:23.109 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55138]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:37:23.109 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55138]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:23.109 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55138]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:23.109 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:22 smithi090 sudo[55138]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:23.408 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 sudo[55236]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:37:23.409 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 sudo[55236]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:23.409 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 sudo[55236]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:23.409 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 sudo[55236]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:23.606 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36248]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-26T04:37:23.607 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36248]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:23.607 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36248]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:23.607 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36248]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:23.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:37:23.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/1719053986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62adc04c-9e7e-4ee5-bb5f-42004b2a615d"}]: dispatch 2024-04-26T04:37:23.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62adc04c-9e7e-4ee5-bb5f-42004b2a615d"}]: dispatch 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62adc04c-9e7e-4ee5-bb5f-42004b2a615d"}]': finished 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:37:23.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:23 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36251]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36251]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36251]: pam_unix(sudo:session): session opened for user root by (uid=0) 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 sudo[36251]: pam_unix(sudo:session): session closed for user root 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/1719053986' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62adc04c-9e7e-4ee5-bb5f-42004b2a615d"}]: dispatch 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:37:23.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:37:23.970 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "62adc04c-9e7e-4ee5-bb5f-42004b2a615d"}]: dispatch 2024-04-26T04:37:23.970 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:37:23.970 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "62adc04c-9e7e-4ee5-bb5f-42004b2a615d"}]': finished 2024-04-26T04:37:23.970 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: osdmap e23: 4 total, 3 up, 4 in 2024-04-26T04:37:23.970 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:23.970 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:37:23.970 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:23 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:37:24.860 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:24 smithi090 ceph-mon[26648]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:37:24.860 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:24 smithi090 ceph-mon[26648]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:24.860 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:24 smithi090 ceph-mon[26648]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:37:24.860 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:24 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:24.860 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:24 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/1631441564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:37:25.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:24 smithi104 ceph-mon[32369]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-26T04:37:25.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:24 smithi104 ceph-mon[32369]: pgmap v81: 1 pgs: 1 creating+peering; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:24 smithi104 ceph-mon[32369]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-26T04:37:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:24 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:24 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/1631441564' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:37:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:26 smithi090 ceph-mon[26648]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:26.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:26.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:26.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:26 smithi104 ceph-mon[32369]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 81 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:28.831 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:28 smithi104 ceph-mon[32369]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:28.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:28 smithi090 ceph-mon[26648]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:30 smithi090 ceph-mon[26648]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:30 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:37:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:30 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:30 smithi104 ceph-mon[32369]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:30 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:37:31.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:30 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:31 smithi090 ceph-mon[26648]: Deploying daemon osd.3 on smithi104 2024-04-26T04:37:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:31 smithi104 ceph-mon[32369]: Deploying daemon osd.3 on smithi104 2024-04-26T04:37:32.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:32 smithi090 ceph-mon[26648]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:33.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:32 smithi104 ceph-mon[32369]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:34 smithi104 ceph-mon[32369]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:34.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:34 smithi090 ceph-mon[26648]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:36.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:36 smithi104 ceph-mon[32369]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:36 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:36 smithi090 ceph-mon[26648]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:36 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:38.480 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.478+0000 7f6ef77fe700 1 -- 172.21.15.104:0/4078888891 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f6f08105d90 con 0x7f6ef006c600 2024-04-26T04:37:38.480 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 3 on host 'smithi104' 2024-04-26T04:37:38.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6ef006c600 msgr2=0x7f6ef006eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:38.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6ef006c600 0x7f6ef006eab0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6f081012a0 tx=0x7f6ef8009260 comp rx=0 tx=0).stop 2024-04-26T04:37:38.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 msgr2=0x7f6f08073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:38.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08073140 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f6efc000c00 tx=0x7f6efc00f690 comp rx=0 tx=0).stop 2024-04-26T04:37:38.482 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 shutdown_connections 2024-04-26T04:37:38.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6ef006c600 0x7f6ef006eab0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:38.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f08100240 0x7f6f08074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:38.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 --2- 172.21.15.104:0/4078888891 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f08101440 0x7f6f08073140 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:38.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 >> 172.21.15.104:0/4078888891 conn(0x7f6f080fb7f0 msgr2=0x7f6f08104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:37:38.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 shutdown_connections 2024-04-26T04:37:38.483 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:38.481+0000 7f6f0cfe0700 1 -- 172.21.15.104:0/4078888891 wait complete. 2024-04-26T04:37:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:38 smithi104 ceph-mon[32369]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:38 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:38 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:38.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:38 smithi090 ceph-mon[26648]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:38.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:38 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:38.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:38 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:39.710 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:39 smithi104 ceph-mon[32369]: from='osd.3 [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:37:39.710 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:39 smithi104 ceph-mon[32369]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:37:39.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:39 smithi090 ceph-mon[26648]: from='osd.3 [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:37:39.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:39 smithi090 ceph-mon[26648]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:37:39.735 DEBUG:teuthology.orchestra.run.smithi104:osd.3> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.3.service 2024-04-26T04:37:39.739 INFO:tasks.cephadm:Deploying osd.4 on smithi104 with /dev/vg_nvme/lv_3... 2024-04-26T04:37:39.739 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-26T04:37:39.980 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:37:40.740 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:40.740 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:37:40.741 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T04:37:40.741 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:40.741 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: from='osd.3 [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:37:40.741 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:37:40.741 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:40.741 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:40.741 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[38332]: 2024-04-26T04:37:40.531+0000 7ff7dcf87700 -1 osd.3 0 waiting for initial osdmap 2024-04-26T04:37:40.741 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:37:40 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[38332]: 2024-04-26T04:37:40.536+0000 7ff7d857b700 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:37:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:37:40.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: osdmap e24: 4 total, 3 up, 4 in 2024-04-26T04:37:40.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:40.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: from='osd.3 [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:37:40.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:37:40.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:40.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:40 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:41.629 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:41 smithi104 ceph-mon[32369]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-26T04:37:41.629 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:41 smithi104 ceph-mon[32369]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T04:37:41.629 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:41.629 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:41 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:41.822 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-26T04:37:41.822 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-26T04:37:41.822 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-26T04:37:41.822 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-26T04:37:41.822 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126362 s, 830 MB/s 2024-04-26T04:37:41.823 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-26T04:37:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:41 smithi090 ceph-mon[26648]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-26T04:37:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:41 smithi090 ceph-mon[26648]: osdmap e25: 4 total, 3 up, 4 in 2024-04-26T04:37:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:41.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:41 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:42.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:42 smithi104 ceph-mon[32369]: purged_snaps scrub starts 2024-04-26T04:37:42.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:42 smithi104 ceph-mon[32369]: purged_snaps scrub ok 2024-04-26T04:37:42.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:42 smithi104 ceph-mon[32369]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:42.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:42 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:42.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:42 smithi104 ceph-mon[32369]: osd.3 [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] boot 2024-04-26T04:37:42.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:42 smithi104 ceph-mon[32369]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T04:37:42.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:42 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:42.803 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_3 2024-04-26T04:37:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:42 smithi090 ceph-mon[26648]: purged_snaps scrub starts 2024-04-26T04:37:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:42 smithi090 ceph-mon[26648]: purged_snaps scrub ok 2024-04-26T04:37:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:42 smithi090 ceph-mon[26648]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 82 MiB used, 268 GiB / 268 GiB avail 2024-04-26T04:37:42.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:42 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:42.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:42 smithi090 ceph-mon[26648]: osd.3 [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] boot 2024-04-26T04:37:42.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:42 smithi090 ceph-mon[26648]: osdmap e26: 4 total, 4 up, 4 in 2024-04-26T04:37:42.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:42 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:37:43.244 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:37:43.750 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:43 smithi104 ceph-mon[32369]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T04:37:43.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:43 smithi090 ceph-mon[26648]: osdmap e27: 4 total, 4 up, 4 in 2024-04-26T04:37:44.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:44 smithi090 ceph-mon[26648]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:37:44.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:44 smithi090 ceph-mon[26648]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T04:37:44.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:44 smithi104 ceph-mon[32369]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:37:44.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:44 smithi104 ceph-mon[32369]: osdmap e28: 4 total, 4 up, 4 in 2024-04-26T04:37:45.692 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 -- 172.21.15.104:0/1234167725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950101440 msgr2=0x7fb950101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:45.692 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 --2- 172.21.15.104:0/1234167725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950101440 0x7fb950101890 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fb94c009910 tx=0x7fb94c009c20 comp rx=0 tx=0).stop 2024-04-26T04:37:45.692 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 -- 172.21.15.104:0/1234167725 shutdown_connections 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 --2- 172.21.15.104:0/1234167725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950101440 0x7fb950101890 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 --2- 172.21.15.104:0/1234167725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb950100240 0x7fb950100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 -- 172.21.15.104:0/1234167725 >> 172.21.15.104:0/1234167725 conn(0x7fb9500fb7f0 msgr2=0x7fb9500fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 -- 172.21.15.104:0/1234167725 shutdown_connections 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.689+0000 7fb9585f6700 1 -- 172.21.15.104:0/1234167725 wait complete. 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9585f6700 1 Processor -- start 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9585f6700 1 -- start start 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9585f6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950100240 0x7fb950195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9585f6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb950101440 0x7fb950195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9585f6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9501965e0 con 0x7fb950101440 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9585f6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb950196720 con 0x7fb950100240 2024-04-26T04:37:45.693 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb956392700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950100240 0x7fb950195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb956392700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950100240 0x7fb950195a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:36762/0 (socket says 172.21.15.104:36762) 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb956392700 1 -- 172.21.15.104:0/2008504839 learned_addr learned my addr 172.21.15.104:0/2008504839 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb955b91700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb950101440 0x7fb950195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb956392700 1 -- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb950101440 msgr2=0x7fb950195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb956392700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb950101440 0x7fb950195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb956392700 1 -- 172.21.15.104:0/2008504839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c0095f0 con 0x7fb950100240 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb956392700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950100240 0x7fb950195a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d900 tx=0x7fb94000dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9477fe700 1 -- 172.21.15.104:0/2008504839 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb9400041f0 con 0x7fb950100240 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb95019b1d0 con 0x7fb950100240 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.690+0000 7fb9477fe700 1 -- 172.21.15.104:0/2008504839 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb940004350 con 0x7fb950100240 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.691+0000 7fb9477fe700 1 -- 172.21.15.104:0/2008504839 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb940003d20 con 0x7fb950100240 2024-04-26T04:37:45.694 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.691+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb95019b720 con 0x7fb950100240 2024-04-26T04:37:45.695 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.692+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb950108d90 con 0x7fb950100240 2024-04-26T04:37:45.695 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.692+0000 7fb9477fe700 1 -- 172.21.15.104:0/2008504839 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb940009730 con 0x7fb950100240 2024-04-26T04:37:45.696 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.693+0000 7fb9477fe700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb93c06c680 0x7fb93c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:37:45.696 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.693+0000 7fb9477fe700 1 -- 172.21.15.104:0/2008504839 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3774+0+0 (secure 0 0 0) 0x7fb94008aff0 con 0x7fb950100240 2024-04-26T04:37:45.696 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.693+0000 7fb955b91700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb93c06c680 0x7fb93c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:37:45.696 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.693+0000 7fb955b91700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb93c06c680 0x7fb93c06eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006010 tx=0x7fb94c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:37:45.698 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.697+0000 7fb9477fe700 1 -- 172.21.15.104:0/2008504839 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb940059880 con 0x7fb950100240 2024-04-26T04:37:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T04:37:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:45.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:45 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.897 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:37:45.895+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb950105d90 con 0x7fb93c06c680 2024-04-26T04:37:45.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: osdmap e29: 4 total, 4 up, 4 in 2024-04-26T04:37:45.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:37:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:45 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:37:46.665 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:46 smithi104 ceph-mon[32369]: Detected new or changed devices on smithi104 2024-04-26T04:37:46.665 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:46 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi104 to 14912M 2024-04-26T04:37:46.665 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:46 smithi104 ceph-mon[32369]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:37:46.665 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:46 smithi104 ceph-mon[32369]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T04:37:46.665 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:46 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:37:46.665 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:46 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:37:46.665 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:46 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:46 smithi090 ceph-mon[26648]: Detected new or changed devices on smithi104 2024-04-26T04:37:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:46 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi104 to 14912M 2024-04-26T04:37:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:46 smithi090 ceph-mon[26648]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:37:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:46 smithi090 ceph-mon[26648]: osdmap e30: 4 total, 4 up, 4 in 2024-04-26T04:37:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:46 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:37:46.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:46 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:37:46.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:46 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:47 smithi090 ceph-mon[26648]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:37:48.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:47 smithi104 ceph-mon[32369]: from='client.24149 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:37:48.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:48 smithi090 ceph-mon[26648]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-26T04:37:48.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:48 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/1829282200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ef2729-84e1-4d2d-868b-3da5fefeb800"}]: dispatch 2024-04-26T04:37:48.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:48 smithi090 ceph-mon[26648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ef2729-84e1-4d2d-868b-3da5fefeb800"}]: dispatch 2024-04-26T04:37:48.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:48 smithi090 ceph-mon[26648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4ef2729-84e1-4d2d-868b-3da5fefeb800"}]': finished 2024-04-26T04:37:48.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:48 smithi090 ceph-mon[26648]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T04:37:48.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:37:48.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:48 smithi104 ceph-mon[32369]: pgmap v100: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 122 KiB/s, 0 objects/s recovering 2024-04-26T04:37:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:48 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/1829282200' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ef2729-84e1-4d2d-868b-3da5fefeb800"}]: dispatch 2024-04-26T04:37:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:48 smithi104 ceph-mon[32369]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a4ef2729-84e1-4d2d-868b-3da5fefeb800"}]: dispatch 2024-04-26T04:37:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:48 smithi104 ceph-mon[32369]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a4ef2729-84e1-4d2d-868b-3da5fefeb800"}]': finished 2024-04-26T04:37:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:48 smithi104 ceph-mon[32369]: osdmap e31: 5 total, 4 up, 5 in 2024-04-26T04:37:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:37:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:37:49.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:49 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/3489415316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:37:50.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:49 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/3489415316' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:37:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:50 smithi090 ceph-mon[26648]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T04:37:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:50 smithi104 ceph-mon[32369]: pgmap v102: 1 pgs: 1 peering; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 101 KiB/s, 0 objects/s recovering 2024-04-26T04:37:52.867 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:52 smithi104 ceph-mon[32369]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T04:37:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:52 smithi090 ceph-mon[26648]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 86 KiB/s, 0 objects/s recovering 2024-04-26T04:37:54.349 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:54 smithi104 ceph-mon[32369]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T04:37:54.397 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:54 smithi090 ceph-mon[26648]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-26T04:37:56.392 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:56 smithi104 ceph-mon[32369]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:37:56.392 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:37:56.392 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:56.392 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:56 smithi104 ceph-mon[32369]: Deploying daemon osd.4 on smithi104 2024-04-26T04:37:56.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:56 smithi090 ceph-mon[26648]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:37:56.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:37:56.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:37:56.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:56 smithi090 ceph-mon[26648]: Deploying daemon osd.4 on smithi104 2024-04-26T04:37:58.460 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:37:58 smithi104 ceph-mon[32369]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:37:58.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:37:58 smithi090 ceph-mon[26648]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:00.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:00 smithi104 ceph-mon[32369]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:00.622 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:00 smithi090 ceph-mon[26648]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:01.966 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:01 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:01.966 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:01 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:01.966 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:01 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:01 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:01 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:01 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:02.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:02 smithi090 ceph-mon[26648]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:03.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:02 smithi104 ceph-mon[32369]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:04.159 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.158+0000 7fb9477fe700 1 -- 172.21.15.104:0/2008504839 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb950105d90 con 0x7fb93c06c680 2024-04-26T04:38:04.159 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 4 on host 'smithi104' 2024-04-26T04:38:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb93c06c680 msgr2=0x7fb93c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb93c06c680 0x7fb93c06eb30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb94c006010 tx=0x7fb94c00b540 comp rx=0 tx=0).stop 2024-04-26T04:38:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950100240 msgr2=0x7fb950195a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950100240 0x7fb950195a80 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb94000d900 tx=0x7fb94000dc10 comp rx=0 tx=0).stop 2024-04-26T04:38:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 shutdown_connections 2024-04-26T04:38:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb93c06c680 0x7fb93c06eb30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:04.161 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb950100240 0x7fb950195a80 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:04.162 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 --2- 172.21.15.104:0/2008504839 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb950101440 0x7fb950195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:04.162 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 >> 172.21.15.104:0/2008504839 conn(0x7fb9500fb7f0 msgr2=0x7fb950104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:04.162 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 shutdown_connections 2024-04-26T04:38:04.162 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:04.160+0000 7fb9585f6700 1 -- 172.21.15.104:0/2008504839 wait complete. 2024-04-26T04:38:04.464 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:04 smithi104 ceph-mon[32369]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:04.464 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:04 smithi090 ceph-mon[26648]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:04.778 DEBUG:teuthology.orchestra.run.smithi104:osd.4> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.4.service 2024-04-26T04:38:04.780 INFO:tasks.cephadm:Deploying osd.5 on smithi104 with /dev/vg_nvme/lv_2... 2024-04-26T04:38:04.781 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-26T04:38:05.157 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:38:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:05 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:05 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:05 smithi090 ceph-mon[26648]: from='osd.4 [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:38:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:05 smithi090 ceph-mon[26648]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:38:05.486 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:05 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:05.486 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:05 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:05.486 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:05 smithi104 ceph-mon[32369]: from='osd.4 [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:38:05.486 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:05 smithi104 ceph-mon[32369]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:38:06.426 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-mon[32369]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:38:06.426 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-mon[32369]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T04:38:06.426 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:06.426 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-mon[32369]: from='osd.4 [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:06.426 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-mon[32369]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:06.426 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-mon[32369]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:06.426 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[43046]: 2024-04-26T04:38:06.168+0000 7f9e3200a700 -1 osd.4 0 waiting for initial osdmap 2024-04-26T04:38:06.427 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:38:06 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[43046]: 2024-04-26T04:38:06.175+0000 7f9e2b5fa700 -1 osd.4 33 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:38:06.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:06 smithi090 ceph-mon[26648]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:38:06.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:06 smithi090 ceph-mon[26648]: osdmap e32: 5 total, 4 up, 5 in 2024-04-26T04:38:06.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:06 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:06.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:06 smithi090 ceph-mon[26648]: from='osd.4 [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:06.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:06 smithi090 ceph-mon[26648]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:06.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:06 smithi090 ceph-mon[26648]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:06.853 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-26T04:38:06.853 INFO:teuthology.orchestra.run.smithi104.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-26T04:38:06.853 INFO:teuthology.orchestra.run.smithi104.stderr: stderr: 10+0 records in 2024-04-26T04:38:06.853 INFO:teuthology.orchestra.run.smithi104.stderr:10+0 records out 2024-04-26T04:38:06.853 INFO:teuthology.orchestra.run.smithi104.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129284 s, 811 MB/s 2024-04-26T04:38:06.853 INFO:teuthology.orchestra.run.smithi104.stderr:--> Zapping successful for: 2024-04-26T04:38:07.421 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:07 smithi104 ceph-mon[32369]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-26T04:38:07.421 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:07 smithi104 ceph-mon[32369]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T04:38:07.421 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:07 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:07.421 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:07 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:07.421 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:07 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:07.421 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:07 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:07.447 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph orch daemon add osd smithi104:vg_nvme/lv_2 2024-04-26T04:38:07.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:07 smithi090 ceph-mon[26648]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-26T04:38:07.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:07 smithi090 ceph-mon[26648]: osdmap e33: 5 total, 4 up, 5 in 2024-04-26T04:38:07.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:07 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:07.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:07 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:07.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:07 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:07.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:07 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:08.134 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:38:08.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:08 smithi090 ceph-mon[26648]: purged_snaps scrub starts 2024-04-26T04:38:08.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:08 smithi090 ceph-mon[26648]: purged_snaps scrub ok 2024-04-26T04:38:08.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:08 smithi090 ceph-mon[26648]: osd.4 [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] boot 2024-04-26T04:38:08.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:08 smithi090 ceph-mon[26648]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T04:38:08.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:08.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:08 smithi090 ceph-mon[26648]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:08.493 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:08 smithi104 ceph-mon[32369]: purged_snaps scrub starts 2024-04-26T04:38:08.493 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:08 smithi104 ceph-mon[32369]: purged_snaps scrub ok 2024-04-26T04:38:08.493 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:08 smithi104 ceph-mon[32369]: osd.4 [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] boot 2024-04-26T04:38:08.493 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:08 smithi104 ceph-mon[32369]: osdmap e34: 5 total, 5 up, 5 in 2024-04-26T04:38:08.494 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:08 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:38:08.494 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:08 smithi104 ceph-mon[32369]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 108 MiB used, 358 GiB / 358 GiB avail 2024-04-26T04:38:09.433 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:09 smithi104 ceph-mon[32369]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T04:38:09.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:09 smithi090 ceph-mon[26648]: osdmap e35: 5 total, 5 up, 5 in 2024-04-26T04:38:10.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:10 smithi090 ceph-mon[26648]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:10.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:10 smithi104 ceph-mon[32369]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- 172.21.15.104:0/698723185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 msgr2=0x7fcb040fe2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 --2- 172.21.15.104:0/698723185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb040fe2e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009910 tx=0x7fcaf0009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- 172.21.15.104:0/698723185 shutdown_connections 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 --2- 172.21.15.104:0/698723185 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb040fe2e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 --2- 172.21.15.104:0/698723185 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb04101840 0x7fcb04101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- 172.21.15.104:0/698723185 >> 172.21.15.104:0/698723185 conn(0x7fcb040f9e50 msgr2=0x7fcb040fc2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- 172.21.15.104:0/698723185 shutdown_connections 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- 172.21.15.104:0/698723185 wait complete. 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 Processor -- start 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- start start 2024-04-26T04:38:11.087 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb04101840 0x7fcb04195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb04196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04196640 con 0x7fcb04101840 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.085+0000 7fcb0b069700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb04196780 con 0x7fcb04102220 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb03fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb04196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb03fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb04196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:34714/0 (socket says 172.21.15.104:34714) 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb03fff700 1 -- 172.21.15.104:0/2228717700 learned_addr learned my addr 172.21.15.104:0/2228717700 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb08e05700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb04101840 0x7fcb04195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb03fff700 1 -- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb04101840 msgr2=0x7fcb04195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb03fff700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb04101840 0x7fcb04195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:11.088 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb03fff700 1 -- 172.21.15.104:0/2228717700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaf00095f0 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb03fff700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb04196020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000bda0 tx=0x7fcaf000bdd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb01ffb700 1 -- 172.21.15.104:0/2228717700 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf001d070 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb01ffb700 1 -- 172.21.15.104:0/2228717700 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcaf000f460 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb0419b1d0 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb01ffb700 1 -- 172.21.15.104:0/2228717700 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaf0021620 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.086+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb0419b6c0 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.087+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb04108da0 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.088+0000 7fcb01ffb700 1 -- 172.21.15.104:0/2228717700 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fcaf000f5d0 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.088+0000 7fcb01ffb700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fcaf406c680 0x7fcaf406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.088+0000 7fcb01ffb700 1 -- 172.21.15.104:0/2228717700 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4206+0+0 (secure 0 0 0) 0x7fcaf008c590 con 0x7fcb04102220 2024-04-26T04:38:11.090 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.088+0000 7fcb08e05700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fcaf406c680 0x7fcaf406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:11.091 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.089+0000 7fcb08e05700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fcaf406c680 0x7fcaf406eb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb04068af0 tx=0x7fcaf800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:11.093 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.092+0000 7fcb01ffb700 1 -- 172.21.15.104:0/2228717700 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fcaf005ac70 con 0x7fcb04102220 2024-04-26T04:38:11.289 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:11.287+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fcb04105f30 con 0x7fcaf406c680 2024-04-26T04:38:12.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: Detected new or changed devices on smithi104 2024-04-26T04:38:12.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi104 to 7456M 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:38:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:38:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:38:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:11 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:12.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: Detected new or changed devices on smithi104 2024-04-26T04:38:12.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi104 to 7456M 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='client.24167 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi104:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:38:12.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-26T04:38:12.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-26T04:38:12.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:11 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:14 smithi090 ceph-mon[26648]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:14 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/1843850789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fa80b6f-ab69-4e6e-9a5f-7e73a898c300"}]: dispatch 2024-04-26T04:38:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:14 smithi090 ceph-mon[26648]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fa80b6f-ab69-4e6e-9a5f-7e73a898c300"}]: dispatch 2024-04-26T04:38:14.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:14 smithi090 ceph-mon[26648]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fa80b6f-ab69-4e6e-9a5f-7e73a898c300"}]': finished 2024-04-26T04:38:14.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:14 smithi090 ceph-mon[26648]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T04:38:14.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:14 smithi104 ceph-mon[32369]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:14 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/1843850789' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fa80b6f-ab69-4e6e-9a5f-7e73a898c300"}]: dispatch 2024-04-26T04:38:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:14 smithi104 ceph-mon[32369]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2fa80b6f-ab69-4e6e-9a5f-7e73a898c300"}]: dispatch 2024-04-26T04:38:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:14 smithi104 ceph-mon[32369]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2fa80b6f-ab69-4e6e-9a5f-7e73a898c300"}]': finished 2024-04-26T04:38:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:14 smithi104 ceph-mon[32369]: osdmap e36: 6 total, 5 up, 6 in 2024-04-26T04:38:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:15.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:15 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/3474761406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:38:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:15 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/3474761406' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-26T04:38:16.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:16 smithi090 ceph-mon[26648]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:16 smithi104 ceph-mon[32369]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:18.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:18 smithi104 ceph-mon[32369]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:18.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:18.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:18 smithi090 ceph-mon[26648]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:18.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:20.527 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:20 smithi104 ceph-mon[32369]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:20 smithi090 ceph-mon[26648]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:21.661 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:38:21.662 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:21 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:21.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:38:21.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:21 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:22.578 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:22 smithi104 ceph-mon[32369]: Deploying daemon osd.5 on smithi104 2024-04-26T04:38:22.578 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:22 smithi104 ceph-mon[32369]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:22.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:22 smithi090 ceph-mon[26648]: Deploying daemon osd.5 on smithi104 2024-04-26T04:38:22.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:22 smithi090 ceph-mon[26648]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:24.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:24 smithi104 ceph-mon[32369]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:24.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:24 smithi090 ceph-mon[26648]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:26.583 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:26 smithi104 ceph-mon[32369]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:26.583 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:26.583 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:26.583 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:26 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:26.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:26 smithi090 ceph-mon[26648]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:26.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:26.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:26.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:26 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:28.428 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:28 smithi104 ceph-mon[32369]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:28.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:28 smithi090 ceph-mon[26648]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:30.003 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.000+0000 7fcb01ffb700 1 -- 172.21.15.104:0/2228717700 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcb04105f30 con 0x7fcaf406c680 2024-04-26T04:38:30.004 INFO:teuthology.orchestra.run.smithi104.stdout:Created osd(s) 5 on host 'smithi104' 2024-04-26T04:38:30.005 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fcaf406c680 msgr2=0x7fcaf406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:30.005 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fcaf406c680 0x7fcaf406eb30 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fcb04068af0 tx=0x7fcaf800b410 comp rx=0 tx=0).stop 2024-04-26T04:38:30.005 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 msgr2=0x7fcb04196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:30.005 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb04196020 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fcaf000bda0 tx=0x7fcaf000bdd0 comp rx=0 tx=0).stop 2024-04-26T04:38:30.005 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 shutdown_connections 2024-04-26T04:38:30.006 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fcaf406c680 0x7fcaf406eb30 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:30.006 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcb04101840 0x7fcb04195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:30.006 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 --2- 172.21.15.104:0/2228717700 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcb04102220 0x7fcb04196020 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:30.006 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 >> 172.21.15.104:0/2228717700 conn(0x7fcb040f9e50 msgr2=0x7fcb04104810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:30.006 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 shutdown_connections 2024-04-26T04:38:30.006 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:30.003+0000 7fcb0b069700 1 -- 172.21.15.104:0/2228717700 wait complete. 2024-04-26T04:38:30.577 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:30 smithi104 ceph-mon[32369]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:30.577 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:30 smithi104 ceph-mon[32369]: from='osd.5 [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:38:30.577 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:30 smithi104 ceph-mon[32369]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:38:30.577 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:30 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:30.578 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:30 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:30.599 DEBUG:teuthology.orchestra.run.smithi104:osd.5> sudo journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.5.service 2024-04-26T04:38:30.601 INFO:tasks.cephadm:Waiting for 6 OSDs to come up... 2024-04-26T04:38:30.602 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-26T04:38:30.629 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:30 smithi090 ceph-mon[26648]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:30.629 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:30 smithi090 ceph-mon[26648]: from='osd.5 [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:38:30.629 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:30 smithi090 ceph-mon[26648]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:38:30.629 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:30 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:30.629 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:30 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:30.820 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:31.653 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-mon[32369]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:38:31.654 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-mon[32369]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T04:38:31.654 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:31.654 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-mon[32369]: from='osd.5 [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:31.654 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-mon[32369]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:31.654 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:31.654 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:31.654 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[47742]: 2024-04-26T04:38:31.344+0000 7f232e316700 -1 osd.5 0 waiting for initial osdmap 2024-04-26T04:38:31.654 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:38:31 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[47742]: 2024-04-26T04:38:31.352+0000 7f232a90c700 -1 osd.5 38 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:38:31.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:31 smithi090 ceph-mon[26648]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:38:31.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:31 smithi090 ceph-mon[26648]: osdmap e37: 6 total, 5 up, 6 in 2024-04-26T04:38:31.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:31 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:31.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:31 smithi090 ceph-mon[26648]: from='osd.5 [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:31.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:31 smithi090 ceph-mon[26648]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:38:31.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:31 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:31.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:31 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 -- 172.21.15.90:0/3221924407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 msgr2=0x7fc9d8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 --2- 172.21.15.90:0/3221924407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8101890 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc009910 tx=0x7fc9cc009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 -- 172.21.15.90:0/3221924407 shutdown_connections 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 --2- 172.21.15.90:0/3221924407 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8101890 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 --2- 172.21.15.90:0/3221924407 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9d8100240 0x7fc9d8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 -- 172.21.15.90:0/3221924407 >> 172.21.15.90:0/3221924407 conn(0x7fc9d80fb7f0 msgr2=0x7fc9d80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 -- 172.21.15.90:0/3221924407 shutdown_connections 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 -- 172.21.15.90:0/3221924407 wait complete. 2024-04-26T04:38:32.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.494+0000 7fc9dd550700 1 Processor -- start 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9dd550700 1 -- start start 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9dd550700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9d8100240 0x7fc9d8195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9dd550700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9dd550700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d81965f0 con 0x7fc9d8101440 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9dd550700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d8196730 con 0x7fc9d8100240 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d67fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d67fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43304/0 (socket says 172.21.15.90:43304) 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d67fc700 1 -- 172.21.15.90:0/724051131 learned_addr learned my addr 172.21.15.90:0/724051131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d6ffd700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9d8100240 0x7fc9d8195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d67fc700 1 -- 172.21.15.90:0/724051131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9d8100240 msgr2=0x7fc9d8195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d67fc700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9d8100240 0x7fc9d8195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d67fc700 1 -- 172.21.15.90:0/724051131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9cc0095f0 con 0x7fc9d8101440 2024-04-26T04:38:32.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.495+0000 7fc9d67fc700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8195fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00bd70 tx=0x7fc9cc00bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.496+0000 7fc9bffff700 1 -- 172.21.15.90:0/724051131 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9cc01d070 con 0x7fc9d8101440 2024-04-26T04:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.496+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d819b180 con 0x7fc9d8101440 2024-04-26T04:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.496+0000 7fc9bffff700 1 -- 172.21.15.90:0/724051131 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc9cc005dc0 con 0x7fc9d8101440 2024-04-26T04:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.496+0000 7fc9bffff700 1 -- 172.21.15.90:0/724051131 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc9cc021410 con 0x7fc9d8101440 2024-04-26T04:38:32.499 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.496+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d819b670 con 0x7fc9d8101440 2024-04-26T04:38:32.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.497+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d8108d90 con 0x7fc9d8101440 2024-04-26T04:38:32.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.497+0000 7fc9bffff700 1 -- 172.21.15.90:0/724051131 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc9cc00f7d0 con 0x7fc9d8101440 2024-04-26T04:38:32.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.497+0000 7fc9bffff700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc9c006c680 0x7fc9c006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:32.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.498+0000 7fc9bffff700 1 -- 172.21.15.90:0/724051131 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc9cc08c830 con 0x7fc9d8101440 2024-04-26T04:38:32.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.498+0000 7fc9d6ffd700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc9c006c680 0x7fc9c006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:32.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.498+0000 7fc9d6ffd700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc9c006c680 0x7fc9c006eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc9d81012a0 tx=0x7fc9c800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:32.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.502+0000 7fc9bffff700 1 -- 172.21.15.90:0/724051131 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc9cc05ace0 con 0x7fc9d8101440 2024-04-26T04:38:32.656 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:32 smithi104 ceph-mon[32369]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:32.656 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:32 smithi104 ceph-mon[32369]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-26T04:38:32.656 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:32 smithi104 ceph-mon[32369]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T04:38:32.656 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:32 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:32.656 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:32 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:32.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.690+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fc9d8108f50 con 0x7fc9d8101440 2024-04-26T04:38:32.691 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:32 smithi090 ceph-mon[26648]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:32.691 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:32 smithi090 ceph-mon[26648]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]': finished 2024-04-26T04:38:32.692 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:32 smithi090 ceph-mon[26648]: osdmap e38: 6 total, 5 up, 6 in 2024-04-26T04:38:32.692 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:32 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:32.692 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:32 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:32.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.690+0000 7fc9bffff700 1 -- 172.21.15.90:0/724051131 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v39) v1 ==== 74+0+130 (secure 0 0 0) 0x7fc9cc00f460 con 0x7fc9d8101440 2024-04-26T04:38:32.692 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc9c006c680 msgr2=0x7fc9c006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc9c006c680 0x7fc9c006eb30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc9d81012a0 tx=0x7fc9c800b410 comp rx=0 tx=0).stop 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 msgr2=0x7fc9d8195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8195fd0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc9cc00bd70 tx=0x7fc9cc00bda0 comp rx=0 tx=0).stop 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 shutdown_connections 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc9c006c680 0x7fc9c006eb30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9d8100240 0x7fc9d8195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 --2- 172.21.15.90:0/724051131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9d8101440 0x7fc9d8195fd0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 >> 172.21.15.90:0/724051131 conn(0x7fc9d80fb7f0 msgr2=0x7fc9d8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 shutdown_connections 2024-04-26T04:38:32.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:32.692+0000 7fc9dd550700 1 -- 172.21.15.90:0/724051131 wait complete. 2024-04-26T04:38:33.279 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":39,"num_osds":6,"num_up_osds":6,"osd_up_since":1714106312,"num_in_osds":6,"osd_in_since":1714106293,"num_remapped_pgs":0} 2024-04-26T04:38:33.279 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T04:38:33.526 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:33.549 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: purged_snaps scrub starts 2024-04-26T04:38:33.549 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: purged_snaps scrub ok 2024-04-26T04:38:33.549 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:33.549 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: osd.5 [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] boot 2024-04-26T04:38:33.549 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T04:38:33.550 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:33.550 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/724051131' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T04:38:33.550 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:33 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: purged_snaps scrub starts 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: purged_snaps scrub ok 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: osd.5 [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] boot 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: osdmap e39: 6 total, 6 up, 6 in 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/724051131' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-26T04:38:33.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:33 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:34.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 135 MiB used, 447 GiB / 447 GiB avail 2024-04-26T04:38:34.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: osdmap e40: 6 total, 6 up, 6 in 2024-04-26T04:38:34.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:34.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:34.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:38:34.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:34.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:34.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:34.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:38:35.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 -- 172.21.15.90:0/2704772831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 msgr2=0x7f12c81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:35.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 --2- 172.21.15.90:0/2704772831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c81008b0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f12b8009910 tx=0x7f12b8009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:35.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 -- 172.21.15.90:0/2704772831 shutdown_connections 2024-04-26T04:38:35.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 --2- 172.21.15.90:0/2704772831 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12c8100e80 0x7f12c8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 --2- 172.21.15.90:0/2704772831 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c81008b0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 -- 172.21.15.90:0/2704772831 >> 172.21.15.90:0/2704772831 conn(0x7f12c80fbda0 msgr2=0x7f12c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 -- 172.21.15.90:0/2704772831 shutdown_connections 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.112+0000 7f12cdd05700 1 -- 172.21.15.90:0/2704772831 wait complete. 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12cdd05700 1 Processor -- start 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12cdd05700 1 -- start start 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12cdd05700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c8195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12cdd05700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12c8100e80 0x7f12c8196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12cdd05700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8196a80 con 0x7f12c81004e0 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12cdd05700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c8196bc0 con 0x7f12c8100e80 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12c77fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c8195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12c77fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c8195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:43320/0 (socket says 172.21.15.90:43320) 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12c77fe700 1 -- 172.21.15.90:0/3027320312 learned_addr learned my addr 172.21.15.90:0/3027320312 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:35.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12c77fe700 1 -- 172.21.15.90:0/3027320312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12c8100e80 msgr2=0x7f12c8196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12c77fe700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12c8100e80 0x7f12c8196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.113+0000 7f12c77fe700 1 -- 172.21.15.90:0/3027320312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12b80095f0 con 0x7f12c81004e0 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.114+0000 7f12c77fe700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c8195ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006010 tx=0x7f12b800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.114+0000 7f12c57fa700 1 -- 172.21.15.90:0/3027320312 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b801d070 con 0x7f12c81004e0 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.114+0000 7f12c57fa700 1 -- 172.21.15.90:0/3027320312 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f12b800f460 con 0x7f12c81004e0 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.114+0000 7f12c57fa700 1 -- 172.21.15.90:0/3027320312 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f12b8021640 con 0x7f12c81004e0 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.114+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c819a9b0 con 0x7f12c81004e0 2024-04-26T04:38:35.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.114+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c819aea0 con 0x7f12c81004e0 2024-04-26T04:38:35.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.115+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c804f2a0 con 0x7f12c81004e0 2024-04-26T04:38:35.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.115+0000 7f12c57fa700 1 -- 172.21.15.90:0/3027320312 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f12b800fab0 con 0x7f12c81004e0 2024-04-26T04:38:35.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.116+0000 7f12c57fa700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f12b406c680 0x7f12b406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:35.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.116+0000 7f12c57fa700 1 -- 172.21.15.90:0/3027320312 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f12b808c8b0 con 0x7f12c81004e0 2024-04-26T04:38:35.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.116+0000 7f12bffff700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f12b406c680 0x7f12b406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:35.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.116+0000 7f12bffff700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f12b406c680 0x7f12b406eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005fd0 tx=0x7f12b0005d00 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:35.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.120+0000 7f12c57fa700 1 -- 172.21.15.90:0/3027320312 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f12b80572d0 con 0x7f12c81004e0 2024-04-26T04:38:35.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.313+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f12c8108200 con 0x7f12c81004e0 2024-04-26T04:38:35.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.314+0000 7f12c57fa700 1 -- 172.21.15.90:0/3027320312 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11335 (secure 0 0 0) 0x7f12b8026090 con 0x7f12c81004e0 2024-04-26T04:38:35.316 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:38:35.316 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":40,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","created":"2024-04-26T04:32:57.928472+0000","modified":"2024-04-26T04:38:33.349137+0000","last_up_change":"2024-04-26T04:38:32.344046+0000","last_in_change":"2024-04-26T04:38:13.654124+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:37:19.554153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57491e88-0d1c-41ac-a1f5-3662e5c0a467","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6803","nonce":4217606471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6805","nonce":4217606471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6809","nonce":4217606471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6807","nonce":4217606471}]},"public_addr":"172.21.15.90:6803/4217606471","cluster_addr":"172.21.15.90:6805/4217606471","heartbeat_back_addr":"172.21.15.90:6809/4217606471","heartbeat_front_addr":"172.21.15.90:6807/4217606471","state":["exists","up"]},{"osd":1,"uuid":"fe237f1b-a589-452c-8e5e-98a9a31593eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6811","nonce":3196794289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6813","nonce":3196794289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6817","nonce":3196794289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6815","nonce":3196794289}]},"public_addr":"172.21.15.90:6811/3196794289","cluster_addr":"172.21.15.90:6813/3196794289","heartbeat_back_addr":"172.21.15.90:6817/3196794289","heartbeat_front_addr":"172.21.15.90:6815/3196794289","state":["exists","up"]},{"osd":2,"uuid":"7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6819","nonce":127929257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6821","nonce":127929257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6825","nonce":127929257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6823","nonce":127929257}]},"public_addr":"172.21.15.90:6819/127929257","cluster_addr":"172.21.15.90:6821/127929257","heartbeat_back_addr":"172.21.15.90:6825/127929257","heartbeat_front_addr":"172.21.15.90:6823/127929257","state":["exists","up"]},{"osd":3,"uuid":"62adc04c-9e7e-4ee5-bb5f-42004b2a615d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6801","nonce":3796716845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6803","nonce":3796716845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6807","nonce":3796716845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6805","nonce":3796716845}]},"public_addr":"172.21.15.104:6801/3796716845","cluster_addr":"172.21.15.104:6803/3796716845","heartbeat_back_addr":"172.21.15.104:6807/3796716845","heartbeat_front_addr":"172.21.15.104:6805/3796716845","state":["exists","up"]},{"osd":4,"uuid":"a4ef2729-84e1-4d2d-868b-3da5fefeb800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6809","nonce":4258788432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6811","nonce":4258788432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6815","nonce":4258788432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6813","nonce":4258788432}]},"public_addr":"172.21.15.104:6809/4258788432","cluster_addr":"172.21.15.104:6811/4258788432","heartbeat_back_addr":"172.21.15.104:6815/4258788432","heartbeat_front_addr":"172.21.15.104:6813/4258788432","state":["exists","up"]},{"osd":5,"uuid":"2fa80b6f-ab69-4e6e-9a5f-7e73a898c300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6817","nonce":2033326841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6819","nonce":2033326841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6823","nonce":2033326841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6821","nonce":2033326841}]},"public_addr":"172.21.15.104:6817/2033326841","cluster_addr":"172.21.15.104:6819/2033326841","heartbeat_back_addr":"172.21.15.104:6823/2033326841","heartbeat_front_addr":"172.21.15.104:6821/2033326841","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:19.861796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:48.482295+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:17.028265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:40.489573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:05.503518+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:30.925548+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3214112922":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/1814853898":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/748209074":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6800/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/50635771":"2024-04-27T04:33:24.670558+0000","172.21.15.90:6800/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/2342166800":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2310050885":"2024-04-27T04:33:57.088900+0000","172.21.15.90:0/424889384":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/3418315403":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2473529454":"2024-04-27T04:33:24.670558+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f12b406c680 msgr2=0x7f12b406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f12b406c680 0x7f12b406eb30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f12b0005fd0 tx=0x7f12b0005d00 comp rx=0 tx=0).stop 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 msgr2=0x7f12c8195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c8195ef0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f12b8006010 tx=0x7f12b800be20 comp rx=0 tx=0).stop 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 shutdown_connections 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f12b406c680 0x7f12b406eb30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f12c81004e0 0x7f12c8195ef0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 --2- 172.21.15.90:0/3027320312 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f12c8100e80 0x7f12c8196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:35.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 >> 172.21.15.90:0/3027320312 conn(0x7f12c80fbda0 msgr2=0x7f12c80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:35.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 shutdown_connections 2024-04-26T04:38:35.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:35.316+0000 7f12cdd05700 1 -- 172.21.15.90:0/3027320312 wait complete. 2024-04-26T04:38:35.617 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:35 smithi090 ceph-mon[26648]: Detected new or changed devices on smithi104 2024-04-26T04:38:35.617 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:35 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi104 to 4970M 2024-04-26T04:38:35.618 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:35.618 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:35.618 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:35 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:35.618 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:35 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3027320312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:38:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:35 smithi104 ceph-mon[32369]: Detected new or changed devices on smithi104 2024-04-26T04:38:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:35 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi104 to 4970M 2024-04-26T04:38:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:35 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:38:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:35 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:38:35.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:35 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:38:35.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:35 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3027320312' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:38:36.042 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:37:19.554153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T04:38:36.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-26T04:38:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:36.662 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:36 smithi104 ceph-mon[32369]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:36.664 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:36 smithi090 ceph-mon[26648]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 -- 172.21.15.90:0/33214491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 msgr2=0x7fbad4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/33214491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4104e50 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fbac4009910 tx=0x7fbac4009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 -- 172.21.15.90:0/33214491 shutdown_connections 2024-04-26T04:38:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/33214491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4104e50 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/33214491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad41004e0 0x7fbad41008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:37.971 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 -- 172.21.15.90:0/33214491 >> 172.21.15.90:0/33214491 conn(0x7fbad40fbda0 msgr2=0x7fbad40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 -- 172.21.15.90:0/33214491 shutdown_connections 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 -- 172.21.15.90:0/33214491 wait complete. 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 Processor -- start 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.968+0000 7fbadb2a6700 1 -- start start 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbadb2a6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad41004e0 0x7fbad4195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbadb2a6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbadb2a6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad4196b50 con 0x7fbad4100e80 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbadb2a6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad419a8e0 con 0x7fbad41004e0 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad8841700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad8841700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46300/0 (socket says 172.21.15.90:46300) 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad8841700 1 -- 172.21.15.90:0/1422468236 learned_addr learned my addr 172.21.15.90:0/1422468236 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad9042700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad41004e0 0x7fbad4195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad8841700 1 -- 172.21.15.90:0/1422468236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad41004e0 msgr2=0x7fbad4195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:37.972 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad8841700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad41004e0 0x7fbad4195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad8841700 1 -- 172.21.15.90:0/1422468236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbac40095f0 con 0x7fbad4100e80 2024-04-26T04:38:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbad8841700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4196470 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbac40092e0 tx=0x7fbac400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbaca7fc700 1 -- 172.21.15.90:0/1422468236 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbac401d070 con 0x7fbad4100e80 2024-04-26T04:38:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad419ab60 con 0x7fbad4100e80 2024-04-26T04:38:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbaca7fc700 1 -- 172.21.15.90:0/1422468236 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbac4005dc0 con 0x7fbad4100e80 2024-04-26T04:38:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbaca7fc700 1 -- 172.21.15.90:0/1422468236 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbac4017400 con 0x7fbad4100e80 2024-04-26T04:38:37.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.969+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad419b050 con 0x7fbad4100e80 2024-04-26T04:38:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.970+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad4108850 con 0x7fbad4100e80 2024-04-26T04:38:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.971+0000 7fbaca7fc700 1 -- 172.21.15.90:0/1422468236 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fbac400f460 con 0x7fbad4100e80 2024-04-26T04:38:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.971+0000 7fbaca7fc700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbac006c680 0x7fbac006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.971+0000 7fbad9042700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbac006c680 0x7fbac006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.971+0000 7fbaca7fc700 1 -- 172.21.15.90:0/1422468236 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbac408c9b0 con 0x7fbad4100e80 2024-04-26T04:38:37.974 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.972+0000 7fbad9042700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbac006c680 0x7fbac006eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009550 tx=0x7fbad00091b0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:37.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:37.976+0000 7fbaca7fc700 1 -- 172.21.15.90:0/1422468236 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbac40573d0 con 0x7fbad4100e80 2024-04-26T04:38:38.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.164+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbad4197290 con 0x7fbad4100e80 2024-04-26T04:38:38.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.164+0000 7fbaca7fc700 1 -- 172.21.15.90:0/1422468236 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v40) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbac405a9f0 con 0x7fbad4100e80 2024-04-26T04:38:38.166 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-26T04:38:38.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.166+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbac006c680 msgr2=0x7fbac006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:38.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbac006c680 0x7fbac006eb30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fbad0009550 tx=0x7fbad00091b0 comp rx=0 tx=0).stop 2024-04-26T04:38:38.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 msgr2=0x7fbad4196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:38.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4196470 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fbac40092e0 tx=0x7fbac400bd70 comp rx=0 tx=0).stop 2024-04-26T04:38:38.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 shutdown_connections 2024-04-26T04:38:38.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbac006c680 0x7fbac006eb30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:38.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbad41004e0 0x7fbad4195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:38.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 --2- 172.21.15.90:0/1422468236 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbad4100e80 0x7fbad4196470 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:38.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 >> 172.21.15.90:0/1422468236 conn(0x7fbad40fbda0 msgr2=0x7fbad40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:38.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 shutdown_connections 2024-04-26T04:38:38.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:38.167+0000 7fbadb2a6700 1 -- 172.21.15.90:0/1422468236 wait complete. 2024-04-26T04:38:38.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:38 smithi090 ceph-mon[26648]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:38.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:38 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1422468236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:38:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:38 smithi104 ceph-mon[32369]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:38 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1422468236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:38:38.827 INFO:tasks.cephadm:Setting up client nodes... 2024-04-26T04:38:38.827 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T04:38:39.030 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:40.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 -- 172.21.15.90:0/729198592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c00ff140 msgr2=0x7ff7c00ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:40.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 --2- 172.21.15.90:0/729198592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c00ff140 0x7ff7c00ff550 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc009910 tx=0x7ff7bc009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:40.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 -- 172.21.15.90:0/729198592 shutdown_connections 2024-04-26T04:38:40.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 --2- 172.21.15.90:0/729198592 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c0100340 0x7ff7c0100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:40.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 --2- 172.21.15.90:0/729198592 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c00ff140 0x7ff7c00ff550 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:40.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 -- 172.21.15.90:0/729198592 >> 172.21.15.90:0/729198592 conn(0x7ff7c00fa6d0 msgr2=0x7ff7c00fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:40.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 -- 172.21.15.90:0/729198592 shutdown_connections 2024-04-26T04:38:40.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.390+0000 7ff7c828b700 1 -- 172.21.15.90:0/729198592 wait complete. 2024-04-26T04:38:40.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c828b700 1 Processor -- start 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c828b700 1 -- start start 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c828b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c00ff140 0x7ff7c0195b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c828b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c0100340 0x7ff7c0196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c828b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01966a0 con 0x7ff7c0100340 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c828b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7c01967e0 con 0x7ff7c00ff140 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c5826700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c0100340 0x7ff7c0196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c5826700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c0100340 0x7ff7c0196080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46324/0 (socket says 172.21.15.90:46324) 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c5826700 1 -- 172.21.15.90:0/1548007860 learned_addr learned my addr 172.21.15.90:0/1548007860 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c6027700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c00ff140 0x7ff7c0195b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c5826700 1 -- 172.21.15.90:0/1548007860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c00ff140 msgr2=0x7ff7c0195b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c5826700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c00ff140 0x7ff7c0195b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:40.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c5826700 1 -- 172.21.15.90:0/1548007860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc0095f0 con 0x7ff7c0100340 2024-04-26T04:38:40.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.391+0000 7ff7c5826700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c0100340 0x7ff7c0196080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b4b0 tx=0x7ff7b000b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:40.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.392+0000 7ff7b77fe700 1 -- 172.21.15.90:0/1548007860 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b0011800 con 0x7ff7c0100340 2024-04-26T04:38:40.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.392+0000 7ff7b77fe700 1 -- 172.21.15.90:0/1548007860 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7b0011e10 con 0x7ff7c0100340 2024-04-26T04:38:40.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.392+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7c019b290 con 0x7ff7c0100340 2024-04-26T04:38:40.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.392+0000 7ff7b77fe700 1 -- 172.21.15.90:0/1548007860 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7b000f2e0 con 0x7ff7c0100340 2024-04-26T04:38:40.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.392+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c019b7e0 con 0x7ff7c0100340 2024-04-26T04:38:40.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.393+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c0107d10 con 0x7ff7c0100340 2024-04-26T04:38:40.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.393+0000 7ff7b77fe700 1 -- 172.21.15.90:0/1548007860 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff7b0010380 con 0x7ff7c0100340 2024-04-26T04:38:40.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.394+0000 7ff7b77fe700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff7ac06c680 0x7ff7ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:40.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.394+0000 7ff7b77fe700 1 -- 172.21.15.90:0/1548007860 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ff7b008b090 con 0x7ff7c0100340 2024-04-26T04:38:40.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.394+0000 7ff7c6027700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff7ac06c680 0x7ff7ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:40.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.394+0000 7ff7c6027700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff7ac06c680 0x7ff7ac06eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:40.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.398+0000 7ff7b77fe700 1 -- 172.21.15.90:0/1548007860 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff7b0059670 con 0x7ff7c0100340 2024-04-26T04:38:40.671 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.669+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7ff7c0107ed0 con 0x7ff7c0100340 2024-04-26T04:38:40.671 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:40 smithi090 ceph-mon[26648]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:40.675 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.673+0000 7ff7b77fe700 1 -- 172.21.15.90:0/1548007860 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7ff7b0059200 con 0x7ff7c0100340 2024-04-26T04:38:40.675 INFO:teuthology.orchestra.run.smithi090.stdout:[client.0] 2024-04-26T04:38:40.676 INFO:teuthology.orchestra.run.smithi090.stdout: key = AQDQLytmpAoAKBAA/xiSYZiq4Uh4Ba/PvQE+Gg== 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff7ac06c680 msgr2=0x7ff7ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff7ac06c680 0x7ff7ac06eb30 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc006010 tx=0x7ff7bc005920 comp rx=0 tx=0).stop 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c0100340 msgr2=0x7ff7c0196080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c0100340 0x7ff7c0196080 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ff7b000b4b0 tx=0x7ff7b000b870 comp rx=0 tx=0).stop 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 shutdown_connections 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff7ac06c680 0x7ff7ac06eb30 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7c00ff140 0x7ff7c0195b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 --2- 172.21.15.90:0/1548007860 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7c0100340 0x7ff7c0196080 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.675+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 >> 172.21.15.90:0/1548007860 conn(0x7ff7c00fa6d0 msgr2=0x7ff7c0068df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:40.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.676+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 shutdown_connections 2024-04-26T04:38:40.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:40.676+0000 7ff7c828b700 1 -- 172.21.15.90:0/1548007860 wait complete. 2024-04-26T04:38:40.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:40 smithi104 ceph-mon[32369]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:41.329 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:38:41.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-26T04:38:41.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-26T04:38:41.366 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-26T04:38:41.580 INFO:teuthology.orchestra.run.smithi104.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi104/config 2024-04-26T04:38:41.701 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:41 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1548007860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:38:41.702 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:41 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1548007860' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:38:41.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:41 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1548007860' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:38:41.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:41 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1548007860' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:38:42.936 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:42 smithi104 ceph-mon[32369]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:42 smithi090 ceph-mon[26648]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:43.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.160+0000 7f4604c46700 1 -- 172.21.15.104:0/478992125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600100740 msgr2=0x7f4600100b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:43.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.160+0000 7f4604c46700 1 --2- 172.21.15.104:0/478992125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600100740 0x7f4600100b50 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f45f0009910 tx=0x7f45f0009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:43.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 -- 172.21.15.104:0/478992125 shutdown_connections 2024-04-26T04:38:43.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 --2- 172.21.15.104:0/478992125 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4600101090 0x7f46000ff490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:43.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 --2- 172.21.15.104:0/478992125 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600100740 0x7f4600100b50 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:43.164 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 -- 172.21.15.104:0/478992125 >> 172.21.15.104:0/478992125 conn(0x7f46000f9f10 msgr2=0x7f46000fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:43.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 -- 172.21.15.104:0/478992125 shutdown_connections 2024-04-26T04:38:43.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 -- 172.21.15.104:0/478992125 wait complete. 2024-04-26T04:38:43.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 Processor -- start 2024-04-26T04:38:43.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 -- start start 2024-04-26T04:38:43.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4600100740 0x7f4600195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:43.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600101090 0x7f4600196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:43.165 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46001965a0 con 0x7f4600100740 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.161+0000 7f4604c46700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46001966e0 con 0x7f4600101090 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600101090 0x7f4600196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fdd9b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600101090 0x7f4600196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.104:51796/0 (socket says 172.21.15.104:51796) 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fdd9b700 1 -- 172.21.15.104:0/871613570 learned_addr learned my addr 172.21.15.104:0/871613570 (peer_addr_for_me v2:172.21.15.104:0/0) 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fe59c700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4600100740 0x7f4600195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fdd9b700 1 -- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4600100740 msgr2=0x7f4600195ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fdd9b700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4600100740 0x7f4600195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:43.166 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fdd9b700 1 -- 172.21.15.104:0/871613570 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45f00095f0 con 0x7f4600101090 2024-04-26T04:38:43.167 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45fdd9b700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600101090 0x7f4600196010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c930 tx=0x7f45f400ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:43.167 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45ef7fe700 1 -- 172.21.15.104:0/871613570 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45f4007a70 con 0x7f4600101090 2024-04-26T04:38:43.167 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45ef7fe700 1 -- 172.21.15.104:0/871613570 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f45f4007bd0 con 0x7f4600101090 2024-04-26T04:38:43.167 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f460019b1a0 con 0x7f4600101090 2024-04-26T04:38:43.167 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f45ef7fe700 1 -- 172.21.15.104:0/871613570 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45f4018630 con 0x7f4600101090 2024-04-26T04:38:43.167 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.162+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f460019b6c0 con 0x7f4600101090 2024-04-26T04:38:43.169 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.163+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4600108d50 con 0x7f4600101090 2024-04-26T04:38:43.170 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.164+0000 7f45ef7fe700 1 -- 172.21.15.104:0/871613570 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f45f4007d40 con 0x7f4600101090 2024-04-26T04:38:43.170 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.164+0000 7f45ef7fe700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f45e806c680 0x7f45e806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:43.170 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.164+0000 7f45ef7fe700 1 -- 172.21.15.104:0/871613570 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f45f408b520 con 0x7f4600101090 2024-04-26T04:38:43.170 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.164+0000 7f45fe59c700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f45e806c680 0x7f45e806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:43.170 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.165+0000 7f45fe59c700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f45e806c680 0x7f45e806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45f000b5c0 tx=0x7f45f0005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:43.170 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.169+0000 7f45ef7fe700 1 -- 172.21.15.104:0/871613570 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f45f4059a50 con 0x7f4600101090 2024-04-26T04:38:43.432 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.431+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f4600108f10 con 0x7f4600101090 2024-04-26T04:38:43.439 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.438+0000 7f45ef7fe700 1 -- 172.21.15.104:0/871613570 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f45f40595e0 con 0x7f4600101090 2024-04-26T04:38:43.440 INFO:teuthology.orchestra.run.smithi104.stdout:[client.1] 2024-04-26T04:38:43.440 INFO:teuthology.orchestra.run.smithi104.stdout: key = AQDTLytm+BrZGRAAieMjpoCqLk2dz7j1pPjU4w== 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.440+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f45e806c680 msgr2=0x7f45e806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.440+0000 7f4604c46700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f45e806c680 0x7f45e806eb30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f45f000b5c0 tx=0x7f45f0005960 comp rx=0 tx=0).stop 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.440+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600101090 msgr2=0x7f4600196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.440+0000 7f4604c46700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600101090 0x7f4600196010 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f45f400c930 tx=0x7f45f400ccf0 comp rx=0 tx=0).stop 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.441+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 shutdown_connections 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.441+0000 7f4604c46700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f45e806c680 0x7f45e806eb30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.441+0000 7f4604c46700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4600100740 0x7f4600195ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.441+0000 7f4604c46700 1 --2- 172.21.15.104:0/871613570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4600101090 0x7f4600196010 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.441+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 >> 172.21.15.104:0/871613570 conn(0x7f46000f9f10 msgr2=0x7f46001036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.441+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 shutdown_connections 2024-04-26T04:38:43.442 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:38:43.441+0000 7f4604c46700 1 -- 172.21.15.104:0/871613570 wait complete. 2024-04-26T04:38:44.137 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:38:44.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-04-26T04:38:44.137 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-26T04:38:44.177 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-26T04:38:44.177 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-26T04:38:44.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-26T04:38:44.394 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:44 smithi090 ceph-mon[26648]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:44 smithi090 ceph-mon[26648]: from='client.? 172.21.15.104:0/871613570' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:38:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:44 smithi090 ceph-mon[26648]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:38:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:44 smithi090 ceph-mon[26648]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:38:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:44 smithi104 ceph-mon[32369]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:44 smithi104 ceph-mon[32369]: from='client.? 172.21.15.104:0/871613570' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:38:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:44 smithi104 ceph-mon[32369]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-26T04:38:44.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:44 smithi104 ceph-mon[32369]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-26T04:38:45.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/4195303996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68104130 msgr2=0x7f0e68104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:45.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/4195303996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68104130 0x7f0e68104500 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009910 tx=0x7f0e58009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:45.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/4195303996 shutdown_connections 2024-04-26T04:38:45.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/4195303996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e680ffbd0 0x7f0e68100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:45.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/4195303996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e68104130 0x7f0e68104500 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:45.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/4195303996 >> 172.21.15.90:0/4195303996 conn(0x7f0e68075130 msgr2=0x7f0e68075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:45.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/4195303996 shutdown_connections 2024-04-26T04:38:45.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.926+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/4195303996 wait complete. 2024-04-26T04:38:45.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6ffb0700 1 Processor -- start 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6ffb0700 1 -- start start 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6ffb0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ffbd0 0x7f0e68195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6ffb0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e68104130 0x7f0e68196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6ffb0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e68196a70 con 0x7f0e680ffbd0 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6ffb0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6819a800 con 0x7f0e68104130 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6dd4c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ffbd0 0x7f0e68195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6dd4c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ffbd0 0x7f0e68195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46340/0 (socket says 172.21.15.90:46340) 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6dd4c700 1 -- 172.21.15.90:0/2726467918 learned_addr learned my addr 172.21.15.90:0/2726467918 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6d54b700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e68104130 0x7f0e68196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6dd4c700 1 -- 172.21.15.90:0/2726467918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e68104130 msgr2=0x7f0e68196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6dd4c700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e68104130 0x7f0e68196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6dd4c700 1 -- 172.21.15.90:0/2726467918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e580095f0 con 0x7f0e680ffbd0 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.927+0000 7f0e6d54b700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e68104130 0x7f0e68196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:38:45.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.928+0000 7f0e6dd4c700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ffbd0 0x7f0e68195e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e58000c00 tx=0x7f0e5800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.928+0000 7f0e5effd700 1 -- 172.21.15.90:0/2726467918 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5801d070 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.928+0000 7f0e5effd700 1 -- 172.21.15.90:0/2726467918 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0e58004fa0 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.928+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6819aa80 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.928+0000 7f0e5effd700 1 -- 172.21.15.90:0/2726467918 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5800f460 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.928+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6819af70 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.929+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e681086a0 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.929+0000 7f0e5effd700 1 -- 172.21.15.90:0/2726467918 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0e58022870 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.929+0000 7f0e5effd700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0e5406c680 0x7f0e5406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.930+0000 7f0e6d54b700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0e5406c680 0x7f0e5406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.930+0000 7f0e5effd700 1 -- 172.21.15.90:0/2726467918 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f0e5808c980 con 0x7f0e680ffbd0 2024-04-26T04:38:45.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.930+0000 7f0e6d54b700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0e5406c680 0x7f0e5406eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e64005950 tx=0x7f0e640058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:45.935 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:45.934+0000 7f0e5effd700 1 -- 172.21.15.90:0/2726467918 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0e580574d0 con 0x7f0e680ffbd0 2024-04-26T04:38:46.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.180+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f0e6804f2a0 con 0x7f0e680ffbd0 2024-04-26T04:38:46.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.183+0000 7f0e5effd700 1 -- 172.21.15.90:0/2726467918 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v19) v1 ==== 74+0+172855 (secure 0 0 0) 0x7f0e58027020 con 0x7f0e680ffbd0 2024-04-26T04:38:46.187 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:38:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0e5406c680 msgr2=0x7f0e5406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0e5406c680 0x7f0e5406eb30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0e64005950 tx=0x7f0e640058e0 comp rx=0 tx=0).stop 2024-04-26T04:38:46.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ffbd0 msgr2=0x7f0e68195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ffbd0 0x7f0e68195e50 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f0e58000c00 tx=0x7f0e5800be20 comp rx=0 tx=0).stop 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 shutdown_connections 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0e5406c680 0x7f0e5406eb30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e680ffbd0 0x7f0e68195e50 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.185+0000 7f0e6ffb0700 1 --2- 172.21.15.90:0/2726467918 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e68104130 0x7f0e68196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.186+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 >> 172.21.15.90:0/2726467918 conn(0x7f0e68075130 msgr2=0x7f0e680fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.186+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 shutdown_connections 2024-04-26T04:38:46.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:46.186+0000 7f0e6ffb0700 1 -- 172.21.15.90:0/2726467918 wait complete. 2024-04-26T04:38:46.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:46 smithi090 ceph-mon[26648]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:46.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:46 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2726467918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T04:38:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:46 smithi104 ceph-mon[32369]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:46 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2726467918' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-26T04:38:46.806 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"active_gid":14213,"active_name":"smithi090.iwrvwa","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6800","nonce":1235211382},{"type":"v1","addr":"172.21.15.90:6801","nonce":1235211382}]},"active_addr":"172.21.15.90:6801/1235211382","active_change":"2024-04-26T04:35:03.230314+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":14236,"name":"smithi104.wwoewi","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph:v18","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.90:8443/","prometheus":"http://172.21.15.90:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2705188709}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2846076831}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":1198706928}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.90:0","nonce":2763746313}]}]} 2024-04-26T04:38:46.810 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-26T04:38:46.810 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-26T04:38:46.811 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T04:38:47.015 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:48.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 -- 172.21.15.90:0/3168283069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c00ffbd0 msgr2=0x7f38c0100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 --2- 172.21.15.90:0/3168283069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c00ffbd0 0x7f38c0100040 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f38a8009910 tx=0x7f38a8009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 -- 172.21.15.90:0/3168283069 shutdown_connections 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 --2- 172.21.15.90:0/3168283069 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c00ffbd0 0x7f38c0100040 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 --2- 172.21.15.90:0/3168283069 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38c0104130 0x7f38c0104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 -- 172.21.15.90:0/3168283069 >> 172.21.15.90:0/3168283069 conn(0x7f38c0075130 msgr2=0x7f38c0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 -- 172.21.15.90:0/3168283069 shutdown_connections 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.579+0000 7f38c5554700 1 -- 172.21.15.90:0/3168283069 wait complete. 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38c5554700 1 Processor -- start 2024-04-26T04:38:48.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38c5554700 1 -- start start 2024-04-26T04:38:48.583 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:48 smithi090 ceph-mon[26648]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:48.583 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38c5554700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38c00ffbd0 0x7f38c0195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38c5554700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c0104130 0x7f38c01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38c5554700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c0196a90 con 0x7f38c0104130 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38c5554700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38c019a820 con 0x7f38c00ffbd0 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38be7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c0104130 0x7f38c01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38be7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c0104130 0x7f38c01963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36284/0 (socket says 172.21.15.90:36284) 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38be7fc700 1 -- 172.21.15.90:0/3914291160 learned_addr learned my addr 172.21.15.90:0/3914291160 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.580+0000 7f38beffd700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38c00ffbd0 0x7f38c0195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38be7fc700 1 -- 172.21.15.90:0/3914291160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38c00ffbd0 msgr2=0x7f38c0195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38be7fc700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38c00ffbd0 0x7f38c0195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:48.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38be7fc700 1 -- 172.21.15.90:0/3914291160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a80095f0 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38be7fc700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c0104130 0x7f38c01963b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f38a8006010 tx=0x7f38a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38b7fff700 1 -- 172.21.15.90:0/3914291160 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38a801d070 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38c019aaa0 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38b7fff700 1 -- 172.21.15.90:0/3914291160 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f38a800f460 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38b7fff700 1 -- 172.21.15.90:0/3914291160 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f38a8017630 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.581+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38c019af90 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.582+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38c01086a0 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.582+0000 7f38b7fff700 1 -- 172.21.15.90:0/3914291160 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f38a8003a60 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.583+0000 7f38b7fff700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f38ac06c680 0x7f38ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.583+0000 7f38b7fff700 1 -- 172.21.15.90:0/3914291160 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f38a808c6d0 con 0x7f38c0104130 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.583+0000 7f38beffd700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f38ac06c680 0x7f38ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:48.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.583+0000 7f38beffd700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f38ac06c680 0x7f38ac06eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009d10 tx=0x7f38b8009040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:48.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.587+0000 7f38b7fff700 1 -- 172.21.15.90:0/3914291160 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f38a80570f0 con 0x7f38c0104130 2024-04-26T04:38:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:48 smithi104 ceph-mon[32369]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:38:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.779+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38c01971d0 con 0x7f38c0104130 2024-04-26T04:38:48.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.779+0000 7f38b7fff700 1 -- 172.21.15.90:0/3914291160 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11335 (secure 0 0 0) 0x7f38a805a710 con 0x7f38c0104130 2024-04-26T04:38:48.781 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:38:48.781 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":40,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","created":"2024-04-26T04:32:57.928472+0000","modified":"2024-04-26T04:38:33.349137+0000","last_up_change":"2024-04-26T04:38:32.344046+0000","last_in_change":"2024-04-26T04:38:13.654124+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:37:19.554153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57491e88-0d1c-41ac-a1f5-3662e5c0a467","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6803","nonce":4217606471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6805","nonce":4217606471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6809","nonce":4217606471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6807","nonce":4217606471}]},"public_addr":"172.21.15.90:6803/4217606471","cluster_addr":"172.21.15.90:6805/4217606471","heartbeat_back_addr":"172.21.15.90:6809/4217606471","heartbeat_front_addr":"172.21.15.90:6807/4217606471","state":["exists","up"]},{"osd":1,"uuid":"fe237f1b-a589-452c-8e5e-98a9a31593eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6811","nonce":3196794289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6813","nonce":3196794289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6817","nonce":3196794289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6815","nonce":3196794289}]},"public_addr":"172.21.15.90:6811/3196794289","cluster_addr":"172.21.15.90:6813/3196794289","heartbeat_back_addr":"172.21.15.90:6817/3196794289","heartbeat_front_addr":"172.21.15.90:6815/3196794289","state":["exists","up"]},{"osd":2,"uuid":"7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6819","nonce":127929257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6821","nonce":127929257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6825","nonce":127929257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6823","nonce":127929257}]},"public_addr":"172.21.15.90:6819/127929257","cluster_addr":"172.21.15.90:6821/127929257","heartbeat_back_addr":"172.21.15.90:6825/127929257","heartbeat_front_addr":"172.21.15.90:6823/127929257","state":["exists","up"]},{"osd":3,"uuid":"62adc04c-9e7e-4ee5-bb5f-42004b2a615d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6801","nonce":3796716845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6803","nonce":3796716845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6807","nonce":3796716845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6805","nonce":3796716845}]},"public_addr":"172.21.15.104:6801/3796716845","cluster_addr":"172.21.15.104:6803/3796716845","heartbeat_back_addr":"172.21.15.104:6807/3796716845","heartbeat_front_addr":"172.21.15.104:6805/3796716845","state":["exists","up"]},{"osd":4,"uuid":"a4ef2729-84e1-4d2d-868b-3da5fefeb800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6809","nonce":4258788432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6811","nonce":4258788432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6815","nonce":4258788432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6813","nonce":4258788432}]},"public_addr":"172.21.15.104:6809/4258788432","cluster_addr":"172.21.15.104:6811/4258788432","heartbeat_back_addr":"172.21.15.104:6815/4258788432","heartbeat_front_addr":"172.21.15.104:6813/4258788432","state":["exists","up"]},{"osd":5,"uuid":"2fa80b6f-ab69-4e6e-9a5f-7e73a898c300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6817","nonce":2033326841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6819","nonce":2033326841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6823","nonce":2033326841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6821","nonce":2033326841}]},"public_addr":"172.21.15.104:6817/2033326841","cluster_addr":"172.21.15.104:6819/2033326841","heartbeat_back_addr":"172.21.15.104:6823/2033326841","heartbeat_front_addr":"172.21.15.104:6821/2033326841","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:19.861796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:48.482295+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:17.028265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:40.489573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:05.503518+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:30.925548+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3214112922":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/1814853898":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/748209074":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6800/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/50635771":"2024-04-27T04:33:24.670558+0000","172.21.15.90:6800/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/2342166800":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2310050885":"2024-04-27T04:33:57.088900+0000","172.21.15.90:0/424889384":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/3418315403":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2473529454":"2024-04-27T04:33:24.670558+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f38ac06c680 msgr2=0x7f38ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f38ac06c680 0x7f38ac06eb30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f38b8009d10 tx=0x7f38b8009040 comp rx=0 tx=0).stop 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c0104130 msgr2=0x7f38c01963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c0104130 0x7f38c01963b0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f38a8006010 tx=0x7f38a800be20 comp rx=0 tx=0).stop 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 shutdown_connections 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f38ac06c680 0x7f38ac06eb30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f38c00ffbd0 0x7f38c0195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 --2- 172.21.15.90:0/3914291160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f38c0104130 0x7f38c01963b0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.781+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 >> 172.21.15.90:0/3914291160 conn(0x7f38c0075130 msgr2=0x7f38c00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.782+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 shutdown_connections 2024-04-26T04:38:48.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:48.782+0000 7f38c5554700 1 -- 172.21.15.90:0/3914291160 wait complete. 2024-04-26T04:38:49.465 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-26T04:38:49.466 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-26T04:38:49.668 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:49.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:49 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3914291160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:38:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:49 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3914291160' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:38:50.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:50 smithi090 ceph-mon[26648]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:50 smithi104 ceph-mon[32369]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 -- 172.21.15.90:0/131264088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 msgr2=0x7f58580ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 --2- 172.21.15.90:0/131264088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f58580ff790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f585000b0a0 tx=0x7f585000b3b0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 -- 172.21.15.90:0/131264088 shutdown_connections 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 --2- 172.21.15.90:0/131264088 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58580ffd60 0x7f5858103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 --2- 172.21.15.90:0/131264088 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f58580ff790 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 -- 172.21.15.90:0/131264088 >> 172.21.15.90:0/131264088 conn(0x7f58580fac80 msgr2=0x7f58580fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 -- 172.21.15.90:0/131264088 shutdown_connections 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.204+0000 7f585ebd7700 1 -- 172.21.15.90:0/131264088 wait complete. 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585ebd7700 1 Processor -- start 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585ebd7700 1 -- start start 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585ebd7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f5858196cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585ebd7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58580ffd60 0x7f5858197230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585ebd7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5858197880 con 0x7f58580ff3c0 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585ebd7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58581979c0 con 0x7f58580ffd60 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585c973700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f5858196cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585c973700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f5858196cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36310/0 (socket says 172.21.15.90:36310) 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585c973700 1 -- 172.21.15.90:0/3758680228 learned_addr learned my addr 172.21.15.90:0/3758680228 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f5857fff700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58580ffd60 0x7f5858197230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585c973700 1 -- 172.21.15.90:0/3758680228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58580ffd60 msgr2=0x7f5858197230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585c973700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58580ffd60 0x7f5858197230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.205+0000 7f585c973700 1 -- 172.21.15.90:0/3758680228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5850009d00 con 0x7f58580ff3c0 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.206+0000 7f585c973700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f5858196cf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f585000b070 tx=0x7f585000cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.206+0000 7f5855ffb700 1 -- 172.21.15.90:0/3758680228 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585000e040 con 0x7f58580ff3c0 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.206+0000 7f5855ffb700 1 -- 172.21.15.90:0/3758680228 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f58500044f0 con 0x7f58580ff3c0 2024-04-26T04:38:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.206+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585819fd60 con 0x7f58580ff3c0 2024-04-26T04:38:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.206+0000 7f5855ffb700 1 -- 172.21.15.90:0/3758680228 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f585001d9a0 con 0x7f58580ff3c0 2024-04-26T04:38:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.206+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58581a0170 con 0x7f58580ff3c0 2024-04-26T04:38:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.207+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58581076d0 con 0x7f58580ff3c0 2024-04-26T04:38:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.207+0000 7f5855ffb700 1 -- 172.21.15.90:0/3758680228 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5850004660 con 0x7f58580ff3c0 2024-04-26T04:38:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.208+0000 7f5855ffb700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f584406c680 0x7f584406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.208+0000 7f5855ffb700 1 -- 172.21.15.90:0/3758680228 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f585008cf50 con 0x7f58580ff3c0 2024-04-26T04:38:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.208+0000 7f5857fff700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f584406c680 0x7f584406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:51.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.208+0000 7f5857fff700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f584406c680 0x7f584406eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5858069eb0 tx=0x7f5840005d50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:51.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.212+0000 7f5855ffb700 1 -- 172.21.15.90:0/3758680228 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f58500bf720 con 0x7f58580ff3c0 2024-04-26T04:38:51.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.405+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f585804f2a0 con 0x7f58580ff3c0 2024-04-26T04:38:51.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.406+0000 7f5855ffb700 1 -- 172.21.15.90:0/3758680228 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v40) v1 ==== 74+0+11335 (secure 0 0 0) 0x7f58500579f0 con 0x7f58580ff3c0 2024-04-26T04:38:51.407 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:38:51.408 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":40,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","created":"2024-04-26T04:32:57.928472+0000","modified":"2024-04-26T04:38:33.349137+0000","last_up_change":"2024-04-26T04:38:32.344046+0000","last_in_change":"2024-04-26T04:38:13.654124+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:37:19.554153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57491e88-0d1c-41ac-a1f5-3662e5c0a467","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6803","nonce":4217606471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6805","nonce":4217606471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6809","nonce":4217606471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6807","nonce":4217606471}]},"public_addr":"172.21.15.90:6803/4217606471","cluster_addr":"172.21.15.90:6805/4217606471","heartbeat_back_addr":"172.21.15.90:6809/4217606471","heartbeat_front_addr":"172.21.15.90:6807/4217606471","state":["exists","up"]},{"osd":1,"uuid":"fe237f1b-a589-452c-8e5e-98a9a31593eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":29,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6811","nonce":3196794289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6813","nonce":3196794289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6817","nonce":3196794289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6815","nonce":3196794289}]},"public_addr":"172.21.15.90:6811/3196794289","cluster_addr":"172.21.15.90:6813/3196794289","heartbeat_back_addr":"172.21.15.90:6817/3196794289","heartbeat_front_addr":"172.21.15.90:6815/3196794289","state":["exists","up"]},{"osd":2,"uuid":"7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6819","nonce":127929257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6821","nonce":127929257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6825","nonce":127929257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6823","nonce":127929257}]},"public_addr":"172.21.15.90:6819/127929257","cluster_addr":"172.21.15.90:6821/127929257","heartbeat_back_addr":"172.21.15.90:6825/127929257","heartbeat_front_addr":"172.21.15.90:6823/127929257","state":["exists","up"]},{"osd":3,"uuid":"62adc04c-9e7e-4ee5-bb5f-42004b2a615d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6801","nonce":3796716845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6803","nonce":3796716845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6807","nonce":3796716845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6805","nonce":3796716845}]},"public_addr":"172.21.15.104:6801/3796716845","cluster_addr":"172.21.15.104:6803/3796716845","heartbeat_back_addr":"172.21.15.104:6807/3796716845","heartbeat_front_addr":"172.21.15.104:6805/3796716845","state":["exists","up"]},{"osd":4,"uuid":"a4ef2729-84e1-4d2d-868b-3da5fefeb800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6809","nonce":4258788432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6811","nonce":4258788432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6815","nonce":4258788432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6813","nonce":4258788432}]},"public_addr":"172.21.15.104:6809/4258788432","cluster_addr":"172.21.15.104:6811/4258788432","heartbeat_back_addr":"172.21.15.104:6815/4258788432","heartbeat_front_addr":"172.21.15.104:6813/4258788432","state":["exists","up"]},{"osd":5,"uuid":"2fa80b6f-ab69-4e6e-9a5f-7e73a898c300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6817","nonce":2033326841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6819","nonce":2033326841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6823","nonce":2033326841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6821","nonce":2033326841}]},"public_addr":"172.21.15.104:6817/2033326841","cluster_addr":"172.21.15.104:6819/2033326841","heartbeat_back_addr":"172.21.15.104:6823/2033326841","heartbeat_front_addr":"172.21.15.104:6821/2033326841","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:19.861796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:48.482295+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:17.028265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:40.489573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:05.503518+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:30.925548+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:0/3214112922":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/1814853898":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/748209074":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6800/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/50635771":"2024-04-27T04:33:24.670558+0000","172.21.15.90:6800/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/2342166800":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2310050885":"2024-04-27T04:33:57.088900+0000","172.21.15.90:0/424889384":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/3418315403":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2473529454":"2024-04-27T04:33:24.670558+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f584406c680 msgr2=0x7f584406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f584406c680 0x7f584406eb30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5858069eb0 tx=0x7f5840005d50 comp rx=0 tx=0).stop 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 msgr2=0x7f5858196cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f5858196cf0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f585000b070 tx=0x7f585000cdb0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 shutdown_connections 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f584406c680 0x7f584406eb30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58580ff3c0 0x7f5858196cf0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 --2- 172.21.15.90:0/3758680228 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58580ffd60 0x7f5858197230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.408+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 >> 172.21.15.90:0/3758680228 conn(0x7f58580fac80 msgr2=0x7f58580fc320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:51.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.409+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 shutdown_connections 2024-04-26T04:38:51.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:51.409+0000 7f585ebd7700 1 -- 172.21.15.90:0/3758680228 wait complete. 2024-04-26T04:38:52.111 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-26T04:38:52.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-26T04:38:52.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-26T04:38:52.112 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-26T04:38:52.113 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-26T04:38:52.113 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-26T04:38:52.381 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:52.403 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:52.641 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:52 smithi090 ceph-mon[26648]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:52.641 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:52 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3758680228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:38:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:52 smithi104 ceph-mon[32369]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:52 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3758680228' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:38:53.166 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:53.167 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:53.167 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:54.354 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:54 smithi090 ceph-mon[26648]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:54.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:54 smithi104 ceph-mon[32369]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:56 smithi090 ceph-mon[26648]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:56.736 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:38:56.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:56 smithi104 ceph-mon[32369]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:57.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 -- 172.21.15.90:0/1432180381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e800ffbd0 msgr2=0x7f5e80100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 --2- 172.21.15.90:0/1432180381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e800ffbd0 0x7f5e80100040 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f5e74009910 tx=0x7f5e74009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:57.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 -- 172.21.15.90:0/1432180381 shutdown_connections 2024-04-26T04:38:57.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 --2- 172.21.15.90:0/1432180381 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e800ffbd0 0x7f5e80100040 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 --2- 172.21.15.90:0/1432180381 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e80104130 0x7f5e80104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 -- 172.21.15.90:0/1432180381 >> 172.21.15.90:0/1432180381 conn(0x7f5e80075130 msgr2=0x7f5e80075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 -- 172.21.15.90:0/1432180381 shutdown_connections 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.485+0000 7f5e863f5700 1 -- 172.21.15.90:0/1432180381 wait complete. 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e863f5700 1 Processor -- start 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e863f5700 1 -- start start 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e863f5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e800ffbd0 0x7f5e80195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e863f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104130 0x7f5e80196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e863f5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e80196a70 con 0x7f5e80104130 2024-04-26T04:38:57.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e863f5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e8019a800 con 0x7f5e800ffbd0 2024-04-26T04:38:57.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104130 0x7f5e80196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e7f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104130 0x7f5e80196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46796/0 (socket says 172.21.15.90:46796) 2024-04-26T04:38:57.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e7f7fe700 1 -- 172.21.15.90:0/4004358102 learned_addr learned my addr 172.21.15.90:0/4004358102 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:57.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e7ffff700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e800ffbd0 0x7f5e80195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e7f7fe700 1 -- 172.21.15.90:0/4004358102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e800ffbd0 msgr2=0x7f5e80195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.491 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.486+0000 7f5e7f7fe700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e800ffbd0 0x7f5e80195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e7f7fe700 1 -- 172.21.15.90:0/4004358102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e740095f0 con 0x7f5e80104130 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e7ffff700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e800ffbd0 0x7f5e80195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e7f7fe700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104130 0x7f5e80196390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400bd70 tx=0x7f5e7400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e7401d070 con 0x7f5e80104130 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5e7400f460 con 0x7f5e80104130 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e8019aa80 con 0x7f5e80104130 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5e74017630 con 0x7f5e80104130 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.487+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e8019af70 con 0x7f5e80104130 2024-04-26T04:38:57.492 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.488+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5e8010c5c0 con 0x7f5e80104130 2024-04-26T04:38:57.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.488+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5e74003a60 con 0x7f5e80104130 2024-04-26T04:38:57.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e7d7fa700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5e6806c680 0x7f5e6806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f5e7408ca00 con 0x7f5e80104130 2024-04-26T04:38:57.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e7ffff700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5e6806c680 0x7f5e6806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e7d7fa700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] conn(0x7f5e68072350 0x7f5e68074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 --> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5e68074e10 con 0x7f5e68072350 2024-04-26T04:38:57.495 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5e740912a0 con 0x7f5e80104130 2024-04-26T04:38:57.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e84992700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] conn(0x7f5e68072350 0x7f5e68074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e7ffff700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5e6806c680 0x7f5e6806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005950 tx=0x7f5e700058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:57.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.489+0000 7f5e84992700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] conn(0x7f5e68072350 0x7f5e68074760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:57.496 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.490+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== osd.1 v2:172.21.15.90:6810/3196794289 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f5e68074e10 con 0x7f5e68072350 2024-04-26T04:38:57.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 --> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5e8004f2a0 con 0x7f5e68072350 2024-04-26T04:38:57.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e7d7fa700 1 -- 172.21.15.90:0/4004358102 <== osd.1 v2:172.21.15.90:6810/3196794289 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5e8004f2a0 con 0x7f5e68072350 2024-04-26T04:38:57.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] conn(0x7f5e68072350 msgr2=0x7f5e68074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e863f5700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] conn(0x7f5e68072350 0x7f5e68074760 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5e6806c680 msgr2=0x7f5e6806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e863f5700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5e6806c680 0x7f5e6806eb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5e70005950 tx=0x7f5e700058e0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104130 msgr2=0x7f5e80196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.506+0000 7f5e863f5700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104130 0x7f5e80196390 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f5e7400bd70 tx=0x7f5e7400be50 comp rx=0 tx=0).stop 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 shutdown_connections 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6810/3196794289,v1:172.21.15.90:6811/3196794289] conn(0x7f5e68072350 0x7f5e68074760 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5e6806c680 0x7f5e6806eb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5e800ffbd0 0x7f5e80195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 --2- 172.21.15.90:0/4004358102 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5e80104130 0x7f5e80196390 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 >> 172.21.15.90:0/4004358102 conn(0x7f5e80075130 msgr2=0x7f5e800febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 shutdown_connections 2024-04-26T04:38:57.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.507+0000 7f5e863f5700 1 -- 172.21.15.90:0/4004358102 wait complete. 2024-04-26T04:38:57.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.635+0000 7fdcece7d700 1 -- 172.21.15.90:0/2283554189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ffc60 msgr2=0x7fdce810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.635+0000 7fdcece7d700 1 --2- 172.21.15.90:0/2283554189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ffc60 0x7fdce810a6d0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fdcdc009910 tx=0x7fdcdc009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 -- 172.21.15.90:0/2283554189 shutdown_connections 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 --2- 172.21.15.90:0/2283554189 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ffc60 0x7fdce810a6d0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 --2- 172.21.15.90:0/2283554189 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce80ff350 0x7fdce80ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 -- 172.21.15.90:0/2283554189 >> 172.21.15.90:0/2283554189 conn(0x7fdce80748b0 msgr2=0x7fdce8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 -- 172.21.15.90:0/2283554189 shutdown_connections 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 -- 172.21.15.90:0/2283554189 wait complete. 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 Processor -- start 2024-04-26T04:38:57.830 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.636+0000 7fdcece7d700 1 -- start start 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcece7d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ff350 0x7fdce8195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcece7d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce80ffc60 0x7fdce8196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcece7d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8196a70 con 0x7fdce80ff350 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcece7d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce8196bb0 con 0x7fdce80ffc60 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ff350 0x7fdce8195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ff350 0x7fdce8195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46824/0 (socket says 172.21.15.90:46824) 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce659c700 1 -- 172.21.15.90:0/992882280 learned_addr learned my addr 172.21.15.90:0/992882280 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce5d9b700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce80ffc60 0x7fdce8196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce659c700 1 -- 172.21.15.90:0/992882280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce80ffc60 msgr2=0x7fdce8196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce659c700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce80ffc60 0x7fdce8196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.831 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce659c700 1 -- 172.21.15.90:0/992882280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcdc0095f0 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdce659c700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ff350 0x7fdce8195ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800c910 tx=0x7fdcd800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd80041f0 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdcd800ce60 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce819aa00 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdcd8003d20 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.637+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce819af50 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.638+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdce810d730 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fdcd8007900 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdcd77fe700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdcd006c680 0x7fdcd006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdce5d9b700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdcd006c680 0x7fdcd006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fdcd808b210 con 0x7fdce80ff350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdcd77fe700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] conn(0x7fdcd0072350 0x7fdcd0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 --> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdcd0074e10 con 0x7fdcd0072350 2024-04-26T04:38:57.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdcd808d8e0 con 0x7fdce80ff350 2024-04-26T04:38:57.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.639+0000 7fdce6d9d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] conn(0x7fdcd0072350 0x7fdcd0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:57.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.640+0000 7fdce5d9b700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdcd006c680 0x7fdcd006eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdce8197500 tx=0x7fdcdc01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:57.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.640+0000 7fdce6d9d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] conn(0x7fdcd0072350 0x7fdcd0074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:57.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.640+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== osd.0 v2:172.21.15.90:6802/4217606471 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fdcd0074e10 con 0x7fdcd0072350 2024-04-26T04:38:57.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 --> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fdce804f2a0 con 0x7fdcd0072350 2024-04-26T04:38:57.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcd77fe700 1 -- 172.21.15.90:0/992882280 <== osd.0 v2:172.21.15.90:6802/4217606471 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fdce804f2a0 con 0x7fdcd0072350 2024-04-26T04:38:57.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] conn(0x7fdcd0072350 msgr2=0x7fdcd0074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcece7d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] conn(0x7fdcd0072350 0x7fdcd0074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdcd006c680 msgr2=0x7fdcd006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcece7d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdcd006c680 0x7fdcd006eb30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdce8197500 tx=0x7fdcdc01a040 comp rx=0 tx=0).stop 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ff350 msgr2=0x7fdce8195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.658+0000 7fdcece7d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ff350 0x7fdce8195ee0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fdcd800c910 tx=0x7fdcd800cc20 comp rx=0 tx=0).stop 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 shutdown_connections 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6802/4217606471,v1:172.21.15.90:6803/4217606471] conn(0x7fdcd0072350 0x7fdcd0074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdcd006c680 0x7fdcd006eb30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdce80ff350 0x7fdce8195ee0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 --2- 172.21.15.90:0/992882280 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdce80ffc60 0x7fdce8196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 >> 172.21.15.90:0/992882280 conn(0x7fdce80748b0 msgr2=0x7fdce8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:57.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 shutdown_connections 2024-04-26T04:38:57.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:57.659+0000 7fdcece7d700 1 -- 172.21.15.90:0/992882280 wait complete. 2024-04-26T04:38:58.431 INFO:teuthology.orchestra.run.smithi090.stdout:38654705699 2024-04-26T04:38:58.431 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-26T04:38:58.454 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:38:58 smithi090 ceph-mon[26648]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:58.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:38:58 smithi104 ceph-mon[32369]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:38:59.617 INFO:teuthology.orchestra.run.smithi090.stdout:60129542172 2024-04-26T04:38:59.617 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-26T04:38:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 -- 172.21.15.90:0/3963413318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ffd60 msgr2=0x7f99c8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3963413318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ffd60 0x7f99c8103d30 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f99bc009910 tx=0x7f99bc009c20 comp rx=0 tx=0).stop 2024-04-26T04:38:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 -- 172.21.15.90:0/3963413318 shutdown_connections 2024-04-26T04:38:59.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3963413318 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ffd60 0x7f99c8103d30 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3963413318 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c80ff3c0 0x7f99c80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 -- 172.21.15.90:0/3963413318 >> 172.21.15.90:0/3963413318 conn(0x7f99c80fac80 msgr2=0x7f99c80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 -- 172.21.15.90:0/3963413318 shutdown_connections 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.295+0000 7f99ce87b700 1 -- 172.21.15.90:0/3963413318 wait complete. 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.296+0000 7f99ce87b700 1 Processor -- start 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.296+0000 7f99ce87b700 1 -- start start 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.296+0000 7f99ce87b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ff3c0 0x7f99c8196e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.296+0000 7f99ce87b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c80ffd60 0x7f99c8197340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.296+0000 7f99ce87b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8197960 con 0x7f99c80ff3c0 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.296+0000 7f99ce87b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99c8197aa0 con 0x7f99c80ffd60 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.296+0000 7f99c7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ff3c0 0x7f99c8196e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ff3c0 0x7f99c8196e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46854/0 (socket says 172.21.15.90:46854) 2024-04-26T04:38:59.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c7fff700 1 -- 172.21.15.90:0/3877058078 learned_addr learned my addr 172.21.15.90:0/3877058078 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c77fe700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c80ffd60 0x7f99c8197340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c7fff700 1 -- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c80ffd60 msgr2=0x7f99c8197340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c7fff700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c80ffd60 0x7f99c8197340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c7fff700 1 -- 172.21.15.90:0/3877058078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99bc0095f0 con 0x7f99c80ff3c0 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c7fff700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ff3c0 0x7f99c8196e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b4b0 tx=0x7f99b800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99b8011800 con 0x7f99c80ff3c0 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f99b8011e10 con 0x7f99c80ff3c0 2024-04-26T04:38:59.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99c819fe70 con 0x7f99c80ff3c0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f99b800f2e0 con 0x7f99c80ff3c0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.297+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99c81a02e0 con 0x7f99c80ff3c0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.298+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f99c810c570 con 0x7f99c80ff3c0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f99b8010380 con 0x7f99c80ff3c0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c57fa700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f99b006c680 0x7f99b006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c77fe700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f99b006c680 0x7f99b006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f99b808b740 con 0x7f99c80ff3c0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c57fa700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] conn(0x7f99b0072350 0x7f99b0074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 --> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f99b0074e10 con 0x7f99b0072350 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f99b808bba0 con 0x7f99c80ff3c0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99c77fe700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f99b006c680 0x7f99b006eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006010 tx=0x7f99bc01a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.299+0000 7f99cce18700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] conn(0x7f99b0072350 0x7f99b0074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.300+0000 7f99cce18700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] conn(0x7f99b0072350 0x7f99b0074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:38:59.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.301+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== osd.3 v2:172.21.15.104:6800/3796716845 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f99b0074e10 con 0x7f99b0072350 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.316+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 --> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f99c804f2a0 con 0x7f99b0072350 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99c57fa700 1 -- 172.21.15.90:0/3877058078 <== osd.3 v2:172.21.15.104:6800/3796716845 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f99c804f2a0 con 0x7f99b0072350 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] conn(0x7f99b0072350 msgr2=0x7f99b0074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] conn(0x7f99b0072350 0x7f99b0074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f99b006c680 msgr2=0x7f99b006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f99b006c680 0x7f99b006eb30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f99bc006010 tx=0x7f99bc01a040 comp rx=0 tx=0).stop 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ff3c0 msgr2=0x7f99c8196e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ff3c0 0x7f99c8196e00 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f99b800b4b0 tx=0x7f99b800b870 comp rx=0 tx=0).stop 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 shutdown_connections 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:6800/3796716845,v1:172.21.15.104:6801/3796716845] conn(0x7f99b0072350 0x7f99b0074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f99b006c680 0x7f99b006eb30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f99c80ff3c0 0x7f99c8196e00 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 --2- 172.21.15.90:0/3877058078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f99c80ffd60 0x7f99c8197340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:38:59.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 >> 172.21.15.90:0/3877058078 conn(0x7f99c80fac80 msgr2=0x7f99c80fc350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:38:59.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 shutdown_connections 2024-04-26T04:38:59.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:38:59.317+0000 7f99ce87b700 1 -- 172.21.15.90:0/3877058078 wait complete. 2024-04-26T04:39:00.506 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:00 smithi090 ceph-mon[26648]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:00.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:00 smithi104 ceph-mon[32369]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:01.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 -- 172.21.15.90:0/1029178584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 msgr2=0x7f7b74104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/1029178584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 0x7f7b74104000 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009910 tx=0x7f7b64009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 -- 172.21.15.90:0/1029178584 shutdown_connections 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/1029178584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 0x7f7b74104000 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/1029178584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 0x7f7b740ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 -- 172.21.15.90:0/1029178584 >> 172.21.15.90:0/1029178584 conn(0x7f7b740fac80 msgr2=0x7f7b740fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 -- 172.21.15.90:0/1029178584 shutdown_connections 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.792+0000 7f7b7bebb700 1 -- 172.21.15.90:0/1029178584 wait complete. 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.793+0000 7f7b7bebb700 1 Processor -- start 2024-04-26T04:39:01.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.793+0000 7f7b7bebb700 1 -- start start 2024-04-26T04:39:01.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.793+0000 7f7b7bebb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 0x7f7b7419c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.793+0000 7f7b7bebb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 0x7f7b7419ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b7bebb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b7419d110 con 0x7f7b740ffb20 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b7bebb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b741a0ea0 con 0x7f7b740ff210 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79c57700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 0x7f7b7419c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79456700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 0x7f7b7419ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79c57700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 0x7f7b7419c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.90:43310/0 (socket says 172.21.15.90:43310) 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79c57700 1 -- 172.21.15.90:0/2110885156 learned_addr learned my addr 172.21.15.90:0/2110885156 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79456700 1 -- 172.21.15.90:0/2110885156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 msgr2=0x7f7b7419c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79456700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 0x7f7b7419c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79456700 1 -- 172.21.15.90:0/2110885156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b640095f0 con 0x7f7b740ffb20 2024-04-26T04:39:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79c57700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 0x7f7b7419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:39:01.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b79456700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 0x7f7b7419ca30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009910 tx=0x7f7b6400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:01.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.794+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b6401d070 con 0x7f7b740ffb20 2024-04-26T04:39:01.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.795+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7b6400f460 con 0x7f7b740ffb20 2024-04-26T04:39:01.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.795+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b64017630 con 0x7f7b740ffb20 2024-04-26T04:39:01.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.795+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b741a1120 con 0x7f7b740ffb20 2024-04-26T04:39:01.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.795+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b741a15c0 con 0x7f7b740ffb20 2024-04-26T04:39:01.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.796+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7b74113c30 con 0x7f7b740ffb20 2024-04-26T04:39:01.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.797+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7b6400fab0 con 0x7f7b740ffb20 2024-04-26T04:39:01.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.797+0000 7f7b6affd700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b6006c680 0x7f7b6006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.797+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7b6408c960 con 0x7f7b740ffb20 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.798+0000 7f7b79c57700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b6006c680 0x7f7b6006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.798+0000 7f7b6affd700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] conn(0x7f7b60072350 0x7f7b60074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.798+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 --> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7b60074e10 con 0x7f7b60072350 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.798+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7b640912a0 con 0x7f7b740ffb20 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.798+0000 7f7b7a458700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] conn(0x7f7b60072350 0x7f7b60074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.798+0000 7f7b79c57700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b6006c680 0x7f7b6006eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b70005950 tx=0x7f7b7000b3a0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.798+0000 7f7b7a458700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] conn(0x7f7b60072350 0x7f7b60074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.799+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== osd.2 v2:172.21.15.90:6818/127929257 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7b60074e10 con 0x7f7b60072350 2024-04-26T04:39:01.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.815+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 --> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7b7404f2a0 con 0x7f7b60072350 2024-04-26T04:39:01.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.816+0000 7f7b6affd700 1 -- 172.21.15.90:0/2110885156 <== osd.2 v2:172.21.15.90:6818/127929257 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7b7404f2a0 con 0x7f7b60072350 2024-04-26T04:39:01.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.816+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] conn(0x7f7b60072350 msgr2=0x7f7b60074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.816+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] conn(0x7f7b60072350 0x7f7b60074760 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.816+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b6006c680 msgr2=0x7f7b6006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.816+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b6006c680 0x7f7b6006eb30 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7b70005950 tx=0x7f7b7000b3a0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.816+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 msgr2=0x7f7b7419ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.816+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 0x7f7b7419ca30 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f7b64009910 tx=0x7f7b6400be20 comp rx=0 tx=0).stop 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 shutdown_connections 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b6006c680 0x7f7b6006eb30 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:6818/127929257,v1:172.21.15.90:6819/127929257] conn(0x7f7b60072350 0x7f7b60074760 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b740ff210 0x7f7b7419c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 --2- 172.21.15.90:0/2110885156 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b740ffb20 0x7f7b7419ca30 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 >> 172.21.15.90:0/2110885156 conn(0x7f7b740fac80 msgr2=0x7f7b74110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 shutdown_connections 2024-04-26T04:39:01.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.817+0000 7f7b7bebb700 1 -- 172.21.15.90:0/2110885156 wait complete. 2024-04-26T04:39:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3603975435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 msgr2=0x7f3d540ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3603975435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d540ff720 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3d44009910 tx=0x7f3d44009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3603975435 shutdown_connections 2024-04-26T04:39:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3603975435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d540ffc60 0x7f3d5410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3603975435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d540ff720 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3603975435 >> 172.21.15.90:0/3603975435 conn(0x7f3d540748b0 msgr2=0x7f3d54074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3603975435 shutdown_connections 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.688+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3603975435 wait complete. 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d5a7e9700 1 Processor -- start 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d5a7e9700 1 -- start start 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d5a7e9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d54195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d5a7e9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d540ffc60 0x7f3d541964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d5a7e9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54196af0 con 0x7f3d540ff350 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d5a7e9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d54196c30 con 0x7f3d540ffc60 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d53fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d54195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d53fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d54195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46874/0 (socket says 172.21.15.90:46874) 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.689+0000 7f3d53fff700 1 -- 172.21.15.90:0/3080568488 learned_addr learned my addr 172.21.15.90:0/3080568488 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d537fe700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d540ffc60 0x7f3d541964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d53fff700 1 -- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d540ffc60 msgr2=0x7f3d541964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d53fff700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d540ffc60 0x7f3d541964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d53fff700 1 -- 172.21.15.90:0/3080568488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d440095f0 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d53fff700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d54195f60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400bda0 tx=0x7f3d4400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d4401d070 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d5419aa20 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3d44022470 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3d4400f690 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.690+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d5419af10 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.691+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3d5410d730 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.692+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3d44022b00 con 0x7f3d540ff350 2024-04-26T04:39:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.692+0000 7f3d517fa700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3d3c06c680 0x7f3d3c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.692+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f3d4408d8c0 con 0x7f3d540ff350 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.692+0000 7f3d537fe700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3d3c06c680 0x7f3d3c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.693+0000 7f3d517fa700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] conn(0x7f3d3c072350 0x7f3d3c074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.693+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 --> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3d3c074e10 con 0x7f3d3c072350 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.693+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d4408dd20 con 0x7f3d540ff350 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.693+0000 7f3d537fe700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3d3c06c680 0x7f3d3c06eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d48005ea0 tx=0x7f3d48005e30 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.693+0000 7f3d58d86700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] conn(0x7f3d3c072350 0x7f3d3c074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.693+0000 7f3d58d86700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] conn(0x7f3d3c072350 0x7f3d3c074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:01.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.694+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== osd.5 v2:172.21.15.104:6816/2033326841 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f3d3c074e10 con 0x7f3d3c072350 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.723+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 --> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3d5404f2a0 con 0x7f3d3c072350 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d517fa700 1 -- 172.21.15.90:0/3080568488 <== osd.5 v2:172.21.15.104:6816/2033326841 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3d5404f2a0 con 0x7f3d3c072350 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] conn(0x7f3d3c072350 msgr2=0x7f3d3c074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] conn(0x7f3d3c072350 0x7f3d3c074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3d3c06c680 msgr2=0x7f3d3c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3d3c06c680 0x7f3d3c06eb30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3d48005ea0 tx=0x7f3d48005e30 comp rx=0 tx=0).stop 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 msgr2=0x7f3d54195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d54195f60 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3d4400bda0 tx=0x7f3d4400be80 comp rx=0 tx=0).stop 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 shutdown_connections 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:6816/2033326841,v1:172.21.15.104:6817/2033326841] conn(0x7f3d3c072350 0x7f3d3c074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3d3c06c680 0x7f3d3c06eb30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d540ff350 0x7f3d54195f60 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 --2- 172.21.15.90:0/3080568488 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d540ffc60 0x7f3d541964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:01.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 >> 172.21.15.90:0/3080568488 conn(0x7f3d540748b0 msgr2=0x7f3d54104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:01.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.724+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 shutdown_connections 2024-04-26T04:39:01.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:01.725+0000 7f3d5a7e9700 1 -- 172.21.15.90:0/3080568488 wait complete. 2024-04-26T04:39:02.538 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:02 smithi090 ceph-mon[26648]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:02.560 INFO:teuthology.orchestra.run.smithi090.stdout:111669149715 2024-04-26T04:39:02.560 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-26T04:39:02.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:02 smithi104 ceph-mon[32369]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:03.129 INFO:teuthology.orchestra.run.smithi090.stdout:81604378648 2024-04-26T04:39:03.129 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-26T04:39:03.822 INFO:teuthology.orchestra.run.smithi090.stdout:167503724553 2024-04-26T04:39:03.822 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-26T04:39:04.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 -- 172.21.15.90:0/3775436745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101d30 msgr2=0x7f455c10a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 --2- 172.21.15.90:0/3775436745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101d30 0x7f455c10a230 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4544009910 tx=0x7f4544009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 -- 172.21.15.90:0/3775436745 shutdown_connections 2024-04-26T04:39:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 --2- 172.21.15.90:0/3775436745 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101d30 0x7f455c10a230 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 --2- 172.21.15.90:0/3775436745 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f455c101420 0x7f455c1017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 -- 172.21.15.90:0/3775436745 >> 172.21.15.90:0/3775436745 conn(0x7f455c0fac80 msgr2=0x7f455c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 -- 172.21.15.90:0/3775436745 shutdown_connections 2024-04-26T04:39:04.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.047+0000 7f45610af700 1 -- 172.21.15.90:0/3775436745 wait complete. 2024-04-26T04:39:04.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f45610af700 1 Processor -- start 2024-04-26T04:39:04.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f45610af700 1 -- start start 2024-04-26T04:39:04.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f45610af700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101420 0x7f455c195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:04.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f45610af700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f455c101d30 0x7f455c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:04.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f45610af700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c196b00 con 0x7f455c101420 2024-04-26T04:39:04.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f45610af700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c19a890 con 0x7f455c101d30 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f455ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101420 0x7f455c195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f455a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f455c101d30 0x7f455c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f455ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101420 0x7f455c195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46928/0 (socket says 172.21.15.90:46928) 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.048+0000 7f455ad9d700 1 -- 172.21.15.90:0/1715007662 learned_addr learned my addr 172.21.15.90:0/1715007662 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f455ad9d700 1 -- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f455c101d30 msgr2=0x7f455c196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f455ad9d700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f455c101d30 0x7f455c196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f455ad9d700 1 -- 172.21.15.90:0/1715007662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45440095f0 con 0x7f455c101420 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f455ad9d700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101420 0x7f455c195ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f454c00ba50 tx=0x7f454c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f454c00c900 con 0x7f455c101420 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f454c014440 con 0x7f455c101420 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f454c0075a0 con 0x7f455c101420 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c19ab70 con 0x7f455c101420 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.049+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c19b0c0 con 0x7f455c101420 2024-04-26T04:39:04.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.050+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f455c10d6b0 con 0x7f455c101420 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.051+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f454c0145b0 con 0x7f455c101420 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.051+0000 7f4553fff700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f454806c680 0x7f454806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.051+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f454c006e40 con 0x7f455c101420 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.052+0000 7f455a59c700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f454806c680 0x7f454806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.052+0000 7f4553fff700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] conn(0x7f4548072350 0x7f4548074760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.052+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 --> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4548074e10 con 0x7f4548072350 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.052+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f454c08b630 con 0x7f455c101420 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.052+0000 7f455b59e700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] conn(0x7f4548072350 0x7f4548074760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:04.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.052+0000 7f455a59c700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f454806c680 0x7f454806eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f455c197500 tx=0x7f454401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:04.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.053+0000 7f455b59e700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] conn(0x7f4548072350 0x7f4548074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:04.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.054+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== osd.4 v2:172.21.15.104:6808/4258788432 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4548074e10 con 0x7f4548072350 2024-04-26T04:39:04.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.070+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 --> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f455c04f2a0 con 0x7f4548072350 2024-04-26T04:39:04.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f4553fff700 1 -- 172.21.15.90:0/1715007662 <== osd.4 v2:172.21.15.104:6808/4258788432 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f455c04f2a0 con 0x7f4548072350 2024-04-26T04:39:04.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] conn(0x7f4548072350 msgr2=0x7f4548074760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:04.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] conn(0x7f4548072350 0x7f4548074760 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f454806c680 msgr2=0x7f454806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:04.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f454806c680 0x7f454806eb30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f455c197500 tx=0x7f454401a040 comp rx=0 tx=0).stop 2024-04-26T04:39:04.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101420 msgr2=0x7f455c195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:04.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101420 0x7f455c195ee0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f454c00ba50 tx=0x7f454c00bd60 comp rx=0 tx=0).stop 2024-04-26T04:39:04.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 shutdown_connections 2024-04-26T04:39:04.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f454806c680 0x7f454806eb30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:6808/4258788432,v1:172.21.15.104:6809/4258788432] conn(0x7f4548072350 0x7f4548074760 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f455c101420 0x7f455c195ee0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 --2- 172.21.15.90:0/1715007662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f455c101d30 0x7f455c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:04.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 >> 172.21.15.90:0/1715007662 conn(0x7f455c0fac80 msgr2=0x7f455c1049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:04.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 shutdown_connections 2024-04-26T04:39:04.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:04.071+0000 7f45610af700 1 -- 172.21.15.90:0/1715007662 wait complete. 2024-04-26T04:39:04.388 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:04 smithi090 ceph-mon[26648]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:04.388 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:04.474 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:04 smithi104 ceph-mon[32369]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:04.474 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:04.677 INFO:teuthology.orchestra.run.smithi090.stdout:146028888079 2024-04-26T04:39:04.677 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-26T04:39:04.722 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:04.725 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:04.725 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:04.726 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:06.582 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:06 smithi104 ceph-mon[32369]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:06.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:06 smithi090 ceph-mon[26648]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:08.078 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:08.631 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:08 smithi090 ceph-mon[26648]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:08.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:08 smithi104 ceph-mon[32369]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:09.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 -- 172.21.15.90:0/1142254856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 msgr2=0x7f8d38100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:09.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/1142254856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38100040 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c009910 tx=0x7f8d2c009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:09.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 -- 172.21.15.90:0/1142254856 shutdown_connections 2024-04-26T04:39:09.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/1142254856 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38100040 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:09.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/1142254856 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d38104130 0x7f8d38104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:09.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 -- 172.21.15.90:0/1142254856 >> 172.21.15.90:0/1142254856 conn(0x7f8d38075130 msgr2=0x7f8d38075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:09.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 -- 172.21.15.90:0/1142254856 shutdown_connections 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.485+0000 7f8d3d89a700 1 -- 172.21.15.90:0/1142254856 wait complete. 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d3d89a700 1 Processor -- start 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d3d89a700 1 -- start start 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d3d89a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d3d89a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d38104130 0x7f8d38197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d3d89a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d38197880 con 0x7f8d380ffbd0 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d3d89a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d381979c0 con 0x7f8d38104130 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:09.900 INFO:teuthology.orchestra.run.smithi090.stdout:60129542174 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d36ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37436/0 (socket says 172.21.15.90:37436) 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.486+0000 7f8d36ffd700 1 -- 172.21.15.90:0/2641201569 learned_addr learned my addr 172.21.15.90:0/2641201569 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d36ffd700 1 -- 172.21.15.90:0/2641201569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d38104130 msgr2=0x7f8d38197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d36ffd700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d38104130 0x7f8d38197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d36ffd700 1 -- 172.21.15.90:0/2641201569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d2c0095f0 con 0x7f8d380ffbd0 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d36ffd700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38196d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000ba50 tx=0x7f8d3000be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d3c898700 1 -- 172.21.15.90:0/2641201569 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d3000ca00 con 0x7f8d380ffbd0 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d3c898700 1 -- 172.21.15.90:0/2641201569 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8d3000cb60 con 0x7f8d380ffbd0 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d3819fd90 con 0x7f8d380ffbd0 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d3c898700 1 -- 172.21.15.90:0/2641201569 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d30007580 con 0x7f8d380ffbd0 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.487+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d381a0250 con 0x7f8d380ffbd0 2024-04-26T04:39:09.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.488+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d381076a0 con 0x7f8d380ffbd0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.488+0000 7f8d3c898700 1 -- 172.21.15.90:0/2641201569 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8d3000ccd0 con 0x7f8d380ffbd0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.489+0000 7f8d3c898700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8d2006c680 0x7f8d2006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.489+0000 7f8d367fc700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8d2006c680 0x7f8d2006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.489+0000 7f8d3c898700 1 -- 172.21.15.90:0/2641201569 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f8d30006e40 con 0x7f8d380ffbd0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.489+0000 7f8d367fc700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8d2006c680 0x7f8d2006eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00b5c0 tx=0x7f8d2c005e20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.493+0000 7f8d3c898700 1 -- 172.21.15.90:0/2641201569 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8d3004e720 con 0x7f8d380ffbd0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.689+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f8d3804f2a0 con 0x7f8d380ffbd0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.689+0000 7f8d3c898700 1 -- 172.21.15.90:0/2641201569 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8d30019020 con 0x7f8d380ffbd0 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8d2006c680 msgr2=0x7f8d2006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8d2006c680 0x7f8d2006eb30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f8d2c00b5c0 tx=0x7f8d2c005e20 comp rx=0 tx=0).stop 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 msgr2=0x7f8d38196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38196d20 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000ba50 tx=0x7f8d3000be10 comp rx=0 tx=0).stop 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 shutdown_connections 2024-04-26T04:39:09.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8d2006c680 0x7f8d2006eb30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:09.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d380ffbd0 0x7f8d38196d20 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:09.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 --2- 172.21.15.90:0/2641201569 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d38104130 0x7f8d38197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:09.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 >> 172.21.15.90:0/2641201569 conn(0x7f8d38075130 msgr2=0x7f8d380fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:09.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 shutdown_connections 2024-04-26T04:39:09.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:09.691+0000 7f8d3d89a700 1 -- 172.21.15.90:0/2641201569 wait complete. 2024-04-26T04:39:10.520 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:10 smithi090 ceph-mon[26648]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:10.520 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:10 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2641201569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T04:39:10.541 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542172 got 60129542174 for osd.1 2024-04-26T04:39:10.541 DEBUG:teuthology.parallel:result is None 2024-04-26T04:39:10.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 -- 172.21.15.90:0/1306045090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0301004e0 msgr2=0x7fb0301008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:10.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 --2- 172.21.15.90:0/1306045090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0301004e0 0x7fb0301008b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb020009910 tx=0x7fb020009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:10.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 -- 172.21.15.90:0/1306045090 shutdown_connections 2024-04-26T04:39:10.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 --2- 172.21.15.90:0/1306045090 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb030100e80 0x7fb030104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:10.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 --2- 172.21.15.90:0/1306045090 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0301004e0 0x7fb0301008b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:10.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 -- 172.21.15.90:0/1306045090 >> 172.21.15.90:0/1306045090 conn(0x7fb0300fbda0 msgr2=0x7fb0300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:10.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 -- 172.21.15.90:0/1306045090 shutdown_connections 2024-04-26T04:39:10.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 -- 172.21.15.90:0/1306045090 wait complete. 2024-04-26T04:39:10.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.729+0000 7fb0357f5700 1 Processor -- start 2024-04-26T04:39:10.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0357f5700 1 -- start start 2024-04-26T04:39:10.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0357f5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0301004e0 0x7fb030195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:10.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0357f5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb030100e80 0x7fb030196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:10.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0357f5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb030196b10 con 0x7fb030100e80 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0357f5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03019a8a0 con 0x7fb0301004e0 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0267fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb030100e80 0x7fb030196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0267fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb030100e80 0x7fb030196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37446/0 (socket says 172.21.15.90:37446) 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0267fc700 1 -- 172.21.15.90:0/3455416792 learned_addr learned my addr 172.21.15.90:0/3455416792 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb02effd700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0301004e0 0x7fb030195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0267fc700 1 -- 172.21.15.90:0/3455416792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0301004e0 msgr2=0x7fb030195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0267fc700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0301004e0 0x7fb030195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0267fc700 1 -- 172.21.15.90:0/3455416792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0200095f0 con 0x7fb030100e80 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0267fc700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb030100e80 0x7fb030196430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb01800d900 tx=0x7fb01800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb02cff9700 1 -- 172.21.15.90:0/3455416792 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb018009900 con 0x7fb030100e80 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb02cff9700 1 -- 172.21.15.90:0/3455416792 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb018010460 con 0x7fb030100e80 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb02cff9700 1 -- 172.21.15.90:0/3455416792 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb01800f5f0 con 0x7fb030100e80 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.730+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb03019ab80 con 0x7fb030100e80 2024-04-26T04:39:10.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.731+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb03019b0d0 con 0x7fb030100e80 2024-04-26T04:39:10.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.731+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0301087f0 con 0x7fb030100e80 2024-04-26T04:39:10.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.732+0000 7fb02cff9700 1 -- 172.21.15.90:0/3455416792 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb0180105d0 con 0x7fb030100e80 2024-04-26T04:39:10.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.732+0000 7fb02cff9700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb01c06c680 0x7fb01c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:10.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.732+0000 7fb02cff9700 1 -- 172.21.15.90:0/3455416792 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb01808b0e0 con 0x7fb030100e80 2024-04-26T04:39:10.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.732+0000 7fb02effd700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb01c06c680 0x7fb01c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:10.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.733+0000 7fb02effd700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb01c06c680 0x7fb01c06eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb020006010 tx=0x7fb02001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:10.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.737+0000 7fb02cff9700 1 -- 172.21.15.90:0/3455416792 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb018055b00 con 0x7fb030100e80 2024-04-26T04:39:10.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:10 smithi104 ceph-mon[32369]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:10.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:10 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2641201569' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-26T04:39:10.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.932+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb03004f2a0 con 0x7fb030100e80 2024-04-26T04:39:10.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.932+0000 7fb02cff9700 1 -- 172.21.15.90:0/3455416792 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb018089e60 con 0x7fb030100e80 2024-04-26T04:39:10.934 INFO:teuthology.orchestra.run.smithi090.stdout:111669149717 2024-04-26T04:39:10.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb01c06c680 msgr2=0x7fb01c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb01c06c680 0x7fb01c06eb30 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb020006010 tx=0x7fb02001a040 comp rx=0 tx=0).stop 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb030100e80 msgr2=0x7fb030196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb030100e80 0x7fb030196430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fb01800d900 tx=0x7fb01800dc10 comp rx=0 tx=0).stop 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 shutdown_connections 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb01c06c680 0x7fb01c06eb30 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0301004e0 0x7fb030195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 --2- 172.21.15.90:0/3455416792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb030100e80 0x7fb030196430 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.934+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 >> 172.21.15.90:0/3455416792 conn(0x7fb0300fbda0 msgr2=0x7fb0300fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.935+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 shutdown_connections 2024-04-26T04:39:10.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:10.935+0000 7fb0357f5700 1 -- 172.21.15.90:0/3455416792 wait complete. 2024-04-26T04:39:11.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 -- 172.21.15.90:0/3597880573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 msgr2=0x7fa11c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:11.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 --2- 172.21.15.90:0/3597880573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100040 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa118009910 tx=0x7fa118009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:11.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 -- 172.21.15.90:0/3597880573 shutdown_connections 2024-04-26T04:39:11.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 --2- 172.21.15.90:0/3597880573 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100040 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:11.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 --2- 172.21.15.90:0/3597880573 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa11c104130 0x7fa11c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 -- 172.21.15.90:0/3597880573 >> 172.21.15.90:0/3597880573 conn(0x7fa11c075130 msgr2=0x7fa11c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 -- 172.21.15.90:0/3597880573 shutdown_connections 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.310+0000 7fa124733700 1 -- 172.21.15.90:0/3597880573 wait complete. 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa124733700 1 Processor -- start 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa124733700 1 -- start start 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa124733700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa124733700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa11c104130 0x7fa11c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa124733700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1018b0 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa124733700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c1019f0 con 0x7fa11c104130 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa1224cf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa1224cf700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37462/0 (socket says 172.21.15.90:37462) 2024-04-26T04:39:11.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa1224cf700 1 -- 172.21.15.90:0/1193319773 learned_addr learned my addr 172.21.15.90:0/1193319773 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa121cce700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa11c104130 0x7fa11c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa1224cf700 1 -- 172.21.15.90:0/1193319773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa11c104130 msgr2=0x7fa11c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa1224cf700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa11c104130 0x7fa11c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa1224cf700 1 -- 172.21.15.90:0/1193319773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1180095f0 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.311+0000 7fa1224cf700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100e30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00d8d0 tx=0x7fa10c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.312+0000 7fa1137fe700 1 -- 172.21.15.90:0/1193319773 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10c009940 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.312+0000 7fa1137fe700 1 -- 172.21.15.90:0/1193319773 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa10c010460 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.312+0000 7fa1137fe700 1 -- 172.21.15.90:0/1193319773 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa10c00f5f0 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.312+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c1a2040 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.312+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c1a23f0 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.313+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c1086a0 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.313+0000 7fa1137fe700 1 -- 172.21.15.90:0/1193319773 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa10c0105d0 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.313+0000 7fa1137fe700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa10806c680 0x7fa10806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.314+0000 7fa121cce700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa10806c680 0x7fa10806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:11.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.314+0000 7fa1137fe700 1 -- 172.21.15.90:0/1193319773 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fa10c08b540 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.314+0000 7fa121cce700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa10806c680 0x7fa10806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1180059d0 tx=0x7fa11801a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:11.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.318+0000 7fa1137fe700 1 -- 172.21.15.90:0/1193319773 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa10c055f60 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:11 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3455416792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T04:39:11.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.512+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa11c04f2a0 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.512+0000 7fa1137fe700 1 -- 172.21.15.90:0/1193319773 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa10c059580 con 0x7fa11c0ffbd0 2024-04-26T04:39:11.514 INFO:teuthology.orchestra.run.smithi090.stdout:167503724555 2024-04-26T04:39:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa10806c680 msgr2=0x7fa10806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:11.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa10806c680 0x7fa10806eb30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa1180059d0 tx=0x7fa11801a040 comp rx=0 tx=0).stop 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 msgr2=0x7fa11c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100e30 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa10c00d8d0 tx=0x7fa10c00dc90 comp rx=0 tx=0).stop 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 shutdown_connections 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa10806c680 0x7fa10806eb30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa11c0ffbd0 0x7fa11c100e30 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 --2- 172.21.15.90:0/1193319773 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa11c104130 0x7fa11c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 >> 172.21.15.90:0/1193319773 conn(0x7fa11c075130 msgr2=0x7fa11c0fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 shutdown_connections 2024-04-26T04:39:11.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:11.514+0000 7fa124733700 1 -- 172.21.15.90:0/1193319773 wait complete. 2024-04-26T04:39:11.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:11 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3455416792' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-26T04:39:11.837 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149715 got 111669149717 for osd.3 2024-04-26T04:39:11.838 DEBUG:teuthology.parallel:result is None 2024-04-26T04:39:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:12 smithi090 ceph-mon[26648]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:12 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1193319773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T04:39:12.579 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724553 got 167503724555 for osd.5 2024-04-26T04:39:12.579 DEBUG:teuthology.parallel:result is None 2024-04-26T04:39:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:12 smithi104 ceph-mon[32369]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:12 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1193319773' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-26T04:39:12.909 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:13.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 -- 172.21.15.90:0/2405964152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ffc60 msgr2=0x7ffb4010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2405964152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ffc60 0x7ffb4010a6d0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000b0a0 tx=0x7ffb3000b3b0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 -- 172.21.15.90:0/2405964152 shutdown_connections 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2405964152 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ffc60 0x7ffb4010a6d0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2405964152 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb400ff350 0x7ffb400ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 -- 172.21.15.90:0/2405964152 >> 172.21.15.90:0/2405964152 conn(0x7ffb400748b0 msgr2=0x7ffb40074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 -- 172.21.15.90:0/2405964152 shutdown_connections 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.515+0000 7ffb46d46700 1 -- 172.21.15.90:0/2405964152 wait complete. 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb46d46700 1 Processor -- start 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb46d46700 1 -- start start 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb46d46700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ff350 0x7ffb40100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb46d46700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb400ffc60 0x7ffb40100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb46d46700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb40100fb0 con 0x7ffb400ff350 2024-04-26T04:39:13.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb46d46700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb401010f0 con 0x7ffb400ffc60 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb44ae2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ff350 0x7ffb40100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb44ae2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ff350 0x7ffb40100530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37488/0 (socket says 172.21.15.90:37488) 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb44ae2700 1 -- 172.21.15.90:0/2564945947 learned_addr learned my addr 172.21.15.90:0/2564945947 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb3ffff700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb400ffc60 0x7ffb40100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb44ae2700 1 -- 172.21.15.90:0/2564945947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb400ffc60 msgr2=0x7ffb40100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb44ae2700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb400ffc60 0x7ffb40100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb44ae2700 1 -- 172.21.15.90:0/2564945947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb30009d00 con 0x7ffb400ff350 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.516+0000 7ffb44ae2700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ff350 0x7ffb40100530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800d8d0 tx=0x7ffb3800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.517+0000 7ffb3dffb700 1 -- 172.21.15.90:0/2564945947 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb38009940 con 0x7ffb400ff350 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.517+0000 7ffb3dffb700 1 -- 172.21.15.90:0/2564945947 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ffb38010460 con 0x7ffb400ff350 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.517+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb401a2070 con 0x7ffb400ff350 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.517+0000 7ffb3dffb700 1 -- 172.21.15.90:0/2564945947 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffb3800f5f0 con 0x7ffb400ff350 2024-04-26T04:39:13.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.517+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb401a2480 con 0x7ffb400ff350 2024-04-26T04:39:13.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.518+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb40107dd0 con 0x7ffb400ff350 2024-04-26T04:39:13.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.518+0000 7ffb3dffb700 1 -- 172.21.15.90:0/2564945947 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ffb380105d0 con 0x7ffb400ff350 2024-04-26T04:39:13.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.518+0000 7ffb3dffb700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ffb2c06c680 0x7ffb2c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:13.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.519+0000 7ffb3dffb700 1 -- 172.21.15.90:0/2564945947 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7ffb3808b540 con 0x7ffb400ff350 2024-04-26T04:39:13.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.519+0000 7ffb3ffff700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ffb2c06c680 0x7ffb2c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:13.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.519+0000 7ffb3ffff700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ffb2c06c680 0x7ffb2c06eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009480 tx=0x7ffb300097c0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:13.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.523+0000 7ffb3dffb700 1 -- 172.21.15.90:0/2564945947 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ffb38055f60 con 0x7ffb400ff350 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 -- 172.21.15.90:0/3722445422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ff3c0 msgr2=0x7f52b00ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 --2- 172.21.15.90:0/3722445422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ff3c0 0x7f52b00ff790 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009910 tx=0x7f52a4009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 -- 172.21.15.90:0/3722445422 shutdown_connections 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 --2- 172.21.15.90:0/3722445422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b00ffd60 0x7f52b0103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 --2- 172.21.15.90:0/3722445422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ff3c0 0x7f52b00ff790 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 -- 172.21.15.90:0/3722445422 >> 172.21.15.90:0/3722445422 conn(0x7f52b00fac60 msgr2=0x7f52b00fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 -- 172.21.15.90:0/3722445422 shutdown_connections 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.706+0000 7f52b603a700 1 -- 172.21.15.90:0/3722445422 wait complete. 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52b603a700 1 Processor -- start 2024-04-26T04:39:13.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52b603a700 1 -- start start 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52b603a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b00ff3c0 0x7f52b0195f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52b603a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ffd60 0x7f52b0196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52b603a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b0196b40 con 0x7f52b00ffd60 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52b603a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52b019a8d0 con 0x7f52b00ff3c0 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52aeffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ffd60 0x7f52b0196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52aeffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ffd60 0x7f52b0196460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37502/0 (socket says 172.21.15.90:37502) 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52aeffd700 1 -- 172.21.15.90:0/2479268096 learned_addr learned my addr 172.21.15.90:0/2479268096 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:13.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52af7fe700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b00ff3c0 0x7f52b0195f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.707+0000 7f52aeffd700 1 -- 172.21.15.90:0/2479268096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b00ff3c0 msgr2=0x7f52b0195f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52aeffd700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b00ff3c0 0x7f52b0195f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52aeffd700 1 -- 172.21.15.90:0/2479268096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52a40095f0 con 0x7f52b00ffd60 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52aeffd700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ffd60 0x7f52b0196460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52a000d8d0 tx=0x7f52a000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52acff9700 1 -- 172.21.15.90:0/2479268096 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a0004a00 con 0x7f52b00ffd60 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52acff9700 1 -- 172.21.15.90:0/2479268096 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f52a00054d0 con 0x7f52b00ffd60 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52b019abb0 con 0x7f52b00ffd60 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52acff9700 1 -- 172.21.15.90:0/2479268096 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f52a000fc20 con 0x7f52b00ffd60 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.708+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52b019b100 con 0x7f52b00ffd60 2024-04-26T04:39:13.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.709+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52b004f2a0 con 0x7f52b00ffd60 2024-04-26T04:39:13.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.709+0000 7f52acff9700 1 -- 172.21.15.90:0/2479268096 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f52a000fd80 con 0x7f52b00ffd60 2024-04-26T04:39:13.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.710+0000 7f52acff9700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f529806c680 0x7f529806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:13.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.710+0000 7f52af7fe700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f529806c680 0x7f529806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:13.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.710+0000 7f52acff9700 1 -- 172.21.15.90:0/2479268096 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f52a008ad80 con 0x7f52b00ffd60 2024-04-26T04:39:13.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.710+0000 7f52af7fe700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f529806c680 0x7f529806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52a400b5c0 tx=0x7f52a401a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:13.715 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.714+0000 7f52acff9700 1 -- 172.21.15.90:0/2479268096 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f52a0055820 con 0x7f52b00ffd60 2024-04-26T04:39:13.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.724+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7ffb401077b0 con 0x7ffb400ff350 2024-04-26T04:39:13.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.724+0000 7ffb3dffb700 1 -- 172.21.15.90:0/2564945947 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ffb38059580 con 0x7ffb400ff350 2024-04-26T04:39:13.725 INFO:teuthology.orchestra.run.smithi090.stdout:81604378650 2024-04-26T04:39:13.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ffb2c06c680 msgr2=0x7ffb2c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ffb2c06c680 0x7ffb2c06eb30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009480 tx=0x7ffb300097c0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ff350 msgr2=0x7ffb40100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ff350 0x7ffb40100530 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ffb3800d8d0 tx=0x7ffb3800dc90 comp rx=0 tx=0).stop 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 shutdown_connections 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ffb2c06c680 0x7ffb2c06eb30 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffb400ff350 0x7ffb40100530 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 --2- 172.21.15.90:0/2564945947 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffb400ffc60 0x7ffb40100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 >> 172.21.15.90:0/2564945947 conn(0x7ffb400748b0 msgr2=0x7ffb40104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 shutdown_connections 2024-04-26T04:39:13.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.726+0000 7ffb46d46700 1 -- 172.21.15.90:0/2564945947 wait complete. 2024-04-26T04:39:13.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.916+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f52b019b370 con 0x7f52b00ffd60 2024-04-26T04:39:13.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.916+0000 7f52acff9700 1 -- 172.21.15.90:0/2479268096 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f52a0017020 con 0x7f52b00ffd60 2024-04-26T04:39:13.918 INFO:teuthology.orchestra.run.smithi090.stdout:38654705701 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f529806c680 msgr2=0x7f529806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f529806c680 0x7f529806eb30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f52a400b5c0 tx=0x7f52a401a040 comp rx=0 tx=0).stop 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ffd60 msgr2=0x7f52b0196460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ffd60 0x7f52b0196460 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f52a000d8d0 tx=0x7f52a000dbe0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 shutdown_connections 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f529806c680 0x7f529806eb30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f52b00ff3c0 0x7f52b0195f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 --2- 172.21.15.90:0/2479268096 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f52b00ffd60 0x7f52b0196460 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.918+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 >> 172.21.15.90:0/2479268096 conn(0x7f52b00fac60 msgr2=0x7f52b0103480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.919+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 shutdown_connections 2024-04-26T04:39:13.920 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:13.919+0000 7f52b603a700 1 -- 172.21.15.90:0/2479268096 wait complete. 2024-04-26T04:39:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:14 smithi090 ceph-mon[26648]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:14 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2564945947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T04:39:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:14 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2479268096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T04:39:14.512 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378648 got 81604378650 for osd.2 2024-04-26T04:39:14.512 DEBUG:teuthology.parallel:result is None 2024-04-26T04:39:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:14 smithi104 ceph-mon[32369]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:14 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2564945947' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-26T04:39:14.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:14 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2479268096' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-26T04:39:16.176 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705699 got 38654705701 for osd.0 2024-04-26T04:39:16.176 DEBUG:teuthology.parallel:result is None 2024-04-26T04:39:16.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:16 smithi090 ceph-mon[26648]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.581+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/2682145529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 msgr2=0x7f9f180ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.581+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/2682145529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f180ff7c0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9f00009910 tx=0x7f9f00009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/2682145529 shutdown_connections 2024-04-26T04:39:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/2682145529 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f180ff7c0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/2682145529 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f181038b0 0x7f9f18103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:16.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/2682145529 >> 172.21.15.90:0/2682145529 conn(0x7f9f180748b0 msgr2=0x7f9f18074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/2682145529 shutdown_connections 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/2682145529 wait complete. 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 Processor -- start 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.582+0000 7f9f1d2d6700 1 -- start start 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f1d2d6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f18195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f1d2d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f181038b0 0x7f9f18196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f1d2d6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f18196a20 con 0x7f9f180ff350 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f1d2d6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f1819a7b0 con 0x7f9f181038b0 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f16ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f18195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:16.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f16ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f18195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37516/0 (socket says 172.21.15.90:37516) 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f16ffd700 1 -- 172.21.15.90:0/745599609 learned_addr learned my addr 172.21.15.90:0/745599609 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f0e7fc700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f181038b0 0x7f9f18196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f16ffd700 1 -- 172.21.15.90:0/745599609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f181038b0 msgr2=0x7f9f18196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f16ffd700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f181038b0 0x7f9f18196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f16ffd700 1 -- 172.21.15.90:0/745599609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f000095f0 con 0x7f9f180ff350 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f16ffd700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f18195e00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800d8d0 tx=0x7f9f0800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f14ff9700 1 -- 172.21.15.90:0/745599609 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f080098a0 con 0x7f9f180ff350 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f14ff9700 1 -- 172.21.15.90:0/745599609 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f08010460 con 0x7f9f180ff350 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f1819aa90 con 0x7f9f180ff350 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f14ff9700 1 -- 172.21.15.90:0/745599609 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f0800f5f0 con 0x7f9f180ff350 2024-04-26T04:39:16.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.583+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f1819afe0 con 0x7f9f180ff350 2024-04-26T04:39:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.584+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f1804f2a0 con 0x7f9f180ff350 2024-04-26T04:39:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.585+0000 7f9f14ff9700 1 -- 172.21.15.90:0/745599609 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9f08009a00 con 0x7f9f180ff350 2024-04-26T04:39:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.585+0000 7f9f14ff9700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9f0406c680 0x7f9f0406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.585+0000 7f9f14ff9700 1 -- 172.21.15.90:0/745599609 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f9f0808b370 con 0x7f9f180ff350 2024-04-26T04:39:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.585+0000 7f9f0e7fc700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9f0406c680 0x7f9f0406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.586+0000 7f9f0e7fc700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9f0406c680 0x7f9f0406eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f18197420 tx=0x7f9f0001a040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:16.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.590+0000 7f9f14ff9700 1 -- 172.21.15.90:0/745599609 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9f08055e10 con 0x7f9f180ff350 2024-04-26T04:39:16.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:16 smithi104 ceph-mon[32369]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:16.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.780+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f9f18197160 con 0x7f9f180ff350 2024-04-26T04:39:16.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.781+0000 7f9f14ff9700 1 -- 172.21.15.90:0/745599609 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f9f08059430 con 0x7f9f180ff350 2024-04-26T04:39:16.782 INFO:teuthology.orchestra.run.smithi090.stdout:146028888081 2024-04-26T04:39:16.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9f0406c680 msgr2=0x7f9f0406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9f0406c680 0x7f9f0406eb30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9f18197420 tx=0x7f9f0001a040 comp rx=0 tx=0).stop 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 msgr2=0x7f9f18195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f18195e00 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9f0800d8d0 tx=0x7f9f0800dbe0 comp rx=0 tx=0).stop 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 shutdown_connections 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9f0406c680 0x7f9f0406eb30 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f180ff350 0x7f9f18195e00 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 --2- 172.21.15.90:0/745599609 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f181038b0 0x7f9f18196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 >> 172.21.15.90:0/745599609 conn(0x7f9f180748b0 msgr2=0x7f9f180fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 shutdown_connections 2024-04-26T04:39:16.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:16.783+0000 7f9f1d2d6700 1 -- 172.21.15.90:0/745599609 wait complete. 2024-04-26T04:39:17.398 INFO:tasks.cephadm.ceph_manager.ceph:need seq 146028888079 got 146028888081 for osd.4 2024-04-26T04:39:17.398 DEBUG:teuthology.parallel:result is None 2024-04-26T04:39:17.398 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-26T04:39:17.398 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T04:39:17.601 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:17.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:17 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/745599609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T04:39:17.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:17 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/745599609' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-26T04:39:18.707 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:18 smithi090 ceph-mon[26648]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:18.707 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:18 smithi104 ceph-mon[32369]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:19.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 -- 172.21.15.90:0/1275952013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584106320 msgr2=0x7f15841066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:19.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 --2- 172.21.15.90:0/1275952013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584106320 0x7f15841066f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f157c00b0a0 tx=0x7f157c00b3b0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 -- 172.21.15.90:0/1275952013 shutdown_connections 2024-04-26T04:39:19.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 --2- 172.21.15.90:0/1275952013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1584068540 0x7f15840689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 --2- 172.21.15.90:0/1275952013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584106320 0x7f15841066f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 -- 172.21.15.90:0/1275952013 >> 172.21.15.90:0/1275952013 conn(0x7f1584075130 msgr2=0x7f1584075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 -- 172.21.15.90:0/1275952013 shutdown_connections 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.164+0000 7f1588de3700 1 -- 172.21.15.90:0/1275952013 wait complete. 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f1588de3700 1 Processor -- start 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f1588de3700 1 -- start start 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f1588de3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584068540 0x7f1584195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f1588de3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1584106320 0x7f15841964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f1588de3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1584196bb0 con 0x7f1584068540 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f1588de3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f158419a940 con 0x7f1584106320 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f158259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584068540 0x7f1584195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f158259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584068540 0x7f1584195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55238/0 (socket says 172.21.15.90:55238) 2024-04-26T04:39:19.168 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f158259c700 1 -- 172.21.15.90:0/1334554277 learned_addr learned my addr 172.21.15.90:0/1334554277 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:19.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f1581d9b700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1584106320 0x7f15841964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:19.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f158259c700 1 -- 172.21.15.90:0/1334554277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1584106320 msgr2=0x7f15841964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:19.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.165+0000 7f158259c700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1584106320 0x7f15841964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f158259c700 1 -- 172.21.15.90:0/1334554277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f157c009d00 con 0x7f1584068540 2024-04-26T04:39:19.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f1581d9b700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1584106320 0x7f15841964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:39:19.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f158259c700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584068540 0x7f1584195f90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f157c000f80 tx=0x7f157c003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:19.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f157b7fe700 1 -- 172.21.15.90:0/1334554277 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f157c00e040 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f158419abc0 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f157b7fe700 1 -- 172.21.15.90:0/1334554277 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f157c007d60 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f157b7fe700 1 -- 172.21.15.90:0/1334554277 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f157c01d7d0 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.166+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f158419b0b0 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.167+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1584108790 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.167+0000 7f157b7fe700 1 -- 172.21.15.90:0/1334554277 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f157c004040 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.168+0000 7f157b7fe700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f157006c680 0x7f157006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.168+0000 7f157b7fe700 1 -- 172.21.15.90:0/1334554277 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f157c08ce80 con 0x7f1584068540 2024-04-26T04:39:19.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.168+0000 7f1581d9b700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f157006c680 0x7f157006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:19.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.168+0000 7f1581d9b700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f157006c680 0x7f157006eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15841975b0 tx=0x7f156c008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:19.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.172+0000 7f157b7fe700 1 -- 172.21.15.90:0/1334554277 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f157c0578a0 con 0x7f1584068540 2024-04-26T04:39:19.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.359+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f158419b2d0 con 0x7f157006c680 2024-04-26T04:39:19.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.361+0000 7f157b7fe700 1 -- 172.21.15.90:0/1334554277 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28079 (secure 0 0 0) 0x7f158419b2d0 con 0x7f157006c680 2024-04-26T04:39:19.362 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:39:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f157006c680 msgr2=0x7f157006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f157006c680 0x7f157006eb30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f15841975b0 tx=0x7f156c008040 comp rx=0 tx=0).stop 2024-04-26T04:39:19.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584068540 msgr2=0x7f1584195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584068540 0x7f1584195f90 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f157c000f80 tx=0x7f157c003ce0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 shutdown_connections 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f157006c680 0x7f157006eb30 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1584068540 0x7f1584195f90 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 --2- 172.21.15.90:0/1334554277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1584106320 0x7f15841964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 >> 172.21.15.90:0/1334554277 conn(0x7f1584075130 msgr2=0x7f15840febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 shutdown_connections 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:19.363+0000 7f1588de3700 1 -- 172.21.15.90:0/1334554277 wait complete. 2024-04-26T04:39:19.365 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T04:39:20.028 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":156,"stamp":"2024-04-26T04:39:19.280626+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2609166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001977"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T04:38:32.350318+0000","last_change":"2024-04-26T04:38:07.175016+0000","last_active":"2024-04-26T04:38:32.350318+0000","last_peered":"2024-04-26T04:38:32.350318+0000","last_clean":"2024-04-26T04:38:32.350318+0000","last_became_active":"2024-04-26T04:38:07.174747+0000","last_became_peered":"2024-04-26T04:38:07.174747+0000","last_unstale":"2024-04-26T04:38:32.350318+0000","last_undegraded":"2024-04-26T04:38:32.350318+0000","last_fullsized":"2024-04-26T04:38:32.350318+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:37:19.770077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:37:19.770077+0000","last_clean_scrub_stamp":"2024-04-26T04:37:19.770077+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:58:13.583424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.499}]},{"osd":1,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.439}]},{"osd":2,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.435},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Apr 26 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Apr 26 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Apr 26 04:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.29299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.309}]},{"osd":1,"last update":"Fri Apr 26 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29699999999999999}]},{"osd":3,"last update":"Fri Apr 26 04:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.499},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Apr 26 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 26 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.28999999999999998,"15min":0.28000000000000003},"min":{"1min":0.245,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.255},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.30199999999999999,"15min":0.28899999999999998},"min":{"1min":0.24299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Apr 26 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.317},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.30199999999999999}]},{"osd":3,"last update":"Fri Apr 26 04:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.436}]},{"osd":4,"last update":"Fri Apr 26 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542176,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.316,"15min":0.31},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.44900000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.314,"15min":0.30599999999999999},"min":{"1min":0.23699999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.309}]},{"osd":2,"last update":"Fri Apr 26 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.34300000000000003}]},{"osd":3,"last update":"Fri Apr 26 04:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.45300000000000001}]},{"osd":4,"last update":"Fri Apr 26 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.53400000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:39:20.029 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-26T04:39:20.237 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:20.658 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:20 smithi090 ceph-mon[26648]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:20.659 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:20 smithi090 ceph-mon[26648]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:39:20.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:20 smithi104 ceph-mon[32369]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:20.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:20 smithi104 ceph-mon[32369]: from='client.14426 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:39:21.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.633+0000 7fb1e94c9700 1 -- 172.21.15.90:0/946071333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 msgr2=0x7fb1e41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.633+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/946071333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e41008b0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4009910 tx=0x7fb1d4009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.634+0000 7fb1e94c9700 1 -- 172.21.15.90:0/946071333 shutdown_connections 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.634+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/946071333 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e4100e80 0x7fb1e4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.634+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/946071333 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e41008b0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.634+0000 7fb1e94c9700 1 -- 172.21.15.90:0/946071333 >> 172.21.15.90:0/946071333 conn(0x7fb1e40fbda0 msgr2=0x7fb1e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.634+0000 7fb1e94c9700 1 -- 172.21.15.90:0/946071333 shutdown_connections 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.634+0000 7fb1e94c9700 1 -- 172.21.15.90:0/946071333 wait complete. 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.634+0000 7fb1e94c9700 1 Processor -- start 2024-04-26T04:39:21.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e94c9700 1 -- start start 2024-04-26T04:39:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e94c9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e4195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e94c9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e4100e80 0x7fb1e4196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e94c9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e4196b10 con 0x7fb1e41004e0 2024-04-26T04:39:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e94c9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1e419a8a0 con 0x7fb1e4100e80 2024-04-26T04:39:21.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e4195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e4195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55250/0 (socket says 172.21.15.90:55250) 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e2ffd700 1 -- 172.21.15.90:0/1324608277 learned_addr learned my addr 172.21.15.90:0/1324608277 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1da7fc700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e4100e80 0x7fb1e4196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e2ffd700 1 -- 172.21.15.90:0/1324608277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e4100e80 msgr2=0x7fb1e4196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e2ffd700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e4100e80 0x7fb1e4196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e2ffd700 1 -- 172.21.15.90:0/1324608277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1d40095f0 con 0x7fb1e41004e0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1da7fc700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e4100e80 0x7fb1e4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.635+0000 7fb1e2ffd700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e4195ef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4006010 tx=0x7fb1d400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.636+0000 7fb1e0ff9700 1 -- 172.21.15.90:0/1324608277 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d401d070 con 0x7fb1e41004e0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.636+0000 7fb1e0ff9700 1 -- 172.21.15.90:0/1324608277 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb1d400f460 con 0x7fb1e41004e0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.636+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1e419ab20 con 0x7fb1e41004e0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.636+0000 7fb1e0ff9700 1 -- 172.21.15.90:0/1324608277 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb1d4021640 con 0x7fb1e41004e0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.636+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1e419b010 con 0x7fb1e41004e0 2024-04-26T04:39:21.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.637+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1e41087f0 con 0x7fb1e41004e0 2024-04-26T04:39:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.637+0000 7fb1e0ff9700 1 -- 172.21.15.90:0/1324608277 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb1d400faf0 con 0x7fb1e41004e0 2024-04-26T04:39:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.638+0000 7fb1e0ff9700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb1d006c680 0x7fb1d006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.638+0000 7fb1da7fc700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb1d006c680 0x7fb1d006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.638+0000 7fb1e0ff9700 1 -- 172.21.15.90:0/1324608277 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fb1d408c780 con 0x7fb1e41004e0 2024-04-26T04:39:21.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.638+0000 7fb1da7fc700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb1d006c680 0x7fb1d006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc005fd0 tx=0x7fb1cc005dc0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:21.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.642+0000 7fb1e0ff9700 1 -- 172.21.15.90:0/1324608277 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb1d4057220 con 0x7fb1e41004e0 2024-04-26T04:39:21.832 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.830+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb1e4197360 con 0x7fb1d006c680 2024-04-26T04:39:21.833 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.832+0000 7fb1e0ff9700 1 -- 172.21.15.90:0/1324608277 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+28051 (secure 0 0 0) 0x7fb1e4197360 con 0x7fb1d006c680 2024-04-26T04:39:21.834 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:39:21.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb1d006c680 msgr2=0x7fb1d006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:21.835 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb1d006c680 0x7fb1d006eb30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb1cc005fd0 tx=0x7fb1cc005dc0 comp rx=0 tx=0).stop 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 msgr2=0x7fb1e4195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e4195ef0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb1d4006010 tx=0x7fb1d400be20 comp rx=0 tx=0).stop 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 shutdown_connections 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb1d006c680 0x7fb1d006eb30 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb1e41004e0 0x7fb1e4195ef0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 --2- 172.21.15.90:0/1324608277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb1e4100e80 0x7fb1e4196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.834+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 >> 172.21.15.90:0/1324608277 conn(0x7fb1e40fbda0 msgr2=0x7fb1e40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.835+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 shutdown_connections 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:21.835+0000 7fb1e94c9700 1 -- 172.21.15.90:0/1324608277 wait complete. 2024-04-26T04:39:21.836 INFO:teuthology.orchestra.run.smithi090.stderr:dumped all 2024-04-26T04:39:22.389 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:22 smithi090 ceph-mon[26648]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:22.411 INFO:teuthology.orchestra.run.smithi090.stdout:{"pg_ready":true,"pg_map":{"version":157,"stamp":"2024-04-26T04:39:21.280927+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":3,"num_osds":6,"num_per_pool_osds":6,"num_per_pool_omap_osds":3,"kb":562470912,"kb_used":165588,"kb_used_data":3876,"kb_used_omap":0,"kb_used_meta":161664,"kb_avail":562305324,"statfs":{"total":575970213888,"available":575800651776,"internally_reserved":0,"allocated":3969024,"data_stored":2609166,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":165543936},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001886"},"pg_stats":[{"pgid":"1.0","version":"23'117","reported_seq":186,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-04-26T04:38:32.350318+0000","last_change":"2024-04-26T04:38:07.175016+0000","last_active":"2024-04-26T04:38:32.350318+0000","last_peered":"2024-04-26T04:38:32.350318+0000","last_clean":"2024-04-26T04:38:32.350318+0000","last_became_active":"2024-04-26T04:38:07.174747+0000","last_became_peered":"2024-04-26T04:38:07.174747+0000","last_unstale":"2024-04-26T04:38:32.350318+0000","last_undegraded":"2024-04-26T04:38:32.350318+0000","last_fullsized":"2024-04-26T04:38:32.350318+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-26T04:37:19.770077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-26T04:37:19.770077+0000","last_clean_scrub_stamp":"2024-04-26T04:37:19.770077+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-27T10:58:13.583424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2375680,"data_stored":2361472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":3,"acting":3,"num_store_stats":4}],"osd_stats":[{"osd":5,"up_from":39,"seq":167503724557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34100000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.375}]}]},{"osd":4,"up_from":34,"seq":146028888082,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"last":0.499}]},{"osd":1,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.439}]},{"osd":2,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51200000000000001,"15min":0.51200000000000001},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"last":0.45800000000000002}]},{"osd":3,"last update":"Fri Apr 26 04:39:09 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48099999999999998}]}]},{"osd":3,"up_from":26,"seq":111669149719,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.435},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41099999999999998}]},{"osd":1,"last update":"Fri Apr 26 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42299999999999999}]},{"osd":2,"last update":"Fri Apr 26 04:38:44 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.46899999999999997}]},{"osd":4,"last update":"Fri Apr 26 04:39:07 2024","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.38,"15min":0.38},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.33200000000000002},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"last":0.29299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.318}]}]},{"osd":2,"up_from":19,"seq":81604378652,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":93745152,"kb_used":27308,"kb_used_data":356,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717844,"statfs":{"total":95995035648,"available":95967072256,"internally_reserved":0,"allocated":364544,"data_stored":139677,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.37,"15min":0.37},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"last":0.27300000000000002},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"min":{"1min":0.22,"5min":0.22,"15min":0.22},"max":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"last":0.309}]},{"osd":1,"last update":"Fri Apr 26 04:38:22 2024","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"last":0.29699999999999999}]},{"osd":3,"last update":"Fri Apr 26 04:38:41 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"last":0.499},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"last":0.50800000000000001}]},{"osd":4,"last update":"Fri Apr 26 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.73599999999999999,"5min":0.73599999999999999,"15min":0.73599999999999999},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]}]},{"osd":0,"up_from":9,"seq":38654705703,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Apr 26 04:38:51 2024","interfaces":[{"interface":"back","average":{"1min":0.34999999999999998,"5min":0.28999999999999998,"15min":0.28000000000000003},"min":{"1min":0.245,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.255},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.30199999999999999,"15min":0.28899999999999998},"min":{"1min":0.24299999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.503,"5min":0.503,"15min":0.503},"last":0.34899999999999998}]},{"osd":2,"last update":"Fri Apr 26 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"last":0.317},{"interface":"front","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"last":0.30199999999999999}]},{"osd":3,"last update":"Fri Apr 26 04:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.436}]},{"osd":4,"last update":"Fri Apr 26 04:39:08 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.496}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]}]},{"osd":1,"up_from":14,"seq":60129542177,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27888,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":93717264,"statfs":{"total":95995035648,"available":95966478336,"internally_reserved":0,"allocated":958464,"data_stored":730045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Apr 26 04:38:57 2024","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.316,"15min":0.31},"min":{"1min":0.254,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.44900000000000001,"5min":0.49099999999999999,"15min":0.49099999999999999},"last":0.372},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.314,"15min":0.30599999999999999},"min":{"1min":0.23699999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.505,"5min":0.505,"15min":0.505},"last":0.34200000000000003}]},{"osd":2,"last update":"Fri Apr 26 04:38:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"last":0.312}]},{"osd":3,"last update":"Fri Apr 26 04:38:43 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.48399999999999999}]},{"osd":4,"last update":"Fri Apr 26 04:39:11 2024","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.47299999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.498}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-26T04:39:22.412 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-26T04:39:22.412 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-26T04:39:22.412 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-26T04:39:22.412 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph health --format=json 2024-04-26T04:39:22.617 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:22.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:22 smithi104 ceph-mon[32369]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:23.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:23 smithi090 ceph-mon[26648]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:39:23.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:23 smithi104 ceph-mon[32369]: from='client.14430 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-26T04:39:24.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 -- 172.21.15.90:0/1251353434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 msgr2=0x7f37b81008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:24.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 --2- 172.21.15.90:0/1251353434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b81008b0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f37a8009910 tx=0x7f37a8009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 -- 172.21.15.90:0/1251353434 shutdown_connections 2024-04-26T04:39:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 --2- 172.21.15.90:0/1251353434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37b8100e80 0x7f37b8104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 --2- 172.21.15.90:0/1251353434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b81008b0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 -- 172.21.15.90:0/1251353434 >> 172.21.15.90:0/1251353434 conn(0x7f37b80fbda0 msgr2=0x7f37b80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 -- 172.21.15.90:0/1251353434 shutdown_connections 2024-04-26T04:39:24.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.259+0000 7f37bf897700 1 -- 172.21.15.90:0/1251353434 wait complete. 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bf897700 1 Processor -- start 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bf897700 1 -- start start 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bf897700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b8195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bf897700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37b8100e80 0x7f37b81964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bf897700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b8196bb0 con 0x7f37b81004e0 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bf897700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37b819a940 con 0x7f37b8100e80 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bd633700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b8195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bd633700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b8195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55266/0 (socket says 172.21.15.90:55266) 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bd633700 1 -- 172.21.15.90:0/2359991626 learned_addr learned my addr 172.21.15.90:0/2359991626 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bce32700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37b8100e80 0x7f37b81964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bd633700 1 -- 172.21.15.90:0/2359991626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37b8100e80 msgr2=0x7f37b81964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:24.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bd633700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37b8100e80 0x7f37b81964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bd633700 1 -- 172.21.15.90:0/2359991626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80095f0 con 0x7f37b81004e0 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.260+0000 7f37bd633700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b8195f90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006010 tx=0x7f37a800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.261+0000 7f37ae7fc700 1 -- 172.21.15.90:0/2359991626 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37a801d070 con 0x7f37b81004e0 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.261+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37b819abc0 con 0x7f37b81004e0 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.261+0000 7f37ae7fc700 1 -- 172.21.15.90:0/2359991626 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f37a800f460 con 0x7f37b81004e0 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.261+0000 7f37ae7fc700 1 -- 172.21.15.90:0/2359991626 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37a8017630 con 0x7f37b81004e0 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.261+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37b819b0b0 con 0x7f37b81004e0 2024-04-26T04:39:24.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.262+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37b81087f0 con 0x7f37b81004e0 2024-04-26T04:39:24.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.262+0000 7f37ae7fc700 1 -- 172.21.15.90:0/2359991626 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f37a800f5d0 con 0x7f37b81004e0 2024-04-26T04:39:24.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.262+0000 7f37ae7fc700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f37a406c680 0x7f37a406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:24.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.263+0000 7f37ae7fc700 1 -- 172.21.15.90:0/2359991626 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f37a808cbd0 con 0x7f37b81004e0 2024-04-26T04:39:24.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.263+0000 7f37bce32700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f37a406c680 0x7f37a406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:24.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.263+0000 7f37bce32700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f37a406c680 0x7f37a406eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f37b4006fd0 tx=0x7f37b4008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:24.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.267+0000 7f37ae7fc700 1 -- 172.21.15.90:0/2359991626 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f37a80575f0 con 0x7f37b81004e0 2024-04-26T04:39:24.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.500+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f37b81973f0 con 0x7f37b81004e0 2024-04-26T04:39:24.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.500+0000 7f37ae7fc700 1 -- 172.21.15.90:0/2359991626 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f37a805ac10 con 0x7f37b81004e0 2024-04-26T04:39:24.502 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:39:24.502 INFO:teuthology.orchestra.run.smithi090.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f37a406c680 msgr2=0x7f37a406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f37a406c680 0x7f37a406eb30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f37b4006fd0 tx=0x7f37b4008040 comp rx=0 tx=0).stop 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 msgr2=0x7f37b8195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b8195f90 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f37a8006010 tx=0x7f37a800be20 comp rx=0 tx=0).stop 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 shutdown_connections 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f37a406c680 0x7f37a406eb30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37b81004e0 0x7f37b8195f90 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 --2- 172.21.15.90:0/2359991626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37b8100e80 0x7f37b81964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.502+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 >> 172.21.15.90:0/2359991626 conn(0x7f37b80fbda0 msgr2=0x7f37b80fd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.503+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 shutdown_connections 2024-04-26T04:39:24.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:24.503+0000 7f37bf897700 1 -- 172.21.15.90:0/2359991626 wait complete. 2024-04-26T04:39:24.626 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:24 smithi104 ceph-mon[32369]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:24.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:24 smithi090 ceph-mon[26648]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:25.126 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-26T04:39:25.126 INFO:tasks.cephadm:Setup complete, yielding 2024-04-26T04:39:25.127 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:39:25.135 INFO:teuthology.task.print:**** done end installing v18.2.0 cephadm ... 2024-04-26T04:39:25.136 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:39:25.154 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:39:25.155 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph config set mgr mgr/cephadm/use_repo_digest true --force' 2024-04-26T04:39:25.357 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:25.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:25 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2359991626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T04:39:25.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:25 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2359991626' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-26T04:39:26.652 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:26 smithi090 ceph-mon[26648]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:26.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 -- 172.21.15.90:0/22581795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec0ff150 msgr2=0x7fbcec0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:26.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/22581795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec0ff150 0x7fbcec0ff560 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fbce8009910 tx=0x7fbce8009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:26.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 -- 172.21.15.90:0/22581795 shutdown_connections 2024-04-26T04:39:26.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/22581795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcec1003f0 0x7fbcec100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:26.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/22581795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec0ff150 0x7fbcec0ff560 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 -- 172.21.15.90:0/22581795 >> 172.21.15.90:0/22581795 conn(0x7fbcec0fa7a0 msgr2=0x7fbcec0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 -- 172.21.15.90:0/22581795 shutdown_connections 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.756+0000 7fbcf3f58700 1 -- 172.21.15.90:0/22581795 wait complete. 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf3f58700 1 Processor -- start 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf3f58700 1 -- start start 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf3f58700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcec0ff150 0x7fbcec069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf3f58700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec1003f0 0x7fbcec19abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf3f58700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec19b180 con 0x7fbcec1003f0 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf3f58700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcec19b2c0 con 0x7fbcec0ff150 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf14f3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec1003f0 0x7fbcec19abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf14f3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec1003f0 0x7fbcec19abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55288/0 (socket says 172.21.15.90:55288) 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf14f3700 1 -- 172.21.15.90:0/1570050795 learned_addr learned my addr 172.21.15.90:0/1570050795 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:26.760 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf1cf4700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcec0ff150 0x7fbcec069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf14f3700 1 -- 172.21.15.90:0/1570050795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcec0ff150 msgr2=0x7fbcec069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf14f3700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcec0ff150 0x7fbcec069930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.757+0000 7fbcf14f3700 1 -- 172.21.15.90:0/1570050795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbce80095f0 con 0x7fbcec1003f0 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.758+0000 7fbcf14f3700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec1003f0 0x7fbcec19abb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b4b0 tx=0x7fbcdc00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.758+0000 7fbce2ffd700 1 -- 172.21.15.90:0/1570050795 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcdc011800 con 0x7fbcec1003f0 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.758+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcec19b5a0 con 0x7fbcec1003f0 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.758+0000 7fbce2ffd700 1 -- 172.21.15.90:0/1570050795 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbcdc011e10 con 0x7fbcec1003f0 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.758+0000 7fbce2ffd700 1 -- 172.21.15.90:0/1570050795 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbcdc00f2e0 con 0x7fbcec1003f0 2024-04-26T04:39:26.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.758+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcec19baf0 con 0x7fbcec1003f0 2024-04-26T04:39:26.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:26 smithi104 ceph-mon[32369]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:26.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.759+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcec068ab0 con 0x7fbcec1003f0 2024-04-26T04:39:26.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.759+0000 7fbce2ffd700 1 -- 172.21.15.90:0/1570050795 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fbcdc010380 con 0x7fbcec1003f0 2024-04-26T04:39:26.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.760+0000 7fbce2ffd700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbcd806c680 0x7fbcd806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:26.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.760+0000 7fbcf1cf4700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbcd806c680 0x7fbcd806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:26.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.760+0000 7fbce2ffd700 1 -- 172.21.15.90:0/1570050795 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fbcdc08b180 con 0x7fbcec1003f0 2024-04-26T04:39:26.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.760+0000 7fbcf1cf4700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbcd806c680 0x7fbcd806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b5c0 tx=0x7fbce8005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:26.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.764+0000 7fbce2ffd700 1 -- 172.21.15.90:0/1570050795 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbcdc059630 con 0x7fbcec1003f0 2024-04-26T04:39:26.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.955+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/use_repo_digest}] v 0) v1 -- 0x7fbcec19bdd0 con 0x7fbcec1003f0 2024-04-26T04:39:26.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.960+0000 7fbce2ffd700 1 -- 172.21.15.90:0/1570050795 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/use_repo_digest}]=0 v20)=0 v20) v1 ==== 143+0+0 (secure 0 0 0) 0x7fbcdc0591c0 con 0x7fbcec1003f0 2024-04-26T04:39:26.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbcd806c680 msgr2=0x7fbcd806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:26.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbcd806c680 0x7fbcd806eb30 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbce800b5c0 tx=0x7fbce8005960 comp rx=0 tx=0).stop 2024-04-26T04:39:26.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec1003f0 msgr2=0x7fbcec19abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:26.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec1003f0 0x7fbcec19abb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbcdc00b4b0 tx=0x7fbcdc00b870 comp rx=0 tx=0).stop 2024-04-26T04:39:26.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 shutdown_connections 2024-04-26T04:39:26.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbcd806c680 0x7fbcd806eb30 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:26.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcec0ff150 0x7fbcec069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:26.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 --2- 172.21.15.90:0/1570050795 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcec1003f0 0x7fbcec19abb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:26.964 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 >> 172.21.15.90:0/1570050795 conn(0x7fbcec0fa7a0 msgr2=0x7fbcec1036a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:26.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 shutdown_connections 2024-04-26T04:39:26.965 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:26.962+0000 7fbcf3f58700 1 -- 172.21.15.90:0/1570050795 wait complete. 2024-04-26T04:39:27.613 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:39:27.622 INFO:teuthology.task.print:**** done cephadm.shell ceph config set mgr... 2024-04-26T04:39:27.622 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:39:27.647 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:39:27.647 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-26T04:39:27.849 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:27 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1570050795' entity='client.admin' 2024-04-26T04:39:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:27 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:39:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:27 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:28.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:27 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:39:28.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:27 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:28.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:27 smithi090 ceph-mon[26648]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:27 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1570050795' entity='client.admin' 2024-04-26T04:39:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:27 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:39:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:27 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:27 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:39:28.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:27 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:28.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:27 smithi104 ceph-mon[32369]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:29.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 -- 172.21.15.90:0/2276815176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338069b00 msgr2=0x7f2338102df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:29.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 --2- 172.21.15.90:0/2276815176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338069b00 0x7f2338102df0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2334009910 tx=0x7f2334009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:29.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 -- 172.21.15.90:0/2276815176 shutdown_connections 2024-04-26T04:39:29.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 --2- 172.21.15.90:0/2276815176 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2338103330 0x7f23381057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 --2- 172.21.15.90:0/2276815176 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338069b00 0x7f2338102df0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 -- 172.21.15.90:0/2276815176 >> 172.21.15.90:0/2276815176 conn(0x7f23380fa7a0 msgr2=0x7f23380fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 -- 172.21.15.90:0/2276815176 shutdown_connections 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.313+0000 7f23404cc700 1 -- 172.21.15.90:0/2276815176 wait complete. 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f23404cc700 1 Processor -- start 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f23404cc700 1 -- start start 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f23404cc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338103330 0x7f2338195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f23404cc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2338196370 0x7f233819b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f23404cc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2338196870 con 0x7f2338103330 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f23404cc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23381969e0 con 0x7f2338196370 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233e268700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338103330 0x7f2338195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:29.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233e268700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338103330 0x7f2338195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38304/0 (socket says 172.21.15.90:38304) 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233e268700 1 -- 172.21.15.90:0/753302969 learned_addr learned my addr 172.21.15.90:0/753302969 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233da67700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2338196370 0x7f233819b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233e268700 1 -- 172.21.15.90:0/753302969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2338196370 msgr2=0x7f233819b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233e268700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2338196370 0x7f233819b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233e268700 1 -- 172.21.15.90:0/753302969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340095f0 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f233e268700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338103330 0x7f2338195e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f233400bda0 tx=0x7f233400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f232f7fe700 1 -- 172.21.15.90:0/753302969 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f233401d070 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.314+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f233819b920 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.315+0000 7f232f7fe700 1 -- 172.21.15.90:0/753302969 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f233400f460 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.315+0000 7f232f7fe700 1 -- 172.21.15.90:0/753302969 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2334021640 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.315+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f233819bde0 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.316+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233818fbf0 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.316+0000 7f232f7fe700 1 -- 172.21.15.90:0/753302969 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f233400fa00 con 0x7f2338103330 2024-04-26T04:39:29.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.316+0000 7f232f7fe700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f232406c680 0x7f232406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:29.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.317+0000 7f233da67700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f232406c680 0x7f232406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:29.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.317+0000 7f232f7fe700 1 -- 172.21.15.90:0/753302969 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f233408c830 con 0x7f2338103330 2024-04-26T04:39:29.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.317+0000 7f233da67700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f232406c680 0x7f232406eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2328005950 tx=0x7f23280058e0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:29.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.321+0000 7f232f7fe700 1 -- 172.21.15.90:0/753302969 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f233405ace0 con 0x7f2338103330 2024-04-26T04:39:29.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.528+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2338061190 con 0x7f232406c680 2024-04-26T04:39:29.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.529+0000 7f232f7fe700 1 -- 172.21.15.90:0/753302969 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2338061190 con 0x7f232406c680 2024-04-26T04:39:29.531 INFO:teuthology.orchestra.run.smithi090.stdout:Backend: cephadm 2024-04-26T04:39:29.531 INFO:teuthology.orchestra.run.smithi090.stdout:Available: Yes 2024-04-26T04:39:29.531 INFO:teuthology.orchestra.run.smithi090.stdout:Paused: No 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f232406c680 msgr2=0x7f232406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f232406c680 0x7f232406eb30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f2328005950 tx=0x7f23280058e0 comp rx=0 tx=0).stop 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338103330 msgr2=0x7f2338195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338103330 0x7f2338195e30 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f233400bda0 tx=0x7f233400be80 comp rx=0 tx=0).stop 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 shutdown_connections 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f232406c680 0x7f232406eb30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2338103330 0x7f2338195e30 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 --2- 172.21.15.90:0/753302969 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2338196370 0x7f233819b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 >> 172.21.15.90:0/753302969 conn(0x7f23380fa7a0 msgr2=0x7f23380fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 shutdown_connections 2024-04-26T04:39:29.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:29.532+0000 7f23404cc700 1 -- 172.21.15.90:0/753302969 wait complete. 2024-04-26T04:39:30.240 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-26T04:39:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:30.536 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:30 smithi090 ceph-mon[26648]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:30.536 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:30 smithi090 ceph-mon[26648]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:30.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:30 smithi104 ceph-mon[32369]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:30.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:30 smithi104 ceph-mon[32369]: from='client.14442 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:31.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 -- 172.21.15.90:0/1173803687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 msgr2=0x7f1d8c0fe320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:31.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 --2- 172.21.15.90:0/1173803687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c0fe320 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009910 tx=0x7f1d80009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:31.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 -- 172.21.15.90:0/1173803687 shutdown_connections 2024-04-26T04:39:31.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 --2- 172.21.15.90:0/1173803687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c0fe320 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:31.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 --2- 172.21.15.90:0/1173803687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c1018a0 0x7f1d8c101cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:31.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 -- 172.21.15.90:0/1173803687 >> 172.21.15.90:0/1173803687 conn(0x7f1d8c0f9f10 msgr2=0x7f1d8c0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 -- 172.21.15.90:0/1173803687 shutdown_connections 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 -- 172.21.15.90:0/1173803687 wait complete. 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.859+0000 7f1d918a9700 1 Processor -- start 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d918a9700 1 -- start start 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d918a9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c1018a0 0x7f1d8c195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d918a9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d918a9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c196630 con 0x7f1d8c1021f0 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d918a9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d8c196770 con 0x7f1d8c1018a0 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38322/0 (socket says 172.21.15.90:38322) 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8affd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c1018a0 0x7f1d8c195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:31.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8a7fc700 1 -- 172.21.15.90:0/3335685744 learned_addr learned my addr 172.21.15.90:0/3335685744 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8affd700 1 -- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 msgr2=0x7f1d8c196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8affd700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8affd700 1 -- 172.21.15.90:0/3335685744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800095f0 con 0x7f1d8c1018a0 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8a7fc700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.860+0000 7f1d8affd700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c1018a0 0x7f1d8c195ad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00e970 tx=0x7f1d7c00ec80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.861+0000 7f1d908a7700 1 -- 172.21.15.90:0/3335685744 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d7c00c9d0 con 0x7f1d8c1018a0 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.861+0000 7f1d908a7700 1 -- 172.21.15.90:0/3335685744 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1d7c010430 con 0x7f1d8c1018a0 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.861+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d8c19b220 con 0x7f1d8c1018a0 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.861+0000 7f1d908a7700 1 -- 172.21.15.90:0/3335685744 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1d7c018650 con 0x7f1d8c1018a0 2024-04-26T04:39:31.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.861+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d8c19b770 con 0x7f1d8c1018a0 2024-04-26T04:39:31.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.862+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d8c108d10 con 0x7f1d8c1018a0 2024-04-26T04:39:31.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.862+0000 7f1d908a7700 1 -- 172.21.15.90:0/3335685744 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1d7c0105a0 con 0x7f1d8c1018a0 2024-04-26T04:39:31.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.863+0000 7f1d908a7700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1d7406c680 0x7f1d7406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:31.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.863+0000 7f1d908a7700 1 -- 172.21.15.90:0/3335685744 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f1d7c014070 con 0x7f1d8c1018a0 2024-04-26T04:39:31.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.863+0000 7f1d8a7fc700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1d7406c680 0x7f1d7406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:31.866 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.863+0000 7f1d8a7fc700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1d7406c680 0x7f1d7406eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009fd0 tx=0x7f1d80005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:31.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:31.867+0000 7f1d908a7700 1 -- 172.21.15.90:0/3335685744 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1d7c05a0c0 con 0x7f1d8c1018a0 2024-04-26T04:39:32.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.073+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1d8c105f90 con 0x7f1d7406c680 2024-04-26T04:39:32.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.082+0000 7f1d908a7700 1 -- 172.21.15.90:0/3335685744 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+2860 (secure 0 0 0) 0x7f1d8c105f90 con 0x7f1d7406c680 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (3m) 2m ago 5m 27.0M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (5m) 2m ago 5m 7004k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (4m) 61s ago 4m 6937k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (5m) 2m ago 5m 6660k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (4m) 61s ago 4m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (3m) 2m ago 4m 76.9M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:9283,8765,8443 running (6m) 2m ago 6m 478M - 18.2.0 dc2bc1663786 a008eb0a2d7e 2024-04-26T04:39:32.084 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (4m) 61s ago 4m 436M - 18.2.0 dc2bc1663786 b809d29d97b5 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (6m) 2m ago 6m 49.1M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (4m) 61s ago 4m 39.6M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (5m) 2m ago 5m 14.5M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (4m) 61s ago 4m 15.4M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (3m) 2m ago 3m 41.4M 3946M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (2m) 2m ago 2m 40.4M 3946M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (2m) 2m ago 2m 42.5M 3946M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (116s) 61s ago 116s 41.6M 4970M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (91s) 61s ago 91s 39.4M 4970M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (66s) 61s ago 66s 11.4M 4970M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:39:32.085 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (3m) 2m ago 4m 41.2M - 2.43.0 a07b618ecd1d 8382d6553d6c 2024-04-26T04:39:32.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1d7406c680 msgr2=0x7f1d7406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:32.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1d7406c680 0x7f1d7406eb30 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1d80009fd0 tx=0x7f1d80005940 comp rx=0 tx=0).stop 2024-04-26T04:39:32.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c1018a0 msgr2=0x7f1d8c195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:32.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c1018a0 0x7f1d8c195ad0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c00e970 tx=0x7f1d7c00ec80 comp rx=0 tx=0).stop 2024-04-26T04:39:32.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 shutdown_connections 2024-04-26T04:39:32.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1d7406c680 0x7f1d7406eb30 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:32.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1d8c1018a0 0x7f1d8c195ad0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:32.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 --2- 172.21.15.90:0/3335685744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1d8c1021f0 0x7f1d8c196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:32.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 >> 172.21.15.90:0/3335685744 conn(0x7f1d8c0f9f10 msgr2=0x7f1d8c104870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:32.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 shutdown_connections 2024-04-26T04:39:32.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:32.084+0000 7f1d918a9700 1 -- 172.21.15.90:0/3335685744 wait complete. 2024-04-26T04:39:32.460 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:32 smithi090 ceph-mon[26648]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:32.741 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-26T04:39:32.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:32 smithi104 ceph-mon[32369]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:32.950 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:33.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:33 smithi090 ceph-mon[26648]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:33.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:33 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:33.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:33 smithi104 ceph-mon[32369]: from='client.24247 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:33.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:33 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:34.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/1553158958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a581036a0 msgr2=0x7f7a58105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:34.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/1553158958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a581036a0 0x7f7a58105a80 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c009910 tx=0x7f7a4c009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:34.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/1553158958 shutdown_connections 2024-04-26T04:39:34.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/1553158958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a581036a0 0x7f7a58105a80 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:34.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/1553158958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a58100d80 0x7f7a58103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/1553158958 >> 172.21.15.90:0/1553158958 conn(0x7f7a580fa7a0 msgr2=0x7f7a580fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/1553158958 shutdown_connections 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.401+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/1553158958 wait complete. 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5cb9c700 1 Processor -- start 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5cb9c700 1 -- start start 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5cb9c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a58100d80 0x7f7a58195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5cb9c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a581036a0 0x7f7a58196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5cb9c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58196920 con 0x7f7a58100d80 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5cb9c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58196a60 con 0x7f7a581036a0 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a58100d80 0x7f7a58195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a58100d80 0x7f7a58195dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38348/0 (socket says 172.21.15.90:38348) 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5659c700 1 -- 172.21.15.90:0/743158507 learned_addr learned my addr 172.21.15.90:0/743158507 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:34.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a55d9b700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a581036a0 0x7f7a58196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5659c700 1 -- 172.21.15.90:0/743158507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a581036a0 msgr2=0x7f7a58196300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5659c700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a581036a0 0x7f7a58196300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5659c700 1 -- 172.21.15.90:0/743158507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0095f0 con 0x7f7a58100d80 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a55d9b700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a581036a0 0x7f7a58196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a5659c700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a58100d80 0x7f7a58195dc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d900 tx=0x7f7a4800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.402+0000 7f7a477fe700 1 -- 172.21.15.90:0/743158507 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a48009900 con 0x7f7a58100d80 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.403+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a5819b510 con 0x7f7a58100d80 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.403+0000 7f7a477fe700 1 -- 172.21.15.90:0/743158507 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7a48010460 con 0x7f7a58100d80 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.403+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a5819ba60 con 0x7f7a58100d80 2024-04-26T04:39:34.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.403+0000 7f7a477fe700 1 -- 172.21.15.90:0/743158507 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7a4800f5f0 con 0x7f7a58100d80 2024-04-26T04:39:34.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.404+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a38005320 con 0x7f7a58100d80 2024-04-26T04:39:34.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.404+0000 7f7a477fe700 1 -- 172.21.15.90:0/743158507 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7a48010a90 con 0x7f7a58100d80 2024-04-26T04:39:34.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.404+0000 7f7a477fe700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7a4006c680 0x7f7a4006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:34.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.405+0000 7f7a55d9b700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7a4006c680 0x7f7a4006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:34.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.405+0000 7f7a477fe700 1 -- 172.21.15.90:0/743158507 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f7a4808b340 con 0x7f7a58100d80 2024-04-26T04:39:34.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.405+0000 7f7a55d9b700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7a4006c680 0x7f7a4006eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b5c0 tx=0x7f7a4c005e20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:34.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.409+0000 7f7a477fe700 1 -- 172.21.15.90:0/743158507 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7a480597f0 con 0x7f7a58100d80 2024-04-26T04:39:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:34 smithi090 ceph-mon[26648]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:34.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.607+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a38000bf0 con 0x7f7a4006c680 2024-04-26T04:39:34.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.611+0000 7f7a477fe700 1 -- 172.21.15.90:0/743158507 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+1300 (secure 0 0 0) 0x7f7a38000bf0 con 0x7f7a4006c680 2024-04-26T04:39:34.612 INFO:teuthology.orchestra.run.smithi090.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-26T04:39:34.612 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager ?:9093,9094 1/1 2m ago 5m count:1 2024-04-26T04:39:34.612 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter 2/2 2m ago 5m * 2024-04-26T04:39:34.613 INFO:teuthology.orchestra.run.smithi090.stdout:crash 2/2 2m ago 5m * 2024-04-26T04:39:34.613 INFO:teuthology.orchestra.run.smithi090.stdout:grafana ?:3000 1/1 2m ago 5m count:1 2024-04-26T04:39:34.613 INFO:teuthology.orchestra.run.smithi090.stdout:mgr 2/2 2m ago 5m count:2 2024-04-26T04:39:34.613 INFO:teuthology.orchestra.run.smithi090.stdout:mon 2/2 2m ago 5m smithi090:172.21.15.90=smithi090;smithi104:172.21.15.104=smithi104;count:2 2024-04-26T04:39:34.613 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter ?:9100 2/2 2m ago 5m * 2024-04-26T04:39:34.613 INFO:teuthology.orchestra.run.smithi090.stdout:osd 6 2m ago - 2024-04-26T04:39:34.613 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus ?:9095 1/1 2m ago 5m count:1 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.613+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7a4006c680 msgr2=0x7f7a4006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.613+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7a4006c680 0x7f7a4006eb30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c00b5c0 tx=0x7f7a4c005e20 comp rx=0 tx=0).stop 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.613+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a58100d80 msgr2=0x7f7a58195dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.613+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a58100d80 0x7f7a58195dc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f7a4800d900 tx=0x7f7a4800dcc0 comp rx=0 tx=0).stop 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.614+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 shutdown_connections 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.614+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7a4006c680 0x7f7a4006eb30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.614+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7a58100d80 0x7f7a58195dc0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.614+0000 7f7a5cb9c700 1 --2- 172.21.15.90:0/743158507 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7a581036a0 0x7f7a58196300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.614+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 >> 172.21.15.90:0/743158507 conn(0x7f7a580fa7a0 msgr2=0x7f7a580fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:34.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.614+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 shutdown_connections 2024-04-26T04:39:34.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:34.614+0000 7f7a5cb9c700 1 -- 172.21.15.90:0/743158507 wait complete. 2024-04-26T04:39:34.702 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:34 smithi104 ceph-mon[32369]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:35.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-26T04:39:35.393 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:35.647 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:35 smithi090 ceph-mon[26648]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:35 smithi104 ceph-mon[32369]: from='client.14450 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:36.663 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:36 smithi104 ceph-mon[32369]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:36.667 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:36 smithi090 ceph-mon[26648]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:36.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 -- 172.21.15.90:0/3285758269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c1003d0 msgr2=0x7f6e6c100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:36.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3285758269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c1003d0 0x7f6e6c100840 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f6e68009910 tx=0x7f6e68009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:36.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 -- 172.21.15.90:0/3285758269 shutdown_connections 2024-04-26T04:39:36.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3285758269 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c1003d0 0x7f6e6c100840 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:36.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3285758269 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c0ff130 0x7f6e6c0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:36.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 -- 172.21.15.90:0/3285758269 >> 172.21.15.90:0/3285758269 conn(0x7f6e6c0fa780 msgr2=0x7f6e6c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:36.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 -- 172.21.15.90:0/3285758269 shutdown_connections 2024-04-26T04:39:36.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.767+0000 7f6e7431f700 1 -- 172.21.15.90:0/3285758269 wait complete. 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e7431f700 1 Processor -- start 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e7431f700 1 -- start start 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e7431f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c0ff130 0x7f6e6c0690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e7431f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c1003d0 0x7f6e6c19cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e7431f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c196630 con 0x7f6e6c0ff130 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e7431f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c19d100 con 0x7f6e6c1003d0 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e720bb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c0ff130 0x7f6e6c0690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e720bb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c0ff130 0x7f6e6c0690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38372/0 (socket says 172.21.15.90:38372) 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e720bb700 1 -- 172.21.15.90:0/3117392078 learned_addr learned my addr 172.21.15.90:0/3117392078 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:36.771 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e718ba700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c1003d0 0x7f6e6c19cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e720bb700 1 -- 172.21.15.90:0/3117392078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c1003d0 msgr2=0x7f6e6c19cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e720bb700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c1003d0 0x7f6e6c19cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e720bb700 1 -- 172.21.15.90:0/3117392078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e680095f0 con 0x7f6e6c0ff130 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e720bb700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c0ff130 0x7f6e6c0690a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e950 tx=0x7f6e5c00ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.768+0000 7f6e637fe700 1 -- 172.21.15.90:0/3117392078 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5c00cba0 con 0x7f6e6c0ff130 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.769+0000 7f6e637fe700 1 -- 172.21.15.90:0/3117392078 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6e5c00eea0 con 0x7f6e6c0ff130 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.769+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c19d3b0 con 0x7f6e6c0ff130 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.769+0000 7f6e637fe700 1 -- 172.21.15.90:0/3117392078 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e5c018650 con 0x7f6e6c0ff130 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.769+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c19d8d0 con 0x7f6e6c0ff130 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.770+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c107c10 con 0x7f6e6c0ff130 2024-04-26T04:39:36.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.770+0000 7f6e637fe700 1 -- 172.21.15.90:0/3117392078 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f6e5c00cd00 con 0x7f6e6c0ff130 2024-04-26T04:39:36.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.770+0000 7f6e637fe700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6e5806c680 0x7f6e5806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:36.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.770+0000 7f6e637fe700 1 -- 172.21.15.90:0/3117392078 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f6e5c014070 con 0x7f6e6c0ff130 2024-04-26T04:39:36.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.770+0000 7f6e718ba700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6e5806c680 0x7f6e5806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:36.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.771+0000 7f6e718ba700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6e5806c680 0x7f6e5806eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800b6f0 tx=0x7f6e68005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:36.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.775+0000 7f6e637fe700 1 -- 172.21.15.90:0/3117392078 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f6e5c051720 con 0x7f6e6c0ff130 2024-04-26T04:39:36.969 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.968+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e6c104d20 con 0x7f6e5806c680 2024-04-26T04:39:36.970 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.969+0000 7f6e637fe700 1 -- 172.21.15.90:0/3117392078 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+148 (secure 0 0 0) 0x7f6e6c104d20 con 0x7f6e5806c680 2024-04-26T04:39:36.970 INFO:teuthology.orchestra.run.smithi090.stdout:HOST ADDR LABELS STATUS 2024-04-26T04:39:36.970 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 172.21.15.90 2024-04-26T04:39:36.970 INFO:teuthology.orchestra.run.smithi090.stdout:smithi104 172.21.15.104 2024-04-26T04:39:36.971 INFO:teuthology.orchestra.run.smithi090.stdout:2 hosts in cluster 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6e5806c680 msgr2=0x7f6e5806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6e5806c680 0x7f6e5806eb30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800b6f0 tx=0x7f6e68005a90 comp rx=0 tx=0).stop 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c0ff130 msgr2=0x7f6e6c0690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c0ff130 0x7f6e6c0690a0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6e5c00e950 tx=0x7f6e5c00ec60 comp rx=0 tx=0).stop 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 shutdown_connections 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f6e5806c680 0x7f6e5806eb30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e6c0ff130 0x7f6e6c0690a0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 --2- 172.21.15.90:0/3117392078 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e6c1003d0 0x7f6e6c19cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 >> 172.21.15.90:0/3117392078 conn(0x7f6e6c0fa780 msgr2=0x7f6e6c103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 shutdown_connections 2024-04-26T04:39:36.973 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:36.971+0000 7f6e7431f700 1 -- 172.21.15.90:0/3117392078 wait complete. 2024-04-26T04:39:37.509 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-26T04:39:37.711 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:38.387 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:38 smithi090 ceph-mon[26648]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:38.444 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:38 smithi090 ceph-mon[26648]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:38 smithi104 ceph-mon[32369]: from='client.14454 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:38 smithi104 ceph-mon[32369]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 -- 172.21.15.90:0/1336418272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 msgr2=0x7fc4e8101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 --2- 172.21.15.90:0/1336418272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8101890 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc009910 tx=0x7fc4dc009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 -- 172.21.15.90:0/1336418272 shutdown_connections 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 --2- 172.21.15.90:0/1336418272 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8101890 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 --2- 172.21.15.90:0/1336418272 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e8100240 0x7fc4e8100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 -- 172.21.15.90:0/1336418272 >> 172.21.15.90:0/1336418272 conn(0x7fc4e80fb7f0 msgr2=0x7fc4e80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 -- 172.21.15.90:0/1336418272 shutdown_connections 2024-04-26T04:39:39.078 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.076+0000 7fc4ed054700 1 -- 172.21.15.90:0/1336418272 wait complete. 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.077+0000 7fc4ed054700 1 Processor -- start 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.077+0000 7fc4ed054700 1 -- start start 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4ed054700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e8100240 0x7fc4e8195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4ed054700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4ed054700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e81965f0 con 0x7fc4e8101440 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4ed054700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e8196730 con 0x7fc4e8100240 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46496/0 (socket says 172.21.15.90:46496) 2024-04-26T04:39:39.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e659c700 1 -- 172.21.15.90:0/3444524086 learned_addr learned my addr 172.21.15.90:0/3444524086 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e6d9d700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e8100240 0x7fc4e8195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e659c700 1 -- 172.21.15.90:0/3444524086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e8100240 msgr2=0x7fc4e8195a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e659c700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e8100240 0x7fc4e8195a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e659c700 1 -- 172.21.15.90:0/3444524086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc0095f0 con 0x7fc4e8101440 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e6d9d700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e8100240 0x7fc4e8195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4e659c700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8195fd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc006010 tx=0x7fc4dc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4cffff700 1 -- 172.21.15.90:0/3444524086 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4dc01d070 con 0x7fc4e8101440 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e819b180 con 0x7fc4e8101440 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4cffff700 1 -- 172.21.15.90:0/3444524086 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc4dc00f460 con 0x7fc4e8101440 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4cffff700 1 -- 172.21.15.90:0/3444524086 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc4dc021640 con 0x7fc4e8101440 2024-04-26T04:39:39.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.078+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e819b670 con 0x7fc4e8101440 2024-04-26T04:39:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.079+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e8108d90 con 0x7fc4e8101440 2024-04-26T04:39:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.080+0000 7fc4cffff700 1 -- 172.21.15.90:0/3444524086 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc4dc00fab0 con 0x7fc4e8101440 2024-04-26T04:39:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.080+0000 7fc4cffff700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc4d006c680 0x7fc4d006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.080+0000 7fc4cffff700 1 -- 172.21.15.90:0/3444524086 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7fc4dc08c980 con 0x7fc4e8101440 2024-04-26T04:39:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.080+0000 7fc4e6d9d700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc4d006c680 0x7fc4d006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:39.082 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.081+0000 7fc4e6d9d700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc4d006c680 0x7fc4d006eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4e81012a0 tx=0x7fc4d8005f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:39.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.085+0000 7fc4cffff700 1 -- 172.21.15.90:0/3444524086 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc4dc05ae30 con 0x7fc4e8101440 2024-04-26T04:39:39.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.282+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4e8105d90 con 0x7fc4d006c680 2024-04-26T04:39:39.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.283+0000 7fc4cffff700 1 -- 172.21.15.90:0/3444524086 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7fc4e8105d90 con 0x7fc4d006c680 2024-04-26T04:39:39.284 INFO:teuthology.orchestra.run.smithi090.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-26T04:39:39.284 INFO:teuthology.orchestra.run.smithi090.stdout:smithi090 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UK400BGN 372G No 2m ago LVM detected, locked 2024-04-26T04:39:39.285 INFO:teuthology.orchestra.run.smithi090.stdout:smithi104 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NW400BGN 372G No 65s ago LVM detected, locked 2024-04-26T04:39:39.286 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc4d006c680 msgr2=0x7fc4d006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc4d006c680 0x7fc4d006eb30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc4e81012a0 tx=0x7fc4d8005f90 comp rx=0 tx=0).stop 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 msgr2=0x7fc4e8195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8195fd0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc006010 tx=0x7fc4dc00be20 comp rx=0 tx=0).stop 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 shutdown_connections 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc4d006c680 0x7fc4d006eb30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4e8100240 0x7fc4e8195a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 --2- 172.21.15.90:0/3444524086 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4e8101440 0x7fc4e8195fd0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 >> 172.21.15.90:0/3444524086 conn(0x7fc4e80fb7f0 msgr2=0x7fc4e8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 shutdown_connections 2024-04-26T04:39:39.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:39.285+0000 7fc4ed054700 1 -- 172.21.15.90:0/3444524086 wait complete. 2024-04-26T04:39:39.836 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:39:39.850 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:39:39.850 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph fs volume create cephfs --placement=4' 2024-04-26T04:39:40.052 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:40.435 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:40 smithi090 ceph-mon[26648]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:40.436 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:40 smithi090 ceph-mon[26648]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:40.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:40 smithi104 ceph-mon[32369]: from='client.14458 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:40.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:40 smithi104 ceph-mon[32369]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 -- 172.21.15.90:0/287749394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e01003d0 msgr2=0x7f44e0100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/287749394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e01003d0 0x7f44e0100840 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f44d0009910 tx=0x7f44d0009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 -- 172.21.15.90:0/287749394 shutdown_connections 2024-04-26T04:39:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/287749394 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e01003d0 0x7f44e0100840 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/287749394 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e00ff130 0x7f44e00ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:41.550 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 -- 172.21.15.90:0/287749394 >> 172.21.15.90:0/287749394 conn(0x7f44e00fa780 msgr2=0x7f44e00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 -- 172.21.15.90:0/287749394 shutdown_connections 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.547+0000 7f44e7bd8700 1 -- 172.21.15.90:0/287749394 wait complete. 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e7bd8700 1 Processor -- start 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e7bd8700 1 -- start start 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e7bd8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e00ff130 0x7f44e0195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e7bd8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e01003d0 0x7f44e0195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e7bd8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e01965c0 con 0x7f44e00ff130 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e7bd8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44e0196700 con 0x7f44e01003d0 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5974700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e00ff130 0x7f44e0195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5974700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e00ff130 0x7f44e0195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46510/0 (socket says 172.21.15.90:46510) 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5974700 1 -- 172.21.15.90:0/2566459749 learned_addr learned my addr 172.21.15.90:0/2566459749 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5173700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e01003d0 0x7f44e0195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5974700 1 -- 172.21.15.90:0/2566459749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e01003d0 msgr2=0x7f44e0195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5974700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e01003d0 0x7f44e0195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5974700 1 -- 172.21.15.90:0/2566459749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d00095f0 con 0x7f44e00ff130 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e5974700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e00ff130 0x7f44e0195a60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44d80060b0 tx=0x7f44d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44d6ffd700 1 -- 172.21.15.90:0/2566459749 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d800dda0 con 0x7f44e00ff130 2024-04-26T04:39:41.552 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44d6ffd700 1 -- 172.21.15.90:0/2566459749 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f44d800f040 con 0x7f44e00ff130 2024-04-26T04:39:41.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.548+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44e019b1b0 con 0x7f44e00ff130 2024-04-26T04:39:41.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.549+0000 7f44d6ffd700 1 -- 172.21.15.90:0/2566459749 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44d80146f0 con 0x7f44e00ff130 2024-04-26T04:39:41.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.549+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44e019b700 con 0x7f44e00ff130 2024-04-26T04:39:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.549+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44e0107c10 con 0x7f44e00ff130 2024-04-26T04:39:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.550+0000 7f44d6ffd700 1 -- 172.21.15.90:0/2566459749 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f44d8015400 con 0x7f44e00ff130 2024-04-26T04:39:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.550+0000 7f44d6ffd700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f44cc06c680 0x7f44cc06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.550+0000 7f44e5173700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f44cc06c680 0x7f44cc06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.550+0000 7f44d6ffd700 1 -- 172.21.15.90:0/2566459749 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 4638+0+0 (secure 0 0 0) 0x7f44d808af90 con 0x7f44e00ff130 2024-04-26T04:39:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.551+0000 7f44e5173700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f44cc06c680 0x7f44cc06eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f44d0006010 tx=0x7f44d000b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.555+0000 7f44d6ffd700 1 -- 172.21.15.90:0/2566459749 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f44d804f720 con 0x7f44e00ff130 2024-04-26T04:39:41.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:41.774+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f44e0068c50 con 0x7f44cc06c680 2024-04-26T04:39:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:42 smithi090 ceph-mon[26648]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:42 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-26T04:39:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:42 smithi104 ceph-mon[32369]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:42 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]: dispatch 2024-04-26T04:39:43.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.384+0000 7f44d6ffd700 1 -- 172.21.15.90:0/2566459749 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f44e0068c50 con 0x7f44cc06c680 2024-04-26T04:39:43.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f44cc06c680 msgr2=0x7f44cc06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f44cc06c680 0x7f44cc06eb30 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f44d0006010 tx=0x7f44d000b540 comp rx=0 tx=0).stop 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e00ff130 msgr2=0x7f44e0195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e00ff130 0x7f44e0195a60 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f44d80060b0 tx=0x7f44d800d3b0 comp rx=0 tx=0).stop 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 shutdown_connections 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f44cc06c680 0x7f44cc06eb30 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44e00ff130 0x7f44e0195a60 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 --2- 172.21.15.90:0/2566459749 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44e01003d0 0x7f44e0195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 >> 172.21.15.90:0/2566459749 conn(0x7f44e00fa780 msgr2=0x7f44e0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 shutdown_connections 2024-04-26T04:39:43.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:43.387+0000 7f44e7bd8700 1 -- 172.21.15.90:0/2566459749 wait complete. 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[26644]: 2024-04-26T04:39:43.361+0000 7fd630dd2700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:39:43.681 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T04:39:43.682 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T04:39:43.682 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-26T04:39:43.682 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: fsmap cephfs:0 2024-04-26T04:39:43.682 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:43.682 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:43 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:39:44.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='client.14462 -' entity='client.admin' cmd=[{"prefix": "fs volume create", "name": "cephfs", "placement": "4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:39:44.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.meta"}]': finished 2024-04-26T04:39:44.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: osdmap e41: 6 total, 6 up, 6 in 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd pool create", "pool": "cephfs.cephfs.data"}]': finished 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: osdmap e42: 6 total, 6 up, 6 in 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]: dispatch 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: osdmap e43: 6 total, 6 up, 6 in 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs.cephfs.meta", "data": "cephfs.cephfs.data"}]': finished 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: fsmap cephfs:0 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:44.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:43 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:39:44.078 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T04:39:44.320 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:44.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: pgmap v169: 33 pgs: 12 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:44.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: Saving service mds.cephfs spec with placement count:4 2024-04-26T04:39:44.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:44.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:39:44.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:44.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:44.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:44.706 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:44.706 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:44 smithi090 ceph-mon[26648]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T04:39:45.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: pgmap v169: 33 pgs: 12 creating+peering, 20 unknown, 1 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: Saving service mds.cephfs spec with placement count:4 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:45.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:44 smithi104 ceph-mon[32369]: osdmap e44: 6 total, 6 up, 6 in 2024-04-26T04:39:45.720 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:45 smithi090 ceph-mon[26648]: Deploying daemon mds.cephfs.smithi090.ehputv on smithi090 2024-04-26T04:39:45.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:45 smithi090 ceph-mon[26648]: osdmap e45: 6 total, 6 up, 6 in 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 -- 172.21.15.90:0/382995551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 msgr2=0x7f75ec0ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 --2- 172.21.15.90:0/382995551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec0ff540 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f75e4009910 tx=0x7f75e4009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 -- 172.21.15.90:0/382995551 shutdown_connections 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 --2- 172.21.15.90:0/382995551 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ec1003d0 0x7f75ec100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 --2- 172.21.15.90:0/382995551 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec0ff540 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 -- 172.21.15.90:0/382995551 >> 172.21.15.90:0/382995551 conn(0x7f75ec0fa780 msgr2=0x7f75ec0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 -- 172.21.15.90:0/382995551 shutdown_connections 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.745+0000 7f75f4608700 1 -- 172.21.15.90:0/382995551 wait complete. 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f4608700 1 Processor -- start 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f4608700 1 -- start start 2024-04-26T04:39:45.748 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f4608700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f4608700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ec1003d0 0x7f75ec1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f4608700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec101cf0 con 0x7f75ec0ff130 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f4608700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75ec101e30 con 0x7f75ec1003d0 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f23a4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f23a4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46536/0 (socket says 172.21.15.90:46536) 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f23a4700 1 -- 172.21.15.90:0/2580005410 learned_addr learned my addr 172.21.15.90:0/2580005410 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f1ba3700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ec1003d0 0x7f75ec1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f23a4700 1 -- 172.21.15.90:0/2580005410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ec1003d0 msgr2=0x7f75ec1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f23a4700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ec1003d0 0x7f75ec1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f23a4700 1 -- 172.21.15.90:0/2580005410 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75e40095f0 con 0x7f75ec0ff130 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f23a4700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec103160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f75e4006010 tx=0x7f75e400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75eb7fe700 1 -- 172.21.15.90:0/2580005410 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e401d070 con 0x7f75ec0ff130 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.746+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75ec1020b0 con 0x7f75ec0ff130 2024-04-26T04:39:45.749 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.747+0000 7f75eb7fe700 1 -- 172.21.15.90:0/2580005410 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f75e400f460 con 0x7f75ec0ff130 2024-04-26T04:39:45.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.747+0000 7f75eb7fe700 1 -- 172.21.15.90:0/2580005410 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75e4021640 con 0x7f75ec0ff130 2024-04-26T04:39:45.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.747+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75ec1025a0 con 0x7f75ec0ff130 2024-04-26T04:39:45.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.747+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75ec107c10 con 0x7f75ec0ff130 2024-04-26T04:39:45.750 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.748+0000 7f75eb7fe700 1 -- 172.21.15.90:0/2580005410 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f75e400fab0 con 0x7f75ec0ff130 2024-04-26T04:39:45.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.748+0000 7f75eb7fe700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f75d406c680 0x7f75d406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:45.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.748+0000 7f75f1ba3700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f75d406c680 0x7f75d406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:45.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.748+0000 7f75eb7fe700 1 -- 172.21.15.90:0/2580005410 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f75e408cc30 con 0x7f75ec0ff130 2024-04-26T04:39:45.751 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.749+0000 7f75f1ba3700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f75d406c680 0x7f75d406eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75e000ac50 tx=0x7f75e000a380 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:45.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.753+0000 7f75eb7fe700 1 -- 172.21.15.90:0/2580005410 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f75e405ada0 con 0x7f75ec0ff130 2024-04-26T04:39:45.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.992+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f75ec19e110 con 0x7f75ec0ff130 2024-04-26T04:39:45.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.993+0000 7f75eb7fe700 1 -- 172.21.15.90:0/2580005410 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 2 v2) v1 ==== 75+0+1093 (secure 0 0 0) 0x7f75e405a930 con 0x7f75ec0ff130 2024-04-26T04:39:45.994 INFO:teuthology.orchestra.run.smithi090.stdout:e2 2024-04-26T04:39:45.994 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:39:45.994 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:39:45.994 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 2 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:39:43.362174+0000 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:39:45.995 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 1 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:in 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:up {} 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:39:45.996 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:39:45.997 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:39:45.997 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 0 2024-04-26T04:39:45.997 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:39:45.997 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f75d406c680 msgr2=0x7f75d406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f75d406c680 0x7f75d406eb30 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f75e000ac50 tx=0x7f75e000a380 comp rx=0 tx=0).stop 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 msgr2=0x7f75ec103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec103160 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f75e4006010 tx=0x7f75e400bd70 comp rx=0 tx=0).stop 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 shutdown_connections 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f75d406c680 0x7f75d406eb30 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75ec0ff130 0x7f75ec103160 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 --2- 172.21.15.90:0/2580005410 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75ec1003d0 0x7f75ec1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 >> 172.21.15.90:0/2580005410 conn(0x7f75ec0fa780 msgr2=0x7f75ec068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:45.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 shutdown_connections 2024-04-26T04:39:45.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:45.995+0000 7f75f4608700 1 -- 172.21.15.90:0/2580005410 wait complete. 2024-04-26T04:39:45.999 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 2 2024-04-26T04:39:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:45 smithi104 ceph-mon[32369]: Deploying daemon mds.cephfs.smithi090.ehputv on smithi090 2024-04-26T04:39:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:45 smithi104 ceph-mon[32369]: osdmap e45: 6 total, 6 up, 6 in 2024-04-26T04:39:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:46 smithi090 ceph-mon[26648]: pgmap v173: 65 pgs: 21 creating+peering, 38 unknown, 6 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:46 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2580005410' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:39:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:46 smithi104 ceph-mon[32369]: pgmap v173: 65 pgs: 21 creating+peering, 38 unknown, 6 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:46 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2580005410' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:39:47.099 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:39:47.109 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:39:47.109 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs max_mds 2' 2024-04-26T04:39:47.655 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:48 smithi090 ceph-mon[26648]: pgmap v175: 65 pgs: 21 creating+peering, 17 unknown, 27 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:48 smithi104 ceph-mon[32369]: pgmap v175: 65 pgs: 21 creating+peering, 17 unknown, 27 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:39:49.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.168+0000 7fbcfe661700 1 -- 172.21.15.90:0/1716662769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf81003d0 msgr2=0x7fbcf8100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 --2- 172.21.15.90:0/1716662769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf81003d0 0x7fbcf8100840 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fbcec009910 tx=0x7fbcec009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 -- 172.21.15.90:0/1716662769 shutdown_connections 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 --2- 172.21.15.90:0/1716662769 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf81003d0 0x7fbcf8100840 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 --2- 172.21.15.90:0/1716662769 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf80ff130 0x7fbcf80ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 -- 172.21.15.90:0/1716662769 >> 172.21.15.90:0/1716662769 conn(0x7fbcf80fa780 msgr2=0x7fbcf80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 -- 172.21.15.90:0/1716662769 shutdown_connections 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 -- 172.21.15.90:0/1716662769 wait complete. 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 Processor -- start 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.169+0000 7fbcfe661700 1 -- start start 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcfe661700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf80ff130 0x7fbcf80690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:49.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcfe661700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf81003d0 0x7fbcf819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcfe661700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf819ad90 con 0x7fbcf80ff130 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcfe661700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcf819aed0 con 0x7fbcf81003d0 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf80ff130 0x7fbcf80690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf80ff130 0x7fbcf80690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33246/0 (socket says 172.21.15.90:33246) 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf7fff700 1 -- 172.21.15.90:0/341991615 learned_addr learned my addr 172.21.15.90:0/341991615 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf7fff700 1 -- 172.21.15.90:0/341991615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf81003d0 msgr2=0x7fbcf819a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf7fff700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf81003d0 0x7fbcf819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf7fff700 1 -- 172.21.15.90:0/341991615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcec0095f0 con 0x7fbcf80ff130 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf7fff700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf80ff130 0x7fbcf80690a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbce800d8d0 tx=0x7fbce800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf57fa700 1 -- 172.21.15.90:0/341991615 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbce8009940 con 0x7fbcf80ff130 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcf819b1b0 con 0x7fbcf80ff130 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf57fa700 1 -- 172.21.15.90:0/341991615 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbce8010460 con 0x7fbcf80ff130 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcf57fa700 1 -- 172.21.15.90:0/341991615 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbce800f5d0 con 0x7fbcf80ff130 2024-04-26T04:39:49.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.170+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcf819b700 con 0x7fbcf80ff130 2024-04-26T04:39:49.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.171+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcf8107c10 con 0x7fbcf80ff130 2024-04-26T04:39:49.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.172+0000 7fbcf57fa700 1 -- 172.21.15.90:0/341991615 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fbce80105d0 con 0x7fbcf80ff130 2024-04-26T04:39:49.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.172+0000 7fbcf57fa700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbce006c680 0x7fbce006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:49.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.172+0000 7fbcf77fe700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbce006c680 0x7fbce006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:49.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.172+0000 7fbcf57fa700 1 -- 172.21.15.90:0/341991615 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fbce808b880 con 0x7fbcf80ff130 2024-04-26T04:39:49.175 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.173+0000 7fbcf77fe700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbce006c680 0x7fbce006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00b5c0 tx=0x7fbcec005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:49.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.177+0000 7fbcf57fa700 1 -- 172.21.15.90:0/341991615 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbce80599f0 con 0x7fbcf80ff130 2024-04-26T04:39:49.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.420+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"} v 0) v1 -- 0x7fbcf819bd40 con 0x7fbcf80ff130 2024-04-26T04:39:49.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.611+0000 7fbcf57fa700 1 -- 172.21.15.90:0/341991615 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]=0 v3) v1 ==== 105+0+0 (secure 0 0 0) 0x7fbce8059580 con 0x7fbcf80ff130 2024-04-26T04:39:49.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbce006c680 msgr2=0x7fbce006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:49.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbce006c680 0x7fbce006eb30 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fbcec00b5c0 tx=0x7fbcec005960 comp rx=0 tx=0).stop 2024-04-26T04:39:49.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf80ff130 msgr2=0x7fbcf80690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:49.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf80ff130 0x7fbcf80690a0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fbce800d8d0 tx=0x7fbce800dc90 comp rx=0 tx=0).stop 2024-04-26T04:39:49.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 shutdown_connections 2024-04-26T04:39:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbce006c680 0x7fbce006eb30 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbcf80ff130 0x7fbcf80690a0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 --2- 172.21.15.90:0/341991615 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbcf81003d0 0x7fbcf819a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.613+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 >> 172.21.15.90:0/341991615 conn(0x7fbcf80fa780 msgr2=0x7fbcf8103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.614+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 shutdown_connections 2024-04-26T04:39:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:49.614+0000 7fbcfe661700 1 -- 172.21.15.90:0/341991615 wait complete. 2024-04-26T04:39:49.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:49 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/341991615' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-26T04:39:50.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:49 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/341991615' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]: dispatch 2024-04-26T04:39:50.179 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:39:50.189 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:39:50.189 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs allow_standby_replay false' 2024-04-26T04:39:50.399 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/341991615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6826/3871906279,v1:172.21.15.90:6827/3871906279] up:boot 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: daemon mds.cephfs.smithi090.ehputv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: fsmap cephfs:0 1 up:standby 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: fsmap cephfs:1 {0=cephfs.smithi090.ehputv=up:creating} 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: daemon mds.cephfs.smithi090.ehputv is now active in filesystem cephfs as rank 0 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:50.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:50.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:50 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: pgmap v176: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/341991615' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "2"}]': finished 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6826/3871906279,v1:172.21.15.90:6827/3871906279] up:boot 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: daemon mds.cephfs.smithi090.ehputv assigned to filesystem cephfs as rank 0 (now has 1 ranks) 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: fsmap cephfs:0 1 up:standby 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:39:50.812 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: fsmap cephfs:1 {0=cephfs.smithi090.ehputv=up:creating} 2024-04-26T04:39:50.813 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: daemon mds.cephfs.smithi090.ehputv is now active in filesystem cephfs as rank 0 2024-04-26T04:39:50.813 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:50.813 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:50.813 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:50.813 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:50.813 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:50.813 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:50 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:51.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:51 smithi090 ceph-mon[26648]: Deploying daemon mds.cephfs.smithi104.wpntdi on smithi104 2024-04-26T04:39:51.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:51 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6826/3871906279,v1:172.21.15.90:6827/3871906279] up:active 2024-04-26T04:39:51.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:51 smithi090 ceph-mon[26648]: fsmap cephfs:1 {0=cephfs.smithi090.ehputv=up:active} 2024-04-26T04:39:51.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.866+0000 7f766ea04700 1 -- 172.21.15.90:0/3363306051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 msgr2=0x7f7668101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:51.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.866+0000 7f766ea04700 1 --2- 172.21.15.90:0/3363306051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668101890 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f765c009910 tx=0x7f765c009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:51.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 -- 172.21.15.90:0/3363306051 shutdown_connections 2024-04-26T04:39:51.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 --2- 172.21.15.90:0/3363306051 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668101890 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:51.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 --2- 172.21.15.90:0/3363306051 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7668100240 0x7f7668100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:51.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 -- 172.21.15.90:0/3363306051 >> 172.21.15.90:0/3363306051 conn(0x7f76680fb7f0 msgr2=0x7f76680fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:51.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 -- 172.21.15.90:0/3363306051 shutdown_connections 2024-04-26T04:39:51.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 -- 172.21.15.90:0/3363306051 wait complete. 2024-04-26T04:39:51.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 Processor -- start 2024-04-26T04:39:51.869 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 -- start start 2024-04-26T04:39:51.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7668100240 0x7f7668074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:51.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.867+0000 7f766ea04700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:51.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f766ea04700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7668073680 con 0x7f7668101440 2024-04-26T04:39:51.871 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f766ea04700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76680737c0 con 0x7f7668100240 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76677fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76677fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33294/0 (socket says 172.21.15.90:33294) 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76677fe700 1 -- 172.21.15.90:0/2660938375 learned_addr learned my addr 172.21.15.90:0/2660938375 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f7667fff700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7668100240 0x7f7668074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76677fe700 1 -- 172.21.15.90:0/2660938375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7668100240 msgr2=0x7f7668074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76677fe700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7668100240 0x7f7668074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76677fe700 1 -- 172.21.15.90:0/2660938375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f765c0095f0 con 0x7f7668101440 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76677fe700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668073140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f765c006010 tx=0x7f765c00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76657fa700 1 -- 172.21.15.90:0/2660938375 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f765c01d070 con 0x7f7668101440 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7668073a40 con 0x7f7668101440 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76657fa700 1 -- 172.21.15.90:0/2660938375 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f765c00f460 con 0x7f7668101440 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f76657fa700 1 -- 172.21.15.90:0/2660938375 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f765c021640 con 0x7f7668101440 2024-04-26T04:39:51.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.868+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7668073f30 con 0x7f7668101440 2024-04-26T04:39:51.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.869+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7668108d90 con 0x7f7668101440 2024-04-26T04:39:51.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.870+0000 7f76657fa700 1 -- 172.21.15.90:0/2660938375 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f765c00fab0 con 0x7f7668101440 2024-04-26T04:39:51.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.870+0000 7f76657fa700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f765006c680 0x7f765006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:51.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.870+0000 7f76657fa700 1 -- 172.21.15.90:0/2660938375 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f765c08cad0 con 0x7f7668101440 2024-04-26T04:39:51.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.870+0000 7f7667fff700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f765006c680 0x7f765006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:51.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.871+0000 7f7667fff700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f765006c680 0x7f765006eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76681012a0 tx=0x7f765800b410 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:51.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:51.875+0000 7f76657fa700 1 -- 172.21.15.90:0/2660938375 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f765c05ac40 con 0x7f7668101440 2024-04-26T04:39:51.939 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:51 smithi104 ceph-mon[32369]: Deploying daemon mds.cephfs.smithi104.wpntdi on smithi104 2024-04-26T04:39:51.939 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:51 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6826/3871906279,v1:172.21.15.90:6827/3871906279] up:active 2024-04-26T04:39:51.939 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:51 smithi104 ceph-mon[32369]: fsmap cephfs:1 {0=cephfs.smithi090.ehputv=up:active} 2024-04-26T04:39:52.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.111+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f7668108f50 con 0x7f7668101440 2024-04-26T04:39:52.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.635+0000 7f76657fa700 1 -- 172.21.15.90:0/2660938375 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f765c05a7d0 con 0x7f7668101440 2024-04-26T04:39:52.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f765006c680 msgr2=0x7f765006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f765006c680 0x7f765006eb30 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f76681012a0 tx=0x7f765800b410 comp rx=0 tx=0).stop 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 msgr2=0x7f7668073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668073140 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f765c006010 tx=0x7f765c00bd70 comp rx=0 tx=0).stop 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 shutdown_connections 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f765006c680 0x7f765006eb30 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7668100240 0x7f7668074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 --2- 172.21.15.90:0/2660938375 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7668101440 0x7f7668073140 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 >> 172.21.15.90:0/2660938375 conn(0x7f76680fb7f0 msgr2=0x7f7668104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 shutdown_connections 2024-04-26T04:39:52.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:52.638+0000 7f766ea04700 1 -- 172.21.15.90:0/2660938375 wait complete. 2024-04-26T04:39:52.958 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 258 B/s wr, 0 op/s 2024-04-26T04:39:52.958 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2660938375' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-26T04:39:52.958 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:52.958 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:52.958 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:52.958 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:52.958 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:52.959 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:52 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: pgmap v177: 65 pgs: 65 active+clean; 577 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 258 B/s wr, 0 op/s 2024-04-26T04:39:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2660938375' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]: dispatch 2024-04-26T04:39:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:53.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:53.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:52 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:53.357 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:39:53.367 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:39:53.367 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph fs set cephfs inline_data false' 2024-04-26T04:39:53.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: Deploying daemon mds.cephfs.smithi090.dbkjuk on smithi090 2024-04-26T04:39:53.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] up:boot 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2660938375' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: daemon mds.cephfs.smithi104.wpntdi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: fsmap cephfs:1 {0=cephfs.smithi090.ehputv=up:active} 1 up:standby 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:creating} 2024-04-26T04:39:53.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:53 smithi090 ceph-mon[26648]: daemon mds.cephfs.smithi104.wpntdi is now active in filesystem cephfs as rank 1 2024-04-26T04:39:54.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: Deploying daemon mds.cephfs.smithi090.dbkjuk on smithi090 2024-04-26T04:39:54.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] up:boot 2024-04-26T04:39:54.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2660938375' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]': finished 2024-04-26T04:39:54.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: daemon mds.cephfs.smithi104.wpntdi assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-26T04:39:54.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: Health check failed: insufficient standby MDS daemons available (MDS_INSUFFICIENT_STANDBY) 2024-04-26T04:39:54.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-26T04:39:54.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: fsmap cephfs:1 {0=cephfs.smithi090.ehputv=up:active} 1 up:standby 2024-04-26T04:39:54.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:39:54.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:creating} 2024-04-26T04:39:54.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:53 smithi104 ceph-mon[32369]: daemon mds.cephfs.smithi104.wpntdi is now active in filesystem cephfs as rank 1 2024-04-26T04:39:54.476 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:54.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:54 smithi090 ceph-mon[26648]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-04-26T04:39:54.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:54 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] up:active 2024-04-26T04:39:54.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:54 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 2024-04-26T04:39:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:54 smithi104 ceph-mon[32369]: pgmap v178: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.6 KiB/s wr, 3 op/s 2024-04-26T04:39:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:54 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] up:active 2024-04-26T04:39:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:54 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 2024-04-26T04:39:56.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.506+0000 7f87b3935700 1 -- 172.21.15.90:0/2194466234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac102210 msgr2=0x7f87ac0fe340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:56.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.506+0000 7f87b3935700 1 --2- 172.21.15.90:0/2194466234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac102210 0x7f87ac0fe340 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f879c009910 tx=0x7f879c009c20 comp rx=0 tx=0).stop 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 -- 172.21.15.90:0/2194466234 shutdown_connections 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 --2- 172.21.15.90:0/2194466234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac102210 0x7f87ac0fe340 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 --2- 172.21.15.90:0/2194466234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87ac1018c0 0x7f87ac101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 -- 172.21.15.90:0/2194466234 >> 172.21.15.90:0/2194466234 conn(0x7f87ac0f9f10 msgr2=0x7f87ac0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 -- 172.21.15.90:0/2194466234 shutdown_connections 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 -- 172.21.15.90:0/2194466234 wait complete. 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 Processor -- start 2024-04-26T04:39:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.507+0000 7f87b3935700 1 -- start start 2024-04-26T04:39:56.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b3935700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac1018c0 0x7f87ac195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:56.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b3935700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87ac102210 0x7f87ac196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:56.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b3935700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac196640 con 0x7f87ac1018c0 2024-04-26T04:39:56.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b3935700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87ac196780 con 0x7f87ac102210 2024-04-26T04:39:56.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b16d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac1018c0 0x7f87ac195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:56.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b16d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac1018c0 0x7f87ac195ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33328/0 (socket says 172.21.15.90:33328) 2024-04-26T04:39:56.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b16d1700 1 -- 172.21.15.90:0/3169755958 learned_addr learned my addr 172.21.15.90:0/3169755958 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b0ed0700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87ac102210 0x7f87ac196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b16d1700 1 -- 172.21.15.90:0/3169755958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87ac102210 msgr2=0x7f87ac196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b16d1700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87ac102210 0x7f87ac196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b16d1700 1 -- 172.21.15.90:0/3169755958 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0095f0 con 0x7f87ac1018c0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b0ed0700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87ac102210 0x7f87ac196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b16d1700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac1018c0 0x7f87ac195ae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87a800d8d0 tx=0x7f87a800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87a27fc700 1 -- 172.21.15.90:0/3169755958 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a8009940 con 0x7f87ac1018c0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87a27fc700 1 -- 172.21.15.90:0/3169755958 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f87a8010460 con 0x7f87ac1018c0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87ac19b230 con 0x7f87ac1018c0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87a27fc700 1 -- 172.21.15.90:0/3169755958 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f87a800f5f0 con 0x7f87ac1018c0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.508+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87ac19b780 con 0x7f87ac1018c0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.509+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87ac108d30 con 0x7f87ac1018c0 2024-04-26T04:39:56.512 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.510+0000 7f87a27fc700 1 -- 172.21.15.90:0/3169755958 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f87a80105d0 con 0x7f87ac1018c0 2024-04-26T04:39:56.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.510+0000 7f87a27fc700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f879806c680 0x7f879806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:39:56.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.510+0000 7f87a27fc700 1 -- 172.21.15.90:0/3169755958 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7f87a808b7b0 con 0x7f87ac1018c0 2024-04-26T04:39:56.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.510+0000 7f87b0ed0700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f879806c680 0x7f879806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:39:56.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.511+0000 7f87b0ed0700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f879806c680 0x7f879806eb30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f879c006010 tx=0x7f879c005ab0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:39:56.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.515+0000 7f87a27fc700 1 -- 172.21.15.90:0/3169755958 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f87a80599a0 con 0x7f87ac1018c0 2024-04-26T04:39:56.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:56.753+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"} v 0) v1 -- 0x7f87ac108ef0 con 0x7f87ac1018c0 2024-04-26T04:39:56.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:56 smithi104 ceph-mon[32369]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-26T04:39:56.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:56.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:56.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:56.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:56.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:56.925 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:56 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:56 smithi090 ceph-mon[26648]: pgmap v179: 65 pgs: 65 active+clean; 579 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-26T04:39:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:39:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-26T04:39:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:56 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:39:57.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.656+0000 7f87a27fc700 1 -- 172.21.15.90:0/3169755958 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]=0 inline data disabled v10) v1 ==== 133+0+0 (secure 0 0 0) 0x7f87a805de60 con 0x7f87ac1018c0 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f879806c680 msgr2=0x7f879806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f879806c680 0x7f879806eb30 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f879c006010 tx=0x7f879c005ab0 comp rx=0 tx=0).stop 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac1018c0 msgr2=0x7f87ac195ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac1018c0 0x7f87ac195ae0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f87a800d8d0 tx=0x7f87a800dc90 comp rx=0 tx=0).stop 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 shutdown_connections 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f879806c680 0x7f879806eb30 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f87ac1018c0 0x7f87ac195ae0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:57.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 --2- 172.21.15.90:0/3169755958 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f87ac102210 0x7f87ac196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:39:57.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 >> 172.21.15.90:0/3169755958 conn(0x7f87ac0f9f10 msgr2=0x7f87ac104890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:39:57.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 shutdown_connections 2024-04-26T04:39:57.661 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:39:57.659+0000 7f87b3935700 1 -- 172.21.15.90:0/3169755958 wait complete. 2024-04-26T04:39:57.661 INFO:teuthology.orchestra.run.smithi090.stderr:inline data disabled 2024-04-26T04:39:57.967 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:57 smithi090 ceph-mon[26648]: Deploying daemon mds.cephfs.smithi104.ndiusl on smithi104 2024-04-26T04:39:57.967 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:57 smithi090 ceph-mon[26648]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-26T04:39:57.967 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:57 smithi090 ceph-mon[26648]: Cluster is now healthy 2024-04-26T04:39:57.967 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:57 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:boot 2024-04-26T04:39:57.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:57 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 1 up:standby 2024-04-26T04:39:57.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:57 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:39:57.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:57 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3169755958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-26T04:39:57.993 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:57 smithi104 ceph-mon[32369]: Deploying daemon mds.cephfs.smithi104.ndiusl on smithi104 2024-04-26T04:39:57.993 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:57 smithi104 ceph-mon[32369]: Health check cleared: MDS_INSUFFICIENT_STANDBY (was: insufficient standby MDS daemons available) 2024-04-26T04:39:57.993 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:57 smithi104 ceph-mon[32369]: Cluster is now healthy 2024-04-26T04:39:57.993 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:57 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:boot 2024-04-26T04:39:57.993 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:57 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 1 up:standby 2024-04-26T04:39:57.993 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:57 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:39:57.993 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:57 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3169755958' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]: dispatch 2024-04-26T04:39:58.320 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:39:58.330 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:39:58.330 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T04:39:58.545 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:39:58.906 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:58 smithi090 ceph-mon[26648]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-26T04:39:58.906 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:58 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3169755958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-26T04:39:58.906 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:58 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 1 up:standby 2024-04-26T04:39:58.906 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:58 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:58 smithi104 ceph-mon[32369]: pgmap v180: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.1 KiB/s wr, 5 op/s 2024-04-26T04:39:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:58 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3169755958' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "inline_data", "val": "false"}]': finished 2024-04-26T04:39:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:58 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 1 up:standby 2024-04-26T04:39:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:58 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:59.796 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] up:boot 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:39:59.797 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:39:59 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:40:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:40:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: pgmap v181: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.2 KiB/s wr, 6 op/s 2024-04-26T04:40:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] up:boot 2024-04-26T04:40:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:39:59 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:40:00.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 -- 172.21.15.90:0/1247068007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 msgr2=0x7fa72c0ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:00.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 --2- 172.21.15.90:0/1247068007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c0ff560 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7fa71c009910 tx=0x7fa71c009c20 comp rx=0 tx=0).stop 2024-04-26T04:40:00.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 -- 172.21.15.90:0/1247068007 shutdown_connections 2024-04-26T04:40:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 --2- 172.21.15.90:0/1247068007 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa72c1003f0 0x7fa72c100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 --2- 172.21.15.90:0/1247068007 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c0ff560 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 -- 172.21.15.90:0/1247068007 >> 172.21.15.90:0/1247068007 conn(0x7fa72c0fa7a0 msgr2=0x7fa72c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 -- 172.21.15.90:0/1247068007 shutdown_connections 2024-04-26T04:40:00.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.012+0000 7fa73135d700 1 -- 172.21.15.90:0/1247068007 wait complete. 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa73135d700 1 Processor -- start 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa73135d700 1 -- start start 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa73135d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa73135d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa72c1003f0 0x7fa72c196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa73135d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c196970 con 0x7fa72c0ff150 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa73135d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c196ab0 con 0x7fa72c1003f0 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa72affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa72affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59686/0 (socket says 172.21.15.90:59686) 2024-04-26T04:40:00.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa72affd700 1 -- 172.21.15.90:0/1443752690 learned_addr learned my addr 172.21.15.90:0/1443752690 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa72a7fc700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa72c1003f0 0x7fa72c196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa72affd700 1 -- 172.21.15.90:0/1443752690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa72c1003f0 msgr2=0x7fa72c196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa72affd700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa72c1003f0 0x7fa72c196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.013+0000 7fa72affd700 1 -- 172.21.15.90:0/1443752690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa71c0095f0 con 0x7fa72c0ff150 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.014+0000 7fa72affd700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c195e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000c00 tx=0x7fa71c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.014+0000 7fa713fff700 1 -- 172.21.15.90:0/1443752690 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa71c01d070 con 0x7fa72c0ff150 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.014+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c19b500 con 0x7fa72c0ff150 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.014+0000 7fa713fff700 1 -- 172.21.15.90:0/1443752690 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa71c005dc0 con 0x7fa72c0ff150 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.014+0000 7fa713fff700 1 -- 172.21.15.90:0/1443752690 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa71c0226a0 con 0x7fa72c0ff150 2024-04-26T04:40:00.017 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.014+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c19b9f0 con 0x7fa72c0ff150 2024-04-26T04:40:00.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.015+0000 7fa713fff700 1 -- 172.21.15.90:0/1443752690 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa71c0038f0 con 0x7fa72c0ff150 2024-04-26T04:40:00.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.016+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c068ab0 con 0x7fa72c0ff150 2024-04-26T04:40:00.018 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.016+0000 7fa713fff700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa71406c680 0x7fa71406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:00.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.016+0000 7fa713fff700 1 -- 172.21.15.90:0/1443752690 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5468+0+0 (secure 0 0 0) 0x7fa71c08cce0 con 0x7fa72c0ff150 2024-04-26T04:40:00.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.016+0000 7fa72a7fc700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa71406c680 0x7fa71406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:00.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.017+0000 7fa72a7fc700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa71406c680 0x7fa71406eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa720009c60 tx=0x7fa720009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:00.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.025+0000 7fa713fff700 1 -- 172.21.15.90:0/1443752690 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa71c05ae50 con 0x7fa72c0ff150 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:e11 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 10 2024-04-26T04:40:00.294 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:39:57.649671+0000 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 0 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:40:00.295 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14474,1=24273} 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{0:14474} state up:active seq 2 addr [v2:172.21.15.90:6826/3871906279,v1:172.21.15.90:6827/3871906279] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 2 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:40:00.296 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:40:00.297 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{-1:14486} state up:standby seq 1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:40:00.297 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:40:00.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.291+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fa72c19bcd0 con 0x7fa72c0ff150 2024-04-26T04:40:00.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.292+0000 7fa713fff700 1 -- 172.21.15.90:0/1443752690 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 11 v11) v1 ==== 76+0+1782 (secure 0 0 0) 0x7fa71c003680 con 0x7fa72c0ff150 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa71406c680 msgr2=0x7fa71406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa71406c680 0x7fa71406eb30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fa720009c60 tx=0x7fa720009260 comp rx=0 tx=0).stop 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 msgr2=0x7fa72c195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c195e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fa71c000c00 tx=0x7fa71c00f690 comp rx=0 tx=0).stop 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 shutdown_connections 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa71406c680 0x7fa71406eb30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa72c0ff150 0x7fa72c195e10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 --2- 172.21.15.90:0/1443752690 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa72c1003f0 0x7fa72c196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 >> 172.21.15.90:0/1443752690 conn(0x7fa72c0fa7a0 msgr2=0x7fa72c068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 shutdown_connections 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:00.294+0000 7fa73135d700 1 -- 172.21.15.90:0/1443752690 wait complete. 2024-04-26T04:40:00.298 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 11 2024-04-26T04:40:01.041 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:00 smithi090 ceph-mon[26648]: overall HEALTH_OK 2024-04-26T04:40:01.041 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:00 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1443752690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:40:01.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:00 smithi104 ceph-mon[32369]: overall HEALTH_OK 2024-04-26T04:40:01.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:00 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1443752690' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:40:01.314 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph --format=json fs dump | jq -e ".filesystems | length == 1"' 2024-04-26T04:40:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:standby 2024-04-26T04:40:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] up:active 2024-04-26T04:40:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: Dropping low affinity active daemon mds.cephfs.smithi090.ehputv in favor of higher affinity standby. 2024-04-26T04:40:01.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: Replacing daemon mds.cephfs.smithi090.ehputv as rank 0 with standby daemon mds.cephfs.smithi090.dbkjuk 2024-04-26T04:40:01.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T04:40:01.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:01.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: osdmap e46: 6 total, 6 up, 6 in 2024-04-26T04:40:01.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:replay,1=cephfs.smithi104.wpntdi=up:active} 1 up:standby 2024-04-26T04:40:01.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:01.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:01.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:01 smithi090 ceph-mon[26648]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-04-26T04:40:02.175 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:standby 2024-04-26T04:40:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] up:active 2024-04-26T04:40:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: Dropping low affinity active daemon mds.cephfs.smithi090.ehputv in favor of higher affinity standby. 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: Replacing daemon mds.cephfs.smithi090.ehputv as rank 0 with standby daemon mds.cephfs.smithi090.dbkjuk 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: osdmap e46: 6 total, 6 up, 6 in 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:replay,1=cephfs.smithi104.wpntdi=up:active} 1 up:standby 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:01 smithi104 ceph-mon[32369]: pgmap v183: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.4 KiB/s wr, 7 op/s 2024-04-26T04:40:03.072 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:02 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:resolve 2024-04-26T04:40:03.072 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:02 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] up:boot 2024-04-26T04:40:03.072 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:02 smithi090 ceph-mon[26648]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:resolve,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:03.072 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:02 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:40:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:02 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:resolve 2024-04-26T04:40:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:02 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] up:boot 2024-04-26T04:40:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:02 smithi104 ceph-mon[32369]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:resolve,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:02 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:40:04.106 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:03 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:reconnect 2024-04-26T04:40:04.106 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:03 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] up:standby 2024-04-26T04:40:04.106 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:03 smithi090 ceph-mon[26648]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:reconnect,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:04.106 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:03 smithi090 ceph-mon[26648]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T04:40:04.106 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:03 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:04.106 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:03 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:04.106 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:03 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:04.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 -- 172.21.15.90:0/222360451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc101440 msgr2=0x7f51fc101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:04.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 --2- 172.21.15.90:0/222360451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc101440 0x7f51fc101890 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009910 tx=0x7f51f0009c20 comp rx=0 tx=0).stop 2024-04-26T04:40:04.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 -- 172.21.15.90:0/222360451 shutdown_connections 2024-04-26T04:40:04.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 --2- 172.21.15.90:0/222360451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc101440 0x7f51fc101890 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 --2- 172.21.15.90:0/222360451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc100240 0x7f51fc100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 -- 172.21.15.90:0/222360451 >> 172.21.15.90:0/222360451 conn(0x7f51fc0fb7f0 msgr2=0x7f51fc0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 -- 172.21.15.90:0/222360451 shutdown_connections 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.122+0000 7f520151a700 1 -- 172.21.15.90:0/222360451 wait complete. 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f520151a700 1 Processor -- start 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f520151a700 1 -- start start 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f520151a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc100240 0x7f51fc195b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f520151a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc101440 0x7f51fc196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f520151a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc196690 con 0x7f51fc100240 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f520151a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc1967d0 con 0x7f51fc101440 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f51faffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc100240 0x7f51fc195b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f51faffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc100240 0x7f51fc195b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59720/0 (socket says 172.21.15.90:59720) 2024-04-26T04:40:04.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.123+0000 7f51faffd700 1 -- 172.21.15.90:0/1000045723 learned_addr learned my addr 172.21.15.90:0/1000045723 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51fa7fc700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc101440 0x7f51fc196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51faffd700 1 -- 172.21.15.90:0/1000045723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc101440 msgr2=0x7f51fc196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51faffd700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc101440 0x7f51fc196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51faffd700 1 -- 172.21.15.90:0/1000045723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f00095f0 con 0x7f51fc100240 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51faffd700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc100240 0x7f51fc195b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b480 tx=0x7f51ec00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51e3fff700 1 -- 172.21.15.90:0/1000045723 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ec0118c0 con 0x7f51fc100240 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51e3fff700 1 -- 172.21.15.90:0/1000045723 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f51ec010380 con 0x7f51fc100240 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc19b280 con 0x7f51fc100240 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f51e3fff700 1 -- 172.21.15.90:0/1000045723 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ec00f4f0 con 0x7f51fc100240 2024-04-26T04:40:04.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.124+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc19b7d0 con 0x7f51fc100240 2024-04-26T04:40:04.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.125+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc108e30 con 0x7f51fc100240 2024-04-26T04:40:04.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.125+0000 7f51e3fff700 1 -- 172.21.15.90:0/1000045723 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f51ec0104f0 con 0x7f51fc100240 2024-04-26T04:40:04.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.126+0000 7f51e3fff700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f51e406c680 0x7f51e406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:04.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.126+0000 7f51e3fff700 1 -- 172.21.15.90:0/1000045723 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f51ec08b530 con 0x7f51fc100240 2024-04-26T04:40:04.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.126+0000 7f51fa7fc700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f51e406c680 0x7f51e406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:04.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.126+0000 7f51fa7fc700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f51e406c680 0x7f51e406eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f51f00059d0 tx=0x7f51f0005920 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:04.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.130+0000 7f51e3fff700 1 -- 172.21.15.90:0/1000045723 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f51ec059640 con 0x7f51fc100240 2024-04-26T04:40:04.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:03 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:reconnect 2024-04-26T04:40:04.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:03 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] up:standby 2024-04-26T04:40:04.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:03 smithi104 ceph-mon[32369]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:reconnect,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:04.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:03 smithi104 ceph-mon[32369]: pgmap v184: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 1.2 KiB/s wr, 6 op/s 2024-04-26T04:40:04.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:03 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:04.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:03 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:04.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:03 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:04.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.378+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51fc19be70 con 0x7f51fc100240 2024-04-26T04:40:04.380 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.379+0000 7f51e3fff700 1 -- 172.21.15.90:0/1000045723 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4754 (secure 0 0 0) 0x7f51ec0591d0 con 0x7f51fc100240 2024-04-26T04:40:04.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f51e406c680 msgr2=0x7f51e406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:04.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f51e406c680 0x7f51e406eb30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f51f00059d0 tx=0x7f51f0005920 comp rx=0 tx=0).stop 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc100240 msgr2=0x7f51fc195b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc100240 0x7f51fc195b30 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00b480 tx=0x7f51ec00b840 comp rx=0 tx=0).stop 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 shutdown_connections 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f51e406c680 0x7f51e406eb30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc100240 0x7f51fc195b30 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 --2- 172.21.15.90:0/1000045723 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc101440 0x7f51fc196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 >> 172.21.15.90:0/1000045723 conn(0x7f51fc0fb7f0 msgr2=0x7f51fc104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:04.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.381+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 shutdown_connections 2024-04-26T04:40:04.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:04.382+0000 7f520151a700 1 -- 172.21.15.90:0/1000045723 wait complete. 2024-04-26T04:40:04.384 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 16 2024-04-26T04:40:04.395 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:40:05.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:04 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:rejoin 2024-04-26T04:40:05.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:04 smithi090 ceph-mon[26648]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:rejoin,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:05.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:04 smithi090 ceph-mon[26648]: daemon mds.cephfs.smithi090.dbkjuk is now active in filesystem cephfs as rank 0 2024-04-26T04:40:05.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:04 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1000045723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:40:05.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:04 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:rejoin 2024-04-26T04:40:05.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:04 smithi104 ceph-mon[32369]: fsmap cephfs:2/2 {0=cephfs.smithi090.dbkjuk=up:rejoin,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:05.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:04 smithi104 ceph-mon[32369]: daemon mds.cephfs.smithi090.dbkjuk is now active in filesystem cephfs as rank 0 2024-04-26T04:40:05.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:04 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1000045723' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:40:05.459 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'while ! ceph --format=json mds versions | jq -e ". | add == 4"; do sleep 1; done' 2024-04-26T04:40:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:05 smithi090 ceph-mon[26648]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T04:40:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:05 smithi090 ceph-mon[26648]: Cluster is now healthy 2024-04-26T04:40:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:05 smithi090 ceph-mon[26648]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:active 2024-04-26T04:40:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:05 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:05 smithi090 ceph-mon[26648]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 511 B/s wr, 3 op/s 2024-04-26T04:40:06.027 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:06.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:05 smithi104 ceph-mon[32369]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T04:40:06.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:05 smithi104 ceph-mon[32369]: Cluster is now healthy 2024-04-26T04:40:06.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:05 smithi104 ceph-mon[32369]: mds.? [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] up:active 2024-04-26T04:40:06.164 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:05 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:40:06.164 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:05 smithi104 ceph-mon[32369]: pgmap v185: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 819 B/s rd, 511 B/s wr, 3 op/s 2024-04-26T04:40:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/3428095842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 msgr2=0x7fa7c41056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/3428095842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c41056d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fa7b8009910 tx=0x7fa7b8009c20 comp rx=0 tx=0).stop 2024-04-26T04:40:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/3428095842 shutdown_connections 2024-04-26T04:40:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/3428095842 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c41056d0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/3428095842 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7c41009d0 0x7fa7c4102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/3428095842 >> 172.21.15.90:0/3428095842 conn(0x7fa7c40fa3f0 msgr2=0x7fa7c40fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/3428095842 shutdown_connections 2024-04-26T04:40:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/3428095842 wait complete. 2024-04-26T04:40:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 Processor -- start 2024-04-26T04:40:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.452+0000 7fa7c8fa2700 1 -- start start 2024-04-26T04:40:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c8fa2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7c41009d0 0x7fa7c4195760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c8fa2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c4195ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c8fa2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c41962c0 con 0x7fa7c41032f0 2024-04-26T04:40:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c8fa2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c4196400 con 0x7fa7c41009d0 2024-04-26T04:40:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c4195ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c4195ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59752/0 (socket says 172.21.15.90:59752) 2024-04-26T04:40:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c2ffd700 1 -- 172.21.15.90:0/258081503 learned_addr learned my addr 172.21.15.90:0/258081503 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c37fe700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7c41009d0 0x7fa7c4195760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c2ffd700 1 -- 172.21.15.90:0/258081503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7c41009d0 msgr2=0x7fa7c4195760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c2ffd700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7c41009d0 0x7fa7c4195760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c2ffd700 1 -- 172.21.15.90:0/258081503 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b80095f0 con 0x7fa7c41032f0 2024-04-26T04:40:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c2ffd700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c4195ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800b5c0 tx=0x7fa7b800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c0ff9700 1 -- 172.21.15.90:0/258081503 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b801d070 con 0x7fa7c41032f0 2024-04-26T04:40:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c0ff9700 1 -- 172.21.15.90:0/258081503 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa7b8022470 con 0x7fa7c41032f0 2024-04-26T04:40:07.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c419ae50 con 0x7fa7c41032f0 2024-04-26T04:40:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.453+0000 7fa7c0ff9700 1 -- 172.21.15.90:0/258081503 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa7b800f690 con 0x7fa7c41032f0 2024-04-26T04:40:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.454+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c419b390 con 0x7fa7c41032f0 2024-04-26T04:40:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.454+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c418f9e0 con 0x7fa7c41032f0 2024-04-26T04:40:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.455+0000 7fa7c0ff9700 1 -- 172.21.15.90:0/258081503 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa7b80225e0 con 0x7fa7c41032f0 2024-04-26T04:40:07.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.455+0000 7fa7c0ff9700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa7ac06c680 0x7fa7ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:07.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.455+0000 7fa7c0ff9700 1 -- 172.21.15.90:0/258081503 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa7b808d070 con 0x7fa7c41032f0 2024-04-26T04:40:07.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.455+0000 7fa7c37fe700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa7ac06c680 0x7fa7ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:07.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.456+0000 7fa7c37fe700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa7ac06c680 0x7fa7ac06eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009a10 tx=0x7fa7b4009190 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:07.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.460+0000 7fa7c0ff9700 1 -- 172.21.15.90:0/258081503 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa7b805b180 con 0x7fa7c41032f0 2024-04-26T04:40:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.753+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "mds versions", "format": "json"} v 0) v1 -- 0x7fa7c402cc30 con 0x7fa7c41032f0 2024-04-26T04:40:07.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.754+0000 7fa7c0ff9700 1 -- 172.21.15.90:0/258081503 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "mds versions", "format": "json"}]=0 v17) v1 ==== 78+0+83 (secure 0 0 0) 0x7fa7b805ad10 con 0x7fa7c41032f0 2024-04-26T04:40:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa7ac06c680 msgr2=0x7fa7ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa7ac06c680 0x7fa7ac06eb30 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009a10 tx=0x7fa7b4009190 comp rx=0 tx=0).stop 2024-04-26T04:40:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 msgr2=0x7fa7c4195ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c4195ca0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800b5c0 tx=0x7fa7b800bd70 comp rx=0 tx=0).stop 2024-04-26T04:40:07.757 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 shutdown_connections 2024-04-26T04:40:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa7ac06c680 0x7fa7ac06eb30 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa7c41009d0 0x7fa7c4195760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 --2- 172.21.15.90:0/258081503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa7c41032f0 0x7fa7c4195ca0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 >> 172.21.15.90:0/258081503 conn(0x7fa7c40fa3f0 msgr2=0x7fa7c40fc820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 shutdown_connections 2024-04-26T04:40:07.758 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:07.756+0000 7fa7c8fa2700 1 -- 172.21.15.90:0/258081503 wait complete. 2024-04-26T04:40:07.768 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:40:08.317 INFO:teuthology.run_tasks:Running task fs.pre_upgrade_save... 2024-04-26T04:40:08.342 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- 172.21.15.90:0/3597839491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000fe570 msgr2=0x7f4c000fe980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/3597839491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000fe570 0x7f4c000fe980 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4009880 tx=0x7f4bf402bce0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- 172.21.15.90:0/3597839491 shutdown_connections 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/3597839491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c000ff760 0x7f4c000ffbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/3597839491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000fe570 0x7f4c000fe980 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- 172.21.15.90:0/3597839491 >> 172.21.15.90:0/3597839491 conn(0x7f4c000f9ad0 msgr2=0x7f4c000fbf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- 172.21.15.90:0/3597839491 shutdown_connections 2024-04-26T04:40:08.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- 172.21.15.90:0/3597839491 wait complete. 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 Processor -- start 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- start start 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c000fe570 0x7f4c00194ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000ff760 0x7f4c001953e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c001959b0 con 0x7f4c000ff760 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4c06ec7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c00195b20 con 0x7f4c000fe570 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.458+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000ff760 0x7f4c001953e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000ff760 0x7f4c001953e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35940/0 (socket says 172.21.15.90:35940) 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfffff640 1 -- 172.21.15.90:0/1531980896 learned_addr learned my addr 172.21.15.90:0/1531980896 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4c04c63640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c000fe570 0x7f4c00194ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfffff640 1 -- 172.21.15.90:0/1531980896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c000fe570 msgr2=0x7f4c00194ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfffff640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c000fe570 0x7f4c00194ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfffff640 1 -- 172.21.15.90:0/1531980896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf4009560 con 0x7f4c000ff760 2024-04-26T04:40:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfffff640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000ff760 0x7f4c001953e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d8d0 tx=0x7f4bec00dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfdffb640 1 -- 172.21.15.90:0/1531980896 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bec00bea0 con 0x7f4c000ff760 2024-04-26T04:40:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfdffb640 1 -- 172.21.15.90:0/1531980896 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f4bec009c00 con 0x7f4c000ff760 2024-04-26T04:40:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c0019a5b0 con 0x7f4c000ff760 2024-04-26T04:40:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4bfdffb640 1 -- 172.21.15.90:0/1531980896 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bec00f710 con 0x7f4c000ff760 2024-04-26T04:40:08.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.459+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c0019ab10 con 0x7f4c000ff760 2024-04-26T04:40:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.460+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c00108380 con 0x7f4c000ff760 2024-04-26T04:40:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.460+0000 7f4bfdffb640 1 -- 172.21.15.90:0/1531980896 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f4bec010460 con 0x7f4c000ff760 2024-04-26T04:40:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.461+0000 7f4bfdffb640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4bf006c6b0 0x7f4bf006eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.461+0000 7f4bfdffb640 1 -- 172.21.15.90:0/1531980896 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f4bec08b530 con 0x7f4c000ff760 2024-04-26T04:40:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.461+0000 7f4c04c63640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4bf006c6b0 0x7f4bf006eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:08.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.461+0000 7f4c04c63640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4bf006c6b0 0x7f4bf006eb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf4005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:08.466 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.465+0000 7f4bfdffb640 1 -- 172.21.15.90:0/1531980896 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f4bec059770 con 0x7f4c000ff760 2024-04-26T04:40:08.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 102 B/s wr, 4 op/s 2024-04-26T04:40:08.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:08.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:08.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:40:08.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:40:08.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:08.615 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:40:08.616 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:08 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/258081503' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-26T04:40:08.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.636+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4c0019ad80 con 0x7f4c000ff760 2024-04-26T04:40:08.638 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.637+0000 7f4bfdffb640 1 -- 172.21.15.90:0/1531980896 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+15128 (secure 0 0 0) 0x7f4bec059300 con 0x7f4c000ff760 2024-04-26T04:40:08.639 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:40:08.639 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":46,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","created":"2024-04-26T04:32:57.928472+0000","modified":"2024-04-26T04:40:00.786372+0000","last_up_change":"2024-04-26T04:38:32.344046+0000","last_in_change":"2024-04-26T04:38:13.654124+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":14,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:37:19.554153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:39:42.011282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:39:42.594560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57491e88-0d1c-41ac-a1f5-3662e5c0a467","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6803","nonce":4217606471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6805","nonce":4217606471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6809","nonce":4217606471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":4217606471},{"type":"v1","addr":"172.21.15.90:6807","nonce":4217606471}]},"public_addr":"172.21.15.90:6803/4217606471","cluster_addr":"172.21.15.90:6805/4217606471","heartbeat_back_addr":"172.21.15.90:6809/4217606471","heartbeat_front_addr":"172.21.15.90:6807/4217606471","state":["exists","up"]},{"osd":1,"uuid":"fe237f1b-a589-452c-8e5e-98a9a31593eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6811","nonce":3196794289}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6813","nonce":3196794289}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6817","nonce":3196794289}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":3196794289},{"type":"v1","addr":"172.21.15.90:6815","nonce":3196794289}]},"public_addr":"172.21.15.90:6811/3196794289","cluster_addr":"172.21.15.90:6813/3196794289","heartbeat_back_addr":"172.21.15.90:6817/3196794289","heartbeat_front_addr":"172.21.15.90:6815/3196794289","state":["exists","up"]},{"osd":2,"uuid":"7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6819","nonce":127929257}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6821","nonce":127929257}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6825","nonce":127929257}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":127929257},{"type":"v1","addr":"172.21.15.90:6823","nonce":127929257}]},"public_addr":"172.21.15.90:6819/127929257","cluster_addr":"172.21.15.90:6821/127929257","heartbeat_back_addr":"172.21.15.90:6825/127929257","heartbeat_front_addr":"172.21.15.90:6823/127929257","state":["exists","up"]},{"osd":3,"uuid":"62adc04c-9e7e-4ee5-bb5f-42004b2a615d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6801","nonce":3796716845}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6803","nonce":3796716845}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6807","nonce":3796716845}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":3796716845},{"type":"v1","addr":"172.21.15.104:6805","nonce":3796716845}]},"public_addr":"172.21.15.104:6801/3796716845","cluster_addr":"172.21.15.104:6803/3796716845","heartbeat_back_addr":"172.21.15.104:6807/3796716845","heartbeat_front_addr":"172.21.15.104:6805/3796716845","state":["exists","up"]},{"osd":4,"uuid":"a4ef2729-84e1-4d2d-868b-3da5fefeb800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":34,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6809","nonce":4258788432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6811","nonce":4258788432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6815","nonce":4258788432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":4258788432},{"type":"v1","addr":"172.21.15.104:6813","nonce":4258788432}]},"public_addr":"172.21.15.104:6809/4258788432","cluster_addr":"172.21.15.104:6811/4258788432","heartbeat_back_addr":"172.21.15.104:6815/4258788432","heartbeat_front_addr":"172.21.15.104:6813/4258788432","state":["exists","up"]},{"osd":5,"uuid":"2fa80b6f-ab69-4e6e-9a5f-7e73a898c300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6817","nonce":2033326841}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6819","nonce":2033326841}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6823","nonce":2033326841}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":2033326841},{"type":"v1","addr":"172.21.15.104:6821","nonce":2033326841}]},"public_addr":"172.21.15.104:6817/2033326841","cluster_addr":"172.21.15.104:6819/2033326841","heartbeat_back_addr":"172.21.15.104:6823/2033326841","heartbeat_front_addr":"172.21.15.104:6821/2033326841","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:19.861796+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:48.482295+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:17.028265+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:40.489573+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:05.503518+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:30.925548+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6827/3871906279":"2024-04-27T04:40:00.786348+0000","172.21.15.90:0/2473529454":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/3418315403":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/424889384":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/2310050885":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2342166800":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:6800/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6801/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:0/748209074":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6801/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/1814853898":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/50635771":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/3214112922":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6826/3871906279":"2024-04-27T04:40:00.786348+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4bf006c6b0 msgr2=0x7f4bf006eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4bf006c6b0 0x7f4bf006eb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4bf4000c00 tx=0x7f4bf4005fd0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000ff760 msgr2=0x7f4c001953e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000ff760 0x7f4c001953e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00d8d0 tx=0x7f4bec00dd90 comp rx=0 tx=0).stop 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 shutdown_connections 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f4bf006c6b0 0x7f4bf006eb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4c000ff760 0x7f4c001953e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 --2- 172.21.15.90:0/1531980896 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4c000fe570 0x7f4c00194ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 >> 172.21.15.90:0/1531980896 conn(0x7f4c000f9ad0 msgr2=0x7f4c00102990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:08.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 shutdown_connections 2024-04-26T04:40:08.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.639+0000 7f4c06ec7640 1 -- 172.21.15.90:0/1531980896 wait complete. 2024-04-26T04:40:08.656 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:37:19.554153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:39:42.011282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:39:42.594560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T04:40:08.656 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T04:40:08.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 -- 172.21.15.90:0/1633425050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 msgr2=0x7fc16c0fe950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1633425050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c0fe950 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009880 tx=0x7fc15c02bce0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 -- 172.21.15.90:0/1633425050 shutdown_connections 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1633425050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc16c0ff730 0x7fc16c0ffba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1633425050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c0fe950 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 -- 172.21.15.90:0/1633425050 >> 172.21.15.90:0/1633425050 conn(0x7fc16c0f9ac0 msgr2=0x7fc16c0fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 -- 172.21.15.90:0/1633425050 shutdown_connections 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.759+0000 7fc1747ac640 1 -- 172.21.15.90:0/1633425050 wait complete. 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc1747ac640 1 Processor -- start 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc1747ac640 1 -- start start 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc1747ac640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc1747ac640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc16c0ff730 0x7fc16c199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:08.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc1747ac640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c19a290 con 0x7fc16c0fe540 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc1747ac640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc16c19a3d0 con 0x7fc16c0ff730 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc172548640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc172548640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c068940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35956/0 (socket says 172.21.15.90:35956) 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc172548640 1 -- 172.21.15.90:0/1176595068 learned_addr learned my addr 172.21.15.90:0/1176595068 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc171d47640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc16c0ff730 0x7fc16c199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc172548640 1 -- 172.21.15.90:0/1176595068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc16c0ff730 msgr2=0x7fc16c199cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc172548640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc16c0ff730 0x7fc16c199cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc172548640 1 -- 172.21.15.90:0/1176595068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc15c009560 con 0x7fc16c0fe540 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc171d47640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc16c0ff730 0x7fc16c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.760+0000 7fc172548640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c068940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009230 tx=0x7fc15c004e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.761+0000 7fc1637fe640 1 -- 172.21.15.90:0/1176595068 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc15c005000 con 0x7fc16c0fe540 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.761+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc16c19a620 con 0x7fc16c0fe540 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.761+0000 7fc1637fe640 1 -- 172.21.15.90:0/1176595068 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc15c005610 con 0x7fc16c0fe540 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.761+0000 7fc1637fe640 1 -- 172.21.15.90:0/1176595068 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc15c034440 con 0x7fc16c0fe540 2024-04-26T04:40:08.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.761+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc16c19ab20 con 0x7fc16c0fe540 2024-04-26T04:40:08.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.762+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc16c107270 con 0x7fc16c0fe540 2024-04-26T04:40:08.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.762+0000 7fc1637fe640 1 -- 172.21.15.90:0/1176595068 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fc15c003880 con 0x7fc16c0fe540 2024-04-26T04:40:08.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.762+0000 7fc1637fe640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc15806c5f0 0x7fc15806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:08.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.763+0000 7fc1637fe640 1 -- 172.21.15.90:0/1176595068 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fc15c0ade50 con 0x7fc16c0fe540 2024-04-26T04:40:08.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.763+0000 7fc171d47640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc15806c5f0 0x7fc15806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:08.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.763+0000 7fc171d47640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc15806c5f0 0x7fc15806eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0686b0 tx=0x7fc168005e50 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:08.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.767+0000 7fc1637fe640 1 -- 172.21.15.90:0/1176595068 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fc15c0784d0 con 0x7fc16c0fe540 2024-04-26T04:40:08.927 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: pgmap v186: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 2.8 KiB/s rd, 102 B/s wr, 4 op/s 2024-04-26T04:40:08.927 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:08.927 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:08.927 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:40:08.928 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:40:08.928 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:08.928 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:40:08.928 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:08 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/258081503' entity='client.admin' cmd=[{"prefix": "mds versions", "format": "json"}]: dispatch 2024-04-26T04:40:08.940 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.939+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc16c19ae00 con 0x7fc16c0fe540 2024-04-26T04:40:08.941 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.940+0000 7fc1637fe640 1 -- 172.21.15.90:0/1176595068 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v46) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc15c041020 con 0x7fc16c0fe540 2024-04-26T04:40:08.941 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-26T04:40:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc15806c5f0 msgr2=0x7fc15806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc15806c5f0 0x7fc15806eaa0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fc16c0686b0 tx=0x7fc168005e50 comp rx=0 tx=0).stop 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 msgr2=0x7fc16c068940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c068940 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fc15c009230 tx=0x7fc15c004e30 comp rx=0 tx=0).stop 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 shutdown_connections 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fc15806c5f0 0x7fc15806eaa0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc16c0ff730 0x7fc16c199cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 --2- 172.21.15.90:0/1176595068 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc16c0fe540 0x7fc16c068940 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 >> 172.21.15.90:0/1176595068 conn(0x7fc16c0f9ac0 msgr2=0x7fc16c102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 shutdown_connections 2024-04-26T04:40:08.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:08.942+0000 7fc1747ac640 1 -- 172.21.15.90:0/1176595068 wait complete. 2024-04-26T04:40:08.955 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T04:40:09.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 -- 172.21.15.90:0/3355427270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 msgr2=0x7f2710101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 --2- 172.21.15.90:0/3355427270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f2710101020 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2700009880 tx=0x7f270002bce0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 -- 172.21.15.90:0/3355427270 shutdown_connections 2024-04-26T04:40:09.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 --2- 172.21.15.90:0/3355427270 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27100fd680 0x7f27100fdaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 --2- 172.21.15.90:0/3355427270 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f2710101020 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 -- 172.21.15.90:0/3355427270 >> 172.21.15.90:0/3355427270 conn(0x7f27100f9210 msgr2=0x7f27100fb660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:09.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 -- 172.21.15.90:0/3355427270 shutdown_connections 2024-04-26T04:40:09.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.062+0000 7f2714955640 1 -- 172.21.15.90:0/3355427270 wait complete. 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f2714955640 1 Processor -- start 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f2714955640 1 -- start start 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f2714955640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27100fd680 0x7f2710194fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f2714955640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f27101954f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f2714955640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710195ac0 con 0x7f2710100c10 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f2714955640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2710195c30 con 0x7f27100fd680 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f27101954f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270dd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f27101954f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35974/0 (socket says 172.21.15.90:35974) 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270dd9b640 1 -- 172.21.15.90:0/3066455172 learned_addr learned my addr 172.21.15.90:0/3066455172 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270e59c640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27100fd680 0x7f2710194fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270dd9b640 1 -- 172.21.15.90:0/3066455172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27100fd680 msgr2=0x7f2710194fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270dd9b640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27100fd680 0x7f2710194fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270dd9b640 1 -- 172.21.15.90:0/3066455172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2700009560 con 0x7f2710100c10 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.063+0000 7f270dd9b640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f27101954f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f270400e970 tx=0x7f270400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.064+0000 7f26ff7fe640 1 -- 172.21.15.90:0/3066455172 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f270400cc90 con 0x7f2710100c10 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.064+0000 7f26ff7fe640 1 -- 172.21.15.90:0/3066455172 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f270400cdf0 con 0x7f2710100c10 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.064+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f271019a6c0 con 0x7f2710100c10 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.064+0000 7f26ff7fe640 1 -- 172.21.15.90:0/3066455172 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2704010430 con 0x7f2710100c10 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.064+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f271019ac20 con 0x7f2710100c10 2024-04-26T04:40:09.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.065+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2710108350 con 0x7f2710100c10 2024-04-26T04:40:09.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.066+0000 7f26ff7fe640 1 -- 172.21.15.90:0/3066455172 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f27040046c0 con 0x7f2710100c10 2024-04-26T04:40:09.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.066+0000 7f26ff7fe640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26f806c5f0 0x7f26f806eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.066+0000 7f26ff7fe640 1 -- 172.21.15.90:0/3066455172 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2704014070 con 0x7f2710100c10 2024-04-26T04:40:09.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.066+0000 7f270e59c640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26f806c5f0 0x7f26f806eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.067+0000 7f270e59c640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26f806c5f0 0x7f26f806eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f2700005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:09.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.071+0000 7f26ff7fe640 1 -- 172.21.15.90:0/3066455172 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2704056570 con 0x7f2710100c10 2024-04-26T04:40:09.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.245+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f271019aea0 con 0x7f2710100c10 2024-04-26T04:40:09.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.245+0000 7f26ff7fe640 1 -- 172.21.15.90:0/3066455172 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7f2704059b90 con 0x7f2710100c10 2024-04-26T04:40:09.247 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26f806c5f0 msgr2=0x7f26f806eaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26f806c5f0 0x7f26f806eaa0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2700000c00 tx=0x7f2700005fd0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 msgr2=0x7f27101954f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f27101954f0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f270400e970 tx=0x7f270400ed70 comp rx=0 tx=0).stop 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 shutdown_connections 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26f806c5f0 0x7f26f806eaa0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2710100c10 0x7f27101954f0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 --2- 172.21.15.90:0/3066455172 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f27100fd680 0x7f2710194fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 >> 172.21.15.90:0/3066455172 conn(0x7f27100f9210 msgr2=0x7f2710103bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:09.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 shutdown_connections 2024-04-26T04:40:09.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.247+0000 7f2714955640 1 -- 172.21.15.90:0/3066455172 wait complete. 2024-04-26T04:40:09.259 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 -- 172.21.15.90:0/3510452887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 msgr2=0x7fb8440ffba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3510452887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb8440ffba0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009880 tx=0x7fb82c02bce0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 -- 172.21.15.90:0/3510452887 shutdown_connections 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3510452887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb8440ffba0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3510452887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8440fe540 0x7fb8440fe950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 -- 172.21.15.90:0/3510452887 >> 172.21.15.90:0/3510452887 conn(0x7fb8440f9a80 msgr2=0x7fb8440fbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 -- 172.21.15.90:0/3510452887 shutdown_connections 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.393+0000 7fb8490e0640 1 -- 172.21.15.90:0/3510452887 wait complete. 2024-04-26T04:40:09.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb8490e0640 1 Processor -- start 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb8490e0640 1 -- start start 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb8490e0640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8440fe540 0x7fb844067e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb8490e0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb844199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb8490e0640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84419a2b0 con 0x7fb8440ff730 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb8490e0640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84419a3f0 con 0x7fb8440fe540 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb83bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb844199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb83bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb844199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35992/0 (socket says 172.21.15.90:35992) 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb83bfff640 1 -- 172.21.15.90:0/3310335101 learned_addr learned my addr 172.21.15.90:0/3310335101 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb842d9d640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8440fe540 0x7fb844067e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb83bfff640 1 -- 172.21.15.90:0/3310335101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8440fe540 msgr2=0x7fb844067e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb83bfff640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8440fe540 0x7fb844067e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb83bfff640 1 -- 172.21.15.90:0/3310335101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c009560 con 0x7fb8440ff730 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb83bfff640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb844199ce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009230 tx=0x7fb82c02cec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb840d99640 1 -- 172.21.15.90:0/3310335101 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb82c004420 con 0x7fb8440ff730 2024-04-26T04:40:09.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb840d99640 1 -- 172.21.15.90:0/3310335101 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb82c0054b0 con 0x7fb8440ff730 2024-04-26T04:40:09.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb840d99640 1 -- 172.21.15.90:0/3310335101 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb82c034440 con 0x7fb8440ff730 2024-04-26T04:40:09.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.394+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84419a640 con 0x7fb8440ff730 2024-04-26T04:40:09.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.395+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84419ab40 con 0x7fb8440ff730 2024-04-26T04:40:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.395+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb844107280 con 0x7fb8440ff730 2024-04-26T04:40:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.396+0000 7fb840d99640 1 -- 172.21.15.90:0/3310335101 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb82c0037d0 con 0x7fb8440ff730 2024-04-26T04:40:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.396+0000 7fb840d99640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb83006c640 0x7fb83006eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.396+0000 7fb840d99640 1 -- 172.21.15.90:0/3310335101 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb82c0add00 con 0x7fb8440ff730 2024-04-26T04:40:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.396+0000 7fb842d9d640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb83006c640 0x7fb83006eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.397+0000 7fb842d9d640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb83006c640 0x7fb83006eaf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8440ff590 tx=0x7fb834005f90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:09.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.401+0000 7fb840d99640 1 -- 172.21.15.90:0/3310335101 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb82c078410 con 0x7fb8440ff730 2024-04-26T04:40:09.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.571+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb84419ae70 con 0x7fb8440ff730 2024-04-26T04:40:09.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.572+0000 7fb840d99640 1 -- 172.21.15.90:0/3310335101 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v46) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb82c04b020 con 0x7fb8440ff730 2024-04-26T04:40:09.573 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.573+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb83006c640 msgr2=0x7fb83006eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.573+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb83006c640 0x7fb83006eaf0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fb8440ff590 tx=0x7fb834005f90 comp rx=0 tx=0).stop 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.573+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 msgr2=0x7fb844199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.573+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb844199ce0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fb82c009230 tx=0x7fb82c02cec0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.574+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 shutdown_connections 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.574+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb83006c640 0x7fb83006eaf0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.574+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8440ff730 0x7fb844199ce0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.574+0000 7fb8490e0640 1 --2- 172.21.15.90:0/3310335101 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8440fe540 0x7fb844067e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.574+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 >> 172.21.15.90:0/3310335101 conn(0x7fb8440f9a80 msgr2=0x7fb844102960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:09.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.574+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 shutdown_connections 2024-04-26T04:40:09.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.574+0000 7fb8490e0640 1 -- 172.21.15.90:0/3310335101 wait complete. 2024-04-26T04:40:09.586 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T04:40:09.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- 172.21.15.90:0/3594396870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a00ff670 msgr2=0x7fa0a00ffa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 --2- 172.21.15.90:0/3594396870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a00ff670 0x7fa0a00ffa80 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009880 tx=0x7fa09c02bce0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- 172.21.15.90:0/3594396870 shutdown_connections 2024-04-26T04:40:09.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 --2- 172.21.15.90:0/3594396870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a0100860 0x7fa0a0100cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 --2- 172.21.15.90:0/3594396870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a00ff670 0x7fa0a00ffa80 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- 172.21.15.90:0/3594396870 >> 172.21.15.90:0/3594396870 conn(0x7fa0a00fabd0 msgr2=0x7fa0a00fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- 172.21.15.90:0/3594396870 shutdown_connections 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- 172.21.15.90:0/3594396870 wait complete. 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 Processor -- start 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- start start 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a00ff670 0x7fa0a01970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a0100860 0x7fa0a0197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0197bf0 con 0x7fa0a0100860 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.684+0000 7fa0a8371640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0a0197d60 con 0x7fa0a00ff670 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a590c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a0100860 0x7fa0a0197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a590c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a0100860 0x7fa0a0197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36004/0 (socket says 172.21.15.90:36004) 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a590c640 1 -- 172.21.15.90:0/4264698727 learned_addr learned my addr 172.21.15.90:0/4264698727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:09.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a610d640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a00ff670 0x7fa0a01970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a590c640 1 -- 172.21.15.90:0/4264698727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a00ff670 msgr2=0x7fa0a01970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a590c640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a00ff670 0x7fa0a01970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a590c640 1 -- 172.21.15.90:0/4264698727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa09c009560 con 0x7fa0a0100860 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a590c640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a0100860 0x7fa0a0197620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e970 tx=0x7fa09000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0977fe640 1 -- 172.21.15.90:0/4264698727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa09000cc90 con 0x7fa0a0100860 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0a019c7f0 con 0x7fa0a0100860 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0977fe640 1 -- 172.21.15.90:0/4264698727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa09000cdf0 con 0x7fa0a0100860 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0977fe640 1 -- 172.21.15.90:0/4264698727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa090010430 con 0x7fa0a0100860 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.685+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0a019cd50 con 0x7fa0a0100860 2024-04-26T04:40:09.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.686+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0a01083a0 con 0x7fa0a0100860 2024-04-26T04:40:09.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.687+0000 7fa0977fe640 1 -- 172.21.15.90:0/4264698727 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa0900046c0 con 0x7fa0a0100860 2024-04-26T04:40:09.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.687+0000 7fa0977fe640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa08c06c6b0 0x7fa08c06eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:09.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.687+0000 7fa0977fe640 1 -- 172.21.15.90:0/4264698727 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa090014070 con 0x7fa0a0100860 2024-04-26T04:40:09.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.687+0000 7fa0a610d640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa08c06c6b0 0x7fa08c06eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:09.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.688+0000 7fa0a610d640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa08c06c6b0 0x7fa08c06eb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:09.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.692+0000 7fa0977fe640 1 -- 172.21.15.90:0/4264698727 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa090056580 con 0x7fa0a0100860 2024-04-26T04:40:09.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.907+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa0a019cfd0 con 0x7fa0a0100860 2024-04-26T04:40:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.907+0000 7fa0977fe640 1 -- 172.21.15.90:0/4264698727 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4754 (secure 0 0 0) 0x7fa090059ba0 con 0x7fa0a0100860 2024-04-26T04:40:09.909 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:40:09.909 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/650878075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":650878075},{"type":"v1","addr":"172.21.15.90:6827","nonce":650878075}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:40:04.813196+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi090.dbkjuk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6829/370183803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":370183803},{"type":"v1","addr":"172.21.15.90:6829","nonce":370183803}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi104.wpntdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6825/1058172006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1058172006},{"type":"v1","addr":"172.21.15.104:6825","nonce":1058172006}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2024-04-26T04:40:09.909 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 17 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa08c06c6b0 msgr2=0x7fa08c06eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa08c06c6b0 0x7fa08c06eb60 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fa09c000c00 tx=0x7fa09c005fd0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a0100860 msgr2=0x7fa0a0197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a0100860 0x7fa0a0197620 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fa09000e970 tx=0x7fa09000ed70 comp rx=0 tx=0).stop 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 shutdown_connections 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa08c06c6b0 0x7fa08c06eb60 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa0a0100860 0x7fa0a0197620 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 --2- 172.21.15.90:0/4264698727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa0a00ff670 0x7fa0a01970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 >> 172.21.15.90:0/4264698727 conn(0x7fa0a00fabd0 msgr2=0x7fa0a00fcfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 shutdown_connections 2024-04-26T04:40:09.911 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:09.909+0000 7fa0a8371640 1 -- 172.21.15.90:0/4264698727 wait complete. 2024-04-26T04:40:09.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:09 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1531980896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:40:09.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:09 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1176595068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:40:09.920 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:09 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3066455172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:40:09.921 DEBUG:tasks.fs:fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-04-26T04:40:09.921 INFO:teuthology.run_tasks:Running task kclient... 2024-04-26T04:40:09.987 INFO:tasks.kclient:Mounting kernel clients... 2024-04-26T04:40:09.988 INFO:tasks.kclient:config is {'client.0': {}, 'client.1': {}} 2024-04-26T04:40:09.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:09.988 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-26T04:40:09.996 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:09 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1531980896' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:40:09.996 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:09 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1176595068' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:40:09.996 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:09 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3066455172' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:40:10.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:10.003 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-04-26T04:40:10.069 INFO:teuthology.orchestra.run.smithi090.stderr:Cannot find device "ceph-brx" 2024-04-26T04:40:10.070 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:40:10.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:10.071 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-04-26T04:40:10.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:10.087 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-04-26T04:40:10.156 INFO:teuthology.orchestra.run.smithi104.stderr:Cannot find device "ceph-brx" 2024-04-26T04:40:10.157 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:40:10.158 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-04-26T04:40:10.158 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T04:40:10.158 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:10.158 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T04:40:10.447 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:10.785 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:10 smithi090 ceph-mon[26648]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T04:40:10.786 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:10 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3310335101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:40:10.786 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:10.786 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:10 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:10.786 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:10 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/4264698727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:40:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:10 smithi104 ceph-mon[32369]: pgmap v187: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 8.2 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T04:40:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:10 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3310335101' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:40:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:10 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:11.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:10 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:11.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:10 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/4264698727' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:40:12.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:12 smithi090 ceph-mon[26648]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T04:40:12.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 -- 172.21.15.90:0/205112753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 msgr2=0x7fd75c1066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:12.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 --2- 172.21.15.90:0/205112753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1066f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fd754009910 tx=0x7fd754009c20 comp rx=0 tx=0).stop 2024-04-26T04:40:12.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 -- 172.21.15.90:0/205112753 shutdown_connections 2024-04-26T04:40:12.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 --2- 172.21.15.90:0/205112753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd75c068540 0x7fd75c0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:12.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 --2- 172.21.15.90:0/205112753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1066f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:12.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 -- 172.21.15.90:0/205112753 >> 172.21.15.90:0/205112753 conn(0x7fd75c075130 msgr2=0x7fd75c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:12.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 -- 172.21.15.90:0/205112753 shutdown_connections 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.991+0000 7fd762210700 1 -- 172.21.15.90:0/205112753 wait complete. 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd762210700 1 Processor -- start 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd762210700 1 -- start start 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd762210700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd75c068540 0x7fd75c195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd762210700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd762210700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c196a00 con 0x7fd75c106320 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd762210700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75c196b40 con 0x7fd75c068540 2024-04-26T04:40:12.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd75affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:13.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd75affd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36018/0 (socket says 172.21.15.90:36018) 2024-04-26T04:40:13.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.992+0000 7fd75affd700 1 -- 172.21.15.90:0/83426870 learned_addr learned my addr 172.21.15.90:0/83426870 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:13.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd75b7fe700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd75c068540 0x7fd75c195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:13.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd75affd700 1 -- 172.21.15.90:0/83426870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd75c068540 msgr2=0x7fd75c195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:13.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd75affd700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd75c068540 0x7fd75c195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:13.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd75affd700 1 -- 172.21.15.90:0/83426870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7540095f0 con 0x7fd75c106320 2024-04-26T04:40:13.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd75affd700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1963b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd75000cfd0 tx=0x7fd75000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:13.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd758ff9700 1 -- 172.21.15.90:0/83426870 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd75000ad70 con 0x7fd75c106320 2024-04-26T04:40:13.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd758ff9700 1 -- 172.21.15.90:0/83426870 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fd7500128d0 con 0x7fd75c106320 2024-04-26T04:40:13.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75c19a990 con 0x7fd75c106320 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd758ff9700 1 -- 172.21.15.90:0/83426870 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd750011880 con 0x7fd75c106320 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.993+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75c19aee0 con 0x7fd75c106320 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.994+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd75c108770 con 0x7fd75c106320 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.994+0000 7fd758ff9700 1 -- 172.21.15.90:0/83426870 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fd750012420 con 0x7fd75c106320 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.995+0000 7fd758ff9700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fd74406c680 0x7fd74406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.995+0000 7fd758ff9700 1 -- 172.21.15.90:0/83426870 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fd75008ba20 con 0x7fd75c106320 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.995+0000 7fd75b7fe700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fd74406c680 0x7fd74406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:13.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.995+0000 7fd75b7fe700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fd74406c680 0x7fd74406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd75400b5c0 tx=0x7fd754005960 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:13.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:12.999+0000 7fd758ff9700 1 -- 172.21.15.90:0/83426870 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fd7500560a0 con 0x7fd75c106320 2024-04-26T04:40:13.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:12 smithi104 ceph-mon[32369]: pgmap v188: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 7.8 KiB/s rd, 0 B/s wr, 7 op/s 2024-04-26T04:40:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.253+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd75c04f2a0 con 0x7fd75c106320 2024-04-26T04:40:13.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.253+0000 7fd758ff9700 1 -- 172.21.15.90:0/83426870 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7fd750016070 con 0x7fd75c106320 2024-04-26T04:40:13.254 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-26T04:40:13.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fd74406c680 msgr2=0x7fd74406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:13.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fd74406c680 0x7fd74406eb30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd75400b5c0 tx=0x7fd754005960 comp rx=0 tx=0).stop 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 msgr2=0x7fd75c1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1963b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fd75000cfd0 tx=0x7fd75000a340 comp rx=0 tx=0).stop 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 shutdown_connections 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fd74406c680 0x7fd74406eb30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd75c068540 0x7fd75c195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 --2- 172.21.15.90:0/83426870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd75c106320 0x7fd75c1963b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 >> 172.21.15.90:0/83426870 conn(0x7fd75c075130 msgr2=0x7fd75c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 shutdown_connections 2024-04-26T04:40:13.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13.255+0000 7fd762210700 1 -- 172.21.15.90:0/83426870 wait complete. 2024-04-26T04:40:13.877 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T04:40:13.877 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T04:40:13.877 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi090.front.sepia.ceph.com 2024-04-26T04:40:13.877 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-26T04:40:13.877 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:13.877 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T04:40:13.877 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T04:40:13.878 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T04:40:13.878 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-26T04:40:13.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:13.878 DEBUG:teuthology.orchestra.run.smithi090:> ip addr 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 ::1/128 scope host 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a8 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f0 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:40:13.894 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:d8:a1:a9 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: altname enp7s0f1 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:66 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: link/ether 0c:c4:7a:8f:d7:67 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: inet 172.21.15.90/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft 42320sec preferred_lft 42320sec 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: inet6 fe80::30a8:3f5b:7da9:df8e/64 scope link noprefixroute 2024-04-26T04:40:13.895 INFO:teuthology.orchestra.run.smithi090.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:40:13.895 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T04:40:13.896 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:13.896 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T04:40:13.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add name ceph-brx type bridge 2024-04-26T04:40:13.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr flush dev ceph-brx 2024-04-26T04:40:13.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set ceph-brx up 2024-04-26T04:40:13.896 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T04:40:13.896 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T04:40:13.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:13 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:14.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:14.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:14.069 DEBUG:teuthology.orchestra.run.smithi090:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T04:40:14.092 INFO:teuthology.orchestra.run.smithi090.stdout:1 2024-04-26T04:40:14.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:14.094 DEBUG:teuthology.orchestra.run.smithi090:> route 2024-04-26T04:40:14.177 INFO:teuthology.orchestra.run.smithi090.stdout:Kernel IP routing table 2024-04-26T04:40:14.177 INFO:teuthology.orchestra.run.smithi090.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T04:40:14.178 INFO:teuthology.orchestra.run.smithi090.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T04:40:14.178 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T04:40:14.178 INFO:teuthology.orchestra.run.smithi090.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T04:40:14.178 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:14.178 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T04:40:14.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T04:40:14.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T04:40:14.179 DEBUG:teuthology.orchestra.run.smithi090:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T04:40:14.179 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T04:40:14.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:14.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:14 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/83426870' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:40:14.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:14.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:14.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:40:14.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:40:14.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:14 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:14.445 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:14 smithi090 ceph-mon[26648]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T04:40:14.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:14.452 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:14.452 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-26T04:40:14.469 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:14.469 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list-id 2024-04-26T04:40:14.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:14 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/83426870' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:40:14.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:40:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:40:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:14 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:14.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:14 smithi104 ceph-mon[32369]: pgmap v189: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.8 KiB/s rd, 0 B/s wr, 6 op/s 2024-04-26T04:40:14.533 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:14.533 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T04:40:14.533 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T04:40:14.533 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-26T04:40:14.533 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T04:40:14.629 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:14.650 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:14.656 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-26T04:40:14.657 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:14.657 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T04:40:14.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-26T04:40:14.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T04:40:14.657 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-26T04:40:14.658 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-26T04:40:14.658 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-26T04:40:14.658 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T04:40:14.745 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:14.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:14.818 DEBUG:teuthology.orchestra.run.smithi090:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:14.818 DEBUG:teuthology.orchestra.run.smithi090:> set -e 2024-04-26T04:40:14.818 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set brx.0 up 2024-04-26T04:40:14.818 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T04:40:14.818 DEBUG:teuthology.orchestra.run.smithi090:> ') 2024-04-26T04:40:14.867 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:14.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:14 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:14.895 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:40:14.895 DEBUG:teuthology.orchestra.run.smithi090:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:14.918 INFO:teuthology.orchestra.run.smithi090.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-26T04:40:14.919 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:40:14.919 DEBUG:teuthology.orchestra.run.smithi090:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:14.978 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T04:40:14.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:14.978 DEBUG:teuthology.orchestra.run.smithi090:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T04:40:15.073 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T04:40:15.074 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T04:40:15.074 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-04-26T04:40:15.177 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T04:40:15.178 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T04:40:15.178 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-04-26T04:40:15.178 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.90:3300,172.21.15.104:3300" 2024-04-26T04:40:15.178 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T04:40:15.183 INFO:teuthology.orchestra.run.smithi090.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:invalid new device string format 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.90:3300,172.21.15.104:3300" 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: resolved to: "172.21.15.90:6789,172.21.15.104:6789" 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.90:6789,172.21.15.104:6789:/ 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=f018e902-0385-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-26T04:40:15.184 INFO:teuthology.orchestra.run.smithi090.stdout: You just mounted a file system that supports labels which does not 2024-04-26T04:40:15.185 INFO:teuthology.orchestra.run.smithi090.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T04:40:15.185 INFO:teuthology.orchestra.run.smithi090.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T04:40:15.185 INFO:teuthology.orchestra.run.smithi090.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T04:40:15.186 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T04:40:15.186 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:15.186 DEBUG:teuthology.orchestra.run.smithi090:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:15.218 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T04:40:15.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T04:40:15.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:15.318 INFO:teuthology.orchestra.run.smithi090.stdout:client24315 2024-04-26T04:40:15.319 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:40:15.319 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T04:40:15.337 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T04:40:15.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T04:40:15.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:15.438 INFO:teuthology.orchestra.run.smithi090.stdout:client24315 2024-04-26T04:40:15.438 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T04:40:15.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T04:40:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:15.499 INFO:teuthology.orchestra.run.smithi090.stdout:client24315 2024-04-26T04:40:15.500 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f018e902-0385-11ef-bc93-c7b262605968.client24315/status') 2024-04-26T04:40:15.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:15.561 INFO:teuthology.orchestra.run.smithi090.stdout:instance: client.24315 (3)172.21.15.90:0/3181179058 2024-04-26T04:40:15.561 INFO:teuthology.orchestra.run.smithi090.stdout:blocklisted: false 2024-04-26T04:40:15.561 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records in 2024-04-26T04:40:15.561 INFO:teuthology.orchestra.run.smithi090.stderr:0+1 records out 2024-04-26T04:40:15.561 INFO:teuthology.orchestra.run.smithi090.stderr:71 bytes copied, 3.8178e-05 s, 1.9 MB/s 2024-04-26T04:40:15.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:15 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:15.569 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-04-26T04:40:15.569 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-26T04:40:15.569 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:15.570 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell --fsid f018e902-0385-11ef-bc93-c7b262605968 -- ceph fs ls 2024-04-26T04:40:15.775 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:16.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:16 smithi090 ceph-mon[26648]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T04:40:16.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:16 smithi104 ceph-mon[32369]: pgmap v190: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T04:40:17.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 -- 172.21.15.90:0/2441177167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ff350 msgr2=0x7f2a400ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:17.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2441177167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ff350 0x7f2a400ff720 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b0a0 tx=0x7f2a3800b3b0 comp rx=0 tx=0).stop 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 -- 172.21.15.90:0/2441177167 shutdown_connections 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2441177167 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a400ffc60 0x7f2a4010a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2441177167 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ff350 0x7f2a400ff720 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 -- 172.21.15.90:0/2441177167 >> 172.21.15.90:0/2441177167 conn(0x7f2a400748b0 msgr2=0x7f2a40074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 -- 172.21.15.90:0/2441177167 shutdown_connections 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 -- 172.21.15.90:0/2441177167 wait complete. 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.291+0000 7f2a4564e700 1 Processor -- start 2024-04-26T04:40:17.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a4564e700 1 -- start start 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a4564e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a400ff350 0x7f2a40100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a4564e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ffc60 0x7f2a40100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a4564e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a40100fb0 con 0x7f2a400ffc60 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a4564e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a401010f0 con 0x7f2a400ff350 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3e7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ffc60 0x7f2a40100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3e7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ffc60 0x7f2a40100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:36036/0 (socket says 172.21.15.90:36036) 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3e7fc700 1 -- 172.21.15.90:0/2642953653 learned_addr learned my addr 172.21.15.90:0/2642953653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3effd700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a400ff350 0x7f2a40100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3e7fc700 1 -- 172.21.15.90:0/2642953653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a400ff350 msgr2=0x7f2a40100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:17.506 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3e7fc700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a400ff350 0x7f2a40100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3e7fc700 1 -- 172.21.15.90:0/2642953653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a38009d00 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.292+0000 7f2a3effd700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a400ff350 0x7f2a40100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.293+0000 7f2a3e7fc700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ffc60 0x7f2a40100a70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800b4b0 tx=0x7f2a2800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.293+0000 7f2a37fff700 1 -- 172.21.15.90:0/2642953653 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a28011800 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.293+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a401a2190 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.293+0000 7f2a37fff700 1 -- 172.21.15.90:0/2642953653 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f2a28011e10 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.293+0000 7f2a37fff700 1 -- 172.21.15.90:0/2642953653 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a2800f2e0 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.293+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a401a25a0 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.294+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a40107dd0 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.294+0000 7f2a37fff700 1 -- 172.21.15.90:0/2642953653 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f2a28010380 con 0x7f2a400ffc60 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.295+0000 7f2a37fff700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f2a2c06c680 0x7f2a2c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:17.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.295+0000 7f2a37fff700 1 -- 172.21.15.90:0/2642953653 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f2a2808b550 con 0x7f2a400ffc60 2024-04-26T04:40:17.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.295+0000 7f2a3effd700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f2a2c06c680 0x7f2a2c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:17.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.295+0000 7f2a3effd700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f2a2c06c680 0x7f2a2c06eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b830 tx=0x7f2a38006000 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:17.508 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.299+0000 7f2a37fff700 1 -- 172.21.15.90:0/2642953653 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f2a28055bd0 con 0x7f2a400ffc60 2024-04-26T04:40:17.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.537+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2a4004f2a0 con 0x7f2a400ffc60 2024-04-26T04:40:17.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.537+0000 7f2a37fff700 1 -- 172.21.15.90:0/2642953653 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+83 (secure 0 0 0) 0x7f2a280591f0 con 0x7f2a400ffc60 2024-04-26T04:40:17.539 INFO:teuthology.orchestra.run.smithi090.stdout:name: cephfs, metadata pool: cephfs.cephfs.meta, data pools: [cephfs.cephfs.data ] 2024-04-26T04:40:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f2a2c06c680 msgr2=0x7f2a2c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f2a2c06c680 0x7f2a2c06eb30 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2a3800b830 tx=0x7f2a38006000 comp rx=0 tx=0).stop 2024-04-26T04:40:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ffc60 msgr2=0x7f2a40100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ffc60 0x7f2a40100a70 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f2a2800b4b0 tx=0x7f2a2800b870 comp rx=0 tx=0).stop 2024-04-26T04:40:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 shutdown_connections 2024-04-26T04:40:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f2a2c06c680 0x7f2a2c06eb30 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:17.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a400ff350 0x7f2a40100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 --2- 172.21.15.90:0/2642953653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a400ffc60 0x7f2a40100a70 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 >> 172.21.15.90:0/2642953653 conn(0x7f2a400748b0 msgr2=0x7f2a40104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 shutdown_connections 2024-04-26T04:40:17.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:17.540+0000 7f2a4564e700 1 -- 172.21.15.90:0/2642953653 wait complete. 2024-04-26T04:40:18.173 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-26T04:40:18.173 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-26T04:40:18.173 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi104.front.sepia.ceph.com 2024-04-26T04:40:18.173 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-04-26T04:40:18.174 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:18.174 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-26T04:40:18.174 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-26T04:40:18.174 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-26T04:40:18.174 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-04-26T04:40:18.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:18.174 DEBUG:teuthology.orchestra.run.smithi104:> ip addr 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: inet 127.0.0.1/8 scope host lo 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 ::1/128 scope host 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1c brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f0 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:d8:a1:1d brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:18.193 INFO:teuthology.orchestra.run.smithi104.stdout: altname enp7s0f1 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:88 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout: link/ether 0c:c4:7a:8f:ce:89 brd ff:ff:ff:ff:ff:ff 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout: inet 172.21.15.104/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft 42270sec preferred_lft 42270sec 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout: inet6 fe80::a0d4:de8:779f:ee1f/64 scope link noprefixroute 2024-04-26T04:40:18.194 INFO:teuthology.orchestra.run.smithi104.stdout: valid_lft forever preferred_lft forever 2024-04-26T04:40:18.194 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-26T04:40:18.194 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:18.195 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-04-26T04:40:18.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add name ceph-brx type bridge 2024-04-26T04:40:18.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr flush dev ceph-brx 2024-04-26T04:40:18.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set ceph-brx up 2024-04-26T04:40:18.195 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-26T04:40:18.195 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-04-26T04:40:18.288 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:18.382 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:18.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:18.389 DEBUG:teuthology.orchestra.run.smithi104:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-26T04:40:18.412 INFO:teuthology.orchestra.run.smithi104.stdout:1 2024-04-26T04:40:18.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:18.414 DEBUG:teuthology.orchestra.run.smithi104:> route 2024-04-26T04:40:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:18 smithi090 ceph-mon[26648]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T04:40:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:18 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2642953653' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:40:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:18.498 INFO:teuthology.orchestra.run.smithi104.stdout:Kernel IP routing table 2024-04-26T04:40:18.498 INFO:teuthology.orchestra.run.smithi104.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-26T04:40:18.499 INFO:teuthology.orchestra.run.smithi104.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-26T04:40:18.499 INFO:teuthology.orchestra.run.smithi104.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-26T04:40:18.499 INFO:teuthology.orchestra.run.smithi104.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-26T04:40:18.499 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:18.499 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-04-26T04:40:18.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-26T04:40:18.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-26T04:40:18.499 DEBUG:teuthology.orchestra.run.smithi104:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-26T04:40:18.499 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-04-26T04:40:18.547 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:18.555 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:18 smithi104 ceph-mon[32369]: pgmap v191: 65 pgs: 65 active+clean; 580 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 6.2 KiB/s rd, 0 B/s wr, 5 op/s 2024-04-26T04:40:18.555 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:18 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2642953653' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-26T04:40:18.555 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:40:18.555 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:18.721 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:18.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:18.728 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-04-26T04:40:18.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:18.743 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list-id 2024-04-26T04:40:18.798 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:18.798 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-04-26T04:40:18.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T04:40:18.798 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-04-26T04:40:18.798 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-04-26T04:40:18.885 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:18.906 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:18 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:18.912 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-04-26T04:40:18.913 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-04-26T04:40:18.960 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:18 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:19.025 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:19.032 DEBUG:teuthology.orchestra.run.smithi104:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-26T04:40:19.032 DEBUG:teuthology.orchestra.run.smithi104:> set -e 2024-04-26T04:40:19.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set brx.0 up 2024-04-26T04:40:19.032 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link set dev brx.0 master ceph-brx 2024-04-26T04:40:19.033 DEBUG:teuthology.orchestra.run.smithi104:> ') 2024-04-26T04:40:19.080 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:19.100 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:19.106 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:40:19.106 DEBUG:teuthology.orchestra.run.smithi104:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:19.121 INFO:teuthology.orchestra.run.smithi104.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-04-26T04:40:19.121 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-26T04:40:19.121 DEBUG:teuthology.orchestra.run.smithi104:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:19.175 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-04-26T04:40:19.175 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:19.176 DEBUG:teuthology.orchestra.run.smithi104:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T04:40:19.274 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T04:40:19.275 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T04:40:19.275 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-04-26T04:40:19.377 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T04:40:19.377 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T04:40:19.377 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-04-26T04:40:19.377 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.90:3300,172.21.15.104:3300" 2024-04-26T04:40:19.377 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:invalid new device string format 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.90:3300,172.21.15.104:3300" 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: switching to using v1 address with old syntax 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: resolved to: "172.21.15.90:6789,172.21.15.104:6789" 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.90:6789,172.21.15.104:6789:/ 2024-04-26T04:40:19.383 INFO:teuthology.orchestra.run.smithi104.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=f018e902-0385-11ef-bc93-c7b262605968" will pass to kernel 2024-04-26T04:40:19.384 INFO:teuthology.orchestra.run.smithi104.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-04-26T04:40:19.384 INFO:teuthology.orchestra.run.smithi104.stdout: You just mounted a file system that supports labels which does not 2024-04-26T04:40:19.384 INFO:teuthology.orchestra.run.smithi104.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-26T04:40:19.384 INFO:teuthology.orchestra.run.smithi104.stdout: applications will generate AVC messages and not be allowed access to 2024-04-26T04:40:19.384 INFO:teuthology.orchestra.run.smithi104.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-26T04:40:19.386 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-26T04:40:19.386 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:40:19.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:19.409 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T04:40:19.497 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T04:40:19.499 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:19.506 INFO:teuthology.orchestra.run.smithi104.stdout:client14540 2024-04-26T04:40:19.506 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:40:19.506 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T04:40:19.526 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T04:40:19.613 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T04:40:19.615 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:19.620 INFO:teuthology.orchestra.run.smithi104.stdout:client14540 2024-04-26T04:40:19.620 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-26T04:40:19.668 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-26T04:40:19.670 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:19.675 INFO:teuthology.orchestra.run.smithi104.stdout:client14540 2024-04-26T04:40:19.676 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f018e902-0385-11ef-bc93-c7b262605968.client14540/status') 2024-04-26T04:40:19.723 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-26T04:40:19.732 INFO:teuthology.orchestra.run.smithi104.stdout:instance: client.14540 (3)172.21.15.104:0/3108279391 2024-04-26T04:40:19.732 INFO:teuthology.orchestra.run.smithi104.stdout:blocklisted: false 2024-04-26T04:40:19.732 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records in 2024-04-26T04:40:19.732 INFO:teuthology.orchestra.run.smithi104.stderr:0+1 records out 2024-04-26T04:40:19.732 INFO:teuthology.orchestra.run.smithi104.stderr:72 bytes copied, 2.136e-05 s, 3.4 MB/s 2024-04-26T04:40:19.733 INFO:teuthology.orchestra.run.smithi104.stderr:2024-04-26T04:40:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-26T04:40:19.739 INFO:teuthology.run_tasks:Running task print... 2024-04-26T04:40:19.751 INFO:teuthology.task.print:**** done client 2024-04-26T04:40:19.751 INFO:teuthology.run_tasks:Running task parallel... 2024-04-26T04:40:19.764 INFO:teuthology.task.parallel:starting parallel... 2024-04-26T04:40:19.764 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-26T04:40:19.765 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-26T04:40:19.765 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:40:19.766 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-26T04:40:19.766 INFO:teuthology.task.parallel:In parallel, running task sequential... 2024-04-26T04:40:19.766 INFO:teuthology.task.sequential:In sequential, running task workunit... 2024-04-26T04:40:19.770 INFO:tasks.workunit:Pulling workunits from ref b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:40:19.770 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-26T04:40:19.770 INFO:tasks.workunit:timeout=3h 2024-04-26T04:40:19.770 INFO:tasks.workunit:cleanup=True 2024-04-26T04:40:19.770 DEBUG:teuthology.orchestra.run.smithi090:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout:Device: 9ch/156d Inode: 1 Links: 2 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout:Modify: 2024-04-26 04:39:49.617508481 +0000 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout:Change: 2024-04-26 04:40:15.214671007 +0000 2024-04-26T04:40:19.791 INFO:teuthology.orchestra.run.smithi090.stdout: Birth: 2024-04-26 04:39:49.617508481 +0000 2024-04-26T04:40:19.792 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-26T04:40:19.792 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-26T04:40:19.862 DEBUG:teuthology.orchestra.run.smithi104:> stat -- /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:19.878 INFO:teuthology.orchestra.run.smithi104.stdout: File: /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:19.878 INFO:teuthology.orchestra.run.smithi104.stdout: Size: 1 Blocks: 0 IO Block: 65536 directory 2024-04-26T04:40:19.878 INFO:teuthology.orchestra.run.smithi104.stdout:Device: 7bh/123d Inode: 1 Links: 3 2024-04-26T04:40:19.878 INFO:teuthology.orchestra.run.smithi104.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-26T04:40:19.879 INFO:teuthology.orchestra.run.smithi104.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-26T04:40:19.879 INFO:teuthology.orchestra.run.smithi104.stdout:Access: 1970-01-01 00:00:00.000000000 +0000 2024-04-26T04:40:19.879 INFO:teuthology.orchestra.run.smithi104.stdout:Modify: 2024-04-26 04:40:19.855598113 +0000 2024-04-26T04:40:19.879 INFO:teuthology.orchestra.run.smithi104.stdout:Change: 2024-04-26 04:40:19.855598113 +0000 2024-04-26T04:40:19.879 INFO:teuthology.orchestra.run.smithi104.stdout: Birth: 2024-04-26 04:39:49.617508481 +0000 2024-04-26T04:40:19.879 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.1 2024-04-26T04:40:19.879 DEBUG:teuthology.orchestra.run.smithi104:> cd -- /home/ubuntu/cephtest/mnt.1 && sudo install -d -m 0755 --owner=ubuntu -- client.1 2024-04-26T04:40:19.952 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:40:19.952 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/clone.client.1 && git clone https://git.ceph.com/ceph.git /home/ubuntu/cephtest/clone.client.1 && cd /home/ubuntu/cephtest/clone.client.1 && git checkout b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:40:19.968 INFO:tasks.workunit.client.0.smithi090.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-26T04:40:19.999 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:20.007 INFO:tasks.workunit.client.1.smithi104.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.1'... 2024-04-26T04:40:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:20 smithi090 ceph-mon[26648]: pgmap v192: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-26T04:40:20.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:20 smithi104 ceph-mon[32369]: pgmap v192: 65 pgs: 65 active+clean; 581 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 4.5 KiB/s rd, 255 B/s wr, 4 op/s 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 -- 172.21.15.90:0/212191043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8100240 msgr2=0x7fa2a8100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 --2- 172.21.15.90:0/212191043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8100240 0x7fa2a8100650 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa29800b0a0 tx=0x7fa29800b3b0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 -- 172.21.15.90:0/212191043 shutdown_connections 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 --2- 172.21.15.90:0/212191043 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2a8101440 0x7fa2a8101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 --2- 172.21.15.90:0/212191043 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8100240 0x7fa2a8100650 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 -- 172.21.15.90:0/212191043 >> 172.21.15.90:0/212191043 conn(0x7fa2a80fb7f0 msgr2=0x7fa2a80fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 -- 172.21.15.90:0/212191043 shutdown_connections 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.369+0000 7fa2adcce700 1 -- 172.21.15.90:0/212191043 wait complete. 2024-04-26T04:40:21.373 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2adcce700 1 Processor -- start 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2adcce700 1 -- start start 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2adcce700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2a8100240 0x7fa2a8195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2adcce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8101440 0x7fa2a8195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2adcce700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8196610 con 0x7fa2a8101440 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2adcce700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2a8196750 con 0x7fa2a8100240 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2a6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8101440 0x7fa2a8195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2a6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8101440 0x7fa2a8195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33378/0 (socket says 172.21.15.90:33378) 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2a6ffd700 1 -- 172.21.15.90:0/1936395687 learned_addr learned my addr 172.21.15.90:0/1936395687 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2a77fe700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2a8100240 0x7fa2a8195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2a6ffd700 1 -- 172.21.15.90:0/1936395687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2a8100240 msgr2=0x7fa2a8195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2a6ffd700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2a8100240 0x7fa2a8195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.370+0000 7fa2a6ffd700 1 -- 172.21.15.90:0/1936395687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa298009d00 con 0x7fa2a8101440 2024-04-26T04:40:21.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.371+0000 7fa2a6ffd700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8101440 0x7fa2a8195ff0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00e920 tx=0x7fa29c00ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:21.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.371+0000 7fa2a4ff9700 1 -- 172.21.15.90:0/1936395687 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29c009790 con 0x7fa2a8101440 2024-04-26T04:40:21.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.371+0000 7fa2a4ff9700 1 -- 172.21.15.90:0/1936395687 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fa29c00cba0 con 0x7fa2a8101440 2024-04-26T04:40:21.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.371+0000 7fa2a4ff9700 1 -- 172.21.15.90:0/1936395687 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa29c018630 con 0x7fa2a8101440 2024-04-26T04:40:21.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.371+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2a819b200 con 0x7fa2a8101440 2024-04-26T04:40:21.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.371+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2a819b750 con 0x7fa2a8101440 2024-04-26T04:40:21.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.372+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2a8108d90 con 0x7fa2a8101440 2024-04-26T04:40:21.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.374+0000 7fa2a4ff9700 1 -- 172.21.15.90:0/1936395687 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fa29c00cd10 con 0x7fa2a8101440 2024-04-26T04:40:21.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.374+0000 7fa2a4ff9700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa29006c680 0x7fa29006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:21.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.375+0000 7fa2a4ff9700 1 -- 172.21.15.90:0/1936395687 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fa29c014070 con 0x7fa2a8101440 2024-04-26T04:40:21.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.375+0000 7fa2a77fe700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa29006c680 0x7fa29006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:21.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.375+0000 7fa2a77fe700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa29006c680 0x7fa29006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa298000f80 tx=0x7fa2980061f0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:21.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.378+0000 7fa2a4ff9700 1 -- 172.21.15.90:0/1936395687 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fa29c05a3a0 con 0x7fa2a8101440 2024-04-26T04:40:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.569+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7fa2a8108f80 con 0x7fa2a8101440 2024-04-26T04:40:21.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.569+0000 7fa2a4ff9700 1 -- 172.21.15.90:0/1936395687 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v21)=0 v21) v1 ==== 155+0+0 (secure 0 0 0) 0x7fa29c059f30 con 0x7fa2a8101440 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa29006c680 msgr2=0x7fa29006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa29006c680 0x7fa29006eb30 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa298000f80 tx=0x7fa2980061f0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8101440 msgr2=0x7fa2a8195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8101440 0x7fa2a8195ff0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fa29c00e920 tx=0x7fa29c00ece0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 shutdown_connections 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fa29006c680 0x7fa29006eb30 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa2a8100240 0x7fa2a8195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 --2- 172.21.15.90:0/1936395687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa2a8101440 0x7fa2a8195ff0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 >> 172.21.15.90:0/1936395687 conn(0x7fa2a80fb7f0 msgr2=0x7fa2a8104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.571+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 shutdown_connections 2024-04-26T04:40:21.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:21.572+0000 7fa2adcce700 1 -- 172.21.15.90:0/1936395687 wait complete. 2024-04-26T04:40:22.126 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-26T04:40:22.374 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:22.682 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:22 smithi090 ceph-mon[26648]: pgmap v193: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:40:22.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:22 smithi104 ceph-mon[32369]: pgmap v193: 65 pgs: 65 active+clean; 582 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 341 B/s wr, 0 op/s 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.776+0000 7f0b58345700 1 -- 172.21.15.90:0/4121006297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50100240 msgr2=0x7f0b50100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.776+0000 7f0b58345700 1 --2- 172.21.15.90:0/4121006297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50100240 0x7f0b50100650 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0b40009910 tx=0x7f0b40009c20 comp rx=0 tx=0).stop 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 -- 172.21.15.90:0/4121006297 shutdown_connections 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 --2- 172.21.15.90:0/4121006297 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b50101440 0x7f0b50101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 --2- 172.21.15.90:0/4121006297 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50100240 0x7f0b50100650 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 -- 172.21.15.90:0/4121006297 >> 172.21.15.90:0/4121006297 conn(0x7f0b500fb7f0 msgr2=0x7f0b500fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 -- 172.21.15.90:0/4121006297 shutdown_connections 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 -- 172.21.15.90:0/4121006297 wait complete. 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 Processor -- start 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 -- start start 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b50100240 0x7f0b50074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:23.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50101440 0x7f0b50073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b50073680 con 0x7f0b50101440 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b58345700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b500737c0 con 0x7f0b50100240 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b558e0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50101440 0x7f0b50073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b558e0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50101440 0x7f0b50073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33392/0 (socket says 172.21.15.90:33392) 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.777+0000 7f0b558e0700 1 -- 172.21.15.90:0/1027283087 learned_addr learned my addr 172.21.15.90:0/1027283087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b560e1700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b50100240 0x7f0b50074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b558e0700 1 -- 172.21.15.90:0/1027283087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b50100240 msgr2=0x7f0b50074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b558e0700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b50100240 0x7f0b50074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b558e0700 1 -- 172.21.15.90:0/1027283087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b400095f0 con 0x7f0b50101440 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b558e0700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50101440 0x7f0b50073140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00d900 tx=0x7f0b4c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b477fe700 1 -- 172.21.15.90:0/1027283087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4c004a00 con 0x7f0b50101440 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b50073aa0 con 0x7f0b50101440 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b477fe700 1 -- 172.21.15.90:0/1027283087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0b4c0054d0 con 0x7f0b50101440 2024-04-26T04:40:23.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b477fe700 1 -- 172.21.15.90:0/1027283087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b4c009d00 con 0x7f0b50101440 2024-04-26T04:40:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.778+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5019db30 con 0x7f0b50101440 2024-04-26T04:40:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.779+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b50108db0 con 0x7f0b50101440 2024-04-26T04:40:23.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.780+0000 7f0b477fe700 1 -- 172.21.15.90:0/1027283087 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0b4c005020 con 0x7f0b50101440 2024-04-26T04:40:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.780+0000 7f0b477fe700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0b3c06c680 0x7f0b3c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.780+0000 7f0b477fe700 1 -- 172.21.15.90:0/1027283087 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0b4c08b870 con 0x7f0b50101440 2024-04-26T04:40:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.780+0000 7f0b560e1700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0b3c06c680 0x7f0b3c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:23.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.781+0000 7f0b560e1700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0b3c06c680 0x7f0b3c06eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b40006010 tx=0x7f0b40005ec0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:23.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.785+0000 7f0b477fe700 1 -- 172.21.15.90:0/1027283087 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0b4c059980 con 0x7f0b50101440 2024-04-26T04:40:23.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.976+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f0b5019e2f0 con 0x7f0b50101440 2024-04-26T04:40:23.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.976+0000 7f0b477fe700 1 -- 172.21.15.90:0/1027283087 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v21)=0 v21) v1 ==== 163+0+0 (secure 0 0 0) 0x7f0b4c0170e0 con 0x7f0b50101440 2024-04-26T04:40:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0b3c06c680 msgr2=0x7f0b3c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0b3c06c680 0x7f0b3c06eb30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f0b40006010 tx=0x7f0b40005ec0 comp rx=0 tx=0).stop 2024-04-26T04:40:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50101440 msgr2=0x7f0b50073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:23.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50101440 0x7f0b50073140 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0b4c00d900 tx=0x7f0b4c00dc10 comp rx=0 tx=0).stop 2024-04-26T04:40:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 shutdown_connections 2024-04-26T04:40:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0b3c06c680 0x7f0b3c06eb30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b50100240 0x7f0b50074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 --2- 172.21.15.90:0/1027283087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b50101440 0x7f0b50073140 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 >> 172.21.15.90:0/1027283087 conn(0x7f0b500fb7f0 msgr2=0x7f0b50104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 shutdown_connections 2024-04-26T04:40:23.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:23.978+0000 7f0b58345700 1 -- 172.21.15.90:0/1027283087 wait complete. 2024-04-26T04:40:24.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:24 smithi090 ceph-mon[26648]: pgmap v194: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-26T04:40:24.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:24 smithi104 ceph-mon[32369]: pgmap v194: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-26T04:40:24.618 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-26T04:40:24.822 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:26.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 -- 172.21.15.90:0/989232124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 msgr2=0x7f63b0103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 --2- 172.21.15.90:0/989232124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0103160 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f63a0009910 tx=0x7f63a0009c20 comp rx=0 tx=0).stop 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 -- 172.21.15.90:0/989232124 shutdown_connections 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 --2- 172.21.15.90:0/989232124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63b01036a0 0x7f63b0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 --2- 172.21.15.90:0/989232124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0103160 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 -- 172.21.15.90:0/989232124 >> 172.21.15.90:0/989232124 conn(0x7f63b00fa7a0 msgr2=0x7f63b00fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 -- 172.21.15.90:0/989232124 shutdown_connections 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.284+0000 7f63b5d63700 1 -- 172.21.15.90:0/989232124 wait complete. 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63b5d63700 1 Processor -- start 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63b5d63700 1 -- start start 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63b5d63700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63b5d63700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63b01036a0 0x7f63b01963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:26.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63b5d63700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b01969d0 con 0x7f63b0100d80 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63b5d63700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63b0196b10 con 0x7f63b01036a0 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63af7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63af7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0195e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:33404/0 (socket says 172.21.15.90:33404) 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63af7fe700 1 -- 172.21.15.90:0/3154736203 learned_addr learned my addr 172.21.15.90:0/3154736203 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63af7fe700 1 -- 172.21.15.90:0/3154736203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63b01036a0 msgr2=0x7f63b01963b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63aeffd700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63b01036a0 0x7f63b01963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.285+0000 7f63af7fe700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63b01036a0 0x7f63b01963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.286+0000 7f63af7fe700 1 -- 172.21.15.90:0/3154736203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63a00095f0 con 0x7f63b0100d80 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.286+0000 7f63af7fe700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0195e70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f63a000bda0 tx=0x7f63a000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.286+0000 7f63acff9700 1 -- 172.21.15.90:0/3154736203 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a001d070 con 0x7f63b0100d80 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.286+0000 7f63acff9700 1 -- 172.21.15.90:0/3154736203 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f63a0004c50 con 0x7f63b0100d80 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.286+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63b019b560 con 0x7f63b0100d80 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.286+0000 7f63acff9700 1 -- 172.21.15.90:0/3154736203 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f63a000f690 con 0x7f63b0100d80 2024-04-26T04:40:26.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.286+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63b019ba50 con 0x7f63b0100d80 2024-04-26T04:40:26.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.287+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63b018fc60 con 0x7f63b0100d80 2024-04-26T04:40:26.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.287+0000 7f63acff9700 1 -- 172.21.15.90:0/3154736203 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f63a0022a20 con 0x7f63b0100d80 2024-04-26T04:40:26.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.288+0000 7f63acff9700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f639c06c680 0x7f639c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:26.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.288+0000 7f63acff9700 1 -- 172.21.15.90:0/3154736203 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f63a008cfb0 con 0x7f63b0100d80 2024-04-26T04:40:26.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.288+0000 7f63aeffd700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f639c06c680 0x7f639c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:26.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.288+0000 7f63aeffd700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f639c06c680 0x7f639c06eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6398006fd0 tx=0x7f6398008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:26.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.292+0000 7f63acff9700 1 -- 172.21.15.90:0/3154736203 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f63a005b170 con 0x7f63b0100d80 2024-04-26T04:40:26.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.484+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7f63b0068ab0 con 0x7f63b0100d80 2024-04-26T04:40:26.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.484+0000 7f63acff9700 1 -- 172.21.15.90:0/3154736203 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v21)=0 v21) v1 ==== 135+0+0 (secure 0 0 0) 0x7f63a005ad00 con 0x7f63b0100d80 2024-04-26T04:40:26.487 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f639c06c680 msgr2=0x7f639c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f639c06c680 0x7f639c06eb30 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6398006fd0 tx=0x7f6398008040 comp rx=0 tx=0).stop 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 msgr2=0x7f63b0195e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0195e70 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f63a000bda0 tx=0x7f63a000be80 comp rx=0 tx=0).stop 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 shutdown_connections 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f639c06c680 0x7f639c06eb30 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f63b0100d80 0x7f63b0195e70 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 --2- 172.21.15.90:0/3154736203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f63b01036a0 0x7f63b01963b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 >> 172.21.15.90:0/3154736203 conn(0x7f63b00fa7a0 msgr2=0x7f63b00fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 shutdown_connections 2024-04-26T04:40:26.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:26.486+0000 7f63b5d63700 1 -- 172.21.15.90:0/3154736203 wait complete. 2024-04-26T04:40:26.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:26 smithi090 ceph-mon[26648]: pgmap v195: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-26T04:40:26.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:26 smithi104 ceph-mon[32369]: pgmap v195: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1023 B/s wr, 0 op/s 2024-04-26T04:40:27.080 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1 --daemon-types mgr' 2024-04-26T04:40:27.285 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:40:28.609 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:28 smithi090 ceph-mon[26648]: pgmap v196: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-26T04:40:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:28 smithi104 ceph-mon[32369]: pgmap v196: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-26T04:40:28.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.908+0000 7fbdf388f700 1 -- 172.21.15.90:0/1297832680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 msgr2=0x7fbdec0ff470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:28.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.908+0000 7fbdf388f700 1 --2- 172.21.15.90:0/1297832680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec0ff470 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbddc009910 tx=0x7fbddc009c20 comp rx=0 tx=0).stop 2024-04-26T04:40:28.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.908+0000 7fbdf388f700 1 -- 172.21.15.90:0/1297832680 shutdown_connections 2024-04-26T04:40:28.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.908+0000 7fbdf388f700 1 --2- 172.21.15.90:0/1297832680 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec0ff470 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:28.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.908+0000 7fbdf388f700 1 --2- 172.21.15.90:0/1297832680 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec100710 0x7fbdec100b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:28.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 -- 172.21.15.90:0/1297832680 >> 172.21.15.90:0/1297832680 conn(0x7fbdec0f9f10 msgr2=0x7fbdec0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 -- 172.21.15.90:0/1297832680 shutdown_connections 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 -- 172.21.15.90:0/1297832680 wait complete. 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 Processor -- start 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 -- start start 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec100710 0x7fbdec0feec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec19a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec19ae70 con 0x7fbdec101060 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.909+0000 7fbdf388f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdec19afb0 con 0x7fbdec100710 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf0e2a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec19a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf0e2a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec19a850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40680/0 (socket says 172.21.15.90:40680) 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf0e2a700 1 -- 172.21.15.90:0/3325629354 learned_addr learned my addr 172.21.15.90:0/3325629354 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf162b700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec100710 0x7fbdec0feec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:28.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf0e2a700 1 -- 172.21.15.90:0/3325629354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec100710 msgr2=0x7fbdec0feec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf0e2a700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec100710 0x7fbdec0feec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf0e2a700 1 -- 172.21.15.90:0/3325629354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbddc0095f0 con 0x7fbdec101060 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf162b700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec100710 0x7fbdec0feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf0e2a700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec19a850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006010 tx=0x7fbddc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbde27fc700 1 -- 172.21.15.90:0/3325629354 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc01d070 con 0x7fbdec101060 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbde27fc700 1 -- 172.21.15.90:0/3325629354 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbddc00f460 con 0x7fbdec101060 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdec19b230 con 0x7fbdec101060 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbde27fc700 1 -- 172.21.15.90:0/3325629354 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbddc021640 con 0x7fbdec101060 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.910+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdec19b720 con 0x7fbdec101060 2024-04-26T04:40:28.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.911+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdec107c90 con 0x7fbdec101060 2024-04-26T04:40:28.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.912+0000 7fbde27fc700 1 -- 172.21.15.90:0/3325629354 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fbddc00fab0 con 0x7fbdec101060 2024-04-26T04:40:28.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.912+0000 7fbde27fc700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbdd806c680 0x7fbdd806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:40:28.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.912+0000 7fbde27fc700 1 -- 172.21.15.90:0/3325629354 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbddc08cce0 con 0x7fbdec101060 2024-04-26T04:40:28.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.912+0000 7fbdf162b700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbdd806c680 0x7fbdd806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:40:28.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.912+0000 7fbdf162b700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbdd806c680 0x7fbdd806eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0688a0 tx=0x7fbde8008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:40:28.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:28.917+0000 7fbde27fc700 1 -- 172.21.15.90:0/3325629354 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbddc05aea0 con 0x7fbdec101060 2024-04-26T04:40:29.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:40:29.123+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "daemon_types": "mgr", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdec104e20 con 0x7fbdd806c680 2024-04-26T04:40:30.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:30 smithi104 ceph-mon[32369]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:40:30.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:30 smithi104 ceph-mon[32369]: pgmap v197: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-26T04:40:30.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:30 smithi090 ceph-mon[26648]: from='client.14556 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "daemon_types": "mgr", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:40:30.674 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:30 smithi090 ceph-mon[26648]: pgmap v197: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 1.2 KiB/s wr, 0 op/s 2024-04-26T04:40:32.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:32 smithi090 ceph-mon[26648]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-26T04:40:32.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:32 smithi104 ceph-mon[32369]: pgmap v198: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 938 B/s wr, 0 op/s 2024-04-26T04:40:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:34 smithi090 ceph-mon[26648]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-04-26T04:40:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:34 smithi104 ceph-mon[32369]: pgmap v199: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 853 B/s wr, 0 op/s 2024-04-26T04:40:36.681 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:36 smithi104 ceph-mon[32369]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-26T04:40:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:36 smithi090 ceph-mon[26648]: pgmap v200: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 255 B/s wr, 0 op/s 2024-04-26T04:40:38.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:38 smithi090 ceph-mon[26648]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:40:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:38 smithi104 ceph-mon[32369]: pgmap v201: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:40:40.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:40 smithi090 ceph-mon[26648]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:40.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:40 smithi104 ceph-mon[32369]: pgmap v202: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:42.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:42 smithi090 ceph-mon[26648]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:42.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:42 smithi104 ceph-mon[32369]: pgmap v203: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:44 smithi090 ceph-mon[26648]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:44 smithi104 ceph-mon[32369]: pgmap v204: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:46.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:46 smithi090 ceph-mon[26648]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:46 smithi104 ceph-mon[32369]: pgmap v205: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:48.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:48 smithi090 ceph-mon[26648]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:48.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:48 smithi104 ceph-mon[32369]: pgmap v206: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:40:50.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:50 smithi090 ceph-mon[26648]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:50 smithi104 ceph-mon[32369]: pgmap v207: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:52 smithi090 ceph-mon[26648]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:52 smithi104 ceph-mon[32369]: pgmap v208: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:54.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:54 smithi090 ceph-mon[26648]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:54.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:54 smithi104 ceph-mon[32369]: pgmap v209: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:56.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:56 smithi090 ceph-mon[26648]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:56.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:56 smithi104 ceph-mon[32369]: pgmap v210: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:58.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:40:58 smithi090 ceph-mon[26648]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:40:58.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:40:58 smithi104 ceph-mon[32369]: pgmap v211: 65 pgs: 65 active+clean; 586 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:00 smithi090 ceph-mon[26648]: pgmap v212: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:41:00.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:00 smithi104 ceph-mon[32369]: pgmap v212: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:41:02.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:02 smithi090 ceph-mon[26648]: pgmap v213: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:41:02.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:02 smithi104 ceph-mon[32369]: pgmap v213: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:41:03.189 INFO:teuthology.orchestra.run.smithi090.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:41:03.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.188+0000 7fbde27fc700 1 -- 172.21.15.90:0/3325629354 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7fbdec104e20 con 0x7fbdd806c680 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbdd806c680 msgr2=0x7fbdd806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbdd806c680 0x7fbdd806eb30 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbdec0688a0 tx=0x7fbde8008040 comp rx=0 tx=0).stop 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 msgr2=0x7fbdec19a850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec19a850 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fbddc006010 tx=0x7fbddc00be20 comp rx=0 tx=0).stop 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 shutdown_connections 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbdd806c680 0x7fbdd806eb30 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdec100710 0x7fbdec0feec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 --2- 172.21.15.90:0/3325629354 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdec101060 0x7fbdec19a850 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.190+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 >> 172.21.15.90:0/3325629354 conn(0x7fbdec0f9f10 msgr2=0x7fbdec103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.191+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 shutdown_connections 2024-04-26T04:41:03.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:03.191+0000 7fbdf388f700 1 -- 172.21.15.90:0/3325629354 wait complete. 2024-04-26T04:41:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:04 smithi090 ceph-mon[26648]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:41:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:41:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:04 smithi090 ceph-mon[26648]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:41:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:04 smithi104 ceph-mon[32369]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:41:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:41:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:04 smithi104 ceph-mon[32369]: pgmap v214: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:41:04.624 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph orch upgrade status ; sleep 30 ; done' 2024-04-26T04:41:04.862 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:41:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:05 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:41:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:05 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:41:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:05 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:05 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:41:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:05 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:41:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:05 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:06.464 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:06 smithi090 ceph-mon[26648]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:41:06.464 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:06 smithi090 ceph-mon[26648]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:41:06.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:06 smithi104 ceph-mon[32369]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:41:06.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:06 smithi104 ceph-mon[32369]: pgmap v215: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:41:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:08 smithi090 ceph-mon[26648]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:41:08.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:08 smithi104 ceph-mon[32369]: pgmap v216: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:41:10.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:10 smithi090 ceph-mon[26648]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:41:10.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:10 smithi104 ceph-mon[32369]: pgmap v217: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 597 B/s wr, 0 op/s 2024-04-26T04:41:12.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:12 smithi090 ceph-mon[26648]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:41:12.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:12 smithi104 ceph-mon[32369]: pgmap v218: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 170 B/s wr, 0 op/s 2024-04-26T04:41:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:14 smithi090 ceph-mon[26648]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:41:14.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:14 smithi104 ceph-mon[32369]: pgmap v219: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:41:16.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:16 smithi090 ceph-mon[26648]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:41:16.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:16 smithi104 ceph-mon[32369]: pgmap v220: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:41:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:18 smithi090 ceph-mon[26648]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:41:18.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 -- 172.21.15.90:0/1423652854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 msgr2=0x7f548c0ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 --2- 172.21.15.90:0/1423652854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c0ff540 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f5488009910 tx=0x7f5488009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 -- 172.21.15.90:0/1423652854 shutdown_connections 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 --2- 172.21.15.90:0/1423652854 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c1003d0 0x7f548c100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 --2- 172.21.15.90:0/1423652854 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c0ff540 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 -- 172.21.15.90:0/1423652854 >> 172.21.15.90:0/1423652854 conn(0x7f548c0fa780 msgr2=0x7f548c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 -- 172.21.15.90:0/1423652854 shutdown_connections 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 -- 172.21.15.90:0/1423652854 wait complete. 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.935+0000 7f54933e2700 1 Processor -- start 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f54933e2700 1 -- start start 2024-04-26T04:41:18.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f54933e2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f54933e2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c1003d0 0x7f548c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f54933e2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c101cf0 con 0x7f548c0ff130 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f549117e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f549117e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c103160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46806/0 (socket says 172.21.15.90:46806) 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f549117e700 1 -- 172.21.15.90:0/2424272938 learned_addr learned my addr 172.21.15.90:0/2424272938 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f549097d700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c1003d0 0x7f548c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f548c101e30 con 0x7f548c1003d0 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f549117e700 1 -- 172.21.15.90:0/2424272938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c1003d0 msgr2=0x7f548c1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f549117e700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c1003d0 0x7f548c1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.936+0000 7f549117e700 1 -- 172.21.15.90:0/2424272938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54880095f0 con 0x7f548c0ff130 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.937+0000 7f549117e700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c103160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5488005b40 tx=0x7f548800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.937+0000 7f54827fc700 1 -- 172.21.15.90:0/2424272938 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f548801d070 con 0x7f548c0ff130 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.937+0000 7f54827fc700 1 -- 172.21.15.90:0/2424272938 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f548800f460 con 0x7f548c0ff130 2024-04-26T04:41:18.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.937+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f548c1020b0 con 0x7f548c0ff130 2024-04-26T04:41:18.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.937+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f548c1025a0 con 0x7f548c0ff130 2024-04-26T04:41:18.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.937+0000 7f54827fc700 1 -- 172.21.15.90:0/2424272938 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5488021620 con 0x7f548c0ff130 2024-04-26T04:41:18.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.938+0000 7f54827fc700 1 -- 172.21.15.90:0/2424272938 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f548800fa40 con 0x7f548c0ff130 2024-04-26T04:41:18.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.939+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f548c18fb00 con 0x7f548c0ff130 2024-04-26T04:41:18.642 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.939+0000 7f54827fc700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f547806c680 0x7f547806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.939+0000 7f54827fc700 1 -- 172.21.15.90:0/2424272938 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f548808cb50 con 0x7f548c0ff130 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.939+0000 7f549097d700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f547806c680 0x7f547806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.940+0000 7f549097d700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f547806c680 0x7f547806eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f547c007be0 tx=0x7f547c0073d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:17.944+0000 7f54827fc700 1 -- 172.21.15.90:0/2424272938 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f548805ac60 con 0x7f548c0ff130 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.182+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f548c061190 con 0x7f547806c680 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.183+0000 7f54827fc700 1 -- 172.21.15.90:0/2424272938 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f548c061190 con 0x7f547806c680 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f547806c680 msgr2=0x7f547806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f547806c680 0x7f547806eb30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f547c007be0 tx=0x7f547c0073d0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 msgr2=0x7f548c103160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c103160 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f5488005b40 tx=0x7f548800bd70 comp rx=0 tx=0).stop 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 shutdown_connections 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f547806c680 0x7f547806eb30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f548c0ff130 0x7f548c103160 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 --2- 172.21.15.90:0/2424272938 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f548c1003d0 0x7f548c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 >> 172.21.15.90:0/2424272938 conn(0x7f548c0fa780 msgr2=0x7f548c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 shutdown_connections 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.186+0000 7f54933e2700 1 -- 172.21.15.90:0/2424272938 wait complete. 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.295+0000 7f50c1d9b700 1 -- 172.21.15.90:0/1371588197 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b40054b0 con 0x7f50c40fe8c0 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.295+0000 7f50c9072700 1 -- 172.21.15.90:0/1371588197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 msgr2=0x7f50c40fecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.295+0000 7f50c9072700 1 --2- 172.21.15.90:0/1371588197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c40fecd0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009910 tx=0x7f50b4009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.295+0000 7f50c9072700 1 -- 172.21.15.90:0/1371588197 shutdown_connections 2024-04-26T04:41:18.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.295+0000 7f50c9072700 1 --2- 172.21.15.90:0/1371588197 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c40ffb60 0x7f50c40fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.295+0000 7f50c9072700 1 --2- 172.21.15.90:0/1371588197 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c40fecd0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.295+0000 7f50c9072700 1 -- 172.21.15.90:0/1371588197 >> 172.21.15.90:0/1371588197 conn(0x7f50c40f9f10 msgr2=0x7f50c40fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.296+0000 7f50c9072700 1 -- 172.21.15.90:0/1371588197 shutdown_connections 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.296+0000 7f50c9072700 1 -- 172.21.15.90:0/1371588197 wait complete. 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c9072700 1 Processor -- start 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c9072700 1 -- start start 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c9072700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c419c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c9072700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c40ffb60 0x7f50c419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c9072700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419cc30 con 0x7f50c40fe8c0 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c9072700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c419cd70 con 0x7f50c40ffb60 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c2d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c419c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c259c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c40ffb60 0x7f50c419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c2d9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c419c0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46826/0 (socket says 172.21.15.90:46826) 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.297+0000 7f50c2d9d700 1 -- 172.21.15.90:0/2103369557 learned_addr learned my addr 172.21.15.90:0/2103369557 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:18.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.298+0000 7f50c2d9d700 1 -- 172.21.15.90:0/2103369557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c40ffb60 msgr2=0x7f50c419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.298+0000 7f50c2d9d700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c40ffb60 0x7f50c419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.298+0000 7f50c2d9d700 1 -- 172.21.15.90:0/2103369557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b40095f0 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.298+0000 7f50c2d9d700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c419c0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50b400bd70 tx=0x7f50b400bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.298+0000 7f50abfff700 1 -- 172.21.15.90:0/2103369557 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b401c070 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.298+0000 7f50abfff700 1 -- 172.21.15.90:0/2103369557 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f50b4005dc0 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.298+0000 7f50abfff700 1 -- 172.21.15.90:0/2103369557 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f50b4020410 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.299+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50c41a17c0 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.299+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c41a1bf0 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.300+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c4106b00 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.301+0000 7f50abfff700 1 -- 172.21.15.90:0/2103369557 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f50b400e7f0 con 0x7f50c40fe8c0 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.301+0000 7f50abfff700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f50ac06c680 0x7f50ac06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.301+0000 7f50abfff700 1 -- 172.21.15.90:0/2103369557 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f50b408c5b0 con 0x7f50c40fe8c0 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.301+0000 7f50c259c700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f50ac06c680 0x7f50ac06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.302+0000 7f50c259c700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f50ac06c680 0x7f50ac06eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005950 tx=0x7f50b800d040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.306+0000 7f50abfff700 1 -- 172.21.15.90:0/2103369557 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f50b405a740 con 0x7f50c40fe8c0 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.530+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f50c41044b0 con 0x7f50ac06c680 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.532+0000 7f50abfff700 1 -- 172.21.15.90:0/2103369557 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+351 (secure 0 0 0) 0x7f50c41044b0 con 0x7f50ac06c680 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f50ac06c680 msgr2=0x7f50ac06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f50ac06c680 0x7f50ac06eb30 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f50b8005950 tx=0x7f50b800d040 comp rx=0 tx=0).stop 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 msgr2=0x7f50c419c0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c419c0d0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f50b400bd70 tx=0x7f50b400bda0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 shutdown_connections 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f50ac06c680 0x7f50ac06eb30 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f50c40fe8c0 0x7f50c419c0d0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 --2- 172.21.15.90:0/2103369557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f50c40ffb60 0x7f50c419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.647 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 >> 172.21.15.90:0/2103369557 conn(0x7f50c40f9f10 msgr2=0x7f50c4102d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:18.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.534+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 shutdown_connections 2024-04-26T04:41:18.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.535+0000 7f50c9072700 1 -- 172.21.15.90:0/2103369557 wait complete. 2024-04-26T04:41:18.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 -- 172.21.15.90:0/3944681203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb301007f0 msgr2=0x7fdb30100c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/3944681203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb301007f0 0x7fdb30100c00 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fdb24009910 tx=0x7fdb24009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 -- 172.21.15.90:0/3944681203 shutdown_connections 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/3944681203 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30101140 0x7fdb30107a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/3944681203 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb301007f0 0x7fdb30100c00 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 -- 172.21.15.90:0/3944681203 >> 172.21.15.90:0/3944681203 conn(0x7fdb300f9f10 msgr2=0x7fdb300fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 -- 172.21.15.90:0/3944681203 shutdown_connections 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 -- 172.21.15.90:0/3944681203 wait complete. 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.687+0000 7fdb36ad9700 1 Processor -- start 2024-04-26T04:41:18.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb36ad9700 1 -- start start 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb36ad9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb301007f0 0x7fdb30197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb36ad9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101140 0x7fdb30198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb36ad9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30198870 con 0x7fdb30101140 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb36ad9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3019d280 con 0x7fdb301007f0 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb2ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101140 0x7fdb30198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb34875700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb301007f0 0x7fdb30197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb2ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101140 0x7fdb30198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46844/0 (socket says 172.21.15.90:46844) 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb2ffff700 1 -- 172.21.15.90:0/33128030 learned_addr learned my addr 172.21.15.90:0/33128030 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb2ffff700 1 -- 172.21.15.90:0/33128030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb301007f0 msgr2=0x7fdb30197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb2ffff700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb301007f0 0x7fdb30197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.688+0000 7fdb2ffff700 1 -- 172.21.15.90:0/33128030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb240095f0 con 0x7fdb30101140 2024-04-26T04:41:18.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.689+0000 7fdb2ffff700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101140 0x7fdb30198250 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00d900 tx=0x7fdb1c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:18.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.689+0000 7fdb2dffb700 1 -- 172.21.15.90:0/33128030 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb1c009900 con 0x7fdb30101140 2024-04-26T04:41:18.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.689+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3019d480 con 0x7fdb30101140 2024-04-26T04:41:18.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.689+0000 7fdb2dffb700 1 -- 172.21.15.90:0/33128030 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdb1c010460 con 0x7fdb30101140 2024-04-26T04:41:18.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.689+0000 7fdb2dffb700 1 -- 172.21.15.90:0/33128030 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb1c00f5f0 con 0x7fdb30101140 2024-04-26T04:41:18.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.690+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3019d9a0 con 0x7fdb30101140 2024-04-26T04:41:18.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.691+0000 7fdb2dffb700 1 -- 172.21.15.90:0/33128030 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fdb1c009c70 con 0x7fdb30101140 2024-04-26T04:41:18.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.692+0000 7fdb2dffb700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdb2006c680 0x7fdb2006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:18.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.692+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb30191e10 con 0x7fdb30101140 2024-04-26T04:41:18.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.692+0000 7fdb2dffb700 1 -- 172.21.15.90:0/33128030 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fdb1c08afa0 con 0x7fdb30101140 2024-04-26T04:41:18.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.692+0000 7fdb34875700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdb2006c680 0x7fdb2006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:18.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.693+0000 7fdb34875700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdb2006c680 0x7fdb2006eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb24006010 tx=0x7fdb24005a90 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:18.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.697+0000 7fdb2dffb700 1 -- 172.21.15.90:0/33128030 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fdb1c055740 con 0x7fdb30101140 2024-04-26T04:41:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:18 smithi104 ceph-mon[32369]: pgmap v221: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:41:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:18.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.902+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb30061190 con 0x7fdb2006c680 2024-04-26T04:41:18.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.914+0000 7fdb2dffb700 1 -- 172.21.15.90:0/33128030 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3480 (secure 0 0 0) 0x7fdb30061190 con 0x7fdb2006c680 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (5m) 65s ago 6m 25.9M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (7m) 65s ago 7m 7407k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (6m) 69s ago 6m 7335k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (7m) 65s ago 7m 6660k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (6m) 69s ago 6m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (5m) 65s ago 6m 83.8M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (83s) 65s ago 82s 15.6M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (89s) 65s ago 89s 11.2M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (80s) 69s ago 80s 12.2M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (87s) 69s ago 86s 18.4M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:9283,8765,8443 running (8m) 65s ago 8m 487M - 18.2.0 dc2bc1663786 a008eb0a2d7e 2024-04-26T04:41:18.916 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (5m) 69s ago 5m 437M - 18.2.0 dc2bc1663786 b809d29d97b5 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (8m) 65s ago 8m 49.1M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (5m) 69s ago 5m 35.3M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (6m) 65s ago 6m 14.3M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (5m) 69s ago 5m 15.5M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (5m) 65s ago 5m 48.1M 3946M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (4m) 65s ago 4m 47.6M 3946M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (4m) 65s ago 4m 47.3M 3946M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (3m) 69s ago 3m 45.4M 4970M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (3m) 69s ago 3m 44.8M 4970M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (2m) 69s ago 2m 44.3M 4970M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:41:18.917 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (5m) 65s ago 6m 60.0M - 2.43.0 a07b618ecd1d 8382d6553d6c 2024-04-26T04:41:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.916+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdb2006c680 msgr2=0x7fdb2006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.916+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdb2006c680 0x7fdb2006eb30 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fdb24006010 tx=0x7fdb24005a90 comp rx=0 tx=0).stop 2024-04-26T04:41:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101140 msgr2=0x7fdb30198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101140 0x7fdb30198250 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00d900 tx=0x7fdb1c00dcc0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 shutdown_connections 2024-04-26T04:41:18.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fdb2006c680 0x7fdb2006eb30 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb301007f0 0x7fdb30197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 --2- 172.21.15.90:0/33128030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101140 0x7fdb30198250 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:18.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 >> 172.21.15.90:0/33128030 conn(0x7fdb300f9f10 msgr2=0x7fdb300fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:18.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 shutdown_connections 2024-04-26T04:41:18.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:18.917+0000 7fdb36ad9700 1 -- 172.21.15.90:0/33128030 wait complete. 2024-04-26T04:41:19.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.026+0000 7f0a39df4700 1 -- 172.21.15.90:0/3868525560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a341003d0 msgr2=0x7f0a34100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.026+0000 7f0a39df4700 1 --2- 172.21.15.90:0/3868525560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a341003d0 0x7f0a34100840 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f0a28009910 tx=0x7f0a28009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:19.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.026+0000 7f0a39df4700 1 -- 172.21.15.90:0/3868525560 shutdown_connections 2024-04-26T04:41:19.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.026+0000 7f0a39df4700 1 --2- 172.21.15.90:0/3868525560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a341003d0 0x7f0a34100840 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.026+0000 7f0a39df4700 1 --2- 172.21.15.90:0/3868525560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a340ff130 0x7f0a340ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.026+0000 7f0a39df4700 1 -- 172.21.15.90:0/3868525560 >> 172.21.15.90:0/3868525560 conn(0x7f0a340fa780 msgr2=0x7f0a340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:19.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.027+0000 7f0a39df4700 1 -- 172.21.15.90:0/3868525560 shutdown_connections 2024-04-26T04:41:19.029 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.027+0000 7f0a39df4700 1 -- 172.21.15.90:0/3868525560 wait complete. 2024-04-26T04:41:19.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.027+0000 7f0a39df4700 1 Processor -- start 2024-04-26T04:41:19.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.027+0000 7f0a39df4700 1 -- start start 2024-04-26T04:41:19.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a39df4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340ff130 0x7f0a340690a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:19.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a39df4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a341003d0 0x7f0a3419a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:19.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a39df4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419ad90 con 0x7f0a340ff130 2024-04-26T04:41:19.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a39df4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a3419aed0 con 0x7f0a341003d0 2024-04-26T04:41:19.031 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340ff130 0x7f0a340690a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a337fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340ff130 0x7f0a340690a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46870/0 (socket says 172.21.15.90:46870) 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a337fe700 1 -- 172.21.15.90:0/1246530037 learned_addr learned my addr 172.21.15.90:0/1246530037 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a32ffd700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a341003d0 0x7f0a3419a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a337fe700 1 -- 172.21.15.90:0/1246530037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a341003d0 msgr2=0x7f0a3419a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a337fe700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a341003d0 0x7f0a3419a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.028+0000 7f0a337fe700 1 -- 172.21.15.90:0/1246530037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a280095f0 con 0x7f0a340ff130 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.029+0000 7f0a337fe700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340ff130 0x7f0a340690a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400d8d0 tx=0x7f0a2400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.029+0000 7f0a30ff9700 1 -- 172.21.15.90:0/1246530037 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a240098a0 con 0x7f0a340ff130 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.029+0000 7f0a30ff9700 1 -- 172.21.15.90:0/1246530037 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0a24010460 con 0x7f0a340ff130 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.029+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a3419b1b0 con 0x7f0a340ff130 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.029+0000 7f0a30ff9700 1 -- 172.21.15.90:0/1246530037 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0a2400f5f0 con 0x7f0a340ff130 2024-04-26T04:41:19.032 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.030+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a3419b640 con 0x7f0a340ff130 2024-04-26T04:41:19.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.030+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a34107c10 con 0x7f0a340ff130 2024-04-26T04:41:19.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.031+0000 7f0a30ff9700 1 -- 172.21.15.90:0/1246530037 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f0a24009a00 con 0x7f0a340ff130 2024-04-26T04:41:19.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.031+0000 7f0a30ff9700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0a1c06c680 0x7f0a1c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:19.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.031+0000 7f0a30ff9700 1 -- 172.21.15.90:0/1246530037 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f0a2408b110 con 0x7f0a340ff130 2024-04-26T04:41:19.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.031+0000 7f0a32ffd700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0a1c06c680 0x7f0a1c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:19.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.032+0000 7f0a32ffd700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0a1c06c680 0x7f0a1c06eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a34069620 tx=0x7f0a2800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:19.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.036+0000 7f0a30ff9700 1 -- 172.21.15.90:0/1246530037 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f0a240592a0 con 0x7f0a340ff130 2024-04-26T04:41:19.320 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.319+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0a3419bd90 con 0x7f0a340ff130 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.319+0000 7f0a30ff9700 1 -- 172.21.15.90:0/1246530037 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f0a24058e30 con 0x7f0a340ff130 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:41:19.321 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:19.322 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:41:19.322 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:41:19.322 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:19.322 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:41:19.322 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-04-26T04:41:19.322 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:41:19.322 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:41:19.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.322+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0a1c06c680 msgr2=0x7f0a1c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.322+0000 7f0a39df4700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0a1c06c680 0x7f0a1c06eb30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f0a34069620 tx=0x7f0a2800b540 comp rx=0 tx=0).stop 2024-04-26T04:41:19.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.322+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340ff130 msgr2=0x7f0a340690a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.322+0000 7f0a39df4700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340ff130 0x7f0a340690a0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f0a2400d8d0 tx=0x7f0a2400dbe0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.322+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 shutdown_connections 2024-04-26T04:41:19.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.322+0000 7f0a39df4700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f0a1c06c680 0x7f0a1c06eb30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.323+0000 7f0a39df4700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0a340ff130 0x7f0a340690a0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.323+0000 7f0a39df4700 1 --2- 172.21.15.90:0/1246530037 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0a341003d0 0x7f0a3419a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.323+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 >> 172.21.15.90:0/1246530037 conn(0x7f0a340fa780 msgr2=0x7f0a34103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:19.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.323+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 shutdown_connections 2024-04-26T04:41:19.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.323+0000 7f0a39df4700 1 -- 172.21.15.90:0/1246530037 wait complete. 2024-04-26T04:41:19.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 -- 172.21.15.90:0/1933622308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 msgr2=0x7f26cc1057b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 --2- 172.21.15.90:0/1933622308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1057b0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f26c0009910 tx=0x7f26c0009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:19.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 -- 172.21.15.90:0/1933622308 shutdown_connections 2024-04-26T04:41:19.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 --2- 172.21.15.90:0/1933622308 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1057b0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 --2- 172.21.15.90:0/1933622308 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26cc069aa0 0x7f26cc102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 -- 172.21.15.90:0/1933622308 >> 172.21.15.90:0/1933622308 conn(0x7f26cc0fa7a0 msgr2=0x7f26cc0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:19.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 -- 172.21.15.90:0/1933622308 shutdown_connections 2024-04-26T04:41:19.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.440+0000 7f26d0812700 1 -- 172.21.15.90:0/1933622308 wait complete. 2024-04-26T04:41:19.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.441+0000 7f26d0812700 1 Processor -- start 2024-04-26T04:41:19.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26d0812700 1 -- start start 2024-04-26T04:41:19.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26d0812700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26cc069aa0 0x7f26cc195da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:19.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26d0812700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:19.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26d0812700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc196900 con 0x7f26cc103390 2024-04-26T04:41:19.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26d0812700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26cc196a40 con 0x7f26cc069aa0 2024-04-26T04:41:19.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26c9d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:19.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26c9d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1962e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46880/0 (socket says 172.21.15.90:46880) 2024-04-26T04:41:19.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26c9d9b700 1 -- 172.21.15.90:0/4088873779 learned_addr learned my addr 172.21.15.90:0/4088873779 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:19.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26ca59c700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26cc069aa0 0x7f26cc195da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:19.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26c9d9b700 1 -- 172.21.15.90:0/4088873779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26cc069aa0 msgr2=0x7f26cc195da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26c9d9b700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26cc069aa0 0x7f26cc195da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26c9d9b700 1 -- 172.21.15.90:0/4088873779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c00095f0 con 0x7f26cc103390 2024-04-26T04:41:19.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.442+0000 7f26c9d9b700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1962e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f26c0000c00 tx=0x7f26c000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:19.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.443+0000 7f26bb7fe700 1 -- 172.21.15.90:0/4088873779 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c001d070 con 0x7f26cc103390 2024-04-26T04:41:19.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.443+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26cc19b490 con 0x7f26cc103390 2024-04-26T04:41:19.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.443+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26cc19b980 con 0x7f26cc103390 2024-04-26T04:41:19.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.443+0000 7f26bb7fe700 1 -- 172.21.15.90:0/4088873779 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f26c0005dc0 con 0x7f26cc103390 2024-04-26T04:41:19.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.443+0000 7f26bb7fe700 1 -- 172.21.15.90:0/4088873779 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f26c0022640 con 0x7f26cc103390 2024-04-26T04:41:19.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.444+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26ac005320 con 0x7f26cc103390 2024-04-26T04:41:19.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.445+0000 7f26bb7fe700 1 -- 172.21.15.90:0/4088873779 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f26c00227a0 con 0x7f26cc103390 2024-04-26T04:41:19.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.445+0000 7f26bb7fe700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26b406c680 0x7f26b406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:19.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.445+0000 7f26bb7fe700 1 -- 172.21.15.90:0/4088873779 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f26c008cd40 con 0x7f26cc103390 2024-04-26T04:41:19.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.445+0000 7f26ca59c700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26b406c680 0x7f26b406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:19.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.446+0000 7f26ca59c700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26b406c680 0x7f26b406eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0697d0 tx=0x7f26bc009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:19.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.450+0000 7f26bb7fe700 1 -- 172.21.15.90:0/4088873779 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f26c005ae30 con 0x7f26cc103390 2024-04-26T04:41:19.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:19.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:19.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:19.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: Upgrade: Target is version 19.0.0-2447-gb22e2ebd (unknown) 2024-04-26T04:41:19.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6'] 2024-04-26T04:41:19.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:41:19.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:19.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: Upgrade: Need to upgrade myself (mgr.smithi090.iwrvwa) 2024-04-26T04:41:19.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:19 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1246530037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:41:19.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.811+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f26ac000bf0 con 0x7f26b406c680 2024-04-26T04:41:19.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.813+0000 7f26bb7fe700 1 -- 172.21.15.90:0/4088873779 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f26ac000bf0 con 0x7f26b406c680 2024-04-26T04:41:19.814 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:41:19.814 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:41:19.814 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:41:19.814 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:41:19.814 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-26T04:41:19.814 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "0/2 daemons upgraded", 2024-04-26T04:41:19.815 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 image on host smithi104", 2024-04-26T04:41:19.815 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:41:19.815 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:41:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26b406c680 msgr2=0x7f26b406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26b406c680 0x7f26b406eb30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f26cc0697d0 tx=0x7f26bc009260 comp rx=0 tx=0).stop 2024-04-26T04:41:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 msgr2=0x7f26cc1962e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:19.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1962e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f26c0000c00 tx=0x7f26c000f740 comp rx=0 tx=0).stop 2024-04-26T04:41:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 shutdown_connections 2024-04-26T04:41:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f26b406c680 0x7f26b406eb30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f26cc069aa0 0x7f26cc195da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 --2- 172.21.15.90:0/4088873779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f26cc103390 0x7f26cc1962e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.816+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 >> 172.21.15.90:0/4088873779 conn(0x7f26cc0fa7a0 msgr2=0x7f26cc0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:19.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.817+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 shutdown_connections 2024-04-26T04:41:19.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:19.817+0000 7f26d0812700 1 -- 172.21.15.90:0/4088873779 wait complete. 2024-04-26T04:41:20.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: from='client.14560 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:20.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: from='client.14564 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:20.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:20.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: Upgrade: Target is version 19.0.0-2447-gb22e2ebd (unknown) 2024-04-26T04:41:20.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6'] 2024-04-26T04:41:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:41:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:41:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: Upgrade: Need to upgrade myself (mgr.smithi090.iwrvwa) 2024-04-26T04:41:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:19 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1246530037' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:41:20.929 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:20 smithi090 ceph-mon[26648]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:20.929 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:20 smithi090 ceph-mon[26648]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 on smithi104 2024-04-26T04:41:20.929 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:20 smithi090 ceph-mon[26648]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:21.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:20 smithi104 ceph-mon[32369]: from='client.14568 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:21.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:20 smithi104 ceph-mon[32369]: Upgrade: Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 on smithi104 2024-04-26T04:41:21.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:20 smithi104 ceph-mon[32369]: pgmap v222: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:21 smithi090 ceph-mon[26648]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:22.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:21 smithi104 ceph-mon[32369]: from='client.14576 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:22.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:22 smithi090 ceph-mon[26648]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:23.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:22 smithi104 ceph-mon[32369]: pgmap v223: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:24.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:24 smithi090 ceph-mon[26648]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:24.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:24 smithi104 ceph-mon[32369]: pgmap v224: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:26.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:26 smithi090 ceph-mon[26648]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:26.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:26 smithi104 ceph-mon[32369]: pgmap v225: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:28.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:28 smithi090 ceph-mon[26648]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:28 smithi104 ceph-mon[32369]: pgmap v226: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:30.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:30 smithi090 ceph-mon[26648]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:30.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:30 smithi104 ceph-mon[32369]: pgmap v227: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:32.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:32 smithi090 ceph-mon[26648]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:32.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:32 smithi104 ceph-mon[32369]: pgmap v228: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:34 smithi090 ceph-mon[26648]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:34 smithi104 ceph-mon[32369]: pgmap v229: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:36 smithi090 ceph-mon[26648]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:36.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:36 smithi104 ceph-mon[32369]: pgmap v230: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:38.658 INFO:tasks.workunit.client.1.smithi104.stderr:Note: switching to 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'. 2024-04-26T04:41:38.658 INFO:tasks.workunit.client.1.smithi104.stderr: 2024-04-26T04:41:38.658 INFO:tasks.workunit.client.1.smithi104.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T04:41:38.658 INFO:tasks.workunit.client.1.smithi104.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T04:41:38.658 INFO:tasks.workunit.client.1.smithi104.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: git switch -c 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr:Or undo this operation with: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: git switch - 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr: 2024-04-26T04:41:38.659 INFO:tasks.workunit.client.1.smithi104.stderr:HEAD is now at b22e2ebdeb2 Merge pull request #57078 from adk3798/squid-cephadm-no-downgrade 2024-04-26T04:41:38.662 DEBUG:teuthology.orchestra.run.smithi104:> cd -- /home/ubuntu/cephtest/clone.client.1/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.1 2024-04-26T04:41:38.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:38 smithi090 ceph-mon[26648]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:38.725 INFO:tasks.workunit.client.1.smithi104.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T04:41:38.727 INFO:tasks.workunit.client.1.smithi104.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-26T04:41:38.727 INFO:tasks.workunit.client.1.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T04:41:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:38 smithi104 ceph-mon[32369]: pgmap v231: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:39.282 INFO:tasks.workunit.client.0.smithi090.stderr:Note: switching to 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'. 2024-04-26T04:41:39.282 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T04:41:39.282 INFO:tasks.workunit.client.0.smithi090.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr:state without impacting any branches by switching back to a branch. 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: git switch -c 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr:Or undo this operation with: 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: git switch - 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-26T04:41:39.283 INFO:tasks.workunit.client.0.smithi090.stderr: 2024-04-26T04:41:39.284 INFO:tasks.workunit.client.0.smithi090.stderr:HEAD is now at b22e2ebdeb2 Merge pull request #57078 from adk3798/squid-cephadm-no-downgrade 2024-04-26T04:41:39.286 DEBUG:teuthology.orchestra.run.smithi090:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-26T04:41:39.315 INFO:tasks.workunit.client.0.smithi090.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-26T04:41:39.317 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T04:41:39.317 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-26T04:41:39.364 INFO:tasks.workunit.client.1.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T04:41:39.421 INFO:tasks.workunit.client.1.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T04:41:39.460 INFO:tasks.workunit.client.1.smithi104.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/direct_io' 2024-04-26T04:41:39.461 INFO:tasks.workunit.client.1.smithi104.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-26T04:41:39.461 INFO:tasks.workunit.client.1.smithi104.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T04:41:39.506 INFO:tasks.workunit.client.1.smithi104.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.1/qa/workunits/fs' 2024-04-26T04:41:39.509 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:41:39.509 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/home/ubuntu/cephtest/workunits.list.client.1 of=/dev/stdout 2024-04-26T04:41:39.567 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.1... 2024-04-26T04:41:39.569 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-26T04:41:39.570 DEBUG:teuthology.orchestra.run.smithi104:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && cd -- /home/ubuntu/cephtest/mnt.1/client.1/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b22e2ebdeb24376882b7bda2a7329c8cccc2276a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="1" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.1 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.1 CEPH_MNT=/home/ubuntu/cephtest/mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.1/qa/workunits/suites/fsstress.sh 2024-04-26T04:41:39.628 INFO:tasks.workunit.client.1.smithi104.stderr:+ mkdir -p fsstress 2024-04-26T04:41:39.630 INFO:tasks.workunit.client.1.smithi104.stderr:+ pushd fsstress 2024-04-26T04:41:39.631 INFO:tasks.workunit.client.1.smithi104.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:41:39.631 INFO:tasks.workunit.client.1.smithi104.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-26T04:41:40.047 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-26T04:41:40.122 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-26T04:41:40.162 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-26T04:41:40.163 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T04:41:40.164 INFO:tasks.workunit.client.0.smithi090.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-26T04:41:40.201 INFO:tasks.workunit.client.0.smithi090.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-26T04:41:40.204 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:41:40.205 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-26T04:41:40.205 INFO:tasks.workunit.client.1.smithi104.stderr:+ tar xzf ltp-full.tgz 2024-04-26T04:41:40.260 INFO:tasks.workunit:Running workunits matching suites/fsstress.sh on client.0... 2024-04-26T04:41:40.261 INFO:tasks.workunit:Running workunit suites/fsstress.sh... 2024-04-26T04:41:40.261 DEBUG:teuthology.orchestra.run.smithi090:workunit test suites/fsstress.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=b22e2ebdeb24376882b7bda2a7329c8cccc2276a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsstress.sh 2024-04-26T04:41:40.320 INFO:tasks.workunit.client.0.smithi090.stderr:+ mkdir -p fsstress 2024-04-26T04:41:40.322 INFO:tasks.workunit.client.0.smithi090.stderr:+ pushd fsstress 2024-04-26T04:41:40.323 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:41:40.323 INFO:tasks.workunit.client.0.smithi090.stderr:+ wget -q -O ltp-full.tgz http://download.ceph.com/qa/ltp-full-20091231.tgz 2024-04-26T04:41:40.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:40 smithi090 ceph-mon[26648]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:40.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:40 smithi104 ceph-mon[32369]: pgmap v232: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:41.244 INFO:tasks.workunit.client.0.smithi090.stderr:+ tar xzf ltp-full.tgz 2024-04-26T04:41:42.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:42 smithi090 ceph-mon[26648]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:42.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:42 smithi104 ceph-mon[32369]: pgmap v233: 65 pgs: 65 active+clean; 589 KiB data, 162 MiB used, 536 GiB / 536 GiB avail 2024-04-26T04:41:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:44 smithi090 ceph-mon[26648]: pgmap v234: 65 pgs: 65 active+clean; 4.6 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-04-26T04:41:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:44 smithi104 ceph-mon[32369]: pgmap v234: 65 pgs: 65 active+clean; 4.6 MiB data, 170 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-04-26T04:41:46.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:46 smithi090 ceph-mon[26648]: pgmap v235: 65 pgs: 65 active+clean; 4.6 MiB data, 177 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-04-26T04:41:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:46 smithi104 ceph-mon[32369]: pgmap v235: 65 pgs: 65 active+clean; 4.6 MiB data, 177 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s wr, 0 op/s 2024-04-26T04:41:47.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:47 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:41:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:47 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:41:48.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:48 smithi090 ceph-mon[26648]: pgmap v236: 65 pgs: 65 active+clean; 28 MiB data, 221 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s wr, 118 op/s 2024-04-26T04:41:48.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:48 smithi104 ceph-mon[32369]: pgmap v236: 65 pgs: 65 active+clean; 28 MiB data, 221 MiB used, 536 GiB / 536 GiB avail; 2.3 MiB/s wr, 118 op/s 2024-04-26T04:41:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:41:49.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 -- 172.21.15.90:0/1052003625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 msgr2=0x7f3a14100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:49.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/1052003625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14100840 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f3a04009910 tx=0x7f3a04009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:49.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 -- 172.21.15.90:0/1052003625 shutdown_connections 2024-04-26T04:41:49.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/1052003625 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14100840 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/1052003625 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a140ff130 0x7f3a140ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 -- 172.21.15.90:0/1052003625 >> 172.21.15.90:0/1052003625 conn(0x7f3a140fa780 msgr2=0x7f3a140fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 -- 172.21.15.90:0/1052003625 shutdown_connections 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.950+0000 7f3a1bff2700 1 -- 172.21.15.90:0/1052003625 wait complete. 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1bff2700 1 Processor -- start 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1bff2700 1 -- start start 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1bff2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a140ff130 0x7f3a14195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1bff2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1bff2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a141965c0 con 0x7f3a141003d0 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1bff2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a14196700 con 0x7f3a140ff130 2024-04-26T04:41:49.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1958d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a19d8e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a140ff130 0x7f3a14195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1958d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34006/0 (socket says 172.21.15.90:34006) 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.951+0000 7f3a1958d700 1 -- 172.21.15.90:0/468194412 learned_addr learned my addr 172.21.15.90:0/468194412 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a1958d700 1 -- 172.21.15.90:0/468194412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a140ff130 msgr2=0x7f3a14195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a1958d700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a140ff130 0x7f3a14195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a1958d700 1 -- 172.21.15.90:0/468194412 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a040095f0 con 0x7f3a141003d0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a19d8e700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a140ff130 0x7f3a14195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a1958d700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14195fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a04000c00 tx=0x7f3a0400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a0affd700 1 -- 172.21.15.90:0/468194412 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a0401d070 con 0x7f3a141003d0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a1419b150 con 0x7f3a141003d0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a0affd700 1 -- 172.21.15.90:0/468194412 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f3a04005dc0 con 0x7f3a141003d0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a0affd700 1 -- 172.21.15.90:0/468194412 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a040226a0 con 0x7f3a141003d0 2024-04-26T04:41:49.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.952+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a1419b640 con 0x7f3a141003d0 2024-04-26T04:41:49.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.953+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a14107c10 con 0x7f3a141003d0 2024-04-26T04:41:49.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.954+0000 7f3a0affd700 1 -- 172.21.15.90:0/468194412 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f3a040038f0 con 0x7f3a141003d0 2024-04-26T04:41:49.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.954+0000 7f3a0affd700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3a0006c680 0x7f3a0006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:49.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.954+0000 7f3a0affd700 1 -- 172.21.15.90:0/468194412 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f3a0408cd40 con 0x7f3a141003d0 2024-04-26T04:41:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.954+0000 7f3a19d8e700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3a0006c680 0x7f3a0006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:49.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.955+0000 7f3a19d8e700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3a0006c680 0x7f3a0006eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a10011490 tx=0x7f3a10009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:49.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:49.962+0000 7f3a0affd700 1 -- 172.21.15.90:0/468194412 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f3a0405ae50 con 0x7f3a141003d0 2024-04-26T04:41:50.304 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.303+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a14068c50 con 0x7f3a0006c680 2024-04-26T04:41:50.305 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.304+0000 7f3a0affd700 1 -- 172.21.15.90:0/468194412 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f3a14068c50 con 0x7f3a0006c680 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3a0006c680 msgr2=0x7f3a0006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3a0006c680 0x7f3a0006eb30 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f3a10011490 tx=0x7f3a10009260 comp rx=0 tx=0).stop 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 msgr2=0x7f3a14195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14195fa0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3a04000c00 tx=0x7f3a0400f690 comp rx=0 tx=0).stop 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 shutdown_connections 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f3a0006c680 0x7f3a0006eb30 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a140ff130 0x7f3a14195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.308 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 --2- 172.21.15.90:0/468194412 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a141003d0 0x7f3a14195fa0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 >> 172.21.15.90:0/468194412 conn(0x7f3a140fa780 msgr2=0x7f3a14068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:50.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 shutdown_connections 2024-04-26T04:41:50.309 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.307+0000 7f3a1bff2700 1 -- 172.21.15.90:0/468194412 wait complete. 2024-04-26T04:41:50.321 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 -- 172.21.15.90:0/3647092855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 msgr2=0x7fac28100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 --2- 172.21.15.90:0/3647092855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28100790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fac18009910 tx=0x7fac18009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 -- 172.21.15.90:0/3647092855 shutdown_connections 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 --2- 172.21.15.90:0/3647092855 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28100790 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 --2- 172.21.15.90:0/3647092855 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac280ff140 0x7fac280ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 -- 172.21.15.90:0/3647092855 >> 172.21.15.90:0/3647092855 conn(0x7fac280fa6d0 msgr2=0x7fac280fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 -- 172.21.15.90:0/3647092855 shutdown_connections 2024-04-26T04:41:50.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.429+0000 7fac2f629700 1 -- 172.21.15.90:0/3647092855 wait complete. 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.430+0000 7fac2f629700 1 Processor -- start 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.430+0000 7fac2f629700 1 -- start start 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.430+0000 7fac2f629700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac280ff140 0x7fac28195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.430+0000 7fac2f629700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.430+0000 7fac2f629700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac28196610 con 0x7fac28100340 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.430+0000 7fac2f629700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac28196750 con 0x7fac280ff140 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.430+0000 7fac2cbc4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2cbc4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34028/0 (socket says 172.21.15.90:34028) 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2cbc4700 1 -- 172.21.15.90:0/427358031 learned_addr learned my addr 172.21.15.90:0/427358031 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2d3c5700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac280ff140 0x7fac28195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2cbc4700 1 -- 172.21.15.90:0/427358031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac280ff140 msgr2=0x7fac28195ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2cbc4700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac280ff140 0x7fac28195ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2cbc4700 1 -- 172.21.15.90:0/427358031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac180095f0 con 0x7fac28100340 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2d3c5700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac280ff140 0x7fac28195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2cbc4700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28195ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac18000c00 tx=0x7fac1800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac1e7fc700 1 -- 172.21.15.90:0/427358031 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac1801d070 con 0x7fac28100340 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac28101560 con 0x7fac28100340 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac1e7fc700 1 -- 172.21.15.90:0/427358031 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fac18005dc0 con 0x7fac28100340 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac1e7fc700 1 -- 172.21.15.90:0/427358031 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fac18022640 con 0x7fac28100340 2024-04-26T04:41:50.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.431+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac28101a50 con 0x7fac28100340 2024-04-26T04:41:50.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.433+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac28107c90 con 0x7fac28100340 2024-04-26T04:41:50.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.433+0000 7fac1e7fc700 1 -- 172.21.15.90:0/427358031 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fac180038f0 con 0x7fac28100340 2024-04-26T04:41:50.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.434+0000 7fac1e7fc700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fac1406c680 0x7fac1406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:50.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.434+0000 7fac1e7fc700 1 -- 172.21.15.90:0/427358031 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fac1808cda0 con 0x7fac28100340 2024-04-26T04:41:50.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.434+0000 7fac2d3c5700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fac1406c680 0x7fac1406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:50.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.434+0000 7fac2d3c5700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fac1406c680 0x7fac1406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fac281001a0 tx=0x7fac24012040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:50.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.441+0000 7fac1e7fc700 1 -- 172.21.15.90:0/427358031 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fac1805aeb0 con 0x7fac28100340 2024-04-26T04:41:50.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.679+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fac280694e0 con 0x7fac1406c680 2024-04-26T04:41:50.680 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:50 smithi090 ceph-mon[26648]: pgmap v237: 65 pgs: 65 active+clean; 63 MiB data, 370 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s wr, 433 op/s 2024-04-26T04:41:50.681 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.680+0000 7fac1e7fc700 1 -- 172.21.15.90:0/427358031 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7fac280694e0 con 0x7fac1406c680 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fac1406c680 msgr2=0x7fac1406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fac1406c680 0x7fac1406eb30 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fac281001a0 tx=0x7fac24012040 comp rx=0 tx=0).stop 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 msgr2=0x7fac28195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28195ff0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fac18000c00 tx=0x7fac1800f690 comp rx=0 tx=0).stop 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 shutdown_connections 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fac1406c680 0x7fac1406eb30 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fac280ff140 0x7fac28195ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.682+0000 7fac2f629700 1 --2- 172.21.15.90:0/427358031 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fac28100340 0x7fac28195ff0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.683+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 >> 172.21.15.90:0/427358031 conn(0x7fac280fa6d0 msgr2=0x7fac28068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.683+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 shutdown_connections 2024-04-26T04:41:50.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.683+0000 7fac2f629700 1 -- 172.21.15.90:0/427358031 wait complete. 2024-04-26T04:41:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:50 smithi104 ceph-mon[32369]: pgmap v237: 65 pgs: 65 active+clean; 63 MiB data, 370 MiB used, 536 GiB / 536 GiB avail; 5.4 MiB/s wr, 433 op/s 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.800+0000 7f88a0b23700 1 -- 172.21.15.90:0/1763342741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c069270 msgr2=0x7f889c102dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.800+0000 7f88a0b23700 1 --2- 172.21.15.90:0/1763342741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c069270 0x7f889c102dd0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f888c009910 tx=0x7f888c009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.800+0000 7f88a0b23700 1 -- 172.21.15.90:0/1763342741 shutdown_connections 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.800+0000 7f88a0b23700 1 --2- 172.21.15.90:0/1763342741 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c103310 0x7f889c105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.800+0000 7f88a0b23700 1 --2- 172.21.15.90:0/1763342741 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c069270 0x7f889c102dd0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.800+0000 7f88a0b23700 1 -- 172.21.15.90:0/1763342741 >> 172.21.15.90:0/1763342741 conn(0x7f889c0fa780 msgr2=0x7f889c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.800+0000 7f88a0b23700 1 -- 172.21.15.90:0/1763342741 shutdown_connections 2024-04-26T04:41:50.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.801+0000 7f88a0b23700 1 -- 172.21.15.90:0/1763342741 wait complete. 2024-04-26T04:41:50.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.801+0000 7f88a0b23700 1 Processor -- start 2024-04-26T04:41:50.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.801+0000 7f88a0b23700 1 -- start start 2024-04-26T04:41:50.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.801+0000 7f88a0b23700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c103310 0x7f889c100720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:50.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.801+0000 7f88a0b23700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c0fed70 0x7f889c0ff1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:50.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f88a0b23700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0ff720 con 0x7f889c0fed70 2024-04-26T04:41:50.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f88a0b23700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f889c0ff890 con 0x7f889c103310 2024-04-26T04:41:50.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f889a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c103310 0x7f889c100720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:50.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f8899d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c0fed70 0x7f889c0ff1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:50.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f889a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c103310 0x7f889c100720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.90:60826/0 (socket says 172.21.15.90:60826) 2024-04-26T04:41:50.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f889a59c700 1 -- 172.21.15.90:0/3097460382 learned_addr learned my addr 172.21.15.90:0/3097460382 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:50.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f8899d9b700 1 -- 172.21.15.90:0/3097460382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c103310 msgr2=0x7f889c100720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:50.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f8899d9b700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c103310 0x7f889c100720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f8899d9b700 1 -- 172.21.15.90:0/3097460382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f888c0095f0 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f8899d9b700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c0fed70 0x7f889c0ff1e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f889000d6d0 tx=0x7f889000d9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f888b7fe700 1 -- 172.21.15.90:0/3097460382 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88900041f0 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f889c0ffb70 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f889c19dd40 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f888b7fe700 1 -- 172.21.15.90:0/3097460382 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8890004ce0 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.802+0000 7f888b7fe700 1 -- 172.21.15.90:0/3097460382 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f889000f690 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.804+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f887c005320 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.804+0000 7f888b7fe700 1 -- 172.21.15.90:0/3097460382 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8890004fc0 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.804+0000 7f888b7fe700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f888406c680 0x7f888406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.804+0000 7f888b7fe700 1 -- 172.21.15.90:0/3097460382 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f889008b2d0 con 0x7f889c0fed70 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.805+0000 7f889a59c700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f888406c680 0x7f888406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:50.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.805+0000 7f889a59c700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f888406c680 0x7f888406eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f888c006010 tx=0x7f888c00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:50.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:50.813+0000 7f888b7fe700 1 -- 172.21.15.90:0/3097460382 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8890059460 con 0x7f889c0fed70 2024-04-26T04:41:51.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.026+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f887c000bf0 con 0x7f888406c680 2024-04-26T04:41:51.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.036+0000 7f888b7fe700 1 -- 172.21.15.90:0/3097460382 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3528 (secure 0 0 0) 0x7f887c000bf0 con 0x7f888406c680 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (5m) 97s ago 7m 25.9M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (7m) 97s ago 7m 7407k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (6m) 101s ago 6m 7335k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (7m) 97s ago 7m 6660k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (6m) 101s ago 6m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (5m) 97s ago 7m 83.8M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (115s) 97s ago 114s 15.6M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (2m) 97s ago 2m 11.2M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (112s) 101s ago 112s 12.2M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (119s) 101s ago 118s 18.4M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:9283,8765,8443 running (8m) 97s ago 8m 487M - 18.2.0 dc2bc1663786 a008eb0a2d7e 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (6m) 101s ago 6m 437M - 18.2.0 dc2bc1663786 b809d29d97b5 2024-04-26T04:41:51.038 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (8m) 97s ago 8m 49.1M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (6m) 101s ago 6m 35.3M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (7m) 97s ago 7m 14.3M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (6m) 101s ago 6m 15.5M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (5m) 97s ago 5m 48.1M 3946M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (5m) 97s ago 5m 47.6M 3946M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (4m) 97s ago 4m 47.3M 3946M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (4m) 101s ago 4m 45.4M 4970M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (3m) 101s ago 3m 44.8M 4970M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (3m) 101s ago 3m 44.3M 4970M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:41:51.039 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (5m) 97s ago 6m 60.0M - 2.43.0 a07b618ecd1d 8382d6553d6c 2024-04-26T04:41:51.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.038+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f888406c680 msgr2=0x7f888406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.038+0000 7f88a0b23700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f888406c680 0x7f888406eb30 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f888c006010 tx=0x7f888c00b540 comp rx=0 tx=0).stop 2024-04-26T04:41:51.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.038+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c0fed70 msgr2=0x7f889c0ff1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.038+0000 7f88a0b23700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c0fed70 0x7f889c0ff1e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f889000d6d0 tx=0x7f889000d9e0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.038+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 shutdown_connections 2024-04-26T04:41:51.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.039+0000 7f88a0b23700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f888406c680 0x7f888406eb30 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.039+0000 7f88a0b23700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f889c103310 0x7f889c100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.039+0000 7f88a0b23700 1 --2- 172.21.15.90:0/3097460382 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f889c0fed70 0x7f889c0ff1e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.039+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 >> 172.21.15.90:0/3097460382 conn(0x7f889c0fa780 msgr2=0x7f889c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:51.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.039+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 shutdown_connections 2024-04-26T04:41:51.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.039+0000 7f88a0b23700 1 -- 172.21.15.90:0/3097460382 wait complete. 2024-04-26T04:41:51.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 -- 172.21.15.90:0/456892240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b41014e0 msgr2=0x7fb0b4101950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 --2- 172.21.15.90:0/456892240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b41014e0 0x7fb0b4101950 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009910 tx=0x7fb0a8009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:51.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 -- 172.21.15.90:0/456892240 shutdown_connections 2024-04-26T04:41:51.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 --2- 172.21.15.90:0/456892240 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b41014e0 0x7fb0b4101950 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 --2- 172.21.15.90:0/456892240 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b4100240 0x7fb0b4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 -- 172.21.15.90:0/456892240 >> 172.21.15.90:0/456892240 conn(0x7fb0b40fb890 msgr2=0x7fb0b40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:51.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 -- 172.21.15.90:0/456892240 shutdown_connections 2024-04-26T04:41:51.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.176+0000 7fb0b957b700 1 -- 172.21.15.90:0/456892240 wait complete. 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b957b700 1 Processor -- start 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b957b700 1 -- start start 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b957b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b4100240 0x7fb0b4074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b957b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b41014e0 0x7fb0b4073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b957b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b4073680 con 0x7fb0b4100240 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b957b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0b40737c0 con 0x7fb0b41014e0 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b4100240 0x7fb0b4074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b4100240 0x7fb0b4074af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34044/0 (socket says 172.21.15.90:34044) 2024-04-26T04:41:51.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b2ffd700 1 -- 172.21.15.90:0/3607726403 learned_addr learned my addr 172.21.15.90:0/3607726403 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.177+0000 7fb0b27fc700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b41014e0 0x7fb0b4073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb0b2ffd700 1 -- 172.21.15.90:0/3607726403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b41014e0 msgr2=0x7fb0b4073140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb0b2ffd700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b41014e0 0x7fb0b4073140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb0b2ffd700 1 -- 172.21.15.90:0/3607726403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0a80095f0 con 0x7fb0b4100240 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb0b2ffd700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b4100240 0x7fb0b4074af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e920 tx=0x7fb0a400ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb09bfff700 1 -- 172.21.15.90:0/3607726403 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0a400c9d0 con 0x7fb0b4100240 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0b4073aa0 con 0x7fb0b4100240 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb09bfff700 1 -- 172.21.15.90:0/3607726403 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb0a4010430 con 0x7fb0b4100240 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb09bfff700 1 -- 172.21.15.90:0/3607726403 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb0a4018650 con 0x7fb0b4100240 2024-04-26T04:41:51.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.178+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0b419daf0 con 0x7fb0b4100240 2024-04-26T04:41:51.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.180+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0b4108d20 con 0x7fb0b4100240 2024-04-26T04:41:51.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.181+0000 7fb09bfff700 1 -- 172.21.15.90:0/3607726403 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb0a40105a0 con 0x7fb0b4100240 2024-04-26T04:41:51.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.181+0000 7fb09bfff700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb09c06c680 0x7fb09c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:51.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.181+0000 7fb09bfff700 1 -- 172.21.15.90:0/3607726403 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb0a4014070 con 0x7fb0b4100240 2024-04-26T04:41:51.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.181+0000 7fb0b27fc700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb09c06c680 0x7fb09c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:51.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.182+0000 7fb0b27fc700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb09c06c680 0x7fb09c06eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009fd0 tx=0x7fb0a8005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:51.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.189+0000 7fb09bfff700 1 -- 172.21.15.90:0/3607726403 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb0a4051720 con 0x7fb0b4100240 2024-04-26T04:41:51.573 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.572+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb0b419e0a0 con 0x7fb0b4100240 2024-04-26T04:41:51.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.572+0000 7fb09bfff700 1 -- 172.21.15.90:0/3607726403 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fb0a408be60 con 0x7fb0b4100240 2024-04-26T04:41:51.574 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:41:51.574 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:41:51.574 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:41:51.574 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:51.574 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:41:51.575 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:41:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb09c06c680 msgr2=0x7fb09c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb09c06c680 0x7fb09c06eb30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fb0a8009fd0 tx=0x7fb0a8005940 comp rx=0 tx=0).stop 2024-04-26T04:41:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b4100240 msgr2=0x7fb0b4074af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b4100240 0x7fb0b4074af0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fb0a400e920 tx=0x7fb0a400ec30 comp rx=0 tx=0).stop 2024-04-26T04:41:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 shutdown_connections 2024-04-26T04:41:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb09c06c680 0x7fb09c06eb30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb0b4100240 0x7fb0b4074af0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 --2- 172.21.15.90:0/3607726403 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb0b41014e0 0x7fb0b4073140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.574+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 >> 172.21.15.90:0/3607726403 conn(0x7fb0b40fb890 msgr2=0x7fb0b4104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.575+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 shutdown_connections 2024-04-26T04:41:51.577 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.575+0000 7fb0b957b700 1 -- 172.21.15.90:0/3607726403 wait complete. 2024-04-26T04:41:51.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 -- 172.21.15.90:0/3752138255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 msgr2=0x7f5d00103140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3752138255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d00103140 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc009910 tx=0x7f5cfc009c20 comp rx=0 tx=0).stop 2024-04-26T04:41:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 -- 172.21.15.90:0/3752138255 shutdown_connections 2024-04-26T04:41:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3752138255 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d00103680 0x7f5d00105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3752138255 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d00103140 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 -- 172.21.15.90:0/3752138255 >> 172.21.15.90:0/3752138255 conn(0x7f5d000fa780 msgr2=0x7f5d000fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:51.700 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:51 smithi090 ceph-mon[26648]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:51.700 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:51 smithi090 ceph-mon[26648]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:51.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 -- 172.21.15.90:0/3752138255 shutdown_connections 2024-04-26T04:41:51.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.698+0000 7f5d0810c700 1 -- 172.21.15.90:0/3752138255 wait complete. 2024-04-26T04:41:51.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d0810c700 1 Processor -- start 2024-04-26T04:41:51.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d0810c700 1 -- start start 2024-04-26T04:41:51.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d0810c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d001006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:51.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d0810c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d00103680 0x7f5d000fed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d0810c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000ff250 con 0x7f5d00100d60 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d0810c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d000ff390 con 0x7f5d00103680 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d05ea8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d001006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d05ea8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d001006c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34066/0 (socket says 172.21.15.90:34066) 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.699+0000 7f5d05ea8700 1 -- 172.21.15.90:0/3293233867 learned_addr learned my addr 172.21.15.90:0/3293233867 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5d056a7700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d00103680 0x7f5d000fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5d05ea8700 1 -- 172.21.15.90:0/3293233867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d00103680 msgr2=0x7f5d000fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5d05ea8700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d00103680 0x7f5d000fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5d05ea8700 1 -- 172.21.15.90:0/3293233867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cfc0095f0 con 0x7f5d00100d60 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5d05ea8700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d001006c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00bda0 tx=0x7f5cfc00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5cf6ffd700 1 -- 172.21.15.90:0/3293233867 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cfc01d070 con 0x7f5d00100d60 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d000ff610 con 0x7f5d00100d60 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5cf6ffd700 1 -- 172.21.15.90:0/3293233867 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f5cfc00f460 con 0x7f5d00100d60 2024-04-26T04:41:51.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5cf6ffd700 1 -- 172.21.15.90:0/3293233867 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5cfc021640 con 0x7f5d00100d60 2024-04-26T04:41:51.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.700+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d000ffb00 con 0x7f5d00100d60 2024-04-26T04:41:51.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.701+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d0018fbc0 con 0x7f5d00100d60 2024-04-26T04:41:51.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.702+0000 7f5cf6ffd700 1 -- 172.21.15.90:0/3293233867 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f5cfc00f5d0 con 0x7f5d00100d60 2024-04-26T04:41:51.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.702+0000 7f5cf6ffd700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5cec06c680 0x7f5cec06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:41:51.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.703+0000 7f5cf6ffd700 1 -- 172.21.15.90:0/3293233867 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f5cfc08cb60 con 0x7f5d00100d60 2024-04-26T04:41:51.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.703+0000 7f5d056a7700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5cec06c680 0x7f5cec06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:41:51.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.703+0000 7f5d056a7700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5cec06c680 0x7f5cec06eb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007be0 tx=0x7f5cf00073d0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:41:51.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.707+0000 7f5cf6ffd700 1 -- 172.21.15.90:0/3293233867 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f5cfc05ac70 con 0x7f5d00100d60 2024-04-26T04:41:51.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:51 smithi104 ceph-mon[32369]: from='client.14580 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:51.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:51 smithi104 ceph-mon[32369]: from='client.14584 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:51.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.947+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d00061190 con 0x7f5cec06c680 2024-04-26T04:41:51.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.948+0000 7f5cf6ffd700 1 -- 172.21.15.90:0/3293233867 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f5d00061190 con 0x7f5cec06c680 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "0/2 daemons upgraded", 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 image on host smithi104", 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:41:51.950 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.951+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5cec06c680 msgr2=0x7f5cec06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5cec06c680 0x7f5cec06eb30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f5cf0007be0 tx=0x7f5cf00073d0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 msgr2=0x7f5d001006c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d001006c0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc00bda0 tx=0x7f5cfc00bdd0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 shutdown_connections 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f5cec06c680 0x7f5cec06eb30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5d00100d60 0x7f5d001006c0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 --2- 172.21.15.90:0/3293233867 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5d00103680 0x7f5d000fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 >> 172.21.15.90:0/3293233867 conn(0x7f5d000fa780 msgr2=0x7f5d000fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:41:51.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 shutdown_connections 2024-04-26T04:41:51.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:41:51.952+0000 7f5d0810c700 1 -- 172.21.15.90:0/3293233867 wait complete. 2024-04-26T04:41:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:52 smithi090 ceph-mon[26648]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:52 smithi090 ceph-mon[26648]: pgmap v238: 65 pgs: 65 active+clean; 66 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s wr, 488 op/s 2024-04-26T04:41:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:52 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3607726403' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:41:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:52 smithi104 ceph-mon[32369]: from='client.14588 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:52 smithi104 ceph-mon[32369]: pgmap v238: 65 pgs: 65 active+clean; 66 MiB data, 412 MiB used, 536 GiB / 536 GiB avail; 5.7 MiB/s wr, 488 op/s 2024-04-26T04:41:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:52 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3607726403' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:41:53.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:53 smithi090 ceph-mon[26648]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:53.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:53 smithi104 ceph-mon[32369]: from='client.14596 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:41:54.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:54 smithi090 ceph-mon[26648]: pgmap v239: 65 pgs: 65 active+clean; 75 MiB data, 434 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s wr, 503 op/s 2024-04-26T04:41:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:54 smithi104 ceph-mon[32369]: pgmap v239: 65 pgs: 65 active+clean; 75 MiB data, 434 MiB used, 536 GiB / 536 GiB avail; 6.5 MiB/s wr, 503 op/s 2024-04-26T04:41:56.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:56 smithi104 ceph-mon[32369]: pgmap v240: 65 pgs: 65 active+clean; 75 MiB data, 448 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s wr, 502 op/s 2024-04-26T04:41:56.684 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:56 smithi090 ceph-mon[26648]: pgmap v240: 65 pgs: 65 active+clean; 75 MiB data, 448 MiB used, 536 GiB / 536 GiB avail; 6.2 MiB/s wr, 502 op/s 2024-04-26T04:41:58.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:41:58 smithi090 ceph-mon[26648]: pgmap v241: 65 pgs: 65 active+clean; 92 MiB data, 483 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s wr, 567 op/s 2024-04-26T04:41:58.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:41:58 smithi104 ceph-mon[32369]: pgmap v241: 65 pgs: 65 active+clean; 92 MiB data, 483 MiB used, 536 GiB / 536 GiB avail; 7.6 MiB/s wr, 567 op/s 2024-04-26T04:41:58.822 INFO:tasks.workunit.client.1.smithi104.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-26T04:41:58.823 INFO:tasks.workunit.client.1.smithi104.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:41:58.823 INFO:tasks.workunit.client.1.smithi104.stderr:+ make 2024-04-26T04:41:58.907 INFO:tasks.workunit.client.1.smithi104.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-26T04:41:59.941 INFO:tasks.workunit.client.1.smithi104.stderr:++ readlink -f fsstress 2024-04-26T04:41:59.942 INFO:tasks.workunit.client.1.smithi104.stderr:+ BIN=/home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-26T04:41:59.942 INFO:tasks.workunit.client.1.smithi104.stderr:+ popd 2024-04-26T04:41:59.942 INFO:tasks.workunit.client.1.smithi104.stdout:~/cephtest/mnt.1/client.1/tmp/fsstress ~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:41:59.942 INFO:tasks.workunit.client.1.smithi104.stderr:+ popd 2024-04-26T04:41:59.943 INFO:tasks.workunit.client.1.smithi104.stdout:~/cephtest/mnt.1/client.1/tmp 2024-04-26T04:41:59.943 INFO:tasks.workunit.client.1.smithi104.stderr:++ mktemp -d -p . 2024-04-26T04:41:59.944 INFO:tasks.workunit.client.1.smithi104.stderr:+ T=./tmp.3rns7cWTtV 2024-04-26T04:41:59.944 INFO:tasks.workunit.client.1.smithi104.stderr:+ /home/ubuntu/cephtest/mnt.1/client.1/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.3rns7cWTtV -l 1 -n 1000 -p 10 -v 2024-04-26T04:41:59.945 INFO:tasks.workunit.client.1.smithi104.stdout:seed = 1713724788 2024-04-26T04:41:59.948 INFO:tasks.workunit.client.1.smithi104.stdout:1/0: rename - no filename 2024-04-26T04:41:59.948 INFO:tasks.workunit.client.1.smithi104.stdout:1/1: dread - no filename 2024-04-26T04:41:59.948 INFO:tasks.workunit.client.1.smithi104.stdout:1/2: unlink - no file 2024-04-26T04:41:59.948 INFO:tasks.workunit.client.1.smithi104.stdout:0/0: mkdir d0 0 2024-04-26T04:41:59.948 INFO:tasks.workunit.client.1.smithi104.stdout:0/1: dwrite - no filename 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:0/2: truncate - no filename 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:0/3: fsync - no filename 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:0/4: chown d0 496 1 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:0/5: rename d0 to d0/d1 22 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:0/6: dread - no filename 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:2/0: rename - no filename 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:1/3: mkdir d0 0 2024-04-26T04:41:59.949 INFO:tasks.workunit.client.1.smithi104.stdout:1/4: write - no filename 2024-04-26T04:41:59.950 INFO:tasks.workunit.client.1.smithi104.stdout:0/7: mkdir d0/d2 0 2024-04-26T04:41:59.950 INFO:tasks.workunit.client.1.smithi104.stdout:0/8: write - no filename 2024-04-26T04:41:59.950 INFO:tasks.workunit.client.1.smithi104.stdout:0/9: write - no filename 2024-04-26T04:41:59.950 INFO:tasks.workunit.client.1.smithi104.stdout:0/10: write - no filename 2024-04-26T04:41:59.950 INFO:tasks.workunit.client.1.smithi104.stdout:0/11: link - no file 2024-04-26T04:41:59.950 INFO:tasks.workunit.client.1.smithi104.stdout:0/12: link - no file 2024-04-26T04:41:59.951 INFO:tasks.workunit.client.1.smithi104.stdout:2/1: symlink l0 0 2024-04-26T04:41:59.951 INFO:tasks.workunit.client.1.smithi104.stdout:1/5: symlink d0/l1 0 2024-04-26T04:41:59.952 INFO:tasks.workunit.client.1.smithi104.stdout:0/13: mknod d0/c3 0 2024-04-26T04:41:59.952 INFO:tasks.workunit.client.1.smithi104.stdout:0/14: dwrite - no filename 2024-04-26T04:41:59.952 INFO:tasks.workunit.client.1.smithi104.stdout:0/15: dread - no filename 2024-04-26T04:41:59.952 INFO:tasks.workunit.client.1.smithi104.stdout:4/0: dread - no filename 2024-04-26T04:41:59.953 INFO:tasks.workunit.client.1.smithi104.stdout:4/1: write - no filename 2024-04-26T04:41:59.953 INFO:tasks.workunit.client.1.smithi104.stdout:3/0: creat f0 x:0 0 0 2024-04-26T04:41:59.953 INFO:tasks.workunit.client.1.smithi104.stdout:3/1: dread - f0 zero size 2024-04-26T04:41:59.953 INFO:tasks.workunit.client.1.smithi104.stdout:3/2: truncate f0 987769 0 2024-04-26T04:41:59.954 INFO:tasks.workunit.client.1.smithi104.stdout:1/6: unlink d0/l1 0 2024-04-26T04:41:59.954 INFO:tasks.workunit.client.1.smithi104.stdout:1/7: stat d0 0 2024-04-26T04:41:59.955 INFO:tasks.workunit.client.1.smithi104.stdout:0/16: mkdir d0/d4 0 2024-04-26T04:41:59.955 INFO:tasks.workunit.client.1.smithi104.stdout:0/17: dread - no filename 2024-04-26T04:41:59.955 INFO:tasks.workunit.client.1.smithi104.stdout:0/18: truncate - no filename 2024-04-26T04:41:59.955 INFO:tasks.workunit.client.1.smithi104.stdout:0/19: read - no filename 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/0: dread - no filename 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/1: rmdir - no directory 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/2: write - no filename 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/3: fsync - no filename 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/4: dread - no filename 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/5: dwrite - no filename 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/6: rmdir - no directory 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/7: dwrite - no filename 2024-04-26T04:41:59.956 INFO:tasks.workunit.client.1.smithi104.stdout:5/8: rmdir - no directory 2024-04-26T04:41:59.957 INFO:tasks.workunit.client.1.smithi104.stdout:0/20: mknod d0/c5 0 2024-04-26T04:41:59.957 INFO:tasks.workunit.client.1.smithi104.stdout:0/21: truncate - no filename 2024-04-26T04:41:59.957 INFO:tasks.workunit.client.1.smithi104.stdout:0/22: dwrite - no filename 2024-04-26T04:41:59.958 INFO:tasks.workunit.client.1.smithi104.stdout:6/0: chown . 5955 1 2024-04-26T04:41:59.958 INFO:tasks.workunit.client.1.smithi104.stdout:6/1: dwrite - no filename 2024-04-26T04:41:59.958 INFO:tasks.workunit.client.1.smithi104.stdout:6/2: dread - no filename 2024-04-26T04:41:59.959 INFO:tasks.workunit.client.1.smithi104.stdout:5/9: mkdir d0 0 2024-04-26T04:41:59.959 INFO:tasks.workunit.client.1.smithi104.stdout:5/10: stat d0 0 2024-04-26T04:41:59.959 INFO:tasks.workunit.client.1.smithi104.stdout:5/11: dwrite - no filename 2024-04-26T04:41:59.959 INFO:tasks.workunit.client.1.smithi104.stdout:5/12: chown d0 1949859 1 2024-04-26T04:41:59.960 INFO:tasks.workunit.client.1.smithi104.stdout:0/23: link d0/c5 d0/d4/c6 0 2024-04-26T04:41:59.961 INFO:tasks.workunit.client.1.smithi104.stdout:7/0: readlink - no filename 2024-04-26T04:41:59.962 INFO:tasks.workunit.client.1.smithi104.stdout:6/3: mkdir d0 0 2024-04-26T04:41:59.962 INFO:tasks.workunit.client.1.smithi104.stdout:6/4: readlink - no filename 2024-04-26T04:41:59.963 INFO:tasks.workunit.client.1.smithi104.stdout:6/5: stat d0 0 2024-04-26T04:41:59.963 INFO:tasks.workunit.client.1.smithi104.stdout:5/13: mkdir d0/d1 0 2024-04-26T04:41:59.963 INFO:tasks.workunit.client.1.smithi104.stdout:5/14: truncate - no filename 2024-04-26T04:41:59.964 INFO:tasks.workunit.client.1.smithi104.stdout:5/15: fdatasync - no filename 2024-04-26T04:41:59.964 INFO:tasks.workunit.client.1.smithi104.stdout:5/16: write - no filename 2024-04-26T04:41:59.965 INFO:tasks.workunit.client.1.smithi104.stdout:0/24: creat d0/d4/f7 x:0 0 0 2024-04-26T04:41:59.969 INFO:tasks.workunit.client.1.smithi104.stdout:8/0: stat - no entries 2024-04-26T04:41:59.970 INFO:tasks.workunit.client.1.smithi104.stdout:6/6: mkdir d0/d1 0 2024-04-26T04:41:59.970 INFO:tasks.workunit.client.1.smithi104.stdout:6/7: dwrite - no filename 2024-04-26T04:41:59.970 INFO:tasks.workunit.client.1.smithi104.stdout:6/8: getdents d0/d1 0 2024-04-26T04:41:59.970 INFO:tasks.workunit.client.1.smithi104.stdout:6/9: read - no filename 2024-04-26T04:41:59.974 INFO:tasks.workunit.client.1.smithi104.stdout:2/2: getdents . 0 2024-04-26T04:41:59.975 INFO:tasks.workunit.client.1.smithi104.stdout:2/3: fdatasync - no filename 2024-04-26T04:41:59.975 INFO:tasks.workunit.client.1.smithi104.stdout:2/4: dread - no filename 2024-04-26T04:41:59.975 INFO:tasks.workunit.client.1.smithi104.stdout:2/5: dwrite - no filename 2024-04-26T04:41:59.975 INFO:tasks.workunit.client.1.smithi104.stdout:2/6: dread - no filename 2024-04-26T04:41:59.977 INFO:tasks.workunit.client.1.smithi104.stdout:5/17: creat d0/d1/f2 x:0 0 0 2024-04-26T04:41:59.977 INFO:tasks.workunit.client.1.smithi104.stdout:5/18: truncate d0/d1/f2 963593 0 2024-04-26T04:41:59.977 INFO:tasks.workunit.client.1.smithi104.stdout:5/19: write d0/d1/f2 [1791871,6631] 0 2024-04-26T04:41:59.982 INFO:tasks.workunit.client.1.smithi104.stdout:8/1: mkdir d0 0 2024-04-26T04:41:59.982 INFO:tasks.workunit.client.1.smithi104.stdout:8/2: dwrite - no filename 2024-04-26T04:41:59.982 INFO:tasks.workunit.client.1.smithi104.stdout:8/3: chown d0 27687425 1 2024-04-26T04:41:59.983 INFO:tasks.workunit.client.1.smithi104.stdout:0/25: rename d0/c3 to d0/d2/c8 0 2024-04-26T04:41:59.983 INFO:tasks.workunit.client.1.smithi104.stdout:0/26: chown d0/d4 8043 1 2024-04-26T04:41:59.983 INFO:tasks.workunit.client.1.smithi104.stdout:0/27: write d0/d4/f7 [539665,96219] 0 2024-04-26T04:41:59.988 INFO:tasks.workunit.client.1.smithi104.stdout:8/4: creat d0/f1 x:0 0 0 2024-04-26T04:41:59.988 INFO:tasks.workunit.client.1.smithi104.stdout:0/28: symlink d0/d2/l9 0 2024-04-26T04:41:59.988 INFO:tasks.workunit.client.1.smithi104.stdout:9/0: symlink l0 0 2024-04-26T04:41:59.988 INFO:tasks.workunit.client.1.smithi104.stdout:6/10: rmdir d0/d1 0 2024-04-26T04:41:59.989 INFO:tasks.workunit.client.1.smithi104.stdout:6/11: dwrite - no filename 2024-04-26T04:41:59.989 INFO:tasks.workunit.client.1.smithi104.stdout:6/12: dread - no filename 2024-04-26T04:41:59.989 INFO:tasks.workunit.client.1.smithi104.stdout:6/13: truncate - no filename 2024-04-26T04:41:59.989 INFO:tasks.workunit.client.1.smithi104.stdout:6/14: dread - no filename 2024-04-26T04:41:59.989 INFO:tasks.workunit.client.1.smithi104.stdout:6/15: dread - no filename 2024-04-26T04:41:59.989 INFO:tasks.workunit.client.1.smithi104.stdout:6/16: chown d0 4 1 2024-04-26T04:41:59.989 INFO:tasks.workunit.client.1.smithi104.stdout:9/1: creat f1 x:0 0 0 2024-04-26T04:41:59.991 INFO:tasks.workunit.client.1.smithi104.stdout:6/17: mkdir d0/d2 0 2024-04-26T04:41:59.991 INFO:tasks.workunit.client.1.smithi104.stdout:6/18: dread - no filename 2024-04-26T04:41:59.991 INFO:tasks.workunit.client.1.smithi104.stdout:6/19: chown d0/d2 1110 1 2024-04-26T04:41:59.991 INFO:tasks.workunit.client.1.smithi104.stdout:6/20: fsync - no filename 2024-04-26T04:41:59.992 INFO:tasks.workunit.client.1.smithi104.stdout:6/21: chown d0/d2 255811803 1 2024-04-26T04:41:59.993 INFO:tasks.workunit.client.1.smithi104.stdout:6/22: mknod d0/d2/c3 0 2024-04-26T04:41:59.993 INFO:tasks.workunit.client.1.smithi104.stdout:6/23: rename d0 to d0/d2/d4 22 2024-04-26T04:41:59.993 INFO:tasks.workunit.client.1.smithi104.stdout:6/24: truncate - no filename 2024-04-26T04:41:59.993 INFO:tasks.workunit.client.1.smithi104.stdout:6/25: chown d0/d2/c3 425 1 2024-04-26T04:41:59.994 INFO:tasks.workunit.client.1.smithi104.stdout:6/26: readlink - no filename 2024-04-26T04:41:59.994 INFO:tasks.workunit.client.1.smithi104.stdout:6/27: dwrite - no filename 2024-04-26T04:42:00.195 INFO:tasks.workunit.client.1.smithi104.stdout:3/3: dwrite f0 [0,4194304] 0 2024-04-26T04:42:00.195 INFO:tasks.workunit.client.1.smithi104.stdout:3/4: truncate f0 4974026 0 2024-04-26T04:42:00.195 INFO:tasks.workunit.client.1.smithi104.stdout:3/5: rmdir - no directory 2024-04-26T04:42:00.252 INFO:tasks.workunit.client.1.smithi104.stdout:6/28: rename d0/d2 to d0/d5 0 2024-04-26T04:42:00.252 INFO:tasks.workunit.client.1.smithi104.stdout:6/29: write - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/30: dread - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/31: fdatasync - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/32: dwrite - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/33: fdatasync - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/34: truncate - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/35: rename d0 to d0/d5/d6 22 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/36: dwrite - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/37: dwrite - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/38: rename d0 to d0/d5/d7 22 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/39: truncate - no filename 2024-04-26T04:42:00.253 INFO:tasks.workunit.client.1.smithi104.stdout:6/40: truncate - no filename 2024-04-26T04:42:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:00 smithi090 ceph-mon[26648]: pgmap v242: 65 pgs: 65 active+clean; 121 MiB data, 559 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s wr, 701 op/s 2024-04-26T04:42:00.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:00 smithi104 ceph-mon[32369]: pgmap v242: 65 pgs: 65 active+clean; 121 MiB data, 559 MiB used, 536 GiB / 536 GiB avail; 8.2 MiB/s wr, 701 op/s 2024-04-26T04:42:00.929 INFO:tasks.workunit.client.1.smithi104.stdout:5/20: fdatasync d0/d1/f2 0 2024-04-26T04:42:00.929 INFO:tasks.workunit.client.1.smithi104.stdout:5/21: readlink - no filename 2024-04-26T04:42:00.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/2: dwrite f1 [0,4194304] 0 2024-04-26T04:42:00.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/3: chown f1 609280 1 2024-04-26T04:42:00.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/4: creat f2 x:0 0 0 2024-04-26T04:42:00.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/5: rmdir - no directory 2024-04-26T04:42:00.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/6: creat f3 x:0 0 0 2024-04-26T04:42:00.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/7: read - f3 zero size 2024-04-26T04:42:00.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/8: dread - f3 zero size 2024-04-26T04:42:01.383 INFO:tasks.workunit.client.1.smithi104.stdout:9/9: dwrite f3 [0,4194304] 0 2024-04-26T04:42:01.396 INFO:tasks.workunit.client.1.smithi104.stdout:9/10: write f3 [1091813,17962] 0 2024-04-26T04:42:01.397 INFO:tasks.workunit.client.1.smithi104.stdout:9/11: creat f4 x:0 0 0 2024-04-26T04:42:01.435 INFO:tasks.workunit.client.1.smithi104.stdout:9/12: dwrite f4 [0,4194304] 0 2024-04-26T04:42:01.578 INFO:tasks.workunit.client.1.smithi104.stdout:0/29: dwrite d0/d4/f7 [0,4194304] 0 2024-04-26T04:42:01.586 INFO:tasks.workunit.client.1.smithi104.stdout:0/30: write d0/d4/f7 [4061498,84131] 0 2024-04-26T04:42:01.805 INFO:tasks.workunit.client.1.smithi104.stdout:9/13: dwrite f2 [0,4194304] 0 2024-04-26T04:42:01.805 INFO:tasks.workunit.client.1.smithi104.stdout:9/14: chown f4 11580544 1 2024-04-26T04:42:02.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:02 smithi090 ceph-mon[26648]: pgmap v243: 65 pgs: 65 active+clean; 126 MiB data, 592 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s wr, 478 op/s 2024-04-26T04:42:02.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:02 smithi104 ceph-mon[32369]: pgmap v243: 65 pgs: 65 active+clean; 126 MiB data, 592 MiB used, 536 GiB / 536 GiB avail; 5.6 MiB/s wr, 478 op/s 2024-04-26T04:42:02.988 INFO:tasks.workunit.client.1.smithi104.stdout:5/22: mkdir d0/d1/d3 0 2024-04-26T04:42:02.988 INFO:tasks.workunit.client.1.smithi104.stdout:5/23: fsync d0/d1/f2 0 2024-04-26T04:42:02.988 INFO:tasks.workunit.client.1.smithi104.stdout:5/24: chown d0/d1/d3 26323683 1 2024-04-26T04:42:02.990 INFO:tasks.workunit.client.1.smithi104.stdout:5/25: write d0/d1/f2 [670235,63767] 0 2024-04-26T04:42:02.990 INFO:tasks.workunit.client.1.smithi104.stdout:5/26: stat d0/d1/d3 0 2024-04-26T04:42:03.008 INFO:tasks.workunit.client.1.smithi104.stdout:5/27: dwrite d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:03.016 INFO:tasks.workunit.client.1.smithi104.stdout:5/28: dread d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:04 smithi090 ceph-mon[26648]: pgmap v244: 65 pgs: 65 active+clean; 167 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 8.7 MiB/s wr, 725 op/s 2024-04-26T04:42:04.480 INFO:tasks.workunit.client.0.smithi090.stderr:+ pushd ltp-full-20091231/testcases/kernel/fs/fsstress 2024-04-26T04:42:04.490 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress ~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:42:04.490 INFO:tasks.workunit.client.0.smithi090.stderr:+ make 2024-04-26T04:42:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:04 smithi104 ceph-mon[32369]: pgmap v244: 65 pgs: 65 active+clean; 167 MiB data, 806 MiB used, 536 GiB / 536 GiB avail; 341 KiB/s rd, 8.7 MiB/s wr, 725 op/s 2024-04-26T04:42:04.516 INFO:tasks.workunit.client.0.smithi090.stdout:cc -DNO_XFS -I/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress -D_LARGEFILE64_SOURCE -D_GNU_SOURCE -I../../../../include -I../../../../include -L../../../../lib fsstress.c -o fsstress 2024-04-26T04:42:05.006 INFO:tasks.workunit.client.1.smithi104.stdout:0/31: creat d0/fa x:0 0 0 2024-04-26T04:42:05.321 INFO:tasks.workunit.client.1.smithi104.stdout:9/15: symlink l5 0 2024-04-26T04:42:05.323 INFO:tasks.workunit.client.1.smithi104.stdout:5/29: rename d0/d1/d3 to d0/d1/d4 0 2024-04-26T04:42:05.327 INFO:tasks.workunit.client.1.smithi104.stdout:9/16: write f1 [4109637,89268] 0 2024-04-26T04:42:05.656 INFO:tasks.workunit.client.0.smithi090.stderr:++ readlink -f fsstress 2024-04-26T04:42:05.657 INFO:tasks.workunit.client.0.smithi090.stderr:+ BIN=/home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress 2024-04-26T04:42:05.657 INFO:tasks.workunit.client.0.smithi090.stderr:+ popd 2024-04-26T04:42:05.661 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp/fsstress ~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:42:05.661 INFO:tasks.workunit.client.0.smithi090.stderr:+ popd 2024-04-26T04:42:05.661 INFO:tasks.workunit.client.0.smithi090.stdout:~/cephtest/mnt.0/client.0/tmp 2024-04-26T04:42:05.661 INFO:tasks.workunit.client.0.smithi090.stderr:++ mktemp -d -p . 2024-04-26T04:42:05.662 INFO:tasks.workunit.client.0.smithi090.stderr:+ T=./tmp.YPJSjuIeJ8 2024-04-26T04:42:05.662 INFO:tasks.workunit.client.0.smithi090.stderr:+ /home/ubuntu/cephtest/mnt.0/client.0/tmp/fsstress/ltp-full-20091231/testcases/kernel/fs/fsstress/fsstress -d ./tmp.YPJSjuIeJ8 -l 1 -n 1000 -p 10 -v 2024-04-26T04:42:05.664 INFO:tasks.workunit.client.0.smithi090.stdout:seed = 1713443341 2024-04-26T04:42:05.667 INFO:tasks.workunit.client.0.smithi090.stdout:1/0: truncate - no filename 2024-04-26T04:42:05.668 INFO:tasks.workunit.client.0.smithi090.stdout:0/0: mkdir d0 0 2024-04-26T04:42:05.668 INFO:tasks.workunit.client.0.smithi090.stdout:0/1: readlink - no filename 2024-04-26T04:42:05.668 INFO:tasks.workunit.client.0.smithi090.stdout:2/0: write - no filename 2024-04-26T04:42:05.668 INFO:tasks.workunit.client.0.smithi090.stdout:2/1: dwrite - no filename 2024-04-26T04:42:05.669 INFO:tasks.workunit.client.0.smithi090.stdout:2/2: write - no filename 2024-04-26T04:42:05.669 INFO:tasks.workunit.client.0.smithi090.stdout:2/3: chown . 0 1 2024-04-26T04:42:05.669 INFO:tasks.workunit.client.0.smithi090.stdout:1/1: mkdir d0 0 2024-04-26T04:42:05.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/2: mkdir d0/d1 0 2024-04-26T04:42:05.670 INFO:tasks.workunit.client.0.smithi090.stdout:2/4: symlink l0 0 2024-04-26T04:42:05.670 INFO:tasks.workunit.client.0.smithi090.stdout:2/5: rmdir - no directory 2024-04-26T04:42:05.671 INFO:tasks.workunit.client.0.smithi090.stdout:1/2: creat d0/f1 x:0 0 0 2024-04-26T04:42:05.671 INFO:tasks.workunit.client.0.smithi090.stdout:1/3: creat d0/f2 x:0 0 0 2024-04-26T04:42:05.671 INFO:tasks.workunit.client.0.smithi090.stdout:0/3: rmdir d0 39 2024-04-26T04:42:05.671 INFO:tasks.workunit.client.0.smithi090.stdout:0/4: write - no filename 2024-04-26T04:42:05.671 INFO:tasks.workunit.client.0.smithi090.stdout:0/5: fsync - no filename 2024-04-26T04:42:05.672 INFO:tasks.workunit.client.0.smithi090.stdout:0/6: dwrite - no filename 2024-04-26T04:42:05.672 INFO:tasks.workunit.client.0.smithi090.stdout:0/7: dwrite - no filename 2024-04-26T04:42:05.672 INFO:tasks.workunit.client.0.smithi090.stdout:0/8: dread - no filename 2024-04-26T04:42:05.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/0: readlink - no filename 2024-04-26T04:42:05.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/1: write - no filename 2024-04-26T04:42:05.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/2: chown . 2114710188 1 2024-04-26T04:42:05.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/3: getdents . 0 2024-04-26T04:42:05.673 INFO:tasks.workunit.client.0.smithi090.stdout:4/4: chown . 4953310 1 2024-04-26T04:42:05.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/0: mknod c0 0 2024-04-26T04:42:05.674 INFO:tasks.workunit.client.0.smithi090.stdout:0/9: rmdir d0/d1 0 2024-04-26T04:42:05.675 INFO:tasks.workunit.client.0.smithi090.stdout:0/10: dwrite - no filename 2024-04-26T04:42:05.675 INFO:tasks.workunit.client.0.smithi090.stdout:5/0: dread - no filename 2024-04-26T04:42:05.676 INFO:tasks.workunit.client.0.smithi090.stdout:5/1: chown . 56223 1 2024-04-26T04:42:05.676 INFO:tasks.workunit.client.0.smithi090.stdout:5/2: chown . 29284 1 2024-04-26T04:42:05.676 INFO:tasks.workunit.client.0.smithi090.stdout:5/3: chown . 21 1 2024-04-26T04:42:05.677 INFO:tasks.workunit.client.0.smithi090.stdout:4/5: creat f0 x:0 0 0 2024-04-26T04:42:05.677 INFO:tasks.workunit.client.0.smithi090.stdout:4/6: chown f0 459579690 1 2024-04-26T04:42:05.677 INFO:tasks.workunit.client.0.smithi090.stdout:4/7: stat f0 0 2024-04-26T04:42:05.677 INFO:tasks.workunit.client.0.smithi090.stdout:3/1: mkdir d1 0 2024-04-26T04:42:05.677 INFO:tasks.workunit.client.0.smithi090.stdout:3/2: dread - no filename 2024-04-26T04:42:05.679 INFO:tasks.workunit.client.0.smithi090.stdout:0/11: symlink d0/l2 0 2024-04-26T04:42:05.679 INFO:tasks.workunit.client.0.smithi090.stdout:0/12: fsync - no filename 2024-04-26T04:42:05.680 INFO:tasks.workunit.client.0.smithi090.stdout:5/4: symlink l0 0 2024-04-26T04:42:05.681 INFO:tasks.workunit.client.0.smithi090.stdout:5/5: chown l0 134 1 2024-04-26T04:42:05.681 INFO:tasks.workunit.client.0.smithi090.stdout:3/3: symlink d1/l2 0 2024-04-26T04:42:05.681 INFO:tasks.workunit.client.0.smithi090.stdout:3/4: truncate - no filename 2024-04-26T04:42:05.681 INFO:tasks.workunit.client.1.smithi104.stdout:0/32: mknod d0/cb 0 2024-04-26T04:42:05.682 INFO:tasks.workunit.client.0.smithi090.stdout:0/13: link d0/l2 d0/l3 0 2024-04-26T04:42:05.683 INFO:tasks.workunit.client.0.smithi090.stdout:7/0: chown . 5954054 1 2024-04-26T04:42:05.684 INFO:tasks.workunit.client.0.smithi090.stdout:7/1: dwrite - no filename 2024-04-26T04:42:05.684 INFO:tasks.workunit.client.0.smithi090.stdout:7/2: chown . 35 1 2024-04-26T04:42:05.684 INFO:tasks.workunit.client.0.smithi090.stdout:6/0: mknod c0 0 2024-04-26T04:42:05.686 INFO:tasks.workunit.client.0.smithi090.stdout:5/6: mkdir d1 0 2024-04-26T04:42:05.687 INFO:tasks.workunit.client.1.smithi104.stdout:5/30: getdents d0 0 2024-04-26T04:42:05.688 INFO:tasks.workunit.client.1.smithi104.stdout:5/31: truncate d0/d1/f2 4979090 0 2024-04-26T04:42:05.688 INFO:tasks.workunit.client.0.smithi090.stdout:3/5: creat d1/f3 x:0 0 0 2024-04-26T04:42:05.688 INFO:tasks.workunit.client.0.smithi090.stdout:3/6: read - d1/f3 zero size 2024-04-26T04:42:05.694 INFO:tasks.workunit.client.0.smithi090.stdout:0/14: symlink d0/l4 0 2024-04-26T04:42:05.694 INFO:tasks.workunit.client.0.smithi090.stdout:7/3: mkdir d0 0 2024-04-26T04:42:05.694 INFO:tasks.workunit.client.0.smithi090.stdout:7/4: dread - no filename 2024-04-26T04:42:05.694 INFO:tasks.workunit.client.0.smithi090.stdout:7/5: chown d0 8317368 1 2024-04-26T04:42:05.695 INFO:tasks.workunit.client.1.smithi104.stdout:9/17: unlink f3 0 2024-04-26T04:42:05.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/0: chown . 9 1 2024-04-26T04:42:05.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/1: dread - no filename 2024-04-26T04:42:05.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/2: write - no filename 2024-04-26T04:42:05.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/3: chown . 16839927 1 2024-04-26T04:42:05.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/4: fsync - no filename 2024-04-26T04:42:05.698 INFO:tasks.workunit.client.0.smithi090.stdout:8/5: rename - no filename 2024-04-26T04:42:05.699 INFO:tasks.workunit.client.1.smithi104.stdout:0/33: symlink d0/d4/lc 0 2024-04-26T04:42:05.699 INFO:tasks.workunit.client.1.smithi104.stdout:0/34: read - d0/fa zero size 2024-04-26T04:42:05.702 INFO:tasks.workunit.client.0.smithi090.stdout:6/1: creat f1 x:0 0 0 2024-04-26T04:42:05.702 INFO:tasks.workunit.client.0.smithi090.stdout:6/2: creat f2 x:0 0 0 2024-04-26T04:42:05.716 INFO:tasks.workunit.client.0.smithi090.stdout:5/7: unlink l0 0 2024-04-26T04:42:05.716 INFO:tasks.workunit.client.0.smithi090.stdout:5/8: chown d1 356399124 1 2024-04-26T04:42:05.720 INFO:tasks.workunit.client.1.smithi104.stdout:9/18: write f2 [2638618,33697] 0 2024-04-26T04:42:05.720 INFO:tasks.workunit.client.1.smithi104.stdout:9/19: creat f6 x:0 0 0 2024-04-26T04:42:05.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/7: unlink c0 0 2024-04-26T04:42:05.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/8: dread - d1/f3 zero size 2024-04-26T04:42:05.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/9: write d1/f3 [98525,18400] 0 2024-04-26T04:42:05.728 INFO:tasks.workunit.client.0.smithi090.stdout:1/4: rmdir d0 39 2024-04-26T04:42:05.728 INFO:tasks.workunit.client.0.smithi090.stdout:0/15: creat d0/f5 x:0 0 0 2024-04-26T04:42:05.729 INFO:tasks.workunit.client.0.smithi090.stdout:1/5: dread - d0/f2 zero size 2024-04-26T04:42:05.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/6: creat f0 x:0 0 0 2024-04-26T04:42:05.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/7: dread - f0 zero size 2024-04-26T04:42:05.927 INFO:tasks.workunit.client.1.smithi104.stdout:5/32: dwrite d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:06.062 INFO:tasks.workunit.client.1.smithi104.stdout:5/33: read d0/d1/f2 [2448425,120430] 0 2024-04-26T04:42:06.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/35: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:06.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/36: write d0/d4/f7 [2733481,106571] 0 2024-04-26T04:42:06.076 INFO:tasks.workunit.client.1.smithi104.stdout:0/37: fsync d0/fa 0 2024-04-26T04:42:06.085 INFO:tasks.workunit.client.1.smithi104.stdout:9/20: fsync f2 0 2024-04-26T04:42:06.110 INFO:tasks.workunit.client.1.smithi104.stdout:0/38: stat d0/d4/c6 0 2024-04-26T04:42:06.110 INFO:tasks.workunit.client.1.smithi104.stdout:0/39: stat d0/d2/c8 0 2024-04-26T04:42:06.213 INFO:tasks.workunit.client.0.smithi090.stdout:9/0: write - no filename 2024-04-26T04:42:06.213 INFO:tasks.workunit.client.0.smithi090.stdout:9/1: link - no file 2024-04-26T04:42:06.213 INFO:tasks.workunit.client.0.smithi090.stdout:9/2: dread - no filename 2024-04-26T04:42:06.388 INFO:tasks.workunit.client.1.smithi104.stdout:9/21: dread f1 [0,4194304] 0 2024-04-26T04:42:06.424 INFO:tasks.workunit.client.1.smithi104.stdout:0/40: truncate d0/fa 3244835 0 2024-04-26T04:42:06.448 INFO:tasks.workunit.client.1.smithi104.stdout:9/22: dread f1 [0,4194304] 0 2024-04-26T04:42:06.449 INFO:tasks.workunit.client.1.smithi104.stdout:9/23: dread - f6 zero size 2024-04-26T04:42:06.464 INFO:tasks.workunit.client.1.smithi104.stdout:9/24: dread f4 [0,4194304] 0 2024-04-26T04:42:06.466 INFO:tasks.workunit.client.1.smithi104.stdout:9/25: unlink l5 0 2024-04-26T04:42:06.466 INFO:tasks.workunit.client.1.smithi104.stdout:9/26: truncate f2 5155451 0 2024-04-26T04:42:06.466 INFO:tasks.workunit.client.1.smithi104.stdout:9/27: write f6 [514392,130094] 0 2024-04-26T04:42:06.529 INFO:tasks.workunit.client.1.smithi104.stdout:9/28: dread f6 [0,4194304] 0 2024-04-26T04:42:06.531 INFO:tasks.workunit.client.1.smithi104.stdout:9/29: rename f2 to f7 0 2024-04-26T04:42:06.643 INFO:tasks.workunit.client.1.smithi104.stdout:9/30: truncate f7 3137878 0 2024-04-26T04:42:06.662 INFO:tasks.workunit.client.1.smithi104.stdout:9/31: dwrite f7 [0,4194304] 0 2024-04-26T04:42:06.663 INFO:tasks.workunit.client.1.smithi104.stdout:9/32: chown f6 53 1 2024-04-26T04:42:06.670 INFO:tasks.workunit.client.1.smithi104.stdout:9/33: write f7 [3745666,119544] 0 2024-04-26T04:42:06.670 INFO:tasks.workunit.client.1.smithi104.stdout:9/34: chown f4 6209676 1 2024-04-26T04:42:06.671 INFO:tasks.workunit.client.1.smithi104.stdout:9/35: link f6 f8 0 2024-04-26T04:42:06.672 INFO:tasks.workunit.client.1.smithi104.stdout:9/36: truncate f4 1940279 0 2024-04-26T04:42:06.684 INFO:tasks.workunit.client.1.smithi104.stdout:9/37: dwrite f8 [0,4194304] 0 2024-04-26T04:42:06.691 INFO:tasks.workunit.client.1.smithi104.stdout:9/38: dread f7 [0,4194304] 0 2024-04-26T04:42:06.691 INFO:tasks.workunit.client.1.smithi104.stdout:9/39: write f4 [2337351,8541] 0 2024-04-26T04:42:06.691 INFO:tasks.workunit.client.1.smithi104.stdout:9/40: chown f7 806 1 2024-04-26T04:42:06.692 INFO:tasks.workunit.client.1.smithi104.stdout:9/41: getdents . 0 2024-04-26T04:42:06.692 INFO:tasks.workunit.client.1.smithi104.stdout:9/42: creat f9 x:0 0 0 2024-04-26T04:42:06.694 INFO:tasks.workunit.client.1.smithi104.stdout:9/43: rename f7 to fa 0 2024-04-26T04:42:06.696 INFO:tasks.workunit.client.1.smithi104.stdout:9/44: rename f1 to fb 0 2024-04-26T04:42:06.699 INFO:tasks.workunit.client.1.smithi104.stdout:9/45: getdents . 0 2024-04-26T04:42:06.714 INFO:tasks.workunit.client.1.smithi104.stdout:9/46: dwrite fa [0,4194304] 0 2024-04-26T04:42:06.718 INFO:tasks.workunit.client.1.smithi104.stdout:9/47: write f4 [619068,55662] 0 2024-04-26T04:42:06.718 INFO:tasks.workunit.client.1.smithi104.stdout:9/48: stat fa 0 2024-04-26T04:42:06.718 INFO:tasks.workunit.client.1.smithi104.stdout:9/49: dread - f9 zero size 2024-04-26T04:42:06.721 INFO:tasks.workunit.client.1.smithi104.stdout:9/50: truncate f4 2118602 0 2024-04-26T04:42:06.721 INFO:tasks.workunit.client.1.smithi104.stdout:9/51: stat f4 0 2024-04-26T04:42:06.722 INFO:tasks.workunit.client.1.smithi104.stdout:9/52: chown f4 24800 1 2024-04-26T04:42:06.722 INFO:tasks.workunit.client.1.smithi104.stdout:9/53: getdents . 0 2024-04-26T04:42:06.722 INFO:tasks.workunit.client.1.smithi104.stdout:9/54: chown fb 0 1 2024-04-26T04:42:06.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:06 smithi090 ceph-mon[26648]: pgmap v245: 65 pgs: 65 active+clean; 201 MiB data, 897 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 11 MiB/s wr, 949 op/s 2024-04-26T04:42:06.723 INFO:tasks.workunit.client.1.smithi104.stdout:9/55: write f4 [2418680,9065] 0 2024-04-26T04:42:06.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:06 smithi104 ceph-mon[32369]: pgmap v245: 65 pgs: 65 active+clean; 201 MiB data, 897 MiB used, 536 GiB / 536 GiB avail; 683 KiB/s rd, 11 MiB/s wr, 949 op/s 2024-04-26T04:42:08.565 INFO:tasks.workunit.client.1.smithi104.stdout:7/1: sync 2024-04-26T04:42:08.565 INFO:tasks.workunit.client.1.smithi104.stdout:3/6: sync 2024-04-26T04:42:08.565 INFO:tasks.workunit.client.1.smithi104.stdout:8/5: sync 2024-04-26T04:42:08.565 INFO:tasks.workunit.client.1.smithi104.stdout:4/2: sync 2024-04-26T04:42:08.565 INFO:tasks.workunit.client.1.smithi104.stdout:1/8: sync 2024-04-26T04:42:08.565 INFO:tasks.workunit.client.1.smithi104.stdout:8/6: chown d0/f1 267661 1 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:3/7: fdatasync f0 0 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:3/8: readlink - no filename 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:5/34: sync 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:6/41: sync 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:9/56: sync 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:2/7: sync 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:0/41: sync 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:2/8: read - no filename 2024-04-26T04:42:08.566 INFO:tasks.workunit.client.1.smithi104.stdout:8/7: write d0/f1 [914417,106667] 0 2024-04-26T04:42:08.567 INFO:tasks.workunit.client.1.smithi104.stdout:5/35: dread d0/d1/f2 [4194304,4194304] 0 2024-04-26T04:42:08.567 INFO:tasks.workunit.client.1.smithi104.stdout:5/36: write d0/d1/f2 [980252,2628] 0 2024-04-26T04:42:08.567 INFO:tasks.workunit.client.1.smithi104.stdout:1/9: mkdir d0/d2 0 2024-04-26T04:42:08.567 INFO:tasks.workunit.client.1.smithi104.stdout:1/10: readlink - no filename 2024-04-26T04:42:08.568 INFO:tasks.workunit.client.1.smithi104.stdout:7/2: creat f0 x:0 0 0 2024-04-26T04:42:08.568 INFO:tasks.workunit.client.1.smithi104.stdout:7/3: dread - f0 zero size 2024-04-26T04:42:08.568 INFO:tasks.workunit.client.1.smithi104.stdout:4/3: creat f0 x:0 0 0 2024-04-26T04:42:08.569 INFO:tasks.workunit.client.1.smithi104.stdout:6/42: creat d0/f8 x:0 0 0 2024-04-26T04:42:08.569 INFO:tasks.workunit.client.1.smithi104.stdout:6/43: chown d0/d5 1 1 2024-04-26T04:42:08.570 INFO:tasks.workunit.client.1.smithi104.stdout:6/44: truncate d0/f8 314240 0 2024-04-26T04:42:08.570 INFO:tasks.workunit.client.1.smithi104.stdout:9/57: rename fa to fc 0 2024-04-26T04:42:08.571 INFO:tasks.workunit.client.1.smithi104.stdout:0/42: creat d0/d2/fd x:0 0 0 2024-04-26T04:42:08.572 INFO:tasks.workunit.client.1.smithi104.stdout:2/9: creat f1 x:0 0 0 2024-04-26T04:42:08.572 INFO:tasks.workunit.client.1.smithi104.stdout:2/10: truncate f1 794307 0 2024-04-26T04:42:08.573 INFO:tasks.workunit.client.1.smithi104.stdout:8/8: symlink d0/l2 0 2024-04-26T04:42:08.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/11: write f1 [69900,84131] 0 2024-04-26T04:42:08.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/12: rmdir - no directory 2024-04-26T04:42:08.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/13: write f1 [1257065,67251] 0 2024-04-26T04:42:08.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/14: creat f2 x:0 0 0 2024-04-26T04:42:08.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/15: readlink l0 0 2024-04-26T04:42:08.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/16: write f2 [181806,45408] 0 2024-04-26T04:42:08.574 INFO:tasks.workunit.client.1.smithi104.stdout:5/37: creat d0/f5 x:0 0 0 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/11: rmdir d0/d2 0 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/12: truncate - no filename 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/13: chown d0 2417 1 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/14: fsync - no filename 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/15: write - no filename 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/16: dwrite - no filename 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/17: fdatasync - no filename 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/18: link - no file 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/19: unlink - no file 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/20: write - no filename 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:3/9: dread f0 [0,4194304] 0 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:3/10: creat f1 x:0 0 0 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:3/11: chown f1 2 1 2024-04-26T04:42:08.576 INFO:tasks.workunit.client.1.smithi104.stdout:7/4: mknod c1 0 2024-04-26T04:42:08.577 INFO:tasks.workunit.client.1.smithi104.stdout:7/5: dread - f0 zero size 2024-04-26T04:42:08.577 INFO:tasks.workunit.client.1.smithi104.stdout:4/4: rename f0 to f1 0 2024-04-26T04:42:08.578 INFO:tasks.workunit.client.1.smithi104.stdout:6/45: unlink d0/d5/c3 0 2024-04-26T04:42:08.579 INFO:tasks.workunit.client.1.smithi104.stdout:9/58: symlink ld 0 2024-04-26T04:42:08.579 INFO:tasks.workunit.client.1.smithi104.stdout:9/59: chown fc 29260053 1 2024-04-26T04:42:08.579 INFO:tasks.workunit.client.1.smithi104.stdout:9/60: chown f9 366383982 1 2024-04-26T04:42:08.579 INFO:tasks.workunit.client.1.smithi104.stdout:9/61: fdatasync f6 0 2024-04-26T04:42:08.579 INFO:tasks.workunit.client.1.smithi104.stdout:6/46: dread d0/f8 [0,4194304] 0 2024-04-26T04:42:08.579 INFO:tasks.workunit.client.1.smithi104.stdout:0/43: truncate d0/d4/f7 370403 0 2024-04-26T04:42:08.580 INFO:tasks.workunit.client.1.smithi104.stdout:8/9: symlink d0/l3 0 2024-04-26T04:42:08.580 INFO:tasks.workunit.client.1.smithi104.stdout:8/10: write d0/f1 [1654629,78245] 0 2024-04-26T04:42:08.580 INFO:tasks.workunit.client.1.smithi104.stdout:8/11: read d0/f1 [1661002,14565] 0 2024-04-26T04:42:08.581 INFO:tasks.workunit.client.1.smithi104.stdout:8/12: truncate d0/f1 1767636 0 2024-04-26T04:42:08.581 INFO:tasks.workunit.client.1.smithi104.stdout:8/13: write d0/f1 [1539422,8666] 0 2024-04-26T04:42:08.582 INFO:tasks.workunit.client.1.smithi104.stdout:2/17: rename l0 to l3 0 2024-04-26T04:42:08.582 INFO:tasks.workunit.client.1.smithi104.stdout:2/18: chown f2 1 1 2024-04-26T04:42:08.584 INFO:tasks.workunit.client.1.smithi104.stdout:1/21: mknod d0/c3 0 2024-04-26T04:42:08.584 INFO:tasks.workunit.client.1.smithi104.stdout:1/22: rename d0 to d0/d4 22 2024-04-26T04:42:08.584 INFO:tasks.workunit.client.1.smithi104.stdout:1/23: dread - no filename 2024-04-26T04:42:08.584 INFO:tasks.workunit.client.1.smithi104.stdout:1/24: dwrite - no filename 2024-04-26T04:42:08.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/12: mknod c2 0 2024-04-26T04:42:08.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/13: creat f3 x:0 0 0 2024-04-26T04:42:08.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/14: creat f4 x:0 0 0 2024-04-26T04:42:08.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/15: write f3 [255838,106339] 0 2024-04-26T04:42:08.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/16: dread - f1 zero size 2024-04-26T04:42:08.588 INFO:tasks.workunit.client.1.smithi104.stdout:7/6: mkdir d2 0 2024-04-26T04:42:08.588 INFO:tasks.workunit.client.1.smithi104.stdout:7/7: readlink - no filename 2024-04-26T04:42:08.588 INFO:tasks.workunit.client.1.smithi104.stdout:7/8: truncate f0 512921 0 2024-04-26T04:42:08.589 INFO:tasks.workunit.client.1.smithi104.stdout:7/9: dread f0 [0,4194304] 0 2024-04-26T04:42:08.589 INFO:tasks.workunit.client.1.smithi104.stdout:9/62: symlink le 0 2024-04-26T04:42:08.592 INFO:tasks.workunit.client.1.smithi104.stdout:6/47: mkdir d0/d9 0 2024-04-26T04:42:08.593 INFO:tasks.workunit.client.1.smithi104.stdout:6/48: dread d0/f8 [0,4194304] 0 2024-04-26T04:42:08.593 INFO:tasks.workunit.client.1.smithi104.stdout:6/49: fdatasync d0/f8 0 2024-04-26T04:42:08.594 INFO:tasks.workunit.client.1.smithi104.stdout:6/50: creat d0/fa x:0 0 0 2024-04-26T04:42:08.594 INFO:tasks.workunit.client.1.smithi104.stdout:6/51: read - d0/fa zero size 2024-04-26T04:42:08.594 INFO:tasks.workunit.client.1.smithi104.stdout:6/52: write d0/f8 [803261,43222] 0 2024-04-26T04:42:08.594 INFO:tasks.workunit.client.1.smithi104.stdout:6/53: readlink - no filename 2024-04-26T04:42:08.594 INFO:tasks.workunit.client.1.smithi104.stdout:6/54: dread - d0/fa zero size 2024-04-26T04:42:08.595 INFO:tasks.workunit.client.1.smithi104.stdout:8/14: symlink d0/l4 0 2024-04-26T04:42:08.595 INFO:tasks.workunit.client.1.smithi104.stdout:8/15: creat d0/f5 x:0 0 0 2024-04-26T04:42:08.596 INFO:tasks.workunit.client.1.smithi104.stdout:2/19: symlink l4 0 2024-04-26T04:42:08.597 INFO:tasks.workunit.client.1.smithi104.stdout:2/20: fdatasync f2 0 2024-04-26T04:42:08.597 INFO:tasks.workunit.client.1.smithi104.stdout:2/21: write f2 [686294,125474] 0 2024-04-26T04:42:08.597 INFO:tasks.workunit.client.1.smithi104.stdout:2/22: read f1 [242869,60421] 0 2024-04-26T04:42:08.597 INFO:tasks.workunit.client.1.smithi104.stdout:5/38: truncate d0/d1/f2 3676656 0 2024-04-26T04:42:08.597 INFO:tasks.workunit.client.1.smithi104.stdout:5/39: creat d0/f6 x:0 0 0 2024-04-26T04:42:08.598 INFO:tasks.workunit.client.1.smithi104.stdout:1/25: unlink d0/c3 0 2024-04-26T04:42:08.599 INFO:tasks.workunit.client.1.smithi104.stdout:1/26: write - no filename 2024-04-26T04:42:08.602 INFO:tasks.workunit.client.1.smithi104.stdout:9/63: creat ff x:0 0 0 2024-04-26T04:42:08.602 INFO:tasks.workunit.client.1.smithi104.stdout:9/64: chown fb 1717 1 2024-04-26T04:42:08.606 INFO:tasks.workunit.client.1.smithi104.stdout:6/55: dread d0/f8 [0,4194304] 0 2024-04-26T04:42:08.606 INFO:tasks.workunit.client.1.smithi104.stdout:6/56: dread - d0/fa zero size 2024-04-26T04:42:08.607 INFO:tasks.workunit.client.1.smithi104.stdout:8/16: symlink d0/l6 0 2024-04-26T04:42:08.607 INFO:tasks.workunit.client.1.smithi104.stdout:8/17: stat d0/l2 0 2024-04-26T04:42:08.607 INFO:tasks.workunit.client.1.smithi104.stdout:8/18: chown d0/l6 1794 1 2024-04-26T04:42:08.607 INFO:tasks.workunit.client.1.smithi104.stdout:8/19: stat d0/l3 0 2024-04-26T04:42:08.607 INFO:tasks.workunit.client.1.smithi104.stdout:8/20: chown d0/f1 33593 1 2024-04-26T04:42:08.609 INFO:tasks.workunit.client.1.smithi104.stdout:2/23: rename l4 to l5 0 2024-04-26T04:42:08.613 INFO:tasks.workunit.client.1.smithi104.stdout:7/10: dwrite f0 [0,4194304] 0 2024-04-26T04:42:08.613 INFO:tasks.workunit.client.1.smithi104.stdout:0/44: dwrite d0/d4/f7 [0,4194304] 0 2024-04-26T04:42:08.613 INFO:tasks.workunit.client.1.smithi104.stdout:0/45: stat d0 0 2024-04-26T04:42:08.614 INFO:tasks.workunit.client.1.smithi104.stdout:1/27: symlink d0/l5 0 2024-04-26T04:42:08.616 INFO:tasks.workunit.client.1.smithi104.stdout:9/65: rename f9 to f10 0 2024-04-26T04:42:08.616 INFO:tasks.workunit.client.1.smithi104.stdout:9/66: readlink ld 0 2024-04-26T04:42:08.616 INFO:tasks.workunit.client.1.smithi104.stdout:9/67: stat f6 0 2024-04-26T04:42:08.617 INFO:tasks.workunit.client.1.smithi104.stdout:2/24: mkdir d6 0 2024-04-26T04:42:08.619 INFO:tasks.workunit.client.1.smithi104.stdout:5/40: rmdir d0/d1 39 2024-04-26T04:42:08.619 INFO:tasks.workunit.client.1.smithi104.stdout:5/41: fsync d0/d1/f2 0 2024-04-26T04:42:08.619 INFO:tasks.workunit.client.1.smithi104.stdout:5/42: chown d0 355377 1 2024-04-26T04:42:08.619 INFO:tasks.workunit.client.1.smithi104.stdout:0/46: creat d0/fe x:0 0 0 2024-04-26T04:42:08.621 INFO:tasks.workunit.client.1.smithi104.stdout:1/28: mkdir d0/d6 0 2024-04-26T04:42:08.621 INFO:tasks.workunit.client.1.smithi104.stdout:2/25: symlink d6/l7 0 2024-04-26T04:42:08.622 INFO:tasks.workunit.client.1.smithi104.stdout:2/26: truncate f2 1496725 0 2024-04-26T04:42:08.622 INFO:tasks.workunit.client.1.smithi104.stdout:2/27: chown d6/l7 15512 1 2024-04-26T04:42:08.623 INFO:tasks.workunit.client.1.smithi104.stdout:1/29: mknod d0/c7 0 2024-04-26T04:42:08.623 INFO:tasks.workunit.client.1.smithi104.stdout:1/30: fdatasync - no filename 2024-04-26T04:42:08.624 INFO:tasks.workunit.client.1.smithi104.stdout:2/28: mkdir d6/d8 0 2024-04-26T04:42:08.627 INFO:tasks.workunit.client.1.smithi104.stdout:1/31: rename d0/l5 to d0/d6/l8 0 2024-04-26T04:42:08.627 INFO:tasks.workunit.client.1.smithi104.stdout:1/32: truncate - no filename 2024-04-26T04:42:08.627 INFO:tasks.workunit.client.1.smithi104.stdout:7/11: write f0 [1379897,127033] 0 2024-04-26T04:42:08.628 INFO:tasks.workunit.client.1.smithi104.stdout:2/29: mknod d6/c9 0 2024-04-26T04:42:08.629 INFO:tasks.workunit.client.1.smithi104.stdout:3/17: fsync f3 0 2024-04-26T04:42:08.629 INFO:tasks.workunit.client.1.smithi104.stdout:3/18: write f1 [574021,78411] 0 2024-04-26T04:42:08.630 INFO:tasks.workunit.client.1.smithi104.stdout:8/21: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:42:08.630 INFO:tasks.workunit.client.1.smithi104.stdout:8/22: chown d0 963259 1 2024-04-26T04:42:08.630 INFO:tasks.workunit.client.1.smithi104.stdout:8/23: chown d0/l3 272740966 1 2024-04-26T04:42:08.630 INFO:tasks.workunit.client.1.smithi104.stdout:1/33: unlink d0/c7 0 2024-04-26T04:42:08.631 INFO:tasks.workunit.client.1.smithi104.stdout:8/24: write d0/f1 [297977,94391] 0 2024-04-26T04:42:08.631 INFO:tasks.workunit.client.1.smithi104.stdout:8/25: creat d0/f7 x:0 0 0 2024-04-26T04:42:08.633 INFO:tasks.workunit.client.1.smithi104.stdout:6/57: dwrite d0/f8 [0,4194304] 0 2024-04-26T04:42:08.633 INFO:tasks.workunit.client.1.smithi104.stdout:9/68: dwrite fb [4194304,4194304] 0 2024-04-26T04:42:08.634 INFO:tasks.workunit.client.1.smithi104.stdout:2/30: write f2 [820431,66294] 0 2024-04-26T04:42:08.634 INFO:tasks.workunit.client.1.smithi104.stdout:7/12: link f0 d2/f3 0 2024-04-26T04:42:08.634 INFO:tasks.workunit.client.1.smithi104.stdout:2/31: readlink d6/l7 0 2024-04-26T04:42:08.637 INFO:tasks.workunit.client.1.smithi104.stdout:3/19: mkdir d5 0 2024-04-26T04:42:08.642 INFO:tasks.workunit.client.1.smithi104.stdout:7/13: dread d2/f3 [0,4194304] 0 2024-04-26T04:42:08.642 INFO:tasks.workunit.client.1.smithi104.stdout:7/14: write f0 [4122355,2346] 0 2024-04-26T04:42:08.643 INFO:tasks.workunit.client.1.smithi104.stdout:8/26: symlink d0/l8 0 2024-04-26T04:42:08.643 INFO:tasks.workunit.client.1.smithi104.stdout:0/47: truncate d0/d4/f7 719697 0 2024-04-26T04:42:08.645 INFO:tasks.workunit.client.1.smithi104.stdout:1/34: link d0/d6/l8 d0/d6/l9 0 2024-04-26T04:42:08.649 INFO:tasks.workunit.client.1.smithi104.stdout:6/58: rename d0/f8 to d0/fb 0 2024-04-26T04:42:08.649 INFO:tasks.workunit.client.1.smithi104.stdout:6/59: chown d0/fb 16534 1 2024-04-26T04:42:08.649 INFO:tasks.workunit.client.1.smithi104.stdout:6/60: fdatasync d0/fa 0 2024-04-26T04:42:08.652 INFO:tasks.workunit.client.1.smithi104.stdout:9/69: mkdir d11 0 2024-04-26T04:42:08.652 INFO:tasks.workunit.client.1.smithi104.stdout:9/70: truncate f10 849786 0 2024-04-26T04:42:08.652 INFO:tasks.workunit.client.1.smithi104.stdout:9/71: write fb [1054497,22] 0 2024-04-26T04:42:08.654 INFO:tasks.workunit.client.1.smithi104.stdout:2/32: rename l5 to d6/d8/la 0 2024-04-26T04:42:08.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/20: link c2 d5/c6 0 2024-04-26T04:42:08.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/21: read f1 [36061,98652] 0 2024-04-26T04:42:08.658 INFO:tasks.workunit.client.1.smithi104.stdout:0/48: mknod d0/d4/cf 0 2024-04-26T04:42:08.658 INFO:tasks.workunit.client.1.smithi104.stdout:0/49: readlink d0/d2/l9 0 2024-04-26T04:42:08.658 INFO:tasks.workunit.client.1.smithi104.stdout:0/50: chown d0 43772 1 2024-04-26T04:42:08.660 INFO:tasks.workunit.client.1.smithi104.stdout:1/35: mknod d0/d6/ca 0 2024-04-26T04:42:08.660 INFO:tasks.workunit.client.1.smithi104.stdout:1/36: dwrite - no filename 2024-04-26T04:42:08.660 INFO:tasks.workunit.client.1.smithi104.stdout:1/37: write - no filename 2024-04-26T04:42:08.664 INFO:tasks.workunit.client.1.smithi104.stdout:9/72: creat d11/f12 x:0 0 0 2024-04-26T04:42:08.666 INFO:tasks.workunit.client.1.smithi104.stdout:2/33: creat d6/d8/fb x:0 0 0 2024-04-26T04:42:08.667 INFO:tasks.workunit.client.1.smithi104.stdout:3/22: unlink d5/c6 0 2024-04-26T04:42:08.668 INFO:tasks.workunit.client.1.smithi104.stdout:8/27: truncate d0/f5 2267510 0 2024-04-26T04:42:08.669 INFO:tasks.workunit.client.1.smithi104.stdout:1/38: creat d0/d6/fb x:0 0 0 2024-04-26T04:42:08.671 INFO:tasks.workunit.client.1.smithi104.stdout:9/73: read f6 [2499727,34984] 0 2024-04-26T04:42:08.672 INFO:tasks.workunit.client.1.smithi104.stdout:7/15: truncate d2/f3 2015783 0 2024-04-26T04:42:08.673 INFO:tasks.workunit.client.1.smithi104.stdout:1/39: fsync d0/d6/fb 0 2024-04-26T04:42:08.673 INFO:tasks.workunit.client.1.smithi104.stdout:1/40: truncate d0/d6/fb 347531 0 2024-04-26T04:42:08.675 INFO:tasks.workunit.client.1.smithi104.stdout:1/41: dread d0/d6/fb [0,4194304] 0 2024-04-26T04:42:08.675 INFO:tasks.workunit.client.1.smithi104.stdout:1/42: creat d0/d6/fc x:0 0 0 2024-04-26T04:42:08.675 INFO:tasks.workunit.client.1.smithi104.stdout:6/61: rmdir d0 39 2024-04-26T04:42:08.677 INFO:tasks.workunit.client.1.smithi104.stdout:2/34: creat d6/fc x:0 0 0 2024-04-26T04:42:08.677 INFO:tasks.workunit.client.1.smithi104.stdout:2/35: chown f2 1396 1 2024-04-26T04:42:08.678 INFO:tasks.workunit.client.1.smithi104.stdout:3/23: link c2 d5/c7 0 2024-04-26T04:42:08.681 INFO:tasks.workunit.client.1.smithi104.stdout:7/16: mkdir d2/d4 0 2024-04-26T04:42:08.681 INFO:tasks.workunit.client.1.smithi104.stdout:7/17: write d2/f3 [1714924,43040] 0 2024-04-26T04:42:08.684 INFO:tasks.workunit.client.1.smithi104.stdout:6/62: link d0/fb d0/d9/fc 0 2024-04-26T04:42:08.684 INFO:tasks.workunit.client.1.smithi104.stdout:6/63: readlink - no filename 2024-04-26T04:42:08.685 INFO:tasks.workunit.client.1.smithi104.stdout:9/74: getdents d11 0 2024-04-26T04:42:08.687 INFO:tasks.workunit.client.1.smithi104.stdout:7/18: symlink d2/l5 0 2024-04-26T04:42:08.687 INFO:tasks.workunit.client.1.smithi104.stdout:7/19: stat d2/f3 0 2024-04-26T04:42:08.688 INFO:tasks.workunit.client.1.smithi104.stdout:1/43: rename d0/d6/fc to d0/fd 0 2024-04-26T04:42:08.690 INFO:tasks.workunit.client.1.smithi104.stdout:1/44: dread d0/d6/fb [0,4194304] 0 2024-04-26T04:42:08.690 INFO:tasks.workunit.client.1.smithi104.stdout:1/45: chown d0/fd 101 1 2024-04-26T04:42:08.690 INFO:tasks.workunit.client.1.smithi104.stdout:1/46: dread - d0/fd zero size 2024-04-26T04:42:08.691 INFO:tasks.workunit.client.1.smithi104.stdout:2/36: rmdir d6 39 2024-04-26T04:42:08.693 INFO:tasks.workunit.client.1.smithi104.stdout:7/20: mknod d2/d4/c6 0 2024-04-26T04:42:08.693 INFO:tasks.workunit.client.1.smithi104.stdout:6/64: write d0/d9/fc [2255553,41688] 0 2024-04-26T04:42:08.693 INFO:tasks.workunit.client.1.smithi104.stdout:7/21: mkdir d2/d4/d7 0 2024-04-26T04:42:08.694 INFO:tasks.workunit.client.1.smithi104.stdout:3/24: dwrite f3 [0,4194304] 0 2024-04-26T04:42:08.694 INFO:tasks.workunit.client.1.smithi104.stdout:3/25: fsync f0 0 2024-04-26T04:42:08.697 INFO:tasks.workunit.client.1.smithi104.stdout:1/47: rmdir d0/d6 39 2024-04-26T04:42:08.699 INFO:tasks.workunit.client.1.smithi104.stdout:9/75: write fc [315627,69738] 0 2024-04-26T04:42:08.700 INFO:tasks.workunit.client.1.smithi104.stdout:9/76: write f4 [2865088,91252] 0 2024-04-26T04:42:08.700 INFO:tasks.workunit.client.1.smithi104.stdout:9/77: stat f6 0 2024-04-26T04:42:08.700 INFO:tasks.workunit.client.1.smithi104.stdout:3/26: creat d5/f8 x:0 0 0 2024-04-26T04:42:08.703 INFO:tasks.workunit.client.1.smithi104.stdout:6/65: dread d0/d9/fc [0,4194304] 0 2024-04-26T04:42:08.703 INFO:tasks.workunit.client.1.smithi104.stdout:1/48: mkdir d0/d6/de 0 2024-04-26T04:42:08.704 INFO:tasks.workunit.client.1.smithi104.stdout:9/78: creat d11/f13 x:0 0 0 2024-04-26T04:42:08.704 INFO:tasks.workunit.client.1.smithi104.stdout:9/79: creat d11/f14 x:0 0 0 2024-04-26T04:42:08.705 INFO:tasks.workunit.client.1.smithi104.stdout:3/27: mkdir d5/d9 0 2024-04-26T04:42:08.705 INFO:tasks.workunit.client.1.smithi104.stdout:3/28: fdatasync f4 0 2024-04-26T04:42:08.706 INFO:tasks.workunit.client.1.smithi104.stdout:6/66: mknod d0/d9/cd 0 2024-04-26T04:42:08.707 INFO:tasks.workunit.client.1.smithi104.stdout:6/67: rename d0 to d0/d9/de 22 2024-04-26T04:42:08.707 INFO:tasks.workunit.client.1.smithi104.stdout:6/68: chown d0/d9/fc 96014 1 2024-04-26T04:42:08.707 INFO:tasks.workunit.client.1.smithi104.stdout:6/69: chown d0/d9 2 1 2024-04-26T04:42:08.707 INFO:tasks.workunit.client.1.smithi104.stdout:6/70: write d0/d9/fc [75516,15064] 0 2024-04-26T04:42:08.707 INFO:tasks.workunit.client.1.smithi104.stdout:2/37: dwrite d6/fc [0,4194304] 0 2024-04-26T04:42:08.709 INFO:tasks.workunit.client.1.smithi104.stdout:1/49: unlink d0/fd 0 2024-04-26T04:42:08.709 INFO:tasks.workunit.client.1.smithi104.stdout:1/50: dread d0/d6/fb [0,4194304] 0 2024-04-26T04:42:08.709 INFO:tasks.workunit.client.1.smithi104.stdout:1/51: read d0/d6/fb [279417,16398] 0 2024-04-26T04:42:08.709 INFO:tasks.workunit.client.1.smithi104.stdout:1/52: chown d0/d6/de 38110 1 2024-04-26T04:42:08.711 INFO:tasks.workunit.client.1.smithi104.stdout:9/80: mkdir d11/d15 0 2024-04-26T04:42:08.711 INFO:tasks.workunit.client.1.smithi104.stdout:9/81: readlink ld 0 2024-04-26T04:42:08.711 INFO:tasks.workunit.client.1.smithi104.stdout:3/29: creat d5/d9/fa x:0 0 0 2024-04-26T04:42:08.712 INFO:tasks.workunit.client.1.smithi104.stdout:3/30: chown d5/f8 0 1 2024-04-26T04:42:08.712 INFO:tasks.workunit.client.1.smithi104.stdout:3/31: dread - d5/d9/fa zero size 2024-04-26T04:42:08.713 INFO:tasks.workunit.client.1.smithi104.stdout:6/71: symlink d0/lf 0 2024-04-26T04:42:08.713 INFO:tasks.workunit.client.1.smithi104.stdout:6/72: fsync d0/fa 0 2024-04-26T04:42:08.713 INFO:tasks.workunit.client.1.smithi104.stdout:6/73: write d0/fa [556749,12325] 0 2024-04-26T04:42:08.714 INFO:tasks.workunit.client.1.smithi104.stdout:2/38: creat d6/fd x:0 0 0 2024-04-26T04:42:08.714 INFO:tasks.workunit.client.1.smithi104.stdout:2/39: truncate f1 1343470 0 2024-04-26T04:42:08.714 INFO:tasks.workunit.client.1.smithi104.stdout:2/40: chown d6/d8 7478 1 2024-04-26T04:42:08.716 INFO:tasks.workunit.client.1.smithi104.stdout:1/53: creat d0/d6/ff x:0 0 0 2024-04-26T04:42:08.716 INFO:tasks.workunit.client.1.smithi104.stdout:1/54: truncate d0/d6/fb 1125129 0 2024-04-26T04:42:08.716 INFO:tasks.workunit.client.1.smithi104.stdout:1/55: read - d0/d6/ff zero size 2024-04-26T04:42:08.719 INFO:tasks.workunit.client.1.smithi104.stdout:3/32: mknod d5/d9/cb 0 2024-04-26T04:42:08.719 INFO:tasks.workunit.client.1.smithi104.stdout:3/33: dread - d5/d9/fa zero size 2024-04-26T04:42:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:08 smithi090 ceph-mon[26648]: pgmap v246: 65 pgs: 65 active+clean; 217 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-04-26T04:42:08.722 INFO:tasks.workunit.client.1.smithi104.stdout:6/74: rename d0/d9/cd to d0/c10 0 2024-04-26T04:42:08.722 INFO:tasks.workunit.client.1.smithi104.stdout:6/75: write d0/fa [555195,5879] 0 2024-04-26T04:42:08.725 INFO:tasks.workunit.client.1.smithi104.stdout:9/82: rmdir d11 39 2024-04-26T04:42:08.725 INFO:tasks.workunit.client.1.smithi104.stdout:9/83: chown d11/f12 25346 1 2024-04-26T04:42:08.725 INFO:tasks.workunit.client.1.smithi104.stdout:9/84: truncate d11/f14 230360 0 2024-04-26T04:42:08.727 INFO:tasks.workunit.client.1.smithi104.stdout:6/76: rename d0/lf to d0/l11 0 2024-04-26T04:42:08.728 INFO:tasks.workunit.client.1.smithi104.stdout:1/56: rmdir d0/d6 39 2024-04-26T04:42:08.728 INFO:tasks.workunit.client.1.smithi104.stdout:1/57: chown d0/d6/ca 160407 1 2024-04-26T04:42:08.729 INFO:tasks.workunit.client.1.smithi104.stdout:9/85: unlink d11/f13 0 2024-04-26T04:42:08.730 INFO:tasks.workunit.client.1.smithi104.stdout:9/86: truncate f6 4841542 0 2024-04-26T04:42:08.730 INFO:tasks.workunit.client.1.smithi104.stdout:9/87: getdents d11/d15 0 2024-04-26T04:42:08.730 INFO:tasks.workunit.client.1.smithi104.stdout:1/58: symlink d0/d6/de/l10 0 2024-04-26T04:42:08.731 INFO:tasks.workunit.client.1.smithi104.stdout:1/59: chown d0 630937947 1 2024-04-26T04:42:08.731 INFO:tasks.workunit.client.1.smithi104.stdout:1/60: dread - d0/d6/ff zero size 2024-04-26T04:42:08.731 INFO:tasks.workunit.client.1.smithi104.stdout:1/61: write d0/d6/ff [223433,91089] 0 2024-04-26T04:42:08.731 INFO:tasks.workunit.client.1.smithi104.stdout:2/41: dwrite f2 [0,4194304] 0 2024-04-26T04:42:08.731 INFO:tasks.workunit.client.1.smithi104.stdout:6/77: fsync d0/d9/fc 0 2024-04-26T04:42:08.731 INFO:tasks.workunit.client.1.smithi104.stdout:1/62: write d0/d6/fb [717851,54336] 0 2024-04-26T04:42:08.732 INFO:tasks.workunit.client.1.smithi104.stdout:9/88: rename d11/f14 to d11/f16 0 2024-04-26T04:42:08.734 INFO:tasks.workunit.client.1.smithi104.stdout:2/42: mkdir d6/de 0 2024-04-26T04:42:08.736 INFO:tasks.workunit.client.1.smithi104.stdout:6/78: getdents d0 0 2024-04-26T04:42:08.737 INFO:tasks.workunit.client.1.smithi104.stdout:1/63: mknod d0/d6/c11 0 2024-04-26T04:42:08.738 INFO:tasks.workunit.client.1.smithi104.stdout:1/64: write d0/d6/fb [941132,98105] 0 2024-04-26T04:42:08.738 INFO:tasks.workunit.client.1.smithi104.stdout:1/65: write d0/d6/fb [737893,36051] 0 2024-04-26T04:42:08.739 INFO:tasks.workunit.client.1.smithi104.stdout:9/89: creat d11/f17 x:0 0 0 2024-04-26T04:42:08.739 INFO:tasks.workunit.client.1.smithi104.stdout:9/90: stat f10 0 2024-04-26T04:42:08.741 INFO:tasks.workunit.client.1.smithi104.stdout:1/66: mknod d0/c12 0 2024-04-26T04:42:08.741 INFO:tasks.workunit.client.1.smithi104.stdout:1/67: chown d0/d6/ca 57664171 1 2024-04-26T04:42:08.742 INFO:tasks.workunit.client.1.smithi104.stdout:6/79: dread d0/d9/fc [0,4194304] 0 2024-04-26T04:42:08.742 INFO:tasks.workunit.client.1.smithi104.stdout:6/80: chown d0/c10 0 1 2024-04-26T04:42:08.742 INFO:tasks.workunit.client.1.smithi104.stdout:1/68: rename d0/d6/c11 to d0/c13 0 2024-04-26T04:42:08.743 INFO:tasks.workunit.client.1.smithi104.stdout:6/81: mknod d0/c12 0 2024-04-26T04:42:08.745 INFO:tasks.workunit.client.1.smithi104.stdout:1/69: fdatasync d0/d6/fb 0 2024-04-26T04:42:08.745 INFO:tasks.workunit.client.1.smithi104.stdout:1/70: fsync d0/d6/ff 0 2024-04-26T04:42:08.745 INFO:tasks.workunit.client.1.smithi104.stdout:6/82: mkdir d0/d9/d13 0 2024-04-26T04:42:08.747 INFO:tasks.workunit.client.1.smithi104.stdout:6/83: dread d0/fa [0,4194304] 0 2024-04-26T04:42:08.747 INFO:tasks.workunit.client.1.smithi104.stdout:6/84: chown d0/d9/d13 323653233 1 2024-04-26T04:42:08.749 INFO:tasks.workunit.client.1.smithi104.stdout:1/71: getdents d0 0 2024-04-26T04:42:08.749 INFO:tasks.workunit.client.1.smithi104.stdout:1/72: truncate d0/d6/ff 906594 0 2024-04-26T04:42:08.750 INFO:tasks.workunit.client.1.smithi104.stdout:9/91: truncate f8 2626603 0 2024-04-26T04:42:08.750 INFO:tasks.workunit.client.1.smithi104.stdout:9/92: stat d11/d15 0 2024-04-26T04:42:08.752 INFO:tasks.workunit.client.1.smithi104.stdout:1/73: link d0/d6/ff d0/f14 0 2024-04-26T04:42:08.752 INFO:tasks.workunit.client.1.smithi104.stdout:1/74: fdatasync d0/d6/fb 0 2024-04-26T04:42:08.755 INFO:tasks.workunit.client.1.smithi104.stdout:1/75: mkdir d0/d15 0 2024-04-26T04:42:08.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:08 smithi104 ceph-mon[32369]: pgmap v246: 65 pgs: 65 active+clean; 217 MiB data, 997 MiB used, 535 GiB / 536 GiB avail; 683 KiB/s rd, 12 MiB/s wr, 1.06k op/s 2024-04-26T04:42:08.775 INFO:tasks.workunit.client.1.smithi104.stdout:6/85: dwrite d0/d9/fc [0,4194304] 0 2024-04-26T04:42:08.775 INFO:tasks.workunit.client.1.smithi104.stdout:6/86: fsync d0/fb 0 2024-04-26T04:42:08.777 INFO:tasks.workunit.client.1.smithi104.stdout:6/87: symlink d0/l14 0 2024-04-26T04:42:08.786 INFO:tasks.workunit.client.1.smithi104.stdout:1/76: dwrite d0/d6/fb [0,4194304] 0 2024-04-26T04:42:08.786 INFO:tasks.workunit.client.1.smithi104.stdout:9/93: dwrite f6 [0,4194304] 0 2024-04-26T04:42:08.787 INFO:tasks.workunit.client.1.smithi104.stdout:6/88: dread d0/fb [0,4194304] 0 2024-04-26T04:42:08.787 INFO:tasks.workunit.client.1.smithi104.stdout:6/89: stat d0/c12 0 2024-04-26T04:42:08.787 INFO:tasks.workunit.client.1.smithi104.stdout:1/77: read d0/f14 [518265,25498] 0 2024-04-26T04:42:08.795 INFO:tasks.workunit.client.1.smithi104.stdout:9/94: write f8 [2429474,102057] 0 2024-04-26T04:42:08.797 INFO:tasks.workunit.client.1.smithi104.stdout:9/95: link fc d11/d15/f18 0 2024-04-26T04:42:08.798 INFO:tasks.workunit.client.1.smithi104.stdout:9/96: write f10 [7971,40533] 0 2024-04-26T04:42:08.798 INFO:tasks.workunit.client.1.smithi104.stdout:9/97: dread - ff zero size 2024-04-26T04:42:08.800 INFO:tasks.workunit.client.1.smithi104.stdout:9/98: unlink ff 0 2024-04-26T04:42:08.800 INFO:tasks.workunit.client.1.smithi104.stdout:9/99: chown d11/f12 3793 1 2024-04-26T04:42:08.801 INFO:tasks.workunit.client.1.smithi104.stdout:9/100: stat f8 0 2024-04-26T04:42:08.801 INFO:tasks.workunit.client.1.smithi104.stdout:9/101: dread - d11/f12 zero size 2024-04-26T04:42:08.805 INFO:tasks.workunit.client.1.smithi104.stdout:9/102: dread d11/d15/f18 [0,4194304] 0 2024-04-26T04:42:08.805 INFO:tasks.workunit.client.1.smithi104.stdout:9/103: write f4 [2907162,34113] 0 2024-04-26T04:42:08.806 INFO:tasks.workunit.client.1.smithi104.stdout:9/104: symlink d11/l19 0 2024-04-26T04:42:08.810 INFO:tasks.workunit.client.1.smithi104.stdout:6/90: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:08.811 INFO:tasks.workunit.client.1.smithi104.stdout:9/105: dread fb [0,4194304] 0 2024-04-26T04:42:08.815 INFO:tasks.workunit.client.1.smithi104.stdout:6/91: write d0/fa [2570666,104792] 0 2024-04-26T04:42:08.818 INFO:tasks.workunit.client.1.smithi104.stdout:6/92: fsync d0/fa 0 2024-04-26T04:42:08.827 INFO:tasks.workunit.client.1.smithi104.stdout:9/106: dwrite d11/f12 [0,4194304] 0 2024-04-26T04:42:08.838 INFO:tasks.workunit.client.1.smithi104.stdout:6/93: dwrite d0/d9/fc [4194304,4194304] 0 2024-04-26T04:42:08.841 INFO:tasks.workunit.client.1.smithi104.stdout:9/107: write d11/f12 [2006069,35535] 0 2024-04-26T04:42:08.841 INFO:tasks.workunit.client.1.smithi104.stdout:9/108: truncate d11/f17 1036027 0 2024-04-26T04:42:08.841 INFO:tasks.workunit.client.1.smithi104.stdout:9/109: read f8 [1960036,110467] 0 2024-04-26T04:42:08.842 INFO:tasks.workunit.client.1.smithi104.stdout:9/110: rename f8 to d11/f1a 0 2024-04-26T04:42:08.842 INFO:tasks.workunit.client.1.smithi104.stdout:9/111: chown d11 31 1 2024-04-26T04:42:08.843 INFO:tasks.workunit.client.1.smithi104.stdout:9/112: mkdir d11/d15/d1b 0 2024-04-26T04:42:08.843 INFO:tasks.workunit.client.1.smithi104.stdout:9/113: chown fc 1840 1 2024-04-26T04:42:08.844 INFO:tasks.workunit.client.1.smithi104.stdout:9/114: symlink d11/l1c 0 2024-04-26T04:42:08.845 INFO:tasks.workunit.client.1.smithi104.stdout:6/94: truncate d0/fb 1381755 0 2024-04-26T04:42:08.846 INFO:tasks.workunit.client.1.smithi104.stdout:9/115: mkdir d11/d1d 0 2024-04-26T04:42:08.847 INFO:tasks.workunit.client.1.smithi104.stdout:6/95: unlink d0/fa 0 2024-04-26T04:42:08.849 INFO:tasks.workunit.client.1.smithi104.stdout:9/116: rename le to d11/d1d/l1e 0 2024-04-26T04:42:08.849 INFO:tasks.workunit.client.1.smithi104.stdout:9/117: chown l0 340 1 2024-04-26T04:42:08.849 INFO:tasks.workunit.client.1.smithi104.stdout:9/118: getdents d11/d15/d1b 0 2024-04-26T04:42:08.849 INFO:tasks.workunit.client.1.smithi104.stdout:6/96: mkdir d0/d9/d13/d15 0 2024-04-26T04:42:08.852 INFO:tasks.workunit.client.1.smithi104.stdout:9/119: creat d11/d15/f1f x:0 0 0 2024-04-26T04:42:08.853 INFO:tasks.workunit.client.1.smithi104.stdout:9/120: truncate fb 7005520 0 2024-04-26T04:42:08.854 INFO:tasks.workunit.client.1.smithi104.stdout:9/121: dread d11/f17 [0,4194304] 0 2024-04-26T04:42:08.867 INFO:tasks.workunit.client.1.smithi104.stdout:6/97: dwrite d0/d9/fc [0,4194304] 0 2024-04-26T04:42:08.868 INFO:tasks.workunit.client.1.smithi104.stdout:6/98: mkdir d0/d9/d13/d16 0 2024-04-26T04:42:08.878 INFO:tasks.workunit.client.1.smithi104.stdout:6/99: dread d0/d9/fc [0,4194304] 0 2024-04-26T04:42:08.879 INFO:tasks.workunit.client.1.smithi104.stdout:6/100: mknod d0/d9/d13/d15/c17 0 2024-04-26T04:42:08.884 INFO:tasks.workunit.client.1.smithi104.stdout:6/101: dread d0/d9/fc [0,4194304] 0 2024-04-26T04:42:08.885 INFO:tasks.workunit.client.1.smithi104.stdout:6/102: creat d0/d9/f18 x:0 0 0 2024-04-26T04:42:08.890 INFO:tasks.workunit.client.1.smithi104.stdout:6/103: truncate d0/fb 3416252 0 2024-04-26T04:42:08.890 INFO:tasks.workunit.client.1.smithi104.stdout:6/104: getdents d0/d9/d13/d16 0 2024-04-26T04:42:08.891 INFO:tasks.workunit.client.1.smithi104.stdout:6/105: creat d0/d9/d13/d16/f19 x:0 0 0 2024-04-26T04:42:08.891 INFO:tasks.workunit.client.1.smithi104.stdout:6/106: rename d0 to d0/d9/d13/d16/d1a 22 2024-04-26T04:42:08.892 INFO:tasks.workunit.client.1.smithi104.stdout:6/107: mkdir d0/d1b 0 2024-04-26T04:42:08.892 INFO:tasks.workunit.client.1.smithi104.stdout:6/108: write d0/d9/fc [4393244,124737] 0 2024-04-26T04:42:08.894 INFO:tasks.workunit.client.1.smithi104.stdout:6/109: rename d0/l11 to d0/d9/d13/l1c 0 2024-04-26T04:42:08.895 INFO:tasks.workunit.client.1.smithi104.stdout:6/110: mknod d0/d9/d13/c1d 0 2024-04-26T04:42:08.899 INFO:tasks.workunit.client.1.smithi104.stdout:6/111: write d0/d9/fc [3740402,50057] 0 2024-04-26T04:42:08.900 INFO:tasks.workunit.client.1.smithi104.stdout:6/112: rename d0/c12 to d0/d9/c1e 0 2024-04-26T04:42:08.901 INFO:tasks.workunit.client.1.smithi104.stdout:6/113: mkdir d0/d9/d13/d1f 0 2024-04-26T04:42:08.921 INFO:tasks.workunit.client.1.smithi104.stdout:6/114: dwrite d0/fb [0,4194304] 0 2024-04-26T04:42:08.921 INFO:tasks.workunit.client.1.smithi104.stdout:6/115: readlink d0/l14 0 2024-04-26T04:42:08.930 INFO:tasks.workunit.client.1.smithi104.stdout:6/116: dread d0/fb [0,4194304] 0 2024-04-26T04:42:08.931 INFO:tasks.workunit.client.1.smithi104.stdout:6/117: creat d0/d9/d13/f20 x:0 0 0 2024-04-26T04:42:08.931 INFO:tasks.workunit.client.1.smithi104.stdout:6/118: read - d0/d9/d13/d16/f19 zero size 2024-04-26T04:42:08.953 INFO:tasks.workunit.client.1.smithi104.stdout:6/119: dwrite d0/d9/d13/f20 [0,4194304] 0 2024-04-26T04:42:08.955 INFO:tasks.workunit.client.1.smithi104.stdout:6/120: getdents d0/d5 0 2024-04-26T04:42:08.955 INFO:tasks.workunit.client.1.smithi104.stdout:6/121: chown d0/d9/f18 0 1 2024-04-26T04:42:08.959 INFO:tasks.workunit.client.1.smithi104.stdout:6/122: truncate d0/d9/fc 1361299 0 2024-04-26T04:42:08.959 INFO:tasks.workunit.client.1.smithi104.stdout:6/123: creat d0/d9/d13/d16/f21 x:0 0 0 2024-04-26T04:42:08.959 INFO:tasks.workunit.client.1.smithi104.stdout:6/124: fdatasync d0/d9/d13/d16/f19 0 2024-04-26T04:42:08.961 INFO:tasks.workunit.client.1.smithi104.stdout:6/125: rmdir d0/d9/d13/d1f 0 2024-04-26T04:42:08.962 INFO:tasks.workunit.client.1.smithi104.stdout:6/126: creat d0/d5/f22 x:0 0 0 2024-04-26T04:42:08.965 INFO:tasks.workunit.client.1.smithi104.stdout:6/127: dread d0/d9/fc [0,4194304] 0 2024-04-26T04:42:08.965 INFO:tasks.workunit.client.1.smithi104.stdout:6/128: chown d0/d1b 246 1 2024-04-26T04:42:08.976 INFO:tasks.workunit.client.1.smithi104.stdout:6/129: dread d0/d9/d13/f20 [0,4194304] 0 2024-04-26T04:42:08.978 INFO:tasks.workunit.client.1.smithi104.stdout:6/130: dread d0/fb [0,4194304] 0 2024-04-26T04:42:08.981 INFO:tasks.workunit.client.1.smithi104.stdout:6/131: dread d0/fb [0,4194304] 0 2024-04-26T04:42:08.982 INFO:tasks.workunit.client.1.smithi104.stdout:6/132: symlink d0/d9/d13/d16/l23 0 2024-04-26T04:42:08.982 INFO:tasks.workunit.client.1.smithi104.stdout:6/133: getdents d0/d5 0 2024-04-26T04:42:08.984 INFO:tasks.workunit.client.1.smithi104.stdout:6/134: creat d0/d1b/f24 x:0 0 0 2024-04-26T04:42:08.985 INFO:tasks.workunit.client.1.smithi104.stdout:6/135: mkdir d0/d9/d25 0 2024-04-26T04:42:08.985 INFO:tasks.workunit.client.1.smithi104.stdout:6/136: write d0/d9/f18 [893908,74453] 0 2024-04-26T04:42:08.986 INFO:tasks.workunit.client.1.smithi104.stdout:6/137: mknod d0/d9/d13/d16/c26 0 2024-04-26T04:42:08.986 INFO:tasks.workunit.client.1.smithi104.stdout:6/138: rename d0/d9 to d0/d9/d27 22 2024-04-26T04:42:08.989 INFO:tasks.workunit.client.1.smithi104.stdout:6/139: getdents d0/d9/d13/d16 0 2024-04-26T04:42:08.990 INFO:tasks.workunit.client.1.smithi104.stdout:6/140: mknod d0/d9/d25/c28 0 2024-04-26T04:42:09.006 INFO:tasks.workunit.client.1.smithi104.stdout:6/141: dwrite d0/d1b/f24 [0,4194304] 0 2024-04-26T04:42:09.008 INFO:tasks.workunit.client.1.smithi104.stdout:6/142: creat d0/f29 x:0 0 0 2024-04-26T04:42:09.010 INFO:tasks.workunit.client.1.smithi104.stdout:6/143: rename d0/fb to d0/d9/f2a 0 2024-04-26T04:42:09.015 INFO:tasks.workunit.client.1.smithi104.stdout:6/144: write d0/d9/f2a [962688,15242] 0 2024-04-26T04:42:09.015 INFO:tasks.workunit.client.1.smithi104.stdout:6/145: getdents d0/d9/d13/d16 0 2024-04-26T04:42:09.015 INFO:tasks.workunit.client.1.smithi104.stdout:6/146: truncate d0/d9/d13/d16/f21 630581 0 2024-04-26T04:42:09.015 INFO:tasks.workunit.client.1.smithi104.stdout:6/147: getdents d0/d1b 0 2024-04-26T04:42:09.015 INFO:tasks.workunit.client.1.smithi104.stdout:6/148: dread - d0/d9/d13/d16/f19 zero size 2024-04-26T04:42:09.015 INFO:tasks.workunit.client.1.smithi104.stdout:6/149: creat d0/d9/d13/d15/f2b x:0 0 0 2024-04-26T04:42:09.034 INFO:tasks.workunit.client.1.smithi104.stdout:6/150: dwrite d0/d9/d13/d16/f21 [0,4194304] 0 2024-04-26T04:42:09.034 INFO:tasks.workunit.client.0.smithi090.stdout:5/9: rmdir d1 0 2024-04-26T04:42:09.035 INFO:tasks.workunit.client.0.smithi090.stdout:0/16: mknod d0/c6 0 2024-04-26T04:42:09.035 INFO:tasks.workunit.client.0.smithi090.stdout:0/17: dread - d0/f5 zero size 2024-04-26T04:42:09.035 INFO:tasks.workunit.client.0.smithi090.stdout:1/6: symlink d0/l3 0 2024-04-26T04:42:09.036 INFO:tasks.workunit.client.0.smithi090.stdout:1/7: fsync d0/f1 0 2024-04-26T04:42:09.036 INFO:tasks.workunit.client.0.smithi090.stdout:8/8: mknod c1 0 2024-04-26T04:42:09.036 INFO:tasks.workunit.client.0.smithi090.stdout:8/9: fdatasync f0 0 2024-04-26T04:42:09.036 INFO:tasks.workunit.client.0.smithi090.stdout:8/10: fsync f0 0 2024-04-26T04:42:09.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/151: creat d0/d1b/f2c x:0 0 0 2024-04-26T04:42:09.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/152: symlink d0/d5/l2d 0 2024-04-26T04:42:09.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/153: dread - d0/d9/d13/d15/f2b zero size 2024-04-26T04:42:09.038 INFO:tasks.workunit.client.1.smithi104.stdout:6/154: mknod d0/d9/d25/c2e 0 2024-04-26T04:42:09.039 INFO:tasks.workunit.client.1.smithi104.stdout:6/155: symlink d0/d9/d25/l2f 0 2024-04-26T04:42:09.040 INFO:tasks.workunit.client.1.smithi104.stdout:6/156: mknod d0/c30 0 2024-04-26T04:42:09.043 INFO:tasks.workunit.client.1.smithi104.stdout:6/157: rename d0/c30 to d0/d5/c31 0 2024-04-26T04:42:09.043 INFO:tasks.workunit.client.1.smithi104.stdout:6/158: rename d0/d9 to d0/d9/d25/d32 22 2024-04-26T04:42:09.053 INFO:tasks.workunit.client.0.smithi090.stdout:8/11: dwrite f0 [0,4194304] 0 2024-04-26T04:42:09.056 INFO:tasks.workunit.client.1.smithi104.stdout:6/159: truncate d0/d9/fc 65251 0 2024-04-26T04:42:09.058 INFO:tasks.workunit.client.1.smithi104.stdout:6/160: symlink d0/d1b/l33 0 2024-04-26T04:42:09.058 INFO:tasks.workunit.client.1.smithi104.stdout:6/161: truncate d0/f29 247188 0 2024-04-26T04:42:09.058 INFO:tasks.workunit.client.1.smithi104.stdout:6/162: chown d0/d9/f18 32 1 2024-04-26T04:42:09.058 INFO:tasks.workunit.client.1.smithi104.stdout:6/163: dread - d0/d9/d13/d16/f19 zero size 2024-04-26T04:42:09.058 INFO:tasks.workunit.client.1.smithi104.stdout:6/164: chown d0/d9/d13/l1c 604 1 2024-04-26T04:42:09.059 INFO:tasks.workunit.client.1.smithi104.stdout:6/165: creat d0/d9/d25/f34 x:0 0 0 2024-04-26T04:42:09.060 INFO:tasks.workunit.client.1.smithi104.stdout:6/166: creat d0/d5/f35 x:0 0 0 2024-04-26T04:42:09.060 INFO:tasks.workunit.client.1.smithi104.stdout:6/167: write d0/d5/f22 [20086,78008] 0 2024-04-26T04:42:09.063 INFO:tasks.workunit.client.1.smithi104.stdout:6/168: rmdir d0/d5 39 2024-04-26T04:42:09.063 INFO:tasks.workunit.client.1.smithi104.stdout:6/169: dread d0/d9/f2a [0,4194304] 0 2024-04-26T04:42:09.065 INFO:tasks.workunit.client.1.smithi104.stdout:6/170: rename d0/c10 to d0/d9/d13/c36 0 2024-04-26T04:42:09.077 INFO:tasks.workunit.client.1.smithi104.stdout:6/171: dwrite d0/d9/d25/f34 [0,4194304] 0 2024-04-26T04:42:09.079 INFO:tasks.workunit.client.1.smithi104.stdout:6/172: mknod d0/d9/d13/d16/c37 0 2024-04-26T04:42:09.080 INFO:tasks.workunit.client.1.smithi104.stdout:6/173: write d0/d5/f35 [476949,112858] 0 2024-04-26T04:42:09.096 INFO:tasks.workunit.client.1.smithi104.stdout:6/174: dwrite d0/d1b/f24 [0,4194304] 0 2024-04-26T04:42:09.096 INFO:tasks.workunit.client.1.smithi104.stdout:6/175: truncate d0/d5/f22 581969 0 2024-04-26T04:42:09.097 INFO:tasks.workunit.client.1.smithi104.stdout:6/176: mknod d0/c38 0 2024-04-26T04:42:09.097 INFO:tasks.workunit.client.1.smithi104.stdout:6/177: creat d0/d9/d25/f39 x:0 0 0 2024-04-26T04:42:09.098 INFO:tasks.workunit.client.1.smithi104.stdout:6/178: symlink d0/d9/d13/l3a 0 2024-04-26T04:42:09.098 INFO:tasks.workunit.client.1.smithi104.stdout:6/179: creat d0/d9/f3b x:0 0 0 2024-04-26T04:42:09.098 INFO:tasks.workunit.client.1.smithi104.stdout:6/180: chown d0/d9/d13/d16/f19 7614 1 2024-04-26T04:42:09.098 INFO:tasks.workunit.client.1.smithi104.stdout:6/181: chown d0/d9/d13/d16/c26 117 1 2024-04-26T04:42:09.103 INFO:tasks.workunit.client.1.smithi104.stdout:6/182: dread d0/d5/f35 [0,4194304] 0 2024-04-26T04:42:09.104 INFO:tasks.workunit.client.1.smithi104.stdout:6/183: symlink d0/d9/d13/d15/l3c 0 2024-04-26T04:42:09.110 INFO:tasks.workunit.client.1.smithi104.stdout:6/184: dread d0/d9/d25/f34 [0,4194304] 0 2024-04-26T04:42:09.111 INFO:tasks.workunit.client.1.smithi104.stdout:6/185: rename d0/d9/f18 to d0/d9/d13/f3d 0 2024-04-26T04:42:09.113 INFO:tasks.workunit.client.1.smithi104.stdout:6/186: link d0/d9/d13/f20 d0/d9/d25/f3e 0 2024-04-26T04:42:09.113 INFO:tasks.workunit.client.1.smithi104.stdout:6/187: dread - d0/d9/f3b zero size 2024-04-26T04:42:09.113 INFO:tasks.workunit.client.1.smithi104.stdout:6/188: truncate d0/d9/d13/f20 1917944 0 2024-04-26T04:42:09.129 INFO:tasks.workunit.client.1.smithi104.stdout:6/189: dwrite d0/d9/d25/f34 [0,4194304] 0 2024-04-26T04:42:09.132 INFO:tasks.workunit.client.1.smithi104.stdout:6/190: dread d0/d9/d25/f3e [0,4194304] 0 2024-04-26T04:42:09.132 INFO:tasks.workunit.client.1.smithi104.stdout:6/191: chown d0/d9/d13/d15/l3c 99 1 2024-04-26T04:42:09.149 INFO:tasks.workunit.client.1.smithi104.stdout:6/192: dwrite d0/d9/fc [0,4194304] 0 2024-04-26T04:42:09.150 INFO:tasks.workunit.client.1.smithi104.stdout:6/193: mkdir d0/d9/d13/d16/d3f 0 2024-04-26T04:42:09.151 INFO:tasks.workunit.client.1.smithi104.stdout:6/194: dread d0/d5/f35 [0,4194304] 0 2024-04-26T04:42:09.151 INFO:tasks.workunit.client.1.smithi104.stdout:6/195: creat d0/d1b/f40 x:0 0 0 2024-04-26T04:42:09.151 INFO:tasks.workunit.client.1.smithi104.stdout:6/196: write d0/d9/d13/d16/f19 [161333,102716] 0 2024-04-26T04:42:09.153 INFO:tasks.workunit.client.1.smithi104.stdout:6/197: mkdir d0/d9/d13/d41 0 2024-04-26T04:42:09.153 INFO:tasks.workunit.client.1.smithi104.stdout:6/198: write d0/d9/d13/d15/f2b [912415,53738] 0 2024-04-26T04:42:09.153 INFO:tasks.workunit.client.1.smithi104.stdout:6/199: creat d0/d9/d13/d15/f42 x:0 0 0 2024-04-26T04:42:09.153 INFO:tasks.workunit.client.1.smithi104.stdout:6/200: fsync d0/d5/f22 0 2024-04-26T04:42:09.153 INFO:tasks.workunit.client.1.smithi104.stdout:6/201: fdatasync d0/d9/d25/f39 0 2024-04-26T04:42:09.153 INFO:tasks.workunit.client.1.smithi104.stdout:6/202: creat d0/d9/d13/f43 x:0 0 0 2024-04-26T04:42:09.170 INFO:tasks.workunit.client.0.smithi090.stdout:6/3: getdents . 0 2024-04-26T04:42:09.180 INFO:tasks.workunit.client.1.smithi104.stdout:6/203: dwrite d0/d9/d13/f20 [0,4194304] 0 2024-04-26T04:42:09.188 INFO:tasks.workunit.client.0.smithi090.stdout:6/4: dwrite f2 [0,4194304] 0 2024-04-26T04:42:09.188 INFO:tasks.workunit.client.0.smithi090.stdout:6/5: chown f2 11 1 2024-04-26T04:42:09.190 INFO:tasks.workunit.client.1.smithi104.stdout:6/204: dread d0/d9/d13/d16/f21 [0,4194304] 0 2024-04-26T04:42:09.191 INFO:tasks.workunit.client.1.smithi104.stdout:6/205: mkdir d0/d9/d13/d44 0 2024-04-26T04:42:09.191 INFO:tasks.workunit.client.1.smithi104.stdout:6/206: chown d0/d9/d13 1318364252 1 2024-04-26T04:42:09.199 INFO:tasks.workunit.client.1.smithi104.stdout:6/207: write d0/d9/fc [1559547,110241] 0 2024-04-26T04:42:09.200 INFO:tasks.workunit.client.1.smithi104.stdout:6/208: mknod d0/d9/d13/d41/c45 0 2024-04-26T04:42:09.200 INFO:tasks.workunit.client.1.smithi104.stdout:6/209: mknod d0/c46 0 2024-04-26T04:42:09.201 INFO:tasks.workunit.client.1.smithi104.stdout:6/210: mkdir d0/d9/d13/d44/d47 0 2024-04-26T04:42:09.203 INFO:tasks.workunit.client.1.smithi104.stdout:6/211: getdents d0/d9/d13/d16 0 2024-04-26T04:42:09.221 INFO:tasks.workunit.client.1.smithi104.stdout:6/212: dwrite d0/d9/f2a [4194304,4194304] 0 2024-04-26T04:42:09.221 INFO:tasks.workunit.client.1.smithi104.stdout:6/213: write d0/d9/d13/d15/f42 [937346,34263] 0 2024-04-26T04:42:09.221 INFO:tasks.workunit.client.1.smithi104.stdout:6/214: fdatasync d0/d1b/f40 0 2024-04-26T04:42:09.229 INFO:tasks.workunit.client.1.smithi104.stdout:6/215: read d0/d9/d13/f20 [3528585,113611] 0 2024-04-26T04:42:09.230 INFO:tasks.workunit.client.1.smithi104.stdout:6/216: rmdir d0/d9/d25 39 2024-04-26T04:42:09.238 INFO:tasks.workunit.client.1.smithi104.stdout:6/217: write d0/d9/f2a [4646744,20464] 0 2024-04-26T04:42:09.239 INFO:tasks.workunit.client.1.smithi104.stdout:6/218: truncate d0/d9/d25/f34 68614 0 2024-04-26T04:42:09.239 INFO:tasks.workunit.client.1.smithi104.stdout:6/219: chown d0/d9/d13/l3a 26888939 1 2024-04-26T04:42:09.240 INFO:tasks.workunit.client.1.smithi104.stdout:6/220: getdents d0/d9/d13/d15 0 2024-04-26T04:42:09.240 INFO:tasks.workunit.client.1.smithi104.stdout:6/221: dread - d0/d1b/f40 zero size 2024-04-26T04:42:09.241 INFO:tasks.workunit.client.1.smithi104.stdout:6/222: symlink d0/d9/d13/d16/d3f/l48 0 2024-04-26T04:42:09.241 INFO:tasks.workunit.client.1.smithi104.stdout:6/223: creat d0/d9/f49 x:0 0 0 2024-04-26T04:42:09.241 INFO:tasks.workunit.client.1.smithi104.stdout:6/224: fsync d0/d1b/f2c 0 2024-04-26T04:42:09.242 INFO:tasks.workunit.client.1.smithi104.stdout:6/225: write d0/d9/f2a [3621137,16827] 0 2024-04-26T04:42:09.244 INFO:tasks.workunit.client.1.smithi104.stdout:6/226: fsync d0/d9/d13/d15/f42 0 2024-04-26T04:42:09.244 INFO:tasks.workunit.client.1.smithi104.stdout:6/227: creat d0/d9/f4a x:0 0 0 2024-04-26T04:42:09.261 INFO:tasks.workunit.client.1.smithi104.stdout:6/228: dwrite d0/d9/d13/f3d [0,4194304] 0 2024-04-26T04:42:09.264 INFO:tasks.workunit.client.1.smithi104.stdout:6/229: creat d0/d9/d13/d16/f4b x:0 0 0 2024-04-26T04:42:09.265 INFO:tasks.workunit.client.1.smithi104.stdout:6/230: creat d0/d9/d13/d44/d47/f4c x:0 0 0 2024-04-26T04:42:09.265 INFO:tasks.workunit.client.1.smithi104.stdout:6/231: write d0/d9/d13/d16/f21 [4633428,18153] 0 2024-04-26T04:42:09.267 INFO:tasks.workunit.client.1.smithi104.stdout:6/232: symlink d0/d9/l4d 0 2024-04-26T04:42:09.267 INFO:tasks.workunit.client.1.smithi104.stdout:6/233: chown d0/d9/d13/d15/l3c 264822 1 2024-04-26T04:42:09.267 INFO:tasks.workunit.client.1.smithi104.stdout:6/234: chown d0/d9/d13/d16/d3f 10090918 1 2024-04-26T04:42:09.268 INFO:tasks.workunit.client.1.smithi104.stdout:6/235: rename d0/d5/f35 to d0/d5/f4e 0 2024-04-26T04:42:09.269 INFO:tasks.workunit.client.1.smithi104.stdout:6/236: mknod d0/d9/d25/c4f 0 2024-04-26T04:42:09.271 INFO:tasks.workunit.client.1.smithi104.stdout:6/237: unlink d0/d1b/f40 0 2024-04-26T04:42:09.271 INFO:tasks.workunit.client.1.smithi104.stdout:6/238: chown d0/d9/d13/d15/c17 3246 1 2024-04-26T04:42:09.271 INFO:tasks.workunit.client.1.smithi104.stdout:6/239: write d0/d9/d13/d15/f2b [619353,20501] 0 2024-04-26T04:42:09.287 INFO:tasks.workunit.client.1.smithi104.stdout:6/240: dwrite d0/d1b/f24 [0,4194304] 0 2024-04-26T04:42:09.289 INFO:tasks.workunit.client.1.smithi104.stdout:6/241: rename d0/d9/d25/c28 to d0/d9/d13/d44/d47/c50 0 2024-04-26T04:42:09.406 INFO:tasks.workunit.client.1.smithi104.stdout:4/5: sync 2024-04-26T04:42:09.406 INFO:tasks.workunit.client.1.smithi104.stdout:4/6: write f1 [919096,71928] 0 2024-04-26T04:42:09.422 INFO:tasks.workunit.client.1.smithi104.stdout:4/7: dwrite f1 [0,4194304] 0 2024-04-26T04:42:09.426 INFO:tasks.workunit.client.1.smithi104.stdout:4/8: rename f1 to f2 0 2024-04-26T04:42:09.431 INFO:tasks.workunit.client.1.smithi104.stdout:4/9: dread f2 [0,4194304] 0 2024-04-26T04:42:09.431 INFO:tasks.workunit.client.1.smithi104.stdout:4/10: creat f3 x:0 0 0 2024-04-26T04:42:09.431 INFO:tasks.workunit.client.1.smithi104.stdout:4/11: dread - f3 zero size 2024-04-26T04:42:09.432 INFO:tasks.workunit.client.1.smithi104.stdout:4/12: mkdir d4 0 2024-04-26T04:42:09.454 INFO:tasks.workunit.client.1.smithi104.stdout:4/13: dwrite f3 [0,4194304] 0 2024-04-26T04:42:09.456 INFO:tasks.workunit.client.1.smithi104.stdout:4/14: mknod d4/c5 0 2024-04-26T04:42:09.457 INFO:tasks.workunit.client.1.smithi104.stdout:4/15: write f3 [4303684,34623] 0 2024-04-26T04:42:09.467 INFO:tasks.workunit.client.1.smithi104.stdout:4/16: read f3 [1931823,61134] 0 2024-04-26T04:42:09.467 INFO:tasks.workunit.client.1.smithi104.stdout:4/17: truncate f2 4206707 0 2024-04-26T04:42:09.468 INFO:tasks.workunit.client.1.smithi104.stdout:4/18: mknod d4/c6 0 2024-04-26T04:42:09.468 INFO:tasks.workunit.client.1.smithi104.stdout:4/19: mkdir d4/d7 0 2024-04-26T04:42:09.469 INFO:tasks.workunit.client.1.smithi104.stdout:4/20: creat d4/d7/f8 x:0 0 0 2024-04-26T04:42:09.469 INFO:tasks.workunit.client.1.smithi104.stdout:4/21: chown f3 120988 1 2024-04-26T04:42:09.469 INFO:tasks.workunit.client.1.smithi104.stdout:4/22: fdatasync d4/d7/f8 0 2024-04-26T04:42:09.472 INFO:tasks.workunit.client.1.smithi104.stdout:4/23: dread f2 [0,4194304] 0 2024-04-26T04:42:09.505 INFO:tasks.workunit.client.1.smithi104.stdout:4/24: dwrite f3 [0,4194304] 0 2024-04-26T04:42:09.508 INFO:tasks.workunit.client.1.smithi104.stdout:0/51: sync 2024-04-26T04:42:09.509 INFO:tasks.workunit.client.1.smithi104.stdout:5/43: sync 2024-04-26T04:42:09.509 INFO:tasks.workunit.client.1.smithi104.stdout:0/52: write d0/d2/fd [616344,103646] 0 2024-04-26T04:42:09.509 INFO:tasks.workunit.client.1.smithi104.stdout:0/53: write d0/d4/f7 [1341271,97614] 0 2024-04-26T04:42:09.512 INFO:tasks.workunit.client.1.smithi104.stdout:0/54: truncate d0/d4/f7 608073 0 2024-04-26T04:42:09.513 INFO:tasks.workunit.client.1.smithi104.stdout:0/55: read - d0/fe zero size 2024-04-26T04:42:09.515 INFO:tasks.workunit.client.1.smithi104.stdout:4/25: dread f3 [0,4194304] 0 2024-04-26T04:42:09.515 INFO:tasks.workunit.client.1.smithi104.stdout:0/56: write d0/d4/f7 [752650,39086] 0 2024-04-26T04:42:09.515 INFO:tasks.workunit.client.1.smithi104.stdout:0/57: chown d0/fa 71396685 1 2024-04-26T04:42:09.516 INFO:tasks.workunit.client.1.smithi104.stdout:0/58: mkdir d0/d2/d10 0 2024-04-26T04:42:09.516 INFO:tasks.workunit.client.1.smithi104.stdout:0/59: creat d0/d4/f11 x:0 0 0 2024-04-26T04:42:09.516 INFO:tasks.workunit.client.1.smithi104.stdout:0/60: write d0/d2/fd [638305,58673] 0 2024-04-26T04:42:09.519 INFO:tasks.workunit.client.1.smithi104.stdout:4/26: write f2 [3098342,86469] 0 2024-04-26T04:42:09.520 INFO:tasks.workunit.client.1.smithi104.stdout:4/27: dread - d4/d7/f8 zero size 2024-04-26T04:42:09.523 INFO:tasks.workunit.client.1.smithi104.stdout:5/44: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:09.527 INFO:tasks.workunit.client.1.smithi104.stdout:5/45: symlink d0/l7 0 2024-04-26T04:42:09.527 INFO:tasks.workunit.client.1.smithi104.stdout:5/46: rename d0/d1 to d0/d1/d8 22 2024-04-26T04:42:09.527 INFO:tasks.workunit.client.1.smithi104.stdout:5/47: link d0/l7 d0/d1/d4/l9 0 2024-04-26T04:42:09.528 INFO:tasks.workunit.client.1.smithi104.stdout:5/48: link d0/d1/d4/l9 d0/d1/d4/la 0 2024-04-26T04:42:09.531 INFO:tasks.workunit.client.1.smithi104.stdout:0/61: dwrite d0/fe [0,4194304] 0 2024-04-26T04:42:09.533 INFO:tasks.workunit.client.1.smithi104.stdout:0/62: creat d0/d2/d10/f12 x:0 0 0 2024-04-26T04:42:09.534 INFO:tasks.workunit.client.1.smithi104.stdout:0/63: mknod d0/d2/c13 0 2024-04-26T04:42:09.535 INFO:tasks.workunit.client.1.smithi104.stdout:0/64: fdatasync d0/d4/f7 0 2024-04-26T04:42:09.535 INFO:tasks.workunit.client.1.smithi104.stdout:0/65: creat d0/f14 x:0 0 0 2024-04-26T04:42:09.536 INFO:tasks.workunit.client.1.smithi104.stdout:0/66: mkdir d0/d4/d15 0 2024-04-26T04:42:09.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/67: rename d0/d2/c8 to d0/d4/d15/c16 0 2024-04-26T04:42:09.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/68: fdatasync d0/d2/d10/f12 0 2024-04-26T04:42:09.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/69: creat d0/f17 x:0 0 0 2024-04-26T04:42:09.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/70: chown d0/f17 2252 1 2024-04-26T04:42:09.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/71: dread - d0/d4/f11 zero size 2024-04-26T04:42:09.541 INFO:tasks.workunit.client.1.smithi104.stdout:0/72: rename d0/f14 to d0/d4/d15/f18 0 2024-04-26T04:42:09.541 INFO:tasks.workunit.client.1.smithi104.stdout:0/73: rmdir d0/d4/d15 39 2024-04-26T04:42:09.543 INFO:tasks.workunit.client.1.smithi104.stdout:0/74: mknod d0/d4/c19 0 2024-04-26T04:42:09.543 INFO:tasks.workunit.client.1.smithi104.stdout:5/49: dwrite d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:09.544 INFO:tasks.workunit.client.1.smithi104.stdout:0/75: symlink d0/l1a 0 2024-04-26T04:42:09.544 INFO:tasks.workunit.client.1.smithi104.stdout:0/76: chown d0/d2/d10/f12 1 1 2024-04-26T04:42:09.544 INFO:tasks.workunit.client.1.smithi104.stdout:0/77: write d0/d2/d10/f12 [4494,118033] 0 2024-04-26T04:42:09.552 INFO:tasks.workunit.client.1.smithi104.stdout:0/78: getdents d0/d4 0 2024-04-26T04:42:09.553 INFO:tasks.workunit.client.1.smithi104.stdout:0/79: creat d0/d4/f1b x:0 0 0 2024-04-26T04:42:09.555 INFO:tasks.workunit.client.1.smithi104.stdout:5/50: dread d0/f6 [0,4194304] 0 2024-04-26T04:42:09.555 INFO:tasks.workunit.client.1.smithi104.stdout:0/80: mknod d0/d2/d10/c1c 0 2024-04-26T04:42:09.555 INFO:tasks.workunit.client.1.smithi104.stdout:0/81: chown d0/d2/d10/c1c 16477960 1 2024-04-26T04:42:09.555 INFO:tasks.workunit.client.1.smithi104.stdout:2/43: sync 2024-04-26T04:42:09.555 INFO:tasks.workunit.client.1.smithi104.stdout:9/122: sync 2024-04-26T04:42:09.556 INFO:tasks.workunit.client.1.smithi104.stdout:1/78: sync 2024-04-26T04:42:09.556 INFO:tasks.workunit.client.1.smithi104.stdout:3/34: sync 2024-04-26T04:42:09.556 INFO:tasks.workunit.client.1.smithi104.stdout:7/22: sync 2024-04-26T04:42:09.556 INFO:tasks.workunit.client.1.smithi104.stdout:2/44: chown d6/de 97268 1 2024-04-26T04:42:09.556 INFO:tasks.workunit.client.1.smithi104.stdout:6/242: sync 2024-04-26T04:42:09.556 INFO:tasks.workunit.client.1.smithi104.stdout:8/28: sync 2024-04-26T04:42:09.556 INFO:tasks.workunit.client.1.smithi104.stdout:0/82: truncate d0/fa 1470314 0 2024-04-26T04:42:09.557 INFO:tasks.workunit.client.1.smithi104.stdout:1/79: dread d0/d6/ff [0,4194304] 0 2024-04-26T04:42:09.557 INFO:tasks.workunit.client.1.smithi104.stdout:1/80: fdatasync d0/f14 0 2024-04-26T04:42:09.557 INFO:tasks.workunit.client.1.smithi104.stdout:2/45: unlink l3 0 2024-04-26T04:42:09.557 INFO:tasks.workunit.client.1.smithi104.stdout:2/46: readlink d6/d8/la 0 2024-04-26T04:42:09.557 INFO:tasks.workunit.client.1.smithi104.stdout:2/47: truncate f2 5156600 0 2024-04-26T04:42:09.557 INFO:tasks.workunit.client.1.smithi104.stdout:2/48: truncate d6/fd 674405 0 2024-04-26T04:42:09.558 INFO:tasks.workunit.client.1.smithi104.stdout:7/23: rename d2/d4/d7 to d2/d8 0 2024-04-26T04:42:09.558 INFO:tasks.workunit.client.1.smithi104.stdout:0/83: truncate d0/d4/d15/f18 754301 0 2024-04-26T04:42:09.559 INFO:tasks.workunit.client.1.smithi104.stdout:9/123: rmdir d11 39 2024-04-26T04:42:09.560 INFO:tasks.workunit.client.1.smithi104.stdout:1/81: getdents d0/d6 0 2024-04-26T04:42:09.561 INFO:tasks.workunit.client.1.smithi104.stdout:2/49: link d6/l7 d6/de/lf 0 2024-04-26T04:42:09.562 INFO:tasks.workunit.client.1.smithi104.stdout:2/50: write d6/d8/fb [674399,64608] 0 2024-04-26T04:42:09.563 INFO:tasks.workunit.client.1.smithi104.stdout:0/84: getdents d0/d2/d10 0 2024-04-26T04:42:09.565 INFO:tasks.workunit.client.1.smithi104.stdout:0/85: mkdir d0/d1d 0 2024-04-26T04:42:09.565 INFO:tasks.workunit.client.1.smithi104.stdout:0/86: fsync d0/d4/f1b 0 2024-04-26T04:42:09.565 INFO:tasks.workunit.client.1.smithi104.stdout:0/87: write d0/f17 [364769,108879] 0 2024-04-26T04:42:09.566 INFO:tasks.workunit.client.1.smithi104.stdout:5/51: write d0/d1/f2 [2503745,42846] 0 2024-04-26T04:42:09.566 INFO:tasks.workunit.client.1.smithi104.stdout:0/88: mkdir d0/d1d/d1e 0 2024-04-26T04:42:09.567 INFO:tasks.workunit.client.1.smithi104.stdout:8/29: write d0/f5 [87443,91252] 0 2024-04-26T04:42:09.570 INFO:tasks.workunit.client.1.smithi104.stdout:3/35: write f3 [1318628,12617] 0 2024-04-26T04:42:09.570 INFO:tasks.workunit.client.1.smithi104.stdout:5/52: symlink d0/d1/d4/lb 0 2024-04-26T04:42:09.572 INFO:tasks.workunit.client.1.smithi104.stdout:0/89: dread d0/d2/fd [0,4194304] 0 2024-04-26T04:42:09.572 INFO:tasks.workunit.client.1.smithi104.stdout:5/53: symlink d0/d1/d4/lc 0 2024-04-26T04:42:09.572 INFO:tasks.workunit.client.1.smithi104.stdout:2/51: write d6/fc [2895961,10444] 0 2024-04-26T04:42:09.572 INFO:tasks.workunit.client.1.smithi104.stdout:3/36: mkdir d5/d9/dc 0 2024-04-26T04:42:09.572 INFO:tasks.workunit.client.1.smithi104.stdout:3/37: chown d5/d9/fa 4887644 1 2024-04-26T04:42:09.574 INFO:tasks.workunit.client.1.smithi104.stdout:0/90: mknod d0/d1d/c1f 0 2024-04-26T04:42:09.574 INFO:tasks.workunit.client.1.smithi104.stdout:5/54: symlink d0/d1/ld 0 2024-04-26T04:42:09.575 INFO:tasks.workunit.client.1.smithi104.stdout:2/52: unlink f1 0 2024-04-26T04:42:09.575 INFO:tasks.workunit.client.1.smithi104.stdout:2/53: creat d6/f10 x:0 0 0 2024-04-26T04:42:09.578 INFO:tasks.workunit.client.1.smithi104.stdout:3/38: rename c2 to d5/d9/cd 0 2024-04-26T04:42:09.578 INFO:tasks.workunit.client.1.smithi104.stdout:3/39: dread - f4 zero size 2024-04-26T04:42:09.578 INFO:tasks.workunit.client.1.smithi104.stdout:3/40: fsync f4 0 2024-04-26T04:42:09.580 INFO:tasks.workunit.client.1.smithi104.stdout:5/55: dread d0/f6 [0,4194304] 0 2024-04-26T04:42:09.580 INFO:tasks.workunit.client.1.smithi104.stdout:2/54: symlink d6/l11 0 2024-04-26T04:42:09.580 INFO:tasks.workunit.client.1.smithi104.stdout:2/55: chown d6/d8/la 871 1 2024-04-26T04:42:09.581 INFO:tasks.workunit.client.1.smithi104.stdout:9/124: dwrite d11/d15/f18 [4194304,4194304] 0 2024-04-26T04:42:09.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/41: dread f1 [0,4194304] 0 2024-04-26T04:42:09.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/42: truncate f4 58594 0 2024-04-26T04:42:09.583 INFO:tasks.workunit.client.1.smithi104.stdout:2/56: rename d6/d8/la to d6/de/l12 0 2024-04-26T04:42:09.586 INFO:tasks.workunit.client.1.smithi104.stdout:3/43: rename f1 to d5/d9/fe 0 2024-04-26T04:42:09.586 INFO:tasks.workunit.client.1.smithi104.stdout:3/44: creat d5/d9/ff x:0 0 0 2024-04-26T04:42:09.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/45: chown f0 647166711 1 2024-04-26T04:42:09.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/46: write f3 [896455,62790] 0 2024-04-26T04:42:09.587 INFO:tasks.workunit.client.1.smithi104.stdout:2/57: mkdir d6/d8/d13 0 2024-04-26T04:42:09.589 INFO:tasks.workunit.client.1.smithi104.stdout:3/47: creat d5/d9/dc/f10 x:0 0 0 2024-04-26T04:42:09.590 INFO:tasks.workunit.client.1.smithi104.stdout:2/58: link d6/f10 d6/f14 0 2024-04-26T04:42:09.591 INFO:tasks.workunit.client.1.smithi104.stdout:3/48: symlink d5/d9/dc/l11 0 2024-04-26T04:42:09.591 INFO:tasks.workunit.client.1.smithi104.stdout:3/49: chown d5/d9/ff 4489635 1 2024-04-26T04:42:09.591 INFO:tasks.workunit.client.1.smithi104.stdout:3/50: truncate d5/d9/fa 45414 0 2024-04-26T04:42:09.591 INFO:tasks.workunit.client.1.smithi104.stdout:3/51: creat d5/d9/f12 x:0 0 0 2024-04-26T04:42:09.592 INFO:tasks.workunit.client.1.smithi104.stdout:3/52: symlink d5/d9/dc/l13 0 2024-04-26T04:42:09.603 INFO:tasks.workunit.client.1.smithi104.stdout:1/82: dwrite d0/d6/fb [0,4194304] 0 2024-04-26T04:42:09.604 INFO:tasks.workunit.client.1.smithi104.stdout:1/83: mknod d0/d6/c16 0 2024-04-26T04:42:09.604 INFO:tasks.workunit.client.1.smithi104.stdout:0/91: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:09.605 INFO:tasks.workunit.client.1.smithi104.stdout:2/59: read d6/d8/fb [470491,9358] 0 2024-04-26T04:42:09.605 INFO:tasks.workunit.client.1.smithi104.stdout:2/60: dread - d6/f14 zero size 2024-04-26T04:42:09.605 INFO:tasks.workunit.client.1.smithi104.stdout:2/61: chown d6/l11 435440 1 2024-04-26T04:42:09.605 INFO:tasks.workunit.client.1.smithi104.stdout:8/30: dwrite d0/f1 [0,4194304] 0 2024-04-26T04:42:09.605 INFO:tasks.workunit.client.1.smithi104.stdout:1/84: write d0/d6/ff [848088,116332] 0 2024-04-26T04:42:09.606 INFO:tasks.workunit.client.1.smithi104.stdout:1/85: stat d0/c13 0 2024-04-26T04:42:09.606 INFO:tasks.workunit.client.1.smithi104.stdout:1/86: chown d0/d6/l9 20064680 1 2024-04-26T04:42:09.607 INFO:tasks.workunit.client.1.smithi104.stdout:5/56: getdents d0/d1/d4 0 2024-04-26T04:42:09.611 INFO:tasks.workunit.client.1.smithi104.stdout:5/57: read d0/f6 [2047779,124661] 0 2024-04-26T04:42:09.613 INFO:tasks.workunit.client.1.smithi104.stdout:0/92: symlink d0/d4/d15/l20 0 2024-04-26T04:42:09.618 INFO:tasks.workunit.client.1.smithi104.stdout:1/87: unlink d0/d6/c16 0 2024-04-26T04:42:09.619 INFO:tasks.workunit.client.1.smithi104.stdout:2/62: dwrite d6/f10 [0,4194304] 0 2024-04-26T04:42:09.620 INFO:tasks.workunit.client.1.smithi104.stdout:0/93: mkdir d0/d2/d21 0 2024-04-26T04:42:09.620 INFO:tasks.workunit.client.1.smithi104.stdout:0/94: dread - d0/d4/f1b zero size 2024-04-26T04:42:09.622 INFO:tasks.workunit.client.1.smithi104.stdout:1/88: mknod d0/d6/c17 0 2024-04-26T04:42:09.624 INFO:tasks.workunit.client.1.smithi104.stdout:2/63: fsync d6/f10 0 2024-04-26T04:42:09.625 INFO:tasks.workunit.client.1.smithi104.stdout:0/95: fdatasync d0/d4/f7 0 2024-04-26T04:42:09.626 INFO:tasks.workunit.client.1.smithi104.stdout:2/64: mknod d6/c15 0 2024-04-26T04:42:09.630 INFO:tasks.workunit.client.1.smithi104.stdout:0/96: unlink d0/d2/d10/f12 0 2024-04-26T04:42:09.630 INFO:tasks.workunit.client.1.smithi104.stdout:0/97: symlink d0/l22 0 2024-04-26T04:42:09.631 INFO:tasks.workunit.client.1.smithi104.stdout:8/31: truncate d0/f5 1703929 0 2024-04-26T04:42:09.632 INFO:tasks.workunit.client.1.smithi104.stdout:5/58: dwrite d0/d1/f2 [4194304,4194304] 0 2024-04-26T04:42:09.634 INFO:tasks.workunit.client.1.smithi104.stdout:1/89: dread d0/d6/fb [0,4194304] 0 2024-04-26T04:42:09.635 INFO:tasks.workunit.client.1.smithi104.stdout:5/59: creat d0/d1/d4/fe x:0 0 0 2024-04-26T04:42:09.636 INFO:tasks.workunit.client.1.smithi104.stdout:1/90: mknod d0/d6/c18 0 2024-04-26T04:42:09.641 INFO:tasks.workunit.client.1.smithi104.stdout:0/98: truncate d0/fa 3222239 0 2024-04-26T04:42:09.642 INFO:tasks.workunit.client.1.smithi104.stdout:1/91: fsync d0/d6/ff 0 2024-04-26T04:42:09.642 INFO:tasks.workunit.client.1.smithi104.stdout:1/92: write d0/f14 [1513756,46719] 0 2024-04-26T04:42:09.643 INFO:tasks.workunit.client.1.smithi104.stdout:8/32: rmdir d0 39 2024-04-26T04:42:09.643 INFO:tasks.workunit.client.1.smithi104.stdout:8/33: write d0/f7 [6345,22484] 0 2024-04-26T04:42:09.646 INFO:tasks.workunit.client.1.smithi104.stdout:1/93: dread d0/d6/fb [0,4194304] 0 2024-04-26T04:42:09.646 INFO:tasks.workunit.client.1.smithi104.stdout:1/94: truncate d0/d6/ff 2383146 0 2024-04-26T04:42:09.647 INFO:tasks.workunit.client.1.smithi104.stdout:1/95: mkdir d0/d6/d19 0 2024-04-26T04:42:09.653 INFO:tasks.workunit.client.1.smithi104.stdout:2/65: dwrite d6/d8/fb [0,4194304] 0 2024-04-26T04:42:09.653 INFO:tasks.workunit.client.1.smithi104.stdout:5/60: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:42:09.654 INFO:tasks.workunit.client.1.smithi104.stdout:1/96: fsync d0/f14 0 2024-04-26T04:42:09.654 INFO:tasks.workunit.client.1.smithi104.stdout:0/99: dread d0/fe [0,4194304] 0 2024-04-26T04:42:09.654 INFO:tasks.workunit.client.1.smithi104.stdout:0/100: creat d0/d2/f23 x:0 0 0 2024-04-26T04:42:09.655 INFO:tasks.workunit.client.1.smithi104.stdout:2/66: mknod d6/de/c16 0 2024-04-26T04:42:09.659 INFO:tasks.workunit.client.1.smithi104.stdout:5/61: mknod d0/cf 0 2024-04-26T04:42:09.659 INFO:tasks.workunit.client.1.smithi104.stdout:5/62: write d0/f6 [403098,112901] 0 2024-04-26T04:42:09.663 INFO:tasks.workunit.client.1.smithi104.stdout:1/97: creat d0/d6/f1a x:0 0 0 2024-04-26T04:42:09.667 INFO:tasks.workunit.client.1.smithi104.stdout:2/67: creat d6/d8/f17 x:0 0 0 2024-04-26T04:42:09.669 INFO:tasks.workunit.client.1.smithi104.stdout:1/98: unlink d0/d6/fb 0 2024-04-26T04:42:09.670 INFO:tasks.workunit.client.1.smithi104.stdout:0/101: unlink d0/d4/c19 0 2024-04-26T04:42:09.670 INFO:tasks.workunit.client.1.smithi104.stdout:0/102: write d0/d2/f23 [596514,37673] 0 2024-04-26T04:42:09.671 INFO:tasks.workunit.client.1.smithi104.stdout:2/68: symlink d6/l18 0 2024-04-26T04:42:09.672 INFO:tasks.workunit.client.1.smithi104.stdout:0/103: link d0/l1a d0/d2/l24 0 2024-04-26T04:42:09.673 INFO:tasks.workunit.client.1.smithi104.stdout:0/104: link d0/l1a d0/d4/d15/l25 0 2024-04-26T04:42:09.674 INFO:tasks.workunit.client.1.smithi104.stdout:0/105: truncate d0/d4/d15/f18 212992 0 2024-04-26T04:42:09.676 INFO:tasks.workunit.client.1.smithi104.stdout:1/99: dread d0/f14 [0,4194304] 0 2024-04-26T04:42:09.677 INFO:tasks.workunit.client.1.smithi104.stdout:1/100: chown d0/d6 268625 1 2024-04-26T04:42:09.677 INFO:tasks.workunit.client.1.smithi104.stdout:8/34: dwrite d0/f7 [0,4194304] 0 2024-04-26T04:42:09.677 INFO:tasks.workunit.client.1.smithi104.stdout:5/63: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:09.677 INFO:tasks.workunit.client.1.smithi104.stdout:0/106: getdents d0 0 2024-04-26T04:42:09.678 INFO:tasks.workunit.client.1.smithi104.stdout:1/101: mkdir d0/d6/d19/d1b 0 2024-04-26T04:42:09.678 INFO:tasks.workunit.client.1.smithi104.stdout:2/69: dread d6/fc [0,4194304] 0 2024-04-26T04:42:09.678 INFO:tasks.workunit.client.1.smithi104.stdout:2/70: creat d6/d8/f19 x:0 0 0 2024-04-26T04:42:09.679 INFO:tasks.workunit.client.1.smithi104.stdout:5/64: mknod d0/d1/c10 0 2024-04-26T04:42:09.680 INFO:tasks.workunit.client.1.smithi104.stdout:1/102: unlink d0/d6/l9 0 2024-04-26T04:42:09.683 INFO:tasks.workunit.client.1.smithi104.stdout:2/71: symlink d6/d8/l1a 0 2024-04-26T04:42:09.684 INFO:tasks.workunit.client.1.smithi104.stdout:2/72: chown d6/fc 52319477 1 2024-04-26T04:42:09.685 INFO:tasks.workunit.client.1.smithi104.stdout:1/103: rmdir d0 39 2024-04-26T04:42:09.687 INFO:tasks.workunit.client.1.smithi104.stdout:2/73: mkdir d6/d1b 0 2024-04-26T04:42:09.687 INFO:tasks.workunit.client.1.smithi104.stdout:0/107: write d0/d4/d15/f18 [1232830,9373] 0 2024-04-26T04:42:09.688 INFO:tasks.workunit.client.1.smithi104.stdout:0/108: creat d0/d2/f26 x:0 0 0 2024-04-26T04:42:09.688 INFO:tasks.workunit.client.1.smithi104.stdout:1/104: unlink d0/d6/c18 0 2024-04-26T04:42:09.689 INFO:tasks.workunit.client.1.smithi104.stdout:1/105: truncate d0/d6/f1a 520695 0 2024-04-26T04:42:09.690 INFO:tasks.workunit.client.1.smithi104.stdout:8/35: dread d0/f1 [0,4194304] 0 2024-04-26T04:42:09.692 INFO:tasks.workunit.client.1.smithi104.stdout:0/109: rename d0/d1d to d0/d2/d21/d27 0 2024-04-26T04:42:09.692 INFO:tasks.workunit.client.1.smithi104.stdout:1/106: dread d0/f14 [0,4194304] 0 2024-04-26T04:42:09.692 INFO:tasks.workunit.client.1.smithi104.stdout:5/65: getdents d0/d1 0 2024-04-26T04:42:09.694 INFO:tasks.workunit.client.1.smithi104.stdout:8/36: creat d0/f9 x:0 0 0 2024-04-26T04:42:09.694 INFO:tasks.workunit.client.1.smithi104.stdout:8/37: chown d0/l8 0 1 2024-04-26T04:42:09.695 INFO:tasks.workunit.client.1.smithi104.stdout:0/110: mkdir d0/d2/d28 0 2024-04-26T04:42:09.696 INFO:tasks.workunit.client.1.smithi104.stdout:0/111: read d0/d4/d15/f18 [448694,127989] 0 2024-04-26T04:42:09.696 INFO:tasks.workunit.client.1.smithi104.stdout:0/112: read - d0/d4/f1b zero size 2024-04-26T04:42:09.696 INFO:tasks.workunit.client.1.smithi104.stdout:0/113: readlink d0/d2/l9 0 2024-04-26T04:42:09.696 INFO:tasks.workunit.client.1.smithi104.stdout:1/107: mknod d0/d6/d19/c1c 0 2024-04-26T04:42:09.697 INFO:tasks.workunit.client.1.smithi104.stdout:5/66: mkdir d0/d11 0 2024-04-26T04:42:09.698 INFO:tasks.workunit.client.1.smithi104.stdout:5/67: chown d0/d1/d4/la 0 1 2024-04-26T04:42:09.698 INFO:tasks.workunit.client.1.smithi104.stdout:5/68: truncate d0/d1/f2 8601979 0 2024-04-26T04:42:09.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/108: symlink d0/d15/l1d 0 2024-04-26T04:42:09.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/109: stat d0/d6/de 0 2024-04-26T04:42:09.700 INFO:tasks.workunit.client.1.smithi104.stdout:0/114: write d0/d4/f7 [744606,24430] 0 2024-04-26T04:42:09.702 INFO:tasks.workunit.client.1.smithi104.stdout:1/110: dread d0/f14 [0,4194304] 0 2024-04-26T04:42:09.702 INFO:tasks.workunit.client.1.smithi104.stdout:0/115: creat d0/d4/f29 x:0 0 0 2024-04-26T04:42:09.702 INFO:tasks.workunit.client.1.smithi104.stdout:0/116: chown d0/d2/f23 255 1 2024-04-26T04:42:09.703 INFO:tasks.workunit.client.1.smithi104.stdout:1/111: mknod d0/d6/d19/c1e 0 2024-04-26T04:42:09.703 INFO:tasks.workunit.client.1.smithi104.stdout:1/112: fdatasync d0/d6/f1a 0 2024-04-26T04:42:09.703 INFO:tasks.workunit.client.1.smithi104.stdout:1/113: chown d0/d6/d19/c1c 0 1 2024-04-26T04:42:09.703 INFO:tasks.workunit.client.1.smithi104.stdout:1/114: chown d0/d6/d19/c1e 1 1 2024-04-26T04:42:09.704 INFO:tasks.workunit.client.1.smithi104.stdout:1/115: mknod d0/d6/d19/c1f 0 2024-04-26T04:42:09.704 INFO:tasks.workunit.client.1.smithi104.stdout:1/116: write d0/f14 [3170171,112577] 0 2024-04-26T04:42:09.704 INFO:tasks.workunit.client.1.smithi104.stdout:1/117: mkdir d0/d6/de/d20 0 2024-04-26T04:42:09.704 INFO:tasks.workunit.client.1.smithi104.stdout:1/118: write d0/f14 [841632,94832] 0 2024-04-26T04:42:09.706 INFO:tasks.workunit.client.1.smithi104.stdout:2/74: dwrite d6/fc [0,4194304] 0 2024-04-26T04:42:09.706 INFO:tasks.workunit.client.1.smithi104.stdout:2/75: fsync d6/f14 0 2024-04-26T04:42:09.710 INFO:tasks.workunit.client.1.smithi104.stdout:8/38: rmdir d0 39 2024-04-26T04:42:09.711 INFO:tasks.workunit.client.1.smithi104.stdout:5/69: read d0/d1/f2 [4285300,46582] 0 2024-04-26T04:42:09.711 INFO:tasks.workunit.client.1.smithi104.stdout:5/70: creat d0/d1/d4/f12 x:0 0 0 2024-04-26T04:42:09.711 INFO:tasks.workunit.client.1.smithi104.stdout:5/71: chown d0/d1/d4/f12 108300012 1 2024-04-26T04:42:09.712 INFO:tasks.workunit.client.1.smithi104.stdout:8/39: mknod d0/ca 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:1/119: rename d0/d15 to d0/d6/d19/d21 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:1/120: mknod d0/d6/d19/c22 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:2/76: dread d6/f14 [0,4194304] 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:2/77: fsync d6/d8/f17 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:8/40: dread d0/f1 [0,4194304] 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:1/121: write d0/d6/f1a [151526,61545] 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:2/78: creat d6/d1b/f1c x:0 0 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:8/41: creat d0/fb x:0 0 0 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:8/42: chown d0/f1 280893 1 2024-04-26T04:42:09.720 INFO:tasks.workunit.client.1.smithi104.stdout:1/122: link d0/d6/d19/d21/l1d d0/l23 0 2024-04-26T04:42:09.721 INFO:tasks.workunit.client.1.smithi104.stdout:8/43: rename d0/f7 to d0/fc 0 2024-04-26T04:42:09.721 INFO:tasks.workunit.client.1.smithi104.stdout:8/44: readlink d0/l6 0 2024-04-26T04:42:09.723 INFO:tasks.workunit.client.1.smithi104.stdout:0/117: dwrite d0/f17 [0,4194304] 0 2024-04-26T04:42:09.723 INFO:tasks.workunit.client.1.smithi104.stdout:0/118: chown d0/d2 0 1 2024-04-26T04:42:09.725 INFO:tasks.workunit.client.1.smithi104.stdout:2/79: truncate d6/fc 4038155 0 2024-04-26T04:42:09.726 INFO:tasks.workunit.client.1.smithi104.stdout:5/72: read d0/f6 [2303096,31797] 0 2024-04-26T04:42:09.726 INFO:tasks.workunit.client.1.smithi104.stdout:5/73: stat d0/cf 0 2024-04-26T04:42:09.726 INFO:tasks.workunit.client.1.smithi104.stdout:1/123: dread d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:09.727 INFO:tasks.workunit.client.1.smithi104.stdout:2/80: mkdir d6/de/d1d 0 2024-04-26T04:42:09.727 INFO:tasks.workunit.client.1.smithi104.stdout:2/81: write d6/d8/f19 [230432,100389] 0 2024-04-26T04:42:09.731 INFO:tasks.workunit.client.1.smithi104.stdout:2/82: stat d6/de/l12 0 2024-04-26T04:42:09.731 INFO:tasks.workunit.client.1.smithi104.stdout:2/83: chown d6/c9 569080 1 2024-04-26T04:42:09.733 INFO:tasks.workunit.client.1.smithi104.stdout:2/84: creat d6/de/f1e x:0 0 0 2024-04-26T04:42:09.734 INFO:tasks.workunit.client.1.smithi104.stdout:0/119: dread d0/f17 [0,4194304] 0 2024-04-26T04:42:09.734 INFO:tasks.workunit.client.1.smithi104.stdout:0/120: write d0/fe [4877814,95966] 0 2024-04-26T04:42:09.734 INFO:tasks.workunit.client.1.smithi104.stdout:0/121: truncate d0/d4/f1b 528352 0 2024-04-26T04:42:09.735 INFO:tasks.workunit.client.1.smithi104.stdout:2/85: rename d6/f14 to d6/d8/d13/f1f 0 2024-04-26T04:42:09.735 INFO:tasks.workunit.client.1.smithi104.stdout:2/86: truncate d6/d1b/f1c 1032748 0 2024-04-26T04:42:09.735 INFO:tasks.workunit.client.1.smithi104.stdout:2/87: write d6/d8/f19 [827308,72769] 0 2024-04-26T04:42:09.735 INFO:tasks.workunit.client.1.smithi104.stdout:2/88: rename d6/d8/d13 to d6/d8/d13/d20 22 2024-04-26T04:42:09.737 INFO:tasks.workunit.client.1.smithi104.stdout:0/122: mknod d0/d2/d21/d27/c2a 0 2024-04-26T04:42:09.737 INFO:tasks.workunit.client.1.smithi104.stdout:0/123: write d0/d2/f26 [287087,40110] 0 2024-04-26T04:42:09.739 INFO:tasks.workunit.client.1.smithi104.stdout:5/74: truncate d0/f6 2772357 0 2024-04-26T04:42:09.739 INFO:tasks.workunit.client.1.smithi104.stdout:5/75: fsync d0/d1/d4/f12 0 2024-04-26T04:42:09.740 INFO:tasks.workunit.client.1.smithi104.stdout:5/76: readlink d0/l7 0 2024-04-26T04:42:09.740 INFO:tasks.workunit.client.1.smithi104.stdout:5/77: creat d0/f13 x:0 0 0 2024-04-26T04:42:09.740 INFO:tasks.workunit.client.1.smithi104.stdout:5/78: fsync d0/d1/d4/f12 0 2024-04-26T04:42:09.740 INFO:tasks.workunit.client.1.smithi104.stdout:1/124: dwrite d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:09.740 INFO:tasks.workunit.client.1.smithi104.stdout:1/125: stat d0/d6/d19 0 2024-04-26T04:42:09.749 INFO:tasks.workunit.client.1.smithi104.stdout:1/126: unlink d0/d6/c17 0 2024-04-26T04:42:09.757 INFO:tasks.workunit.client.1.smithi104.stdout:1/127: write d0/d6/f1a [479287,48747] 0 2024-04-26T04:42:09.764 INFO:tasks.workunit.client.1.smithi104.stdout:5/79: dwrite d0/d1/d4/f12 [0,4194304] 0 2024-04-26T04:42:09.774 INFO:tasks.workunit.client.1.smithi104.stdout:1/128: dwrite d0/d6/ff [0,4194304] 0 2024-04-26T04:42:09.775 INFO:tasks.workunit.client.1.smithi104.stdout:1/129: symlink d0/d6/de/l24 0 2024-04-26T04:42:09.786 INFO:tasks.workunit.client.1.smithi104.stdout:5/80: dwrite d0/f13 [0,4194304] 0 2024-04-26T04:42:09.789 INFO:tasks.workunit.client.1.smithi104.stdout:1/130: write d0/d6/ff [2865103,48188] 0 2024-04-26T04:42:09.791 INFO:tasks.workunit.client.1.smithi104.stdout:5/81: dread d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:09.791 INFO:tasks.workunit.client.1.smithi104.stdout:5/82: dread - d0/d1/d4/fe zero size 2024-04-26T04:42:09.791 INFO:tasks.workunit.client.1.smithi104.stdout:5/83: stat d0/d1/d4/lc 0 2024-04-26T04:42:09.791 INFO:tasks.workunit.client.1.smithi104.stdout:1/131: fdatasync d0/d6/ff 0 2024-04-26T04:42:09.793 INFO:tasks.workunit.client.1.smithi104.stdout:1/132: rmdir d0/d6 39 2024-04-26T04:42:09.793 INFO:tasks.workunit.client.1.smithi104.stdout:5/84: dread d0/f6 [0,4194304] 0 2024-04-26T04:42:09.793 INFO:tasks.workunit.client.1.smithi104.stdout:5/85: fdatasync d0/d1/d4/fe 0 2024-04-26T04:42:09.794 INFO:tasks.workunit.client.1.smithi104.stdout:1/133: symlink d0/d6/d19/d21/l25 0 2024-04-26T04:42:09.795 INFO:tasks.workunit.client.1.smithi104.stdout:1/134: symlink d0/d6/d19/d1b/l26 0 2024-04-26T04:42:09.795 INFO:tasks.workunit.client.1.smithi104.stdout:1/135: write d0/f14 [3417106,111690] 0 2024-04-26T04:42:09.796 INFO:tasks.workunit.client.1.smithi104.stdout:1/136: symlink d0/d6/l27 0 2024-04-26T04:42:09.796 INFO:tasks.workunit.client.1.smithi104.stdout:1/137: truncate d0/f14 4814251 0 2024-04-26T04:42:09.796 INFO:tasks.workunit.client.1.smithi104.stdout:1/138: readlink d0/d6/d19/d1b/l26 0 2024-04-26T04:42:09.796 INFO:tasks.workunit.client.1.smithi104.stdout:1/139: read d0/d6/f1a [1670149,25583] 0 2024-04-26T04:42:09.796 INFO:tasks.workunit.client.1.smithi104.stdout:1/140: readlink d0/d6/l8 0 2024-04-26T04:42:09.796 INFO:tasks.workunit.client.1.smithi104.stdout:1/141: mkdir d0/d6/d19/d21/d28 0 2024-04-26T04:42:09.799 INFO:tasks.workunit.client.1.smithi104.stdout:1/142: fsync d0/f14 0 2024-04-26T04:42:09.801 INFO:tasks.workunit.client.1.smithi104.stdout:1/143: mkdir d0/d6/de/d29 0 2024-04-26T04:42:09.801 INFO:tasks.workunit.client.1.smithi104.stdout:1/144: write d0/f14 [755123,125651] 0 2024-04-26T04:42:09.801 INFO:tasks.workunit.client.1.smithi104.stdout:1/145: readlink d0/d6/d19/d1b/l26 0 2024-04-26T04:42:09.801 INFO:tasks.workunit.client.1.smithi104.stdout:1/146: read d0/d6/f1a [3900207,41113] 0 2024-04-26T04:42:09.801 INFO:tasks.workunit.client.1.smithi104.stdout:1/147: chown d0/d6/d19/d21/l25 50137273 1 2024-04-26T04:42:09.801 INFO:tasks.workunit.client.1.smithi104.stdout:1/148: chown d0/l23 2603 1 2024-04-26T04:42:09.802 INFO:tasks.workunit.client.1.smithi104.stdout:5/86: dread d0/f13 [0,4194304] 0 2024-04-26T04:42:09.802 INFO:tasks.workunit.client.1.smithi104.stdout:5/87: fsync d0/f13 0 2024-04-26T04:42:09.802 INFO:tasks.workunit.client.1.smithi104.stdout:1/149: mkdir d0/d6/de/d20/d2a 0 2024-04-26T04:42:09.803 INFO:tasks.workunit.client.1.smithi104.stdout:5/88: mknod d0/d1/c14 0 2024-04-26T04:42:09.803 INFO:tasks.workunit.client.1.smithi104.stdout:5/89: read - d0/d1/d4/fe zero size 2024-04-26T04:42:09.805 INFO:tasks.workunit.client.1.smithi104.stdout:5/90: dread d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:09.807 INFO:tasks.workunit.client.1.smithi104.stdout:5/91: link d0/d1/c10 d0/d1/c15 0 2024-04-26T04:42:09.807 INFO:tasks.workunit.client.1.smithi104.stdout:5/92: symlink d0/l16 0 2024-04-26T04:42:09.809 INFO:tasks.workunit.client.1.smithi104.stdout:5/93: link d0/d1/d4/fe d0/d1/d4/f17 0 2024-04-26T04:42:09.809 INFO:tasks.workunit.client.1.smithi104.stdout:5/94: chown d0/d1/d4/lc 11248952 1 2024-04-26T04:42:09.809 INFO:tasks.workunit.client.1.smithi104.stdout:1/150: dread d0/f14 [0,4194304] 0 2024-04-26T04:42:09.811 INFO:tasks.workunit.client.1.smithi104.stdout:1/151: mknod d0/d6/c2b 0 2024-04-26T04:42:09.812 INFO:tasks.workunit.client.1.smithi104.stdout:1/152: creat d0/d6/de/d20/d2a/f2c x:0 0 0 2024-04-26T04:42:09.813 INFO:tasks.workunit.client.1.smithi104.stdout:1/153: rmdir d0/d6/d19/d1b 39 2024-04-26T04:42:09.817 INFO:tasks.workunit.client.1.smithi104.stdout:5/95: dread d0/d1/d4/f12 [0,4194304] 0 2024-04-26T04:42:09.817 INFO:tasks.workunit.client.1.smithi104.stdout:5/96: chown d0/d1/c10 19138 1 2024-04-26T04:42:09.819 INFO:tasks.workunit.client.1.smithi104.stdout:5/97: rename d0/d1/d4/fe to d0/f18 0 2024-04-26T04:42:09.835 INFO:tasks.workunit.client.1.smithi104.stdout:5/98: dwrite d0/f18 [0,4194304] 0 2024-04-26T04:42:09.844 INFO:tasks.workunit.client.1.smithi104.stdout:5/99: fsync d0/f18 0 2024-04-26T04:42:09.844 INFO:tasks.workunit.client.1.smithi104.stdout:5/100: truncate d0/f13 502619 0 2024-04-26T04:42:09.860 INFO:tasks.workunit.client.1.smithi104.stdout:5/101: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:09.861 INFO:tasks.workunit.client.1.smithi104.stdout:5/102: creat d0/d11/f19 x:0 0 0 2024-04-26T04:42:09.862 INFO:tasks.workunit.client.1.smithi104.stdout:5/103: symlink d0/d11/l1a 0 2024-04-26T04:42:09.863 INFO:tasks.workunit.client.1.smithi104.stdout:5/104: getdents d0/d1/d4 0 2024-04-26T04:42:09.863 INFO:tasks.workunit.client.1.smithi104.stdout:5/105: write d0/d11/f19 [564535,118980] 0 2024-04-26T04:42:09.873 INFO:tasks.workunit.client.1.smithi104.stdout:5/106: write d0/f6 [2923207,22663] 0 2024-04-26T04:42:09.893 INFO:tasks.workunit.client.1.smithi104.stdout:4/28: sync 2024-04-26T04:42:09.893 INFO:tasks.workunit.client.1.smithi104.stdout:4/29: creat d4/d7/f9 x:0 0 0 2024-04-26T04:42:09.893 INFO:tasks.workunit.client.1.smithi104.stdout:4/30: chown d4/d7/f9 63 1 2024-04-26T04:42:09.894 INFO:tasks.workunit.client.1.smithi104.stdout:4/31: rename d4/d7/f9 to d4/d7/fa 0 2024-04-26T04:42:09.894 INFO:tasks.workunit.client.1.smithi104.stdout:4/32: chown f2 8295021 1 2024-04-26T04:42:09.895 INFO:tasks.workunit.client.1.smithi104.stdout:4/33: mkdir d4/db 0 2024-04-26T04:42:09.896 INFO:tasks.workunit.client.1.smithi104.stdout:4/34: link d4/c5 d4/db/cc 0 2024-04-26T04:42:09.897 INFO:tasks.workunit.client.1.smithi104.stdout:5/107: dwrite d0/d1/d4/f12 [0,4194304] 0 2024-04-26T04:42:09.918 INFO:tasks.workunit.client.1.smithi104.stdout:5/108: dwrite d0/d11/f19 [0,4194304] 0 2024-04-26T04:42:09.918 INFO:tasks.workunit.client.1.smithi104.stdout:4/35: dwrite d4/d7/fa [0,4194304] 0 2024-04-26T04:42:09.919 INFO:tasks.workunit.client.1.smithi104.stdout:5/109: mknod d0/d1/c1b 0 2024-04-26T04:42:09.920 INFO:tasks.workunit.client.1.smithi104.stdout:4/36: stat d4/db/cc 0 2024-04-26T04:42:09.921 INFO:tasks.workunit.client.1.smithi104.stdout:4/37: mknod d4/cd 0 2024-04-26T04:42:09.922 INFO:tasks.workunit.client.1.smithi104.stdout:4/38: creat d4/db/fe x:0 0 0 2024-04-26T04:42:09.932 INFO:tasks.workunit.client.1.smithi104.stdout:4/39: dread f3 [0,4194304] 0 2024-04-26T04:42:09.932 INFO:tasks.workunit.client.1.smithi104.stdout:4/40: truncate f2 4592750 0 2024-04-26T04:42:09.932 INFO:tasks.workunit.client.1.smithi104.stdout:4/41: creat d4/d7/ff x:0 0 0 2024-04-26T04:42:09.940 INFO:tasks.workunit.client.1.smithi104.stdout:5/110: dwrite d0/f18 [0,4194304] 0 2024-04-26T04:42:09.941 INFO:tasks.workunit.client.1.smithi104.stdout:4/42: dread d4/d7/fa [0,4194304] 0 2024-04-26T04:42:09.941 INFO:tasks.workunit.client.1.smithi104.stdout:5/111: symlink d0/d1/d4/l1c 0 2024-04-26T04:42:09.941 INFO:tasks.workunit.client.1.smithi104.stdout:4/43: link d4/db/cc d4/d7/c10 0 2024-04-26T04:42:09.942 INFO:tasks.workunit.client.1.smithi104.stdout:4/44: creat d4/f11 x:0 0 0 2024-04-26T04:42:09.943 INFO:tasks.workunit.client.1.smithi104.stdout:4/45: link f2 d4/db/f12 0 2024-04-26T04:42:09.948 INFO:tasks.workunit.client.1.smithi104.stdout:5/112: dread d0/d1/d4/f17 [0,4194304] 0 2024-04-26T04:42:09.949 INFO:tasks.workunit.client.1.smithi104.stdout:5/113: stat d0/d1/d4/lb 0 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:4/46: dwrite d4/d7/f8 [0,4194304] 0 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:4/47: write d4/db/fe [529481,130452] 0 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:4/48: write d4/f11 [143223,6795] 0 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:4/49: chown d4/db/fe 178488791 1 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:4/50: readlink - no filename 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:4/51: stat d4/d7/ff 0 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:5/114: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:09.968 INFO:tasks.workunit.client.1.smithi104.stdout:4/52: chown d4/c5 1493360287 1 2024-04-26T04:42:09.969 INFO:tasks.workunit.client.1.smithi104.stdout:4/53: mkdir d4/db/d13 0 2024-04-26T04:42:09.970 INFO:tasks.workunit.client.1.smithi104.stdout:5/115: link d0/d1/d4/f17 d0/f1d 0 2024-04-26T04:42:09.971 INFO:tasks.workunit.client.1.smithi104.stdout:5/116: mkdir d0/d1/d1e 0 2024-04-26T04:42:09.972 INFO:tasks.workunit.client.1.smithi104.stdout:5/117: symlink d0/d11/l1f 0 2024-04-26T04:42:09.972 INFO:tasks.workunit.client.1.smithi104.stdout:5/118: truncate d0/f1d 4629209 0 2024-04-26T04:42:09.974 INFO:tasks.workunit.client.1.smithi104.stdout:6/243: sync 2024-04-26T04:42:09.974 INFO:tasks.workunit.client.1.smithi104.stdout:6/244: write d0/d9/d13/d16/f4b [194108,104148] 0 2024-04-26T04:42:09.977 INFO:tasks.workunit.client.1.smithi104.stdout:6/245: mknod d0/d9/d13/d41/c51 0 2024-04-26T04:42:09.979 INFO:tasks.workunit.client.1.smithi104.stdout:6/246: mkdir d0/d9/d52 0 2024-04-26T04:42:09.979 INFO:tasks.workunit.client.1.smithi104.stdout:6/247: fsync d0/d5/f22 0 2024-04-26T04:42:09.979 INFO:tasks.workunit.client.1.smithi104.stdout:6/248: write d0/d9/d13/d15/f2b [872701,91188] 0 2024-04-26T04:42:09.979 INFO:tasks.workunit.client.1.smithi104.stdout:6/249: write d0/d9/d13/d16/f19 [959201,74947] 0 2024-04-26T04:42:09.979 INFO:tasks.workunit.client.1.smithi104.stdout:4/54: write d4/d7/f8 [2185990,4499] 0 2024-04-26T04:42:09.980 INFO:tasks.workunit.client.1.smithi104.stdout:6/250: mknod d0/c53 0 2024-04-26T04:42:09.980 INFO:tasks.workunit.client.1.smithi104.stdout:6/251: write d0/d9/d13/d44/d47/f4c [195518,76727] 0 2024-04-26T04:42:09.981 INFO:tasks.workunit.client.1.smithi104.stdout:5/119: write d0/f6 [4177657,37075] 0 2024-04-26T04:42:09.981 INFO:tasks.workunit.client.1.smithi104.stdout:4/55: link d4/db/fe d4/db/d13/f14 0 2024-04-26T04:42:09.982 INFO:tasks.workunit.client.1.smithi104.stdout:6/252: symlink d0/d9/d13/d15/l54 0 2024-04-26T04:42:09.983 INFO:tasks.workunit.client.1.smithi104.stdout:5/120: dread d0/f13 [0,4194304] 0 2024-04-26T04:42:09.983 INFO:tasks.workunit.client.1.smithi104.stdout:5/121: write d0/f18 [5499149,30251] 0 2024-04-26T04:42:09.984 INFO:tasks.workunit.client.1.smithi104.stdout:6/253: rename d0/d9/d13/d41/c51 to d0/d5/c55 0 2024-04-26T04:42:09.984 INFO:tasks.workunit.client.1.smithi104.stdout:6/254: write d0/d9/d13/f3d [4534326,40058] 0 2024-04-26T04:42:09.985 INFO:tasks.workunit.client.1.smithi104.stdout:6/255: symlink d0/d9/d13/d44/l56 0 2024-04-26T04:42:09.985 INFO:tasks.workunit.client.1.smithi104.stdout:6/256: rmdir d0/d9/d25 39 2024-04-26T04:42:09.986 INFO:tasks.workunit.client.1.smithi104.stdout:4/56: dread d4/db/f12 [0,4194304] 0 2024-04-26T04:42:09.988 INFO:tasks.workunit.client.1.smithi104.stdout:4/57: truncate d4/db/f12 2446071 0 2024-04-26T04:42:09.989 INFO:tasks.workunit.client.1.smithi104.stdout:4/58: mkdir d4/db/d13/d15 0 2024-04-26T04:42:09.990 INFO:tasks.workunit.client.1.smithi104.stdout:5/122: write d0/d11/f19 [3793854,48856] 0 2024-04-26T04:42:09.992 INFO:tasks.workunit.client.1.smithi104.stdout:4/59: rename d4/d7/c10 to d4/db/d13/d15/c16 0 2024-04-26T04:42:09.992 INFO:tasks.workunit.client.1.smithi104.stdout:4/60: creat d4/f17 x:0 0 0 2024-04-26T04:42:09.995 INFO:tasks.workunit.client.1.smithi104.stdout:4/61: mkdir d4/db/d13/d18 0 2024-04-26T04:42:09.996 INFO:tasks.workunit.client.1.smithi104.stdout:5/123: dread d0/d1/d4/f17 [4194304,4194304] 0 2024-04-26T04:42:09.996 INFO:tasks.workunit.client.1.smithi104.stdout:5/124: chown d0/d11 577263670 1 2024-04-26T04:42:09.996 INFO:tasks.workunit.client.1.smithi104.stdout:5/125: creat d0/f20 x:0 0 0 2024-04-26T04:42:09.998 INFO:tasks.workunit.client.1.smithi104.stdout:5/126: creat d0/d1/d1e/f21 x:0 0 0 2024-04-26T04:42:09.998 INFO:tasks.workunit.client.1.smithi104.stdout:5/127: write d0/d11/f19 [1258778,74233] 0 2024-04-26T04:42:09.999 INFO:tasks.workunit.client.1.smithi104.stdout:4/62: read d4/d7/fa [3590197,1419] 0 2024-04-26T04:42:09.999 INFO:tasks.workunit.client.1.smithi104.stdout:4/63: stat d4/c6 0 2024-04-26T04:42:09.999 INFO:tasks.workunit.client.1.smithi104.stdout:4/64: truncate d4/d7/fa 4866285 0 2024-04-26T04:42:09.999 INFO:tasks.workunit.client.1.smithi104.stdout:4/65: stat d4/db/d13 0 2024-04-26T04:42:09.999 INFO:tasks.workunit.client.1.smithi104.stdout:4/66: dread - d4/d7/ff zero size 2024-04-26T04:42:09.999 INFO:tasks.workunit.client.0.smithi090.stdout:5/10: symlink l2 0 2024-04-26T04:42:10.000 INFO:tasks.workunit.client.1.smithi104.stdout:5/128: unlink d0/d1/c10 0 2024-04-26T04:42:10.000 INFO:tasks.workunit.client.0.smithi090.stdout:0/18: mknod d0/c7 0 2024-04-26T04:42:10.001 INFO:tasks.workunit.client.0.smithi090.stdout:0/19: chown d0/c7 6 1 2024-04-26T04:42:10.003 INFO:tasks.workunit.client.0.smithi090.stdout:1/8: mkdir d0/d4 0 2024-04-26T04:42:10.004 INFO:tasks.workunit.client.1.smithi104.stdout:5/129: dread d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:10.004 INFO:tasks.workunit.client.1.smithi104.stdout:5/130: chown d0/f18 233259 1 2024-04-26T04:42:10.004 INFO:tasks.workunit.client.1.smithi104.stdout:6/257: dwrite d0/d9/d25/f39 [0,4194304] 0 2024-04-26T04:42:10.004 INFO:tasks.workunit.client.1.smithi104.stdout:6/258: fsync d0/d9/fc 0 2024-04-26T04:42:10.004 INFO:tasks.workunit.client.1.smithi104.stdout:6/259: chown d0/d9/d13/d15/f42 358 1 2024-04-26T04:42:10.004 INFO:tasks.workunit.client.1.smithi104.stdout:4/67: dread d4/d7/fa [0,4194304] 0 2024-04-26T04:42:10.004 INFO:tasks.workunit.client.1.smithi104.stdout:4/68: readlink - no filename 2024-04-26T04:42:10.007 INFO:tasks.workunit.client.0.smithi090.stdout:8/12: mknod c2 0 2024-04-26T04:42:10.014 INFO:tasks.workunit.client.0.smithi090.stdout:6/6: creat f3 x:0 0 0 2024-04-26T04:42:10.014 INFO:tasks.workunit.client.0.smithi090.stdout:6/7: creat f4 x:0 0 0 2024-04-26T04:42:10.014 INFO:tasks.workunit.client.0.smithi090.stdout:6/8: fdatasync f3 0 2024-04-26T04:42:10.015 INFO:tasks.workunit.client.0.smithi090.stdout:5/11: unlink l2 0 2024-04-26T04:42:10.015 INFO:tasks.workunit.client.0.smithi090.stdout:5/12: rename - no filename 2024-04-26T04:42:10.015 INFO:tasks.workunit.client.0.smithi090.stdout:5/13: dwrite - no filename 2024-04-26T04:42:10.016 INFO:tasks.workunit.client.0.smithi090.stdout:1/9: symlink d0/l5 0 2024-04-26T04:42:10.016 INFO:tasks.workunit.client.0.smithi090.stdout:1/10: write d0/f1 [375896,48137] 0 2024-04-26T04:42:10.018 INFO:tasks.workunit.client.0.smithi090.stdout:5/14: mknod c3 0 2024-04-26T04:42:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/15: dread - no filename 2024-04-26T04:42:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/16: write - no filename 2024-04-26T04:42:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/17: dwrite - no filename 2024-04-26T04:42:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/18: stat c3 0 2024-04-26T04:42:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/19: dwrite - no filename 2024-04-26T04:42:10.019 INFO:tasks.workunit.client.0.smithi090.stdout:5/20: fsync - no filename 2024-04-26T04:42:10.020 INFO:tasks.workunit.client.0.smithi090.stdout:1/11: unlink d0/f1 0 2024-04-26T04:42:10.064 INFO:tasks.workunit.client.0.smithi090.stdout:6/9: dwrite f1 [0,4194304] 0 2024-04-26T04:42:10.066 INFO:tasks.workunit.client.1.smithi104.stdout:4/69: dwrite d4/f11 [0,4194304] 0 2024-04-26T04:42:10.067 INFO:tasks.workunit.client.1.smithi104.stdout:6/260: dwrite d0/d9/d13/d15/f42 [0,4194304] 0 2024-04-26T04:42:10.067 INFO:tasks.workunit.client.0.smithi090.stdout:8/13: dwrite f0 [0,4194304] 0 2024-04-26T04:42:10.067 INFO:tasks.workunit.client.0.smithi090.stdout:0/20: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:42:10.067 INFO:tasks.workunit.client.0.smithi090.stdout:1/12: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:10.068 INFO:tasks.workunit.client.0.smithi090.stdout:1/13: fsync d0/f2 0 2024-04-26T04:42:10.068 INFO:tasks.workunit.client.0.smithi090.stdout:6/10: unlink f4 0 2024-04-26T04:42:10.068 INFO:tasks.workunit.client.1.smithi104.stdout:5/131: dwrite d0/d1/d1e/f21 [0,4194304] 0 2024-04-26T04:42:10.068 INFO:tasks.workunit.client.0.smithi090.stdout:8/14: symlink l3 0 2024-04-26T04:42:10.071 INFO:tasks.workunit.client.0.smithi090.stdout:0/21: rename d0/c6 to d0/c8 0 2024-04-26T04:42:10.074 INFO:tasks.workunit.client.1.smithi104.stdout:6/261: symlink d0/d9/d25/l57 0 2024-04-26T04:42:10.077 INFO:tasks.workunit.client.0.smithi090.stdout:8/15: read f0 [395072,95325] 0 2024-04-26T04:42:10.079 INFO:tasks.workunit.client.1.smithi104.stdout:6/262: creat d0/d9/d13/d44/f58 x:0 0 0 2024-04-26T04:42:10.080 INFO:tasks.workunit.client.1.smithi104.stdout:6/263: symlink d0/d5/l59 0 2024-04-26T04:42:10.080 INFO:tasks.workunit.client.1.smithi104.stdout:5/132: write d0/d1/d4/f17 [5020072,77619] 0 2024-04-26T04:42:10.081 INFO:tasks.workunit.client.1.smithi104.stdout:6/264: mknod d0/d9/d13/c5a 0 2024-04-26T04:42:10.081 INFO:tasks.workunit.client.1.smithi104.stdout:6/265: write d0/d9/d25/f39 [4484844,92315] 0 2024-04-26T04:42:10.082 INFO:tasks.workunit.client.1.smithi104.stdout:6/266: symlink d0/d5/l5b 0 2024-04-26T04:42:10.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/14: symlink d0/l6 0 2024-04-26T04:42:10.085 INFO:tasks.workunit.client.1.smithi104.stdout:6/267: rename d0/d9/d13/d16 to d0/d9/d5c 0 2024-04-26T04:42:10.086 INFO:tasks.workunit.client.0.smithi090.stdout:8/16: rename f0 to f4 0 2024-04-26T04:42:10.086 INFO:tasks.workunit.client.0.smithi090.stdout:8/17: readlink l3 0 2024-04-26T04:42:10.086 INFO:tasks.workunit.client.0.smithi090.stdout:8/18: read f4 [3205335,99479] 0 2024-04-26T04:42:10.089 INFO:tasks.workunit.client.0.smithi090.stdout:8/19: dread f4 [0,4194304] 0 2024-04-26T04:42:10.089 INFO:tasks.workunit.client.0.smithi090.stdout:8/20: creat f5 x:0 0 0 2024-04-26T04:42:10.090 INFO:tasks.workunit.client.1.smithi104.stdout:0/124: sync 2024-04-26T04:42:10.090 INFO:tasks.workunit.client.1.smithi104.stdout:7/24: sync 2024-04-26T04:42:10.090 INFO:tasks.workunit.client.1.smithi104.stdout:7/25: write d2/f3 [2660195,127326] 0 2024-04-26T04:42:10.091 INFO:tasks.workunit.client.1.smithi104.stdout:2/89: sync 2024-04-26T04:42:10.091 INFO:tasks.workunit.client.1.smithi104.stdout:2/90: stat d6/l18 0 2024-04-26T04:42:10.091 INFO:tasks.workunit.client.0.smithi090.stdout:1/15: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:10.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/11: dwrite f1 [0,4194304] 0 2024-04-26T04:42:10.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/12: creat f5 x:0 0 0 2024-04-26T04:42:10.092 INFO:tasks.workunit.client.1.smithi104.stdout:8/45: sync 2024-04-26T04:42:10.092 INFO:tasks.workunit.client.1.smithi104.stdout:3/53: sync 2024-04-26T04:42:10.092 INFO:tasks.workunit.client.1.smithi104.stdout:9/125: sync 2024-04-26T04:42:10.092 INFO:tasks.workunit.client.1.smithi104.stdout:0/125: write d0/d2/fd [139497,110474] 0 2024-04-26T04:42:10.096 INFO:tasks.workunit.client.1.smithi104.stdout:2/91: mkdir d6/d8/d13/d21 0 2024-04-26T04:42:10.097 INFO:tasks.workunit.client.1.smithi104.stdout:2/92: chown d6/f10 84079767 1 2024-04-26T04:42:10.097 INFO:tasks.workunit.client.1.smithi104.stdout:2/93: chown d6/l7 4533 1 2024-04-26T04:42:10.097 INFO:tasks.workunit.client.1.smithi104.stdout:8/46: symlink d0/ld 0 2024-04-26T04:42:10.098 INFO:tasks.workunit.client.1.smithi104.stdout:3/54: dread f0 [0,4194304] 0 2024-04-26T04:42:10.098 INFO:tasks.workunit.client.1.smithi104.stdout:3/55: write d5/d9/ff [111406,29440] 0 2024-04-26T04:42:10.098 INFO:tasks.workunit.client.1.smithi104.stdout:3/56: stat d5/d9/cb 0 2024-04-26T04:42:10.099 INFO:tasks.workunit.client.1.smithi104.stdout:2/94: mknod d6/d8/d13/d21/c22 0 2024-04-26T04:42:10.099 INFO:tasks.workunit.client.1.smithi104.stdout:2/95: fdatasync d6/de/f1e 0 2024-04-26T04:42:10.099 INFO:tasks.workunit.client.1.smithi104.stdout:2/96: fsync d6/d1b/f1c 0 2024-04-26T04:42:10.099 INFO:tasks.workunit.client.1.smithi104.stdout:2/97: stat d6/de/f1e 0 2024-04-26T04:42:10.100 INFO:tasks.workunit.client.1.smithi104.stdout:2/98: truncate d6/fd 913901 0 2024-04-26T04:42:10.100 INFO:tasks.workunit.client.1.smithi104.stdout:2/99: creat d6/d8/f23 x:0 0 0 2024-04-26T04:42:10.100 INFO:tasks.workunit.client.0.smithi090.stdout:6/13: write f1 [1960267,80888] 0 2024-04-26T04:42:10.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/14: creat f6 x:0 0 0 2024-04-26T04:42:10.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/15: write f6 [500620,119856] 0 2024-04-26T04:42:10.102 INFO:tasks.workunit.client.1.smithi104.stdout:8/47: rename d0/fc to d0/fe 0 2024-04-26T04:42:10.102 INFO:tasks.workunit.client.1.smithi104.stdout:8/48: chown d0/l3 77287 1 2024-04-26T04:42:10.102 INFO:tasks.workunit.client.1.smithi104.stdout:8/49: write d0/f5 [429335,48500] 0 2024-04-26T04:42:10.102 INFO:tasks.workunit.client.1.smithi104.stdout:8/50: chown d0/f5 2282 1 2024-04-26T04:42:10.102 INFO:tasks.workunit.client.1.smithi104.stdout:8/51: stat d0/l4 0 2024-04-26T04:42:10.103 INFO:tasks.workunit.client.1.smithi104.stdout:8/52: dread - d0/fb zero size 2024-04-26T04:42:10.104 INFO:tasks.workunit.client.1.smithi104.stdout:6/268: dwrite d0/d9/d13/d15/f2b [0,4194304] 0 2024-04-26T04:42:10.104 INFO:tasks.workunit.client.1.smithi104.stdout:6/269: fdatasync d0/d9/d13/f43 0 2024-04-26T04:42:10.105 INFO:tasks.workunit.client.1.smithi104.stdout:3/57: unlink d5/d9/dc/f10 0 2024-04-26T04:42:10.105 INFO:tasks.workunit.client.1.smithi104.stdout:3/58: fdatasync d5/d9/f12 0 2024-04-26T04:42:10.106 INFO:tasks.workunit.client.1.smithi104.stdout:3/59: dread f0 [4194304,4194304] 0 2024-04-26T04:42:10.106 INFO:tasks.workunit.client.1.smithi104.stdout:3/60: creat d5/d9/f14 x:0 0 0 2024-04-26T04:42:10.106 INFO:tasks.workunit.client.1.smithi104.stdout:3/61: readlink d5/d9/dc/l13 0 2024-04-26T04:42:10.109 INFO:tasks.workunit.client.1.smithi104.stdout:8/53: symlink d0/lf 0 2024-04-26T04:42:10.109 INFO:tasks.workunit.client.1.smithi104.stdout:8/54: chown d0/l2 2 1 2024-04-26T04:42:10.115 INFO:tasks.workunit.client.1.smithi104.stdout:6/270: creat d0/d9/d5c/d3f/f5d x:0 0 0 2024-04-26T04:42:10.118 INFO:tasks.workunit.client.1.smithi104.stdout:8/55: rename d0/l8 to d0/l10 0 2024-04-26T04:42:10.118 INFO:tasks.workunit.client.1.smithi104.stdout:8/56: readlink d0/l4 0 2024-04-26T04:42:10.142 INFO:tasks.workunit.client.1.smithi104.stdout:6/271: write d0/d9/d25/f39 [803711,120418] 0 2024-04-26T04:42:10.142 INFO:tasks.workunit.client.1.smithi104.stdout:6/272: chown d0/d9/d52 58773863 1 2024-04-26T04:42:10.142 INFO:tasks.workunit.client.1.smithi104.stdout:6/273: stat d0/d9/l4d 0 2024-04-26T04:42:10.142 INFO:tasks.workunit.client.1.smithi104.stdout:6/274: write d0/f29 [1293366,89684] 0 2024-04-26T04:42:10.142 INFO:tasks.workunit.client.1.smithi104.stdout:6/275: creat d0/d9/d13/d15/f5e x:0 0 0 2024-04-26T04:42:10.142 INFO:tasks.workunit.client.1.smithi104.stdout:6/276: symlink d0/d1b/l5f 0 2024-04-26T04:42:10.143 INFO:tasks.workunit.client.1.smithi104.stdout:6/277: creat d0/d9/d13/d15/f60 x:0 0 0 2024-04-26T04:42:10.144 INFO:tasks.workunit.client.1.smithi104.stdout:6/278: dread d0/d9/d25/f34 [0,4194304] 0 2024-04-26T04:42:10.144 INFO:tasks.workunit.client.1.smithi104.stdout:6/279: truncate d0/d9/d13/d15/f5e 965420 0 2024-04-26T04:42:10.144 INFO:tasks.workunit.client.1.smithi104.stdout:6/280: write d0/d9/d5c/f19 [1517027,91813] 0 2024-04-26T04:42:10.145 INFO:tasks.workunit.client.1.smithi104.stdout:6/281: read d0/d9/d5c/f4b [2935,113378] 0 2024-04-26T04:42:10.145 INFO:tasks.workunit.client.1.smithi104.stdout:6/282: fdatasync d0/d9/d13/d44/d47/f4c 0 2024-04-26T04:42:10.145 INFO:tasks.workunit.client.1.smithi104.stdout:6/283: creat d0/d9/d52/f61 x:0 0 0 2024-04-26T04:42:10.146 INFO:tasks.workunit.client.1.smithi104.stdout:6/284: symlink d0/d9/d13/d41/l62 0 2024-04-26T04:42:10.146 INFO:tasks.workunit.client.0.smithi090.stdout:8/21: dwrite f5 [0,4194304] 0 2024-04-26T04:42:10.146 INFO:tasks.workunit.client.0.smithi090.stdout:8/22: fdatasync f4 0 2024-04-26T04:42:10.147 INFO:tasks.workunit.client.0.smithi090.stdout:8/23: creat f6 x:0 0 0 2024-04-26T04:42:10.147 INFO:tasks.workunit.client.0.smithi090.stdout:6/16: dwrite f6 [0,4194304] 0 2024-04-26T04:42:10.148 INFO:tasks.workunit.client.1.smithi104.stdout:0/126: dwrite d0/d2/fd [0,4194304] 0 2024-04-26T04:42:10.148 INFO:tasks.workunit.client.1.smithi104.stdout:9/126: dwrite d11/f12 [0,4194304] 0 2024-04-26T04:42:10.148 INFO:tasks.workunit.client.1.smithi104.stdout:7/26: dwrite f0 [0,4194304] 0 2024-04-26T04:42:10.148 INFO:tasks.workunit.client.1.smithi104.stdout:2/100: dwrite d6/de/f1e [0,4194304] 0 2024-04-26T04:42:10.148 INFO:tasks.workunit.client.1.smithi104.stdout:6/285: mknod d0/d9/d13/c63 0 2024-04-26T04:42:10.151 INFO:tasks.workunit.client.1.smithi104.stdout:0/127: read - d0/d4/f11 zero size 2024-04-26T04:42:10.151 INFO:tasks.workunit.client.1.smithi104.stdout:0/128: chown d0/d2/fd 1 1 2024-04-26T04:42:10.151 INFO:tasks.workunit.client.1.smithi104.stdout:0/129: readlink d0/l1a 0 2024-04-26T04:42:10.155 INFO:tasks.workunit.client.1.smithi104.stdout:3/62: rmdir d5/d9/dc 39 2024-04-26T04:42:10.165 INFO:tasks.workunit.client.1.smithi104.stdout:2/101: rename d6/d8/f23 to d6/de/f24 0 2024-04-26T04:42:10.166 INFO:tasks.workunit.client.1.smithi104.stdout:2/102: dread d6/d1b/f1c [0,4194304] 0 2024-04-26T04:42:10.167 INFO:tasks.workunit.client.1.smithi104.stdout:7/27: dread f0 [0,4194304] 0 2024-04-26T04:42:10.168 INFO:tasks.workunit.client.1.smithi104.stdout:2/103: creat d6/de/d1d/f25 x:0 0 0 2024-04-26T04:42:10.170 INFO:tasks.workunit.client.1.smithi104.stdout:2/104: truncate d6/fc 1987871 0 2024-04-26T04:42:10.171 INFO:tasks.workunit.client.1.smithi104.stdout:2/105: write d6/d8/f19 [97135,119614] 0 2024-04-26T04:42:10.173 INFO:tasks.workunit.client.1.smithi104.stdout:2/106: creat d6/d8/d13/f26 x:0 0 0 2024-04-26T04:42:10.173 INFO:tasks.workunit.client.1.smithi104.stdout:7/28: dread f0 [0,4194304] 0 2024-04-26T04:42:10.173 INFO:tasks.workunit.client.1.smithi104.stdout:7/29: chown f0 20784629 1 2024-04-26T04:42:10.176 INFO:tasks.workunit.client.1.smithi104.stdout:8/57: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:42:10.182 INFO:tasks.workunit.client.1.smithi104.stdout:3/63: dwrite d5/f8 [0,4194304] 0 2024-04-26T04:42:10.182 INFO:tasks.workunit.client.1.smithi104.stdout:3/64: fsync f0 0 2024-04-26T04:42:10.182 INFO:tasks.workunit.client.0.smithi090.stdout:6/17: dwrite f5 [0,4194304] 0 2024-04-26T04:42:10.183 INFO:tasks.workunit.client.1.smithi104.stdout:9/127: dwrite d11/f1a [4194304,4194304] 0 2024-04-26T04:42:10.183 INFO:tasks.workunit.client.1.smithi104.stdout:9/128: fdatasync fb 0 2024-04-26T04:42:10.183 INFO:tasks.workunit.client.1.smithi104.stdout:3/65: dread f4 [0,4194304] 0 2024-04-26T04:42:10.193 INFO:tasks.workunit.client.1.smithi104.stdout:3/66: creat d5/d9/dc/f15 x:0 0 0 2024-04-26T04:42:10.193 INFO:tasks.workunit.client.1.smithi104.stdout:3/67: stat d5/d9/f12 0 2024-04-26T04:42:10.198 INFO:tasks.workunit.client.1.smithi104.stdout:2/107: dwrite d6/d8/fb [0,4194304] 0 2024-04-26T04:42:10.199 INFO:tasks.workunit.client.1.smithi104.stdout:3/68: dread f3 [0,4194304] 0 2024-04-26T04:42:10.200 INFO:tasks.workunit.client.1.smithi104.stdout:3/69: creat d5/d9/dc/f16 x:0 0 0 2024-04-26T04:42:10.200 INFO:tasks.workunit.client.1.smithi104.stdout:2/108: getdents d6/d8/d13 0 2024-04-26T04:42:10.200 INFO:tasks.workunit.client.1.smithi104.stdout:3/70: write d5/d9/fa [311697,78845] 0 2024-04-26T04:42:10.200 INFO:tasks.workunit.client.1.smithi104.stdout:2/109: write d6/de/f24 [179131,85300] 0 2024-04-26T04:42:10.200 INFO:tasks.workunit.client.1.smithi104.stdout:2/110: write d6/de/d1d/f25 [860836,63576] 0 2024-04-26T04:42:10.203 INFO:tasks.workunit.client.1.smithi104.stdout:3/71: mkdir d5/d9/d17 0 2024-04-26T04:42:10.203 INFO:tasks.workunit.client.1.smithi104.stdout:3/72: fsync d5/f8 0 2024-04-26T04:42:10.203 INFO:tasks.workunit.client.1.smithi104.stdout:3/73: fdatasync d5/d9/dc/f15 0 2024-04-26T04:42:10.203 INFO:tasks.workunit.client.1.smithi104.stdout:3/74: symlink d5/d9/dc/l18 0 2024-04-26T04:42:10.204 INFO:tasks.workunit.client.1.smithi104.stdout:8/58: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:42:10.205 INFO:tasks.workunit.client.1.smithi104.stdout:3/75: dread d5/d9/ff [0,4194304] 0 2024-04-26T04:42:10.205 INFO:tasks.workunit.client.1.smithi104.stdout:7/30: dwrite f0 [0,4194304] 0 2024-04-26T04:42:10.206 INFO:tasks.workunit.client.1.smithi104.stdout:2/111: rmdir d6 39 2024-04-26T04:42:10.212 INFO:tasks.workunit.client.1.smithi104.stdout:7/31: symlink d2/l9 0 2024-04-26T04:42:10.212 INFO:tasks.workunit.client.1.smithi104.stdout:7/32: stat d2/d8 0 2024-04-26T04:42:10.212 INFO:tasks.workunit.client.1.smithi104.stdout:2/112: dread d6/d8/d13/f1f [0,4194304] 0 2024-04-26T04:42:10.213 INFO:tasks.workunit.client.1.smithi104.stdout:2/113: stat d6/c9 0 2024-04-26T04:42:10.213 INFO:tasks.workunit.client.1.smithi104.stdout:2/114: readlink d6/l11 0 2024-04-26T04:42:10.213 INFO:tasks.workunit.client.1.smithi104.stdout:2/115: fdatasync d6/d8/f19 0 2024-04-26T04:42:10.215 INFO:tasks.workunit.client.1.smithi104.stdout:3/76: rename d5/f8 to d5/d9/dc/f19 0 2024-04-26T04:42:10.215 INFO:tasks.workunit.client.1.smithi104.stdout:3/77: fdatasync d5/d9/dc/f16 0 2024-04-26T04:42:10.216 INFO:tasks.workunit.client.1.smithi104.stdout:2/116: creat d6/d8/d13/d21/f27 x:0 0 0 2024-04-26T04:42:10.217 INFO:tasks.workunit.client.1.smithi104.stdout:2/117: write d6/d1b/f1c [271717,90454] 0 2024-04-26T04:42:10.226 INFO:tasks.workunit.client.1.smithi104.stdout:2/118: rmdir d6/d8/d13/d21 39 2024-04-26T04:42:10.226 INFO:tasks.workunit.client.1.smithi104.stdout:2/119: write d6/de/d1d/f25 [1410750,79846] 0 2024-04-26T04:42:10.227 INFO:tasks.workunit.client.1.smithi104.stdout:2/120: symlink d6/d1b/l28 0 2024-04-26T04:42:10.229 INFO:tasks.workunit.client.1.smithi104.stdout:2/121: write d6/de/d1d/f25 [177374,112949] 0 2024-04-26T04:42:10.230 INFO:tasks.workunit.client.1.smithi104.stdout:2/122: mkdir d6/d8/d13/d21/d29 0 2024-04-26T04:42:10.231 INFO:tasks.workunit.client.1.smithi104.stdout:2/123: rmdir d6/d1b 39 2024-04-26T04:42:10.232 INFO:tasks.workunit.client.1.smithi104.stdout:7/33: dwrite f0 [0,4194304] 0 2024-04-26T04:42:10.233 INFO:tasks.workunit.client.1.smithi104.stdout:3/78: dwrite d5/d9/fe [0,4194304] 0 2024-04-26T04:42:10.235 INFO:tasks.workunit.client.1.smithi104.stdout:7/34: rename d2/l9 to d2/la 0 2024-04-26T04:42:10.235 INFO:tasks.workunit.client.1.smithi104.stdout:7/35: fdatasync f0 0 2024-04-26T04:42:10.242 INFO:tasks.workunit.client.1.smithi104.stdout:7/36: mknod d2/cb 0 2024-04-26T04:42:10.247 INFO:tasks.workunit.client.1.smithi104.stdout:1/154: sync 2024-04-26T04:42:10.248 INFO:tasks.workunit.client.1.smithi104.stdout:1/155: symlink d0/d6/d19/d21/l2d 0 2024-04-26T04:42:10.248 INFO:tasks.workunit.client.1.smithi104.stdout:7/37: read d2/f3 [522360,116688] 0 2024-04-26T04:42:10.254 INFO:tasks.workunit.client.1.smithi104.stdout:7/38: dread f0 [0,4194304] 0 2024-04-26T04:42:10.256 INFO:tasks.workunit.client.1.smithi104.stdout:2/124: dwrite d6/fc [0,4194304] 0 2024-04-26T04:42:10.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/79: dwrite f4 [0,4194304] 0 2024-04-26T04:42:10.259 INFO:tasks.workunit.client.1.smithi104.stdout:2/125: rename d6/d1b/f1c to d6/d8/d13/f2a 0 2024-04-26T04:42:10.262 INFO:tasks.workunit.client.1.smithi104.stdout:3/80: symlink d5/l1a 0 2024-04-26T04:42:10.263 INFO:tasks.workunit.client.1.smithi104.stdout:2/126: unlink d6/d1b/l28 0 2024-04-26T04:42:10.264 INFO:tasks.workunit.client.1.smithi104.stdout:2/127: truncate d6/de/d1d/f25 1229177 0 2024-04-26T04:42:10.268 INFO:tasks.workunit.client.1.smithi104.stdout:2/128: dread d6/f10 [0,4194304] 0 2024-04-26T04:42:10.270 INFO:tasks.workunit.client.1.smithi104.stdout:2/129: creat d6/d8/d13/d21/d29/f2b x:0 0 0 2024-04-26T04:42:10.272 INFO:tasks.workunit.client.1.smithi104.stdout:2/130: rename d6/de/f1e to d6/d8/d13/d21/f2c 0 2024-04-26T04:42:10.272 INFO:tasks.workunit.client.1.smithi104.stdout:1/156: dwrite d0/d6/ff [0,4194304] 0 2024-04-26T04:42:10.273 INFO:tasks.workunit.client.1.smithi104.stdout:2/131: creat d6/d1b/f2d x:0 0 0 2024-04-26T04:42:10.273 INFO:tasks.workunit.client.1.smithi104.stdout:2/132: dread - d6/d8/d13/d21/d29/f2b zero size 2024-04-26T04:42:10.273 INFO:tasks.workunit.client.1.smithi104.stdout:2/133: readlink d6/l18 0 2024-04-26T04:42:10.273 INFO:tasks.workunit.client.1.smithi104.stdout:2/134: dread - d6/d8/d13/d21/f27 zero size 2024-04-26T04:42:10.275 INFO:tasks.workunit.client.1.smithi104.stdout:1/157: mkdir d0/d6/d2e 0 2024-04-26T04:42:10.275 INFO:tasks.workunit.client.1.smithi104.stdout:1/158: write d0/d6/f1a [3305701,28853] 0 2024-04-26T04:42:10.276 INFO:tasks.workunit.client.1.smithi104.stdout:2/135: rename d6/d8/d13/d21/f27 to d6/d8/d13/d21/f2e 0 2024-04-26T04:42:10.277 INFO:tasks.workunit.client.1.smithi104.stdout:1/159: mknod d0/d6/c2f 0 2024-04-26T04:42:10.277 INFO:tasks.workunit.client.1.smithi104.stdout:2/136: dread d6/fd [0,4194304] 0 2024-04-26T04:42:10.277 INFO:tasks.workunit.client.1.smithi104.stdout:2/137: truncate d6/d8/d13/d21/f2c 1725143 0 2024-04-26T04:42:10.278 INFO:tasks.workunit.client.1.smithi104.stdout:2/138: getdents d6/d8/d13 0 2024-04-26T04:42:10.279 INFO:tasks.workunit.client.1.smithi104.stdout:3/81: dwrite d5/d9/f12 [0,4194304] 0 2024-04-26T04:42:10.279 INFO:tasks.workunit.client.1.smithi104.stdout:3/82: read - d5/d9/dc/f16 zero size 2024-04-26T04:42:10.279 INFO:tasks.workunit.client.1.smithi104.stdout:3/83: write d5/d9/dc/f16 [793215,79143] 0 2024-04-26T04:42:10.279 INFO:tasks.workunit.client.1.smithi104.stdout:3/84: fsync f4 0 2024-04-26T04:42:10.285 INFO:tasks.workunit.client.1.smithi104.stdout:3/85: write f0 [1554644,107458] 0 2024-04-26T04:42:10.285 INFO:tasks.workunit.client.1.smithi104.stdout:2/139: dread d6/d8/f19 [0,4194304] 0 2024-04-26T04:42:10.287 INFO:tasks.workunit.client.1.smithi104.stdout:3/86: creat d5/d9/d17/f1b x:0 0 0 2024-04-26T04:42:10.297 INFO:tasks.workunit.client.1.smithi104.stdout:1/160: dwrite d0/d6/de/d20/d2a/f2c [0,4194304] 0 2024-04-26T04:42:10.299 INFO:tasks.workunit.client.1.smithi104.stdout:1/161: mkdir d0/d6/de/d30 0 2024-04-26T04:42:10.299 INFO:tasks.workunit.client.1.smithi104.stdout:1/162: chown d0/d6/d19 10330 1 2024-04-26T04:42:10.306 INFO:tasks.workunit.client.1.smithi104.stdout:1/163: dread d0/d6/de/d20/d2a/f2c [0,4194304] 0 2024-04-26T04:42:10.307 INFO:tasks.workunit.client.1.smithi104.stdout:1/164: rmdir d0/d6/d19/d1b 39 2024-04-26T04:42:10.307 INFO:tasks.workunit.client.1.smithi104.stdout:1/165: fsync d0/d6/de/d20/d2a/f2c 0 2024-04-26T04:42:10.312 INFO:tasks.workunit.client.1.smithi104.stdout:2/140: dwrite d6/d8/fb [0,4194304] 0 2024-04-26T04:42:10.313 INFO:tasks.workunit.client.1.smithi104.stdout:3/87: dwrite f4 [4194304,4194304] 0 2024-04-26T04:42:10.314 INFO:tasks.workunit.client.1.smithi104.stdout:2/141: symlink d6/de/l2f 0 2024-04-26T04:42:10.314 INFO:tasks.workunit.client.1.smithi104.stdout:2/142: fsync d6/d8/f17 0 2024-04-26T04:42:10.314 INFO:tasks.workunit.client.1.smithi104.stdout:2/143: chown d6/de/lf 0 1 2024-04-26T04:42:10.318 INFO:tasks.workunit.client.1.smithi104.stdout:3/88: rmdir d5/d9 39 2024-04-26T04:42:10.319 INFO:tasks.workunit.client.1.smithi104.stdout:2/144: rmdir d6/d8/d13/d21/d29 39 2024-04-26T04:42:10.319 INFO:tasks.workunit.client.1.smithi104.stdout:2/145: write d6/fc [4597713,17326] 0 2024-04-26T04:42:10.319 INFO:tasks.workunit.client.1.smithi104.stdout:2/146: chown f2 5764 1 2024-04-26T04:42:10.319 INFO:tasks.workunit.client.1.smithi104.stdout:3/89: creat d5/d9/dc/f1c x:0 0 0 2024-04-26T04:42:10.321 INFO:tasks.workunit.client.1.smithi104.stdout:2/147: unlink d6/d8/f19 0 2024-04-26T04:42:10.321 INFO:tasks.workunit.client.1.smithi104.stdout:3/90: creat d5/d9/f1d x:0 0 0 2024-04-26T04:42:10.322 INFO:tasks.workunit.client.1.smithi104.stdout:3/91: fdatasync d5/d9/dc/f1c 0 2024-04-26T04:42:10.322 INFO:tasks.workunit.client.1.smithi104.stdout:2/148: write d6/fd [166771,65078] 0 2024-04-26T04:42:10.322 INFO:tasks.workunit.client.1.smithi104.stdout:3/92: rmdir d5/d9/d17 39 2024-04-26T04:42:10.324 INFO:tasks.workunit.client.1.smithi104.stdout:1/166: dwrite d0/f14 [0,4194304] 0 2024-04-26T04:42:10.325 INFO:tasks.workunit.client.1.smithi104.stdout:1/167: creat d0/d6/de/d20/d2a/f31 x:0 0 0 2024-04-26T04:42:10.326 INFO:tasks.workunit.client.1.smithi104.stdout:1/168: mkdir d0/d6/d2e/d32 0 2024-04-26T04:42:10.333 INFO:tasks.workunit.client.1.smithi104.stdout:3/93: truncate f0 4365253 0 2024-04-26T04:42:10.333 INFO:tasks.workunit.client.1.smithi104.stdout:3/94: creat d5/d9/dc/f1e x:0 0 0 2024-04-26T04:42:10.336 INFO:tasks.workunit.client.1.smithi104.stdout:3/95: rename d5/c7 to d5/d9/d17/c1f 0 2024-04-26T04:42:10.336 INFO:tasks.workunit.client.1.smithi104.stdout:2/149: dwrite d6/d8/d13/f1f [4194304,4194304] 0 2024-04-26T04:42:10.336 INFO:tasks.workunit.client.1.smithi104.stdout:2/150: dread - d6/d8/d13/d21/f2e zero size 2024-04-26T04:42:10.343 INFO:tasks.workunit.client.1.smithi104.stdout:2/151: mknod d6/d1b/c30 0 2024-04-26T04:42:10.345 INFO:tasks.workunit.client.1.smithi104.stdout:1/169: dwrite d0/f14 [0,4194304] 0 2024-04-26T04:42:10.345 INFO:tasks.workunit.client.1.smithi104.stdout:1/170: write d0/d6/de/d20/d2a/f31 [168228,102265] 0 2024-04-26T04:42:10.345 INFO:tasks.workunit.client.1.smithi104.stdout:2/152: unlink d6/d1b/f2d 0 2024-04-26T04:42:10.350 INFO:tasks.workunit.client.1.smithi104.stdout:1/171: dread d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:10.353 INFO:tasks.workunit.client.1.smithi104.stdout:3/96: dwrite d5/d9/dc/f15 [0,4194304] 0 2024-04-26T04:42:10.353 INFO:tasks.workunit.client.1.smithi104.stdout:3/97: stat d5/d9/dc/f1c 0 2024-04-26T04:42:10.353 INFO:tasks.workunit.client.1.smithi104.stdout:3/98: write d5/d9/f14 [595082,129813] 0 2024-04-26T04:42:10.354 INFO:tasks.workunit.client.1.smithi104.stdout:1/172: dread d0/d6/de/d20/d2a/f31 [0,4194304] 0 2024-04-26T04:42:10.356 INFO:tasks.workunit.client.1.smithi104.stdout:3/99: getdents d5/d9/d17 0 2024-04-26T04:42:10.356 INFO:tasks.workunit.client.1.smithi104.stdout:3/100: chown d5/d9 5842 1 2024-04-26T04:42:10.356 INFO:tasks.workunit.client.1.smithi104.stdout:2/153: rmdir d6/d1b 39 2024-04-26T04:42:10.356 INFO:tasks.workunit.client.1.smithi104.stdout:2/154: chown d6/de/d1d 36 1 2024-04-26T04:42:10.357 INFO:tasks.workunit.client.1.smithi104.stdout:3/101: symlink d5/l20 0 2024-04-26T04:42:10.357 INFO:tasks.workunit.client.1.smithi104.stdout:3/102: chown f4 15 1 2024-04-26T04:42:10.358 INFO:tasks.workunit.client.1.smithi104.stdout:2/155: truncate d6/d8/fb 4118013 0 2024-04-26T04:42:10.358 INFO:tasks.workunit.client.1.smithi104.stdout:2/156: chown d6/d8/d13/f26 459 1 2024-04-26T04:42:10.358 INFO:tasks.workunit.client.1.smithi104.stdout:2/157: dread - d6/d8/f17 zero size 2024-04-26T04:42:10.358 INFO:tasks.workunit.client.1.smithi104.stdout:2/158: chown d6/l18 70211 1 2024-04-26T04:42:10.359 INFO:tasks.workunit.client.1.smithi104.stdout:2/159: creat d6/d1b/f31 x:0 0 0 2024-04-26T04:42:10.360 INFO:tasks.workunit.client.1.smithi104.stdout:3/103: getdents d5/d9/dc 0 2024-04-26T04:42:10.361 INFO:tasks.workunit.client.1.smithi104.stdout:2/160: symlink d6/de/l32 0 2024-04-26T04:42:10.362 INFO:tasks.workunit.client.1.smithi104.stdout:3/104: dread d5/d9/dc/f16 [0,4194304] 0 2024-04-26T04:42:10.363 INFO:tasks.workunit.client.1.smithi104.stdout:2/161: stat d6/l7 0 2024-04-26T04:42:10.363 INFO:tasks.workunit.client.1.smithi104.stdout:2/162: chown d6/d8/d13/d21/f2c 164 1 2024-04-26T04:42:10.364 INFO:tasks.workunit.client.1.smithi104.stdout:3/105: creat d5/d9/dc/f21 x:0 0 0 2024-04-26T04:42:10.364 INFO:tasks.workunit.client.1.smithi104.stdout:1/173: write d0/d6/ff [3927911,49573] 0 2024-04-26T04:42:10.364 INFO:tasks.workunit.client.1.smithi104.stdout:1/174: truncate d0/f14 5429504 0 2024-04-26T04:42:10.364 INFO:tasks.workunit.client.1.smithi104.stdout:2/163: creat d6/f33 x:0 0 0 2024-04-26T04:42:10.365 INFO:tasks.workunit.client.1.smithi104.stdout:1/175: getdents d0/d6/d19/d1b 0 2024-04-26T04:42:10.365 INFO:tasks.workunit.client.1.smithi104.stdout:1/176: chown d0/d6/de 7 1 2024-04-26T04:42:10.367 INFO:tasks.workunit.client.1.smithi104.stdout:1/177: creat d0/d6/de/f33 x:0 0 0 2024-04-26T04:42:10.377 INFO:tasks.workunit.client.1.smithi104.stdout:1/178: dread d0/f14 [0,4194304] 0 2024-04-26T04:42:10.381 INFO:tasks.workunit.client.1.smithi104.stdout:1/179: dread d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:10.381 INFO:tasks.workunit.client.1.smithi104.stdout:1/180: dread - d0/d6/de/f33 zero size 2024-04-26T04:42:10.398 INFO:tasks.workunit.client.1.smithi104.stdout:3/106: dwrite d5/d9/f1d [0,4194304] 0 2024-04-26T04:42:10.398 INFO:tasks.workunit.client.1.smithi104.stdout:2/164: dwrite d6/de/d1d/f25 [0,4194304] 0 2024-04-26T04:42:10.402 INFO:tasks.workunit.client.1.smithi104.stdout:2/165: mknod d6/c34 0 2024-04-26T04:42:10.403 INFO:tasks.workunit.client.1.smithi104.stdout:1/181: dwrite d0/d6/de/d20/d2a/f31 [0,4194304] 0 2024-04-26T04:42:10.404 INFO:tasks.workunit.client.1.smithi104.stdout:2/166: rename d6/d8/d13/f1f to d6/f35 0 2024-04-26T04:42:10.407 INFO:tasks.workunit.client.1.smithi104.stdout:2/167: creat d6/de/f36 x:0 0 0 2024-04-26T04:42:10.409 INFO:tasks.workunit.client.1.smithi104.stdout:2/168: creat d6/d8/f37 x:0 0 0 2024-04-26T04:42:10.409 INFO:tasks.workunit.client.1.smithi104.stdout:2/169: write d6/de/d1d/f25 [4834426,7399] 0 2024-04-26T04:42:10.409 INFO:tasks.workunit.client.1.smithi104.stdout:2/170: truncate d6/d8/f17 644434 0 2024-04-26T04:42:10.420 INFO:tasks.workunit.client.1.smithi104.stdout:2/171: read d6/f35 [7846373,107895] 0 2024-04-26T04:42:10.421 INFO:tasks.workunit.client.1.smithi104.stdout:3/107: dwrite d5/d9/f12 [4194304,4194304] 0 2024-04-26T04:42:10.425 INFO:tasks.workunit.client.1.smithi104.stdout:1/182: dwrite d0/d6/de/d20/d2a/f31 [0,4194304] 0 2024-04-26T04:42:10.428 INFO:tasks.workunit.client.1.smithi104.stdout:1/183: write d0/d6/de/d20/d2a/f31 [4018176,55326] 0 2024-04-26T04:42:10.428 INFO:tasks.workunit.client.1.smithi104.stdout:1/184: chown d0/d6/f1a 703660 1 2024-04-26T04:42:10.431 INFO:tasks.workunit.client.1.smithi104.stdout:1/185: mknod d0/d6/c34 0 2024-04-26T04:42:10.433 INFO:tasks.workunit.client.1.smithi104.stdout:1/186: mkdir d0/d6/d19/d1b/d35 0 2024-04-26T04:42:10.450 INFO:tasks.workunit.client.1.smithi104.stdout:2/172: dwrite f2 [0,4194304] 0 2024-04-26T04:42:10.451 INFO:tasks.workunit.client.1.smithi104.stdout:3/108: dwrite d5/d9/dc/f15 [0,4194304] 0 2024-04-26T04:42:10.451 INFO:tasks.workunit.client.1.smithi104.stdout:2/173: chown d6/f10 14319 1 2024-04-26T04:42:10.451 INFO:tasks.workunit.client.1.smithi104.stdout:3/109: creat d5/f22 x:0 0 0 2024-04-26T04:42:10.451 INFO:tasks.workunit.client.1.smithi104.stdout:3/110: creat d5/f23 x:0 0 0 2024-04-26T04:42:10.451 INFO:tasks.workunit.client.1.smithi104.stdout:1/187: dwrite d0/f14 [4194304,4194304] 0 2024-04-26T04:42:10.455 INFO:tasks.workunit.client.1.smithi104.stdout:1/188: dread d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:10.455 INFO:tasks.workunit.client.1.smithi104.stdout:1/189: readlink d0/d6/d19/d21/l1d 0 2024-04-26T04:42:10.455 INFO:tasks.workunit.client.1.smithi104.stdout:1/190: dread - d0/d6/de/f33 zero size 2024-04-26T04:42:10.456 INFO:tasks.workunit.client.1.smithi104.stdout:1/191: rename d0/d6/de/f33 to d0/d6/d19/d21/d28/f36 0 2024-04-26T04:42:10.456 INFO:tasks.workunit.client.1.smithi104.stdout:1/192: fsync d0/d6/f1a 0 2024-04-26T04:42:10.456 INFO:tasks.workunit.client.1.smithi104.stdout:1/193: fdatasync d0/d6/ff 0 2024-04-26T04:42:10.457 INFO:tasks.workunit.client.1.smithi104.stdout:1/194: chown d0/d6/l27 60 1 2024-04-26T04:42:10.469 INFO:tasks.workunit.client.1.smithi104.stdout:3/111: write d5/d9/f1d [1231683,15886] 0 2024-04-26T04:42:10.469 INFO:tasks.workunit.client.1.smithi104.stdout:3/112: creat d5/f24 x:0 0 0 2024-04-26T04:42:10.470 INFO:tasks.workunit.client.1.smithi104.stdout:2/174: dwrite d6/d8/d13/f2a [0,4194304] 0 2024-04-26T04:42:10.470 INFO:tasks.workunit.client.1.smithi104.stdout:2/175: creat d6/d1b/f38 x:0 0 0 2024-04-26T04:42:10.470 INFO:tasks.workunit.client.1.smithi104.stdout:2/176: creat d6/d8/f39 x:0 0 0 2024-04-26T04:42:10.473 INFO:tasks.workunit.client.1.smithi104.stdout:3/113: dread f3 [0,4194304] 0 2024-04-26T04:42:10.473 INFO:tasks.workunit.client.1.smithi104.stdout:3/114: creat d5/f25 x:0 0 0 2024-04-26T04:42:10.473 INFO:tasks.workunit.client.1.smithi104.stdout:2/177: symlink d6/d1b/l3a 0 2024-04-26T04:42:10.476 INFO:tasks.workunit.client.1.smithi104.stdout:1/195: dwrite d0/d6/ff [0,4194304] 0 2024-04-26T04:42:10.490 INFO:tasks.workunit.client.1.smithi104.stdout:2/178: dwrite d6/d8/f37 [0,4194304] 0 2024-04-26T04:42:10.492 INFO:tasks.workunit.client.1.smithi104.stdout:2/179: symlink d6/d8/d13/l3b 0 2024-04-26T04:42:10.492 INFO:tasks.workunit.client.1.smithi104.stdout:2/180: truncate d6/d8/d13/d21/d29/f2b 104513 0 2024-04-26T04:42:10.493 INFO:tasks.workunit.client.1.smithi104.stdout:2/181: mknod d6/d8/d13/d21/c3c 0 2024-04-26T04:42:10.494 INFO:tasks.workunit.client.1.smithi104.stdout:2/182: creat d6/d8/d13/f3d x:0 0 0 2024-04-26T04:42:10.499 INFO:tasks.workunit.client.1.smithi104.stdout:2/183: rename d6/d8/d13 to d6/d8/d3e 0 2024-04-26T04:42:10.500 INFO:tasks.workunit.client.1.smithi104.stdout:2/184: rename d6/l11 to d6/d8/l3f 0 2024-04-26T04:42:10.500 INFO:tasks.workunit.client.1.smithi104.stdout:2/185: write d6/fd [1607934,24324] 0 2024-04-26T04:42:10.501 INFO:tasks.workunit.client.1.smithi104.stdout:2/186: dread d6/d8/f17 [0,4194304] 0 2024-04-26T04:42:10.501 INFO:tasks.workunit.client.1.smithi104.stdout:2/187: readlink d6/l18 0 2024-04-26T04:42:10.504 INFO:tasks.workunit.client.1.smithi104.stdout:2/188: rename d6/d8/d3e/d21/d29/f2b to d6/de/f40 0 2024-04-26T04:42:10.504 INFO:tasks.workunit.client.1.smithi104.stdout:2/189: write d6/de/f36 [576074,85891] 0 2024-04-26T04:42:10.504 INFO:tasks.workunit.client.1.smithi104.stdout:2/190: fsync d6/de/d1d/f25 0 2024-04-26T04:42:10.505 INFO:tasks.workunit.client.1.smithi104.stdout:2/191: link d6/de/l32 d6/d1b/l41 0 2024-04-26T04:42:10.506 INFO:tasks.workunit.client.1.smithi104.stdout:2/192: truncate d6/f10 4846972 0 2024-04-26T04:42:10.508 INFO:tasks.workunit.client.1.smithi104.stdout:5/133: sync 2024-04-26T04:42:10.508 INFO:tasks.workunit.client.1.smithi104.stdout:4/70: sync 2024-04-26T04:42:10.508 INFO:tasks.workunit.client.1.smithi104.stdout:4/71: truncate d4/d7/ff 410887 0 2024-04-26T04:42:10.511 INFO:tasks.workunit.client.1.smithi104.stdout:4/72: link d4/cd d4/db/d13/c19 0 2024-04-26T04:42:10.518 INFO:tasks.workunit.client.1.smithi104.stdout:4/73: dread d4/d7/f8 [0,4194304] 0 2024-04-26T04:42:10.518 INFO:tasks.workunit.client.1.smithi104.stdout:4/74: chown d4/db/d13/d18 8 1 2024-04-26T04:42:10.528 INFO:tasks.workunit.client.1.smithi104.stdout:2/193: dwrite d6/d8/d3e/f26 [0,4194304] 0 2024-04-26T04:42:10.530 INFO:tasks.workunit.client.1.smithi104.stdout:5/134: dwrite d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:10.530 INFO:tasks.workunit.client.1.smithi104.stdout:5/135: creat d0/f22 x:0 0 0 2024-04-26T04:42:10.530 INFO:tasks.workunit.client.1.smithi104.stdout:5/136: creat d0/d11/f23 x:0 0 0 2024-04-26T04:42:10.530 INFO:tasks.workunit.client.1.smithi104.stdout:2/194: creat d6/de/d1d/f42 x:0 0 0 2024-04-26T04:42:10.531 INFO:tasks.workunit.client.1.smithi104.stdout:4/75: write d4/f11 [3379632,11700] 0 2024-04-26T04:42:10.531 INFO:tasks.workunit.client.1.smithi104.stdout:4/76: stat d4/db/d13/f14 0 2024-04-26T04:42:10.531 INFO:tasks.workunit.client.1.smithi104.stdout:4/77: chown d4 13608 1 2024-04-26T04:42:10.531 INFO:tasks.workunit.client.1.smithi104.stdout:4/78: stat d4/db 0 2024-04-26T04:42:10.531 INFO:tasks.workunit.client.1.smithi104.stdout:4/79: dread - d4/f17 zero size 2024-04-26T04:42:10.533 INFO:tasks.workunit.client.1.smithi104.stdout:5/137: creat d0/d1/d4/f24 x:0 0 0 2024-04-26T04:42:10.535 INFO:tasks.workunit.client.1.smithi104.stdout:4/80: creat d4/db/d13/d15/f1a x:0 0 0 2024-04-26T04:42:10.536 INFO:tasks.workunit.client.1.smithi104.stdout:4/81: mknod d4/db/d13/d15/c1b 0 2024-04-26T04:42:10.538 INFO:tasks.workunit.client.1.smithi104.stdout:5/138: mkdir d0/d1/d25 0 2024-04-26T04:42:10.539 INFO:tasks.workunit.client.1.smithi104.stdout:4/82: creat d4/db/d13/d18/f1c x:0 0 0 2024-04-26T04:42:10.550 INFO:tasks.workunit.client.1.smithi104.stdout:2/195: dwrite f2 [0,4194304] 0 2024-04-26T04:42:10.550 INFO:tasks.workunit.client.1.smithi104.stdout:5/139: dread d0/d1/d4/f12 [0,4194304] 0 2024-04-26T04:42:10.550 INFO:tasks.workunit.client.1.smithi104.stdout:2/196: dread - d6/de/d1d/f42 zero size 2024-04-26T04:42:10.554 INFO:tasks.workunit.client.1.smithi104.stdout:4/83: rename d4/db/d13/d18 to d4/d7/d1d 0 2024-04-26T04:42:10.554 INFO:tasks.workunit.client.1.smithi104.stdout:4/84: readlink - no filename 2024-04-26T04:42:10.554 INFO:tasks.workunit.client.1.smithi104.stdout:4/85: write d4/d7/f8 [1325321,12536] 0 2024-04-26T04:42:10.557 INFO:tasks.workunit.client.1.smithi104.stdout:5/140: write d0/d1/d1e/f21 [759074,130801] 0 2024-04-26T04:42:10.560 INFO:tasks.workunit.client.1.smithi104.stdout:4/86: rename d4/db/d13 to d4/d7/d1d/d1e 0 2024-04-26T04:42:10.560 INFO:tasks.workunit.client.1.smithi104.stdout:4/87: dread - d4/d7/d1d/f1c zero size 2024-04-26T04:42:10.560 INFO:tasks.workunit.client.1.smithi104.stdout:4/88: write f2 [3258409,126886] 0 2024-04-26T04:42:10.561 INFO:tasks.workunit.client.1.smithi104.stdout:4/89: link d4/f11 d4/d7/f1f 0 2024-04-26T04:42:10.562 INFO:tasks.workunit.client.1.smithi104.stdout:5/141: getdents d0/d1/d1e 0 2024-04-26T04:42:10.562 INFO:tasks.workunit.client.1.smithi104.stdout:5/142: getdents d0/d1/d1e 0 2024-04-26T04:42:10.568 INFO:tasks.workunit.client.1.smithi104.stdout:2/197: dwrite d6/d8/d3e/d21/f2c [0,4194304] 0 2024-04-26T04:42:10.568 INFO:tasks.workunit.client.1.smithi104.stdout:2/198: creat d6/de/d1d/f43 x:0 0 0 2024-04-26T04:42:10.568 INFO:tasks.workunit.client.1.smithi104.stdout:2/199: creat d6/de/f44 x:0 0 0 2024-04-26T04:42:10.568 INFO:tasks.workunit.client.1.smithi104.stdout:2/200: readlink d6/d8/l1a 0 2024-04-26T04:42:10.570 INFO:tasks.workunit.client.1.smithi104.stdout:4/90: fsync d4/f11 0 2024-04-26T04:42:10.570 INFO:tasks.workunit.client.1.smithi104.stdout:5/143: write d0/f5 [2203768,12130] 0 2024-04-26T04:42:10.571 INFO:tasks.workunit.client.1.smithi104.stdout:2/201: symlink d6/d8/d3e/d21/d29/l45 0 2024-04-26T04:42:10.571 INFO:tasks.workunit.client.1.smithi104.stdout:2/202: creat d6/d8/f46 x:0 0 0 2024-04-26T04:42:10.572 INFO:tasks.workunit.client.1.smithi104.stdout:4/91: rmdir d4/d7/d1d/d1e 39 2024-04-26T04:42:10.572 INFO:tasks.workunit.client.1.smithi104.stdout:4/92: chown d4 49221941 1 2024-04-26T04:42:10.572 INFO:tasks.workunit.client.1.smithi104.stdout:4/93: chown d4/d7/f8 1565 1 2024-04-26T04:42:10.572 INFO:tasks.workunit.client.1.smithi104.stdout:4/94: fdatasync d4/d7/d1d/f1c 0 2024-04-26T04:42:10.572 INFO:tasks.workunit.client.1.smithi104.stdout:4/95: chown d4/d7/d1d/d1e 999046 1 2024-04-26T04:42:10.573 INFO:tasks.workunit.client.1.smithi104.stdout:5/144: mkdir d0/d11/d26 0 2024-04-26T04:42:10.577 INFO:tasks.workunit.client.1.smithi104.stdout:2/203: rename d6/d8/d3e/d21/d29/l45 to d6/l47 0 2024-04-26T04:42:10.578 INFO:tasks.workunit.client.1.smithi104.stdout:4/96: mkdir d4/d20 0 2024-04-26T04:42:10.579 INFO:tasks.workunit.client.1.smithi104.stdout:5/145: creat d0/d1/d1e/f27 x:0 0 0 2024-04-26T04:42:10.579 INFO:tasks.workunit.client.1.smithi104.stdout:2/204: symlink d6/d8/d3e/d21/l48 0 2024-04-26T04:42:10.586 INFO:tasks.workunit.client.1.smithi104.stdout:2/205: read d6/fc [2905296,49474] 0 2024-04-26T04:42:10.601 INFO:tasks.workunit.client.1.smithi104.stdout:2/206: dread d6/d8/fb [0,4194304] 0 2024-04-26T04:42:10.602 INFO:tasks.workunit.client.1.smithi104.stdout:5/146: dwrite d0/f13 [0,4194304] 0 2024-04-26T04:42:10.602 INFO:tasks.workunit.client.1.smithi104.stdout:5/147: write d0/f6 [2438238,31415] 0 2024-04-26T04:42:10.602 INFO:tasks.workunit.client.1.smithi104.stdout:2/207: creat d6/d8/d3e/f49 x:0 0 0 2024-04-26T04:42:10.611 INFO:tasks.workunit.client.1.smithi104.stdout:4/97: dwrite f3 [0,4194304] 0 2024-04-26T04:42:10.612 INFO:tasks.workunit.client.1.smithi104.stdout:4/98: creat d4/d7/d1d/d1e/d15/f21 x:0 0 0 2024-04-26T04:42:10.612 INFO:tasks.workunit.client.1.smithi104.stdout:4/99: creat d4/d7/d1d/d1e/d15/f22 x:0 0 0 2024-04-26T04:42:10.615 INFO:tasks.workunit.client.1.smithi104.stdout:2/208: rmdir d6/d8 39 2024-04-26T04:42:10.615 INFO:tasks.workunit.client.1.smithi104.stdout:2/209: stat d6/d8/d3e/d21/l48 0 2024-04-26T04:42:10.616 INFO:tasks.workunit.client.1.smithi104.stdout:2/210: write d6/de/f24 [766745,1032] 0 2024-04-26T04:42:10.616 INFO:tasks.workunit.client.1.smithi104.stdout:2/211: chown d6/d1b/l3a 1357 1 2024-04-26T04:42:10.616 INFO:tasks.workunit.client.1.smithi104.stdout:2/212: truncate d6/f35 3771591 0 2024-04-26T04:42:10.620 INFO:tasks.workunit.client.1.smithi104.stdout:4/100: truncate d4/d7/f8 28482 0 2024-04-26T04:42:10.621 INFO:tasks.workunit.client.1.smithi104.stdout:4/101: getdents d4/db 0 2024-04-26T04:42:10.623 INFO:tasks.workunit.client.1.smithi104.stdout:4/102: creat d4/d7/d1d/f23 x:0 0 0 2024-04-26T04:42:10.626 INFO:tasks.workunit.client.1.smithi104.stdout:5/148: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:42:10.628 INFO:tasks.workunit.client.1.smithi104.stdout:5/149: mknod d0/d11/d26/c28 0 2024-04-26T04:42:10.628 INFO:tasks.workunit.client.1.smithi104.stdout:2/213: dread d6/fd [0,4194304] 0 2024-04-26T04:42:10.628 INFO:tasks.workunit.client.1.smithi104.stdout:2/214: creat d6/de/d1d/f4a x:0 0 0 2024-04-26T04:42:10.629 INFO:tasks.workunit.client.1.smithi104.stdout:5/150: mknod d0/d1/d25/c29 0 2024-04-26T04:42:10.629 INFO:tasks.workunit.client.1.smithi104.stdout:5/151: readlink d0/d1/d4/lb 0 2024-04-26T04:42:10.633 INFO:tasks.workunit.client.1.smithi104.stdout:5/152: mkdir d0/d1/d4/d2a 0 2024-04-26T04:42:10.633 INFO:tasks.workunit.client.1.smithi104.stdout:5/153: truncate d0/d1/d4/f12 4607519 0 2024-04-26T04:42:10.642 INFO:tasks.workunit.client.1.smithi104.stdout:2/215: read d6/d8/d3e/d21/f2c [39374,118126] 0 2024-04-26T04:42:10.642 INFO:tasks.workunit.client.1.smithi104.stdout:2/216: read - d6/d8/f46 zero size 2024-04-26T04:42:10.642 INFO:tasks.workunit.client.1.smithi104.stdout:2/217: creat d6/f4b x:0 0 0 2024-04-26T04:42:10.642 INFO:tasks.workunit.client.1.smithi104.stdout:2/218: stat d6/f10 0 2024-04-26T04:42:10.643 INFO:tasks.workunit.client.1.smithi104.stdout:2/219: chown d6/fd 61 1 2024-04-26T04:42:10.643 INFO:tasks.workunit.client.1.smithi104.stdout:2/220: truncate d6/fd 2059778 0 2024-04-26T04:42:10.643 INFO:tasks.workunit.client.1.smithi104.stdout:4/103: dwrite d4/db/f12 [0,4194304] 0 2024-04-26T04:42:10.643 INFO:tasks.workunit.client.1.smithi104.stdout:2/221: creat d6/d8/d3e/d21/d29/f4c x:0 0 0 2024-04-26T04:42:10.650 INFO:tasks.workunit.client.1.smithi104.stdout:4/104: mkdir d4/d7/d1d/d1e/d24 0 2024-04-26T04:42:10.650 INFO:tasks.workunit.client.1.smithi104.stdout:5/154: dwrite d0/f20 [0,4194304] 0 2024-04-26T04:42:10.651 INFO:tasks.workunit.client.1.smithi104.stdout:2/222: dread d6/d8/d3e/f26 [0,4194304] 0 2024-04-26T04:42:10.651 INFO:tasks.workunit.client.1.smithi104.stdout:4/105: symlink d4/d7/d1d/d1e/d15/l25 0 2024-04-26T04:42:10.653 INFO:tasks.workunit.client.0.smithi090.stdout:6/18: write f2 [4791251,33207] 0 2024-04-26T04:42:10.654 INFO:tasks.workunit.client.1.smithi104.stdout:2/223: rename d6/d8/l3f to d6/d1b/l4d 0 2024-04-26T04:42:10.655 INFO:tasks.workunit.client.1.smithi104.stdout:2/224: mknod d6/d8/d3e/c4e 0 2024-04-26T04:42:10.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/21: getdents . 0 2024-04-26T04:42:10.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/22: mkdir d4 0 2024-04-26T04:42:10.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/23: fsync - no filename 2024-04-26T04:42:10.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/24: read - no filename 2024-04-26T04:42:10.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/25: mkdir d4/d5 0 2024-04-26T04:42:10.661 INFO:tasks.workunit.client.0.smithi090.stdout:5/26: mknod d4/d5/c6 0 2024-04-26T04:42:10.661 INFO:tasks.workunit.client.0.smithi090.stdout:5/27: write - no filename 2024-04-26T04:42:10.662 INFO:tasks.workunit.client.1.smithi104.stdout:4/106: dread f2 [0,4194304] 0 2024-04-26T04:42:10.662 INFO:tasks.workunit.client.1.smithi104.stdout:4/107: chown d4/db/cc 52 1 2024-04-26T04:42:10.663 INFO:tasks.workunit.client.1.smithi104.stdout:4/108: mknod d4/c26 0 2024-04-26T04:42:10.663 INFO:tasks.workunit.client.1.smithi104.stdout:4/109: stat f3 0 2024-04-26T04:42:10.663 INFO:tasks.workunit.client.1.smithi104.stdout:2/225: rmdir d6/d1b 39 2024-04-26T04:42:10.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/28: rmdir d4 39 2024-04-26T04:42:10.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/29: readlink - no filename 2024-04-26T04:42:10.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/30: rename d4/d5 to d4/d5/d7 22 2024-04-26T04:42:10.665 INFO:tasks.workunit.client.0.smithi090.stdout:5/31: rename c3 to d4/d5/c8 0 2024-04-26T04:42:10.665 INFO:tasks.workunit.client.0.smithi090.stdout:5/32: dwrite - no filename 2024-04-26T04:42:10.665 INFO:tasks.workunit.client.0.smithi090.stdout:5/33: dwrite - no filename 2024-04-26T04:42:10.666 INFO:tasks.workunit.client.1.smithi104.stdout:4/110: write f2 [3916984,114311] 0 2024-04-26T04:42:10.667 INFO:tasks.workunit.client.0.smithi090.stdout:5/34: mkdir d4/d9 0 2024-04-26T04:42:10.668 INFO:tasks.workunit.client.1.smithi104.stdout:4/111: symlink d4/d7/d1d/l27 0 2024-04-26T04:42:10.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/35: creat d4/d9/fa x:0 0 0 2024-04-26T04:42:10.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/36: chown d4 10101 1 2024-04-26T04:42:10.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/22: getdents d0 0 2024-04-26T04:42:10.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/23: fdatasync d0/f5 0 2024-04-26T04:42:10.672 INFO:tasks.workunit.client.1.smithi104.stdout:4/112: creat d4/db/f28 x:0 0 0 2024-04-26T04:42:10.672 INFO:tasks.workunit.client.1.smithi104.stdout:4/113: read d4/d7/f1f [170060,12081] 0 2024-04-26T04:42:10.672 INFO:tasks.workunit.client.1.smithi104.stdout:4/114: chown d4/db/cc 11835 1 2024-04-26T04:42:10.673 INFO:tasks.workunit.client.1.smithi104.stdout:4/115: dread d4/d7/ff [0,4194304] 0 2024-04-26T04:42:10.673 INFO:tasks.workunit.client.1.smithi104.stdout:5/155: dwrite d0/d1/d1e/f21 [0,4194304] 0 2024-04-26T04:42:10.674 INFO:tasks.workunit.client.1.smithi104.stdout:4/116: symlink d4/d7/d1d/d1e/d24/l29 0 2024-04-26T04:42:10.676 INFO:tasks.workunit.client.1.smithi104.stdout:5/156: mkdir d0/d1/d4/d2a/d2b 0 2024-04-26T04:42:10.676 INFO:tasks.workunit.client.0.smithi090.stdout:8/24: truncate f4 2005562 0 2024-04-26T04:42:10.677 INFO:tasks.workunit.client.1.smithi104.stdout:6/286: sync 2024-04-26T04:42:10.677 INFO:tasks.workunit.client.1.smithi104.stdout:5/157: symlink d0/d11/d26/l2c 0 2024-04-26T04:42:10.679 INFO:tasks.workunit.client.1.smithi104.stdout:5/158: symlink d0/l2d 0 2024-04-26T04:42:10.679 INFO:tasks.workunit.client.1.smithi104.stdout:2/226: write d6/d8/d3e/f2a [526688,65415] 0 2024-04-26T04:42:10.691 INFO:tasks.workunit.client.0.smithi090.stdout:0/24: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:10.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/25: fsync d0/f5 0 2024-04-26T04:42:10.693 INFO:tasks.workunit.client.1.smithi104.stdout:2/227: dread d6/d8/fb [0,4194304] 0 2024-04-26T04:42:10.694 INFO:tasks.workunit.client.1.smithi104.stdout:6/287: dread d0/d9/d13/d15/f42 [0,4194304] 0 2024-04-26T04:42:10.696 INFO:tasks.workunit.client.1.smithi104.stdout:2/228: rename d6/l47 to d6/d8/d3e/d21/l4f 0 2024-04-26T04:42:10.697 INFO:tasks.workunit.client.1.smithi104.stdout:6/288: link d0/d9/d13/c36 d0/d9/d13/d15/c64 0 2024-04-26T04:42:10.697 INFO:tasks.workunit.client.1.smithi104.stdout:6/289: fsync d0/d9/d52/f61 0 2024-04-26T04:42:10.697 INFO:tasks.workunit.client.1.smithi104.stdout:5/159: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:10.698 INFO:tasks.workunit.client.1.smithi104.stdout:2/229: link d6/c9 d6/d8/d3e/c50 0 2024-04-26T04:42:10.698 INFO:tasks.workunit.client.1.smithi104.stdout:2/230: read - d6/d8/d3e/f49 zero size 2024-04-26T04:42:10.700 INFO:tasks.workunit.client.1.smithi104.stdout:5/160: rename d0/f13 to d0/d1/d1e/f2e 0 2024-04-26T04:42:10.700 INFO:tasks.workunit.client.0.smithi090.stdout:5/37: dwrite d4/d9/fa [0,4194304] 0 2024-04-26T04:42:10.700 INFO:tasks.workunit.client.0.smithi090.stdout:5/38: rename d4 to d4/db 22 2024-04-26T04:42:10.701 INFO:tasks.workunit.client.1.smithi104.stdout:2/231: mkdir d6/d1b/d51 0 2024-04-26T04:42:10.701 INFO:tasks.workunit.client.1.smithi104.stdout:2/232: creat d6/de/f52 x:0 0 0 2024-04-26T04:42:10.702 INFO:tasks.workunit.client.1.smithi104.stdout:6/290: getdents d0/d9/d5c/d3f 0 2024-04-26T04:42:10.704 INFO:tasks.workunit.client.1.smithi104.stdout:2/233: unlink d6/de/lf 0 2024-04-26T04:42:10.704 INFO:tasks.workunit.client.1.smithi104.stdout:6/291: stat d0/d9/d13/d44/d47/c50 0 2024-04-26T04:42:10.704 INFO:tasks.workunit.client.1.smithi104.stdout:6/292: chown d0/d9/d5c/l23 35036161 1 2024-04-26T04:42:10.705 INFO:tasks.workunit.client.0.smithi090.stdout:0/26: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:10.705 INFO:tasks.workunit.client.1.smithi104.stdout:5/161: write d0/f1d [3137196,37864] 0 2024-04-26T04:42:10.705 INFO:tasks.workunit.client.0.smithi090.stdout:0/27: stat d0/l2 0 2024-04-26T04:42:10.706 INFO:tasks.workunit.client.1.smithi104.stdout:6/293: rename d0/d9/d13/c1d to d0/d1b/c65 0 2024-04-26T04:42:10.708 INFO:tasks.workunit.client.0.smithi090.stdout:6/19: dwrite f5 [4194304,4194304] 0 2024-04-26T04:42:10.709 INFO:tasks.workunit.client.1.smithi104.stdout:4/117: dwrite d4/d7/fa [0,4194304] 0 2024-04-26T04:42:10.711 INFO:tasks.workunit.client.0.smithi090.stdout:8/25: dwrite f5 [0,4194304] 0 2024-04-26T04:42:10.712 INFO:tasks.workunit.client.0.smithi090.stdout:6/20: rename f6 to f7 0 2024-04-26T04:42:10.713 INFO:tasks.workunit.client.0.smithi090.stdout:5/39: read d4/d9/fa [3927246,23410] 0 2024-04-26T04:42:10.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/26: rename f6 to f7 0 2024-04-26T04:42:10.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/27: creat f8 x:0 0 0 2024-04-26T04:42:10.716 INFO:tasks.workunit.client.0.smithi090.stdout:6/21: mknod c8 0 2024-04-26T04:42:10.718 INFO:tasks.workunit.client.0.smithi090.stdout:5/40: creat d4/d5/fc x:0 0 0 2024-04-26T04:42:10.718 INFO:tasks.workunit.client.0.smithi090.stdout:0/28: write d0/f5 [3121065,105810] 0 2024-04-26T04:42:10.719 INFO:tasks.workunit.client.0.smithi090.stdout:0/29: chown d0/l3 1 1 2024-04-26T04:42:10.719 INFO:tasks.workunit.client.0.smithi090.stdout:0/30: getdents d0 0 2024-04-26T04:42:10.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:10 smithi090 ceph-mon[26648]: pgmap v247: 65 pgs: 65 active+clean; 274 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-04-26T04:42:10.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/41: creat d4/fd x:0 0 0 2024-04-26T04:42:10.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/42: creat d4/fe x:0 0 0 2024-04-26T04:42:10.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/43: truncate d4/fe 39299 0 2024-04-26T04:42:10.723 INFO:tasks.workunit.client.1.smithi104.stdout:5/162: dread d0/d11/f19 [0,4194304] 0 2024-04-26T04:42:10.724 INFO:tasks.workunit.client.0.smithi090.stdout:0/31: mkdir d0/d9 0 2024-04-26T04:42:10.726 INFO:tasks.workunit.client.1.smithi104.stdout:4/118: dread f3 [0,4194304] 0 2024-04-26T04:42:10.727 INFO:tasks.workunit.client.1.smithi104.stdout:5/163: creat d0/d1/f2f x:0 0 0 2024-04-26T04:42:10.728 INFO:tasks.workunit.client.0.smithi090.stdout:6/22: write f5 [3553595,75747] 0 2024-04-26T04:42:10.729 INFO:tasks.workunit.client.1.smithi104.stdout:4/119: getdents d4/d7/d1d 0 2024-04-26T04:42:10.729 INFO:tasks.workunit.client.1.smithi104.stdout:4/120: getdents d4/d7/d1d 0 2024-04-26T04:42:10.729 INFO:tasks.workunit.client.1.smithi104.stdout:4/121: truncate d4/d7/f8 985429 0 2024-04-26T04:42:10.732 INFO:tasks.workunit.client.1.smithi104.stdout:6/294: dwrite d0/d5/f22 [0,4194304] 0 2024-04-26T04:42:10.733 INFO:tasks.workunit.client.1.smithi104.stdout:9/129: sync 2024-04-26T04:42:10.733 INFO:tasks.workunit.client.1.smithi104.stdout:0/130: sync 2024-04-26T04:42:10.733 INFO:tasks.workunit.client.1.smithi104.stdout:8/59: sync 2024-04-26T04:42:10.733 INFO:tasks.workunit.client.1.smithi104.stdout:0/131: truncate d0/fa 3870962 0 2024-04-26T04:42:10.733 INFO:tasks.workunit.client.1.smithi104.stdout:2/234: dwrite d6/fc [0,4194304] 0 2024-04-26T04:42:10.735 INFO:tasks.workunit.client.1.smithi104.stdout:6/295: mknod d0/d5/c66 0 2024-04-26T04:42:10.735 INFO:tasks.workunit.client.1.smithi104.stdout:9/130: creat d11/f20 x:0 0 0 2024-04-26T04:42:10.735 INFO:tasks.workunit.client.1.smithi104.stdout:9/131: write f10 [268803,15782] 0 2024-04-26T04:42:10.737 INFO:tasks.workunit.client.1.smithi104.stdout:5/164: read d0/d1/d1e/f21 [1819404,32274] 0 2024-04-26T04:42:10.737 INFO:tasks.workunit.client.1.smithi104.stdout:8/60: symlink d0/l11 0 2024-04-26T04:42:10.737 INFO:tasks.workunit.client.1.smithi104.stdout:0/132: mknod d0/d2/d10/c2b 0 2024-04-26T04:42:10.738 INFO:tasks.workunit.client.1.smithi104.stdout:0/133: truncate d0/d4/d15/f18 1531701 0 2024-04-26T04:42:10.738 INFO:tasks.workunit.client.1.smithi104.stdout:0/134: truncate d0/d4/f7 1430548 0 2024-04-26T04:42:10.738 INFO:tasks.workunit.client.0.smithi090.stdout:8/28: getdents . 0 2024-04-26T04:42:10.740 INFO:tasks.workunit.client.1.smithi104.stdout:4/122: dread f2 [0,4194304] 0 2024-04-26T04:42:10.740 INFO:tasks.workunit.client.1.smithi104.stdout:2/235: rename d6/de/f24 to d6/d8/d3e/d21/f53 0 2024-04-26T04:42:10.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/29: mknod c9 0 2024-04-26T04:42:10.741 INFO:tasks.workunit.client.1.smithi104.stdout:6/296: creat d0/d5/f67 x:0 0 0 2024-04-26T04:42:10.743 INFO:tasks.workunit.client.1.smithi104.stdout:9/132: symlink d11/d15/l21 0 2024-04-26T04:42:10.743 INFO:tasks.workunit.client.1.smithi104.stdout:9/133: chown d11/l19 243227 1 2024-04-26T04:42:10.743 INFO:tasks.workunit.client.1.smithi104.stdout:8/61: mkdir d0/d12 0 2024-04-26T04:42:10.744 INFO:tasks.workunit.client.0.smithi090.stdout:5/44: dwrite d4/d5/fc [0,4194304] 0 2024-04-26T04:42:10.744 INFO:tasks.workunit.client.0.smithi090.stdout:5/45: fdatasync d4/d9/fa 0 2024-04-26T04:42:10.744 INFO:tasks.workunit.client.0.smithi090.stdout:5/46: chown d4/d5 2611021 1 2024-04-26T04:42:10.744 INFO:tasks.workunit.client.0.smithi090.stdout:5/47: write d4/fd [691029,64449] 0 2024-04-26T04:42:10.745 INFO:tasks.workunit.client.0.smithi090.stdout:8/30: creat fa x:0 0 0 2024-04-26T04:42:10.745 INFO:tasks.workunit.client.1.smithi104.stdout:0/135: creat d0/d2/d21/d27/d1e/f2c x:0 0 0 2024-04-26T04:42:10.747 INFO:tasks.workunit.client.1.smithi104.stdout:4/123: link d4/d7/d1d/d1e/d15/f21 d4/d7/f2a 0 2024-04-26T04:42:10.747 INFO:tasks.workunit.client.1.smithi104.stdout:2/236: creat d6/d8/d3e/d21/f54 x:0 0 0 2024-04-26T04:42:10.749 INFO:tasks.workunit.client.1.smithi104.stdout:8/62: mkdir d0/d13 0 2024-04-26T04:42:10.749 INFO:tasks.workunit.client.1.smithi104.stdout:8/63: chown d0/f5 0 1 2024-04-26T04:42:10.749 INFO:tasks.workunit.client.0.smithi090.stdout:6/23: truncate f1 3498228 0 2024-04-26T04:42:10.751 INFO:tasks.workunit.client.1.smithi104.stdout:0/136: link d0/d4/f29 d0/d2/d21/d27/d1e/f2d 0 2024-04-26T04:42:10.752 INFO:tasks.workunit.client.0.smithi090.stdout:8/31: mknod cb 0 2024-04-26T04:42:10.754 INFO:tasks.workunit.client.1.smithi104.stdout:4/124: mknod d4/d7/d1d/d1e/d15/c2b 0 2024-04-26T04:42:10.755 INFO:tasks.workunit.client.1.smithi104.stdout:9/134: write f6 [6838047,130838] 0 2024-04-26T04:42:10.755 INFO:tasks.workunit.client.1.smithi104.stdout:9/135: read - d11/d15/f1f zero size 2024-04-26T04:42:10.757 INFO:tasks.workunit.client.1.smithi104.stdout:2/237: mknod d6/de/c55 0 2024-04-26T04:42:10.757 INFO:tasks.workunit.client.1.smithi104.stdout:2/238: chown d6/d8/fb 1 1 2024-04-26T04:42:10.759 INFO:tasks.workunit.client.1.smithi104.stdout:0/137: rename d0/l22 to d0/l2e 0 2024-04-26T04:42:10.760 INFO:tasks.workunit.client.1.smithi104.stdout:7/39: sync 2024-04-26T04:42:10.760 INFO:tasks.workunit.client.1.smithi104.stdout:3/115: sync 2024-04-26T04:42:10.760 INFO:tasks.workunit.client.1.smithi104.stdout:3/116: read - d5/d9/dc/f1c zero size 2024-04-26T04:42:10.760 INFO:tasks.workunit.client.1.smithi104.stdout:3/117: creat d5/d9/f26 x:0 0 0 2024-04-26T04:42:10.760 INFO:tasks.workunit.client.0.smithi090.stdout:6/24: write f7 [4127846,82702] 0 2024-04-26T04:42:10.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:10 smithi104 ceph-mon[32369]: pgmap v247: 65 pgs: 65 active+clean; 274 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.6 MiB/s rd, 17 MiB/s wr, 1.20k op/s 2024-04-26T04:42:10.761 INFO:tasks.workunit.client.1.smithi104.stdout:4/125: unlink d4/d7/d1d/d1e/d15/f21 0 2024-04-26T04:42:10.761 INFO:tasks.workunit.client.1.smithi104.stdout:1/196: sync 2024-04-26T04:42:10.762 INFO:tasks.workunit.client.1.smithi104.stdout:9/136: mknod d11/d15/d1b/c22 0 2024-04-26T04:42:10.762 INFO:tasks.workunit.client.1.smithi104.stdout:9/137: write f6 [6034035,100777] 0 2024-04-26T04:42:10.762 INFO:tasks.workunit.client.0.smithi090.stdout:0/32: dwrite d0/f5 [0,4194304] 0 2024-04-26T04:42:10.764 INFO:tasks.workunit.client.1.smithi104.stdout:0/138: link d0/d2/f23 d0/f2f 0 2024-04-26T04:42:10.765 INFO:tasks.workunit.client.1.smithi104.stdout:0/139: dread d0/d4/f1b [0,4194304] 0 2024-04-26T04:42:10.766 INFO:tasks.workunit.client.1.smithi104.stdout:4/126: unlink d4/d7/fa 0 2024-04-26T04:42:10.768 INFO:tasks.workunit.client.1.smithi104.stdout:1/197: link d0/d6/de/l24 d0/d6/d19/d21/l37 0 2024-04-26T04:42:10.769 INFO:tasks.workunit.client.1.smithi104.stdout:3/118: rmdir d5/d9/dc 39 2024-04-26T04:42:10.769 INFO:tasks.workunit.client.1.smithi104.stdout:3/119: truncate d5/d9/f14 1091816 0 2024-04-26T04:42:10.769 INFO:tasks.workunit.client.1.smithi104.stdout:1/198: mknod d0/d6/d2e/d32/c38 0 2024-04-26T04:42:10.769 INFO:tasks.workunit.client.1.smithi104.stdout:1/199: write d0/d6/f1a [2669498,13543] 0 2024-04-26T04:42:10.770 INFO:tasks.workunit.client.1.smithi104.stdout:0/140: write d0/fe [270757,121665] 0 2024-04-26T04:42:10.770 INFO:tasks.workunit.client.1.smithi104.stdout:3/120: symlink d5/d9/d17/l27 0 2024-04-26T04:42:10.771 INFO:tasks.workunit.client.1.smithi104.stdout:7/40: dread f0 [0,4194304] 0 2024-04-26T04:42:10.773 INFO:tasks.workunit.client.1.smithi104.stdout:5/165: dwrite d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:10.773 INFO:tasks.workunit.client.1.smithi104.stdout:5/166: read - d0/d1/f2f zero size 2024-04-26T04:42:10.776 INFO:tasks.workunit.client.1.smithi104.stdout:7/41: rmdir d2/d8 0 2024-04-26T04:42:10.776 INFO:tasks.workunit.client.1.smithi104.stdout:3/121: truncate d5/d9/dc/f19 3455580 0 2024-04-26T04:42:10.777 INFO:tasks.workunit.client.0.smithi090.stdout:0/33: write d0/f5 [1664809,63949] 0 2024-04-26T04:42:10.785 INFO:tasks.workunit.client.1.smithi104.stdout:1/200: dread d0/f14 [4194304,4194304] 0 2024-04-26T04:42:10.786 INFO:tasks.workunit.client.0.smithi090.stdout:5/48: truncate d4/fd 630630 0 2024-04-26T04:42:10.786 INFO:tasks.workunit.client.1.smithi104.stdout:8/64: dwrite d0/f1 [4194304,4194304] 0 2024-04-26T04:42:10.787 INFO:tasks.workunit.client.1.smithi104.stdout:8/65: write d0/f9 [954615,120896] 0 2024-04-26T04:42:10.787 INFO:tasks.workunit.client.1.smithi104.stdout:2/239: rename d6/d8/d3e/d21/f53 to d6/d8/f56 0 2024-04-26T04:42:10.788 INFO:tasks.workunit.client.1.smithi104.stdout:1/201: creat d0/d6/de/d29/f39 x:0 0 0 2024-04-26T04:42:10.791 INFO:tasks.workunit.client.1.smithi104.stdout:9/138: dwrite d11/d15/f1f [0,4194304] 0 2024-04-26T04:42:10.791 INFO:tasks.workunit.client.1.smithi104.stdout:8/66: unlink d0/lf 0 2024-04-26T04:42:10.794 INFO:tasks.workunit.client.1.smithi104.stdout:4/127: rename d4/c26 to d4/db/c2c 0 2024-04-26T04:42:10.794 INFO:tasks.workunit.client.1.smithi104.stdout:7/42: rename d2/d4 to d2/d4/dc 22 2024-04-26T04:42:10.794 INFO:tasks.workunit.client.1.smithi104.stdout:9/139: rename d11 to d11/d23 22 2024-04-26T04:42:10.794 INFO:tasks.workunit.client.1.smithi104.stdout:9/140: write d11/f16 [604190,79575] 0 2024-04-26T04:42:10.795 INFO:tasks.workunit.client.1.smithi104.stdout:4/128: write d4/d7/d1d/d1e/f14 [302477,103690] 0 2024-04-26T04:42:10.797 INFO:tasks.workunit.client.1.smithi104.stdout:2/240: dread d6/d8/d3e/d21/f2c [0,4194304] 0 2024-04-26T04:42:10.797 INFO:tasks.workunit.client.1.smithi104.stdout:1/202: creat d0/d6/d19/f3a x:0 0 0 2024-04-26T04:42:10.800 INFO:tasks.workunit.client.0.smithi090.stdout:0/34: fsync d0/f5 0 2024-04-26T04:42:10.800 INFO:tasks.workunit.client.0.smithi090.stdout:0/35: chown d0/f5 85643189 1 2024-04-26T04:42:10.800 INFO:tasks.workunit.client.0.smithi090.stdout:0/36: write d0/f5 [2332970,129156] 0 2024-04-26T04:42:10.802 INFO:tasks.workunit.client.0.smithi090.stdout:0/37: symlink d0/d9/la 0 2024-04-26T04:42:10.805 INFO:tasks.workunit.client.1.smithi104.stdout:5/167: dwrite d0/f22 [0,4194304] 0 2024-04-26T04:42:10.805 INFO:tasks.workunit.client.1.smithi104.stdout:5/168: dread - d0/d1/f2f zero size 2024-04-26T04:42:10.805 INFO:tasks.workunit.client.1.smithi104.stdout:5/169: creat d0/d1/d1e/f30 x:0 0 0 2024-04-26T04:42:10.806 INFO:tasks.workunit.client.1.smithi104.stdout:8/67: link d0/l6 d0/d13/l14 0 2024-04-26T04:42:10.807 INFO:tasks.workunit.client.1.smithi104.stdout:7/43: dread d2/f3 [0,4194304] 0 2024-04-26T04:42:10.807 INFO:tasks.workunit.client.1.smithi104.stdout:7/44: chown d2/f3 21427 1 2024-04-26T04:42:10.810 INFO:tasks.workunit.client.1.smithi104.stdout:4/129: symlink d4/d20/l2d 0 2024-04-26T04:42:10.810 INFO:tasks.workunit.client.1.smithi104.stdout:4/130: dread - d4/f17 zero size 2024-04-26T04:42:10.810 INFO:tasks.workunit.client.1.smithi104.stdout:2/241: symlink d6/d8/d3e/d21/d29/l57 0 2024-04-26T04:42:10.812 INFO:tasks.workunit.client.0.smithi090.stdout:5/49: dwrite d4/fe [0,4194304] 0 2024-04-26T04:42:10.812 INFO:tasks.workunit.client.0.smithi090.stdout:5/50: truncate d4/fe 4621489 0 2024-04-26T04:42:10.813 INFO:tasks.workunit.client.1.smithi104.stdout:1/203: symlink d0/d6/de/d29/l3b 0 2024-04-26T04:42:10.813 INFO:tasks.workunit.client.1.smithi104.stdout:3/122: dwrite f4 [0,4194304] 0 2024-04-26T04:42:10.813 INFO:tasks.workunit.client.1.smithi104.stdout:3/123: write d5/f23 [441986,114161] 0 2024-04-26T04:42:10.817 INFO:tasks.workunit.client.0.smithi090.stdout:6/25: truncate f1 427057 0 2024-04-26T04:42:10.818 INFO:tasks.workunit.client.0.smithi090.stdout:5/51: unlink d4/fd 0 2024-04-26T04:42:10.818 INFO:tasks.workunit.client.0.smithi090.stdout:5/52: readlink - no filename 2024-04-26T04:42:10.818 INFO:tasks.workunit.client.0.smithi090.stdout:5/53: chown d4/d9/fa 10539096 1 2024-04-26T04:42:10.819 INFO:tasks.workunit.client.0.smithi090.stdout:6/26: mkdir d9 0 2024-04-26T04:42:10.821 INFO:tasks.workunit.client.1.smithi104.stdout:7/45: creat d2/d4/fd x:0 0 0 2024-04-26T04:42:10.822 INFO:tasks.workunit.client.1.smithi104.stdout:3/124: dread f3 [0,4194304] 0 2024-04-26T04:42:10.822 INFO:tasks.workunit.client.1.smithi104.stdout:3/125: chown d5/d9/fe 19590 1 2024-04-26T04:42:10.828 INFO:tasks.workunit.client.0.smithi090.stdout:0/38: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:10.829 INFO:tasks.workunit.client.0.smithi090.stdout:0/39: mknod d0/d9/cb 0 2024-04-26T04:42:10.830 INFO:tasks.workunit.client.0.smithi090.stdout:0/40: chown d0/d9/cb 63224782 1 2024-04-26T04:42:10.830 INFO:tasks.workunit.client.0.smithi090.stdout:0/41: fdatasync d0/f5 0 2024-04-26T04:42:10.832 INFO:tasks.workunit.client.1.smithi104.stdout:7/46: dread d2/f3 [0,4194304] 0 2024-04-26T04:42:10.833 INFO:tasks.workunit.client.1.smithi104.stdout:2/242: unlink d6/d8/d3e/f3d 0 2024-04-26T04:42:10.833 INFO:tasks.workunit.client.1.smithi104.stdout:2/243: truncate d6/de/f40 833171 0 2024-04-26T04:42:10.834 INFO:tasks.workunit.client.1.smithi104.stdout:1/204: mkdir d0/d6/de/d3c 0 2024-04-26T04:42:10.843 INFO:tasks.workunit.client.1.smithi104.stdout:3/126: link d5/d9/f26 d5/d9/dc/f28 0 2024-04-26T04:42:10.843 INFO:tasks.workunit.client.1.smithi104.stdout:3/127: chown d5/d9/f26 12216082 1 2024-04-26T04:42:10.843 INFO:tasks.workunit.client.1.smithi104.stdout:4/131: truncate d4/db/fe 526036 0 2024-04-26T04:42:10.844 INFO:tasks.workunit.client.1.smithi104.stdout:2/244: creat d6/d1b/d51/f58 x:0 0 0 2024-04-26T04:42:10.845 INFO:tasks.workunit.client.1.smithi104.stdout:3/128: mkdir d5/d9/dc/d29 0 2024-04-26T04:42:10.847 INFO:tasks.workunit.client.1.smithi104.stdout:2/245: rename d6/d8/d3e/d21/f2c to d6/d8/d3e/d21/f59 0 2024-04-26T04:42:10.847 INFO:tasks.workunit.client.1.smithi104.stdout:1/205: dread d0/d6/de/d20/d2a/f31 [0,4194304] 0 2024-04-26T04:42:10.847 INFO:tasks.workunit.client.1.smithi104.stdout:3/129: creat d5/d9/d17/f2a x:0 0 0 2024-04-26T04:42:10.849 INFO:tasks.workunit.client.1.smithi104.stdout:1/206: link d0/d6/d19/d1b/l26 d0/d6/de/l3d 0 2024-04-26T04:42:10.849 INFO:tasks.workunit.client.1.smithi104.stdout:9/141: dwrite f6 [4194304,4194304] 0 2024-04-26T04:42:10.850 INFO:tasks.workunit.client.0.smithi090.stdout:0/42: dwrite d0/f5 [4194304,4194304] 0 2024-04-26T04:42:10.855 INFO:tasks.workunit.client.1.smithi104.stdout:5/170: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:10.855 INFO:tasks.workunit.client.1.smithi104.stdout:5/171: fdatasync d0/f20 0 2024-04-26T04:42:10.855 INFO:tasks.workunit.client.1.smithi104.stdout:5/172: fsync d0/d1/d4/f12 0 2024-04-26T04:42:10.855 INFO:tasks.workunit.client.1.smithi104.stdout:5/173: write d0/d1/d4/f24 [753421,20159] 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/174: chown d0/d1/d4/lc 6 1 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/175: chown d0/l7 12 1 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/176: chown d0/d1/d25/c29 15631 1 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/177: creat d0/d11/f31 x:0 0 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/178: chown d0/d11/d26/l2c 0 1 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/179: write d0/d11/f31 [699909,56627] 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:3/130: mknod d5/d9/d17/c2b 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:3/131: creat d5/d9/f2c x:0 0 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:3/132: readlink d5/l20 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:3/133: write d5/d9/dc/f1c [570629,106725] 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:8/68: dwrite d0/fb [0,4194304] 0 2024-04-26T04:42:10.856 INFO:tasks.workunit.client.1.smithi104.stdout:1/207: symlink d0/d6/de/d3c/l3e 0 2024-04-26T04:42:10.857 INFO:tasks.workunit.client.1.smithi104.stdout:1/208: write d0/d6/d19/f3a [556394,62315] 0 2024-04-26T04:42:10.857 INFO:tasks.workunit.client.0.smithi090.stdout:6/27: dwrite f3 [0,4194304] 0 2024-04-26T04:42:10.857 INFO:tasks.workunit.client.0.smithi090.stdout:0/43: creat d0/fc x:0 0 0 2024-04-26T04:42:10.857 INFO:tasks.workunit.client.0.smithi090.stdout:0/44: rename d0/c7 to d0/cd 0 2024-04-26T04:42:10.858 INFO:tasks.workunit.client.1.smithi104.stdout:8/69: dread d0/f9 [0,4194304] 0 2024-04-26T04:42:10.859 INFO:tasks.workunit.client.1.smithi104.stdout:9/142: creat d11/d15/d1b/f24 x:0 0 0 2024-04-26T04:42:10.859 INFO:tasks.workunit.client.1.smithi104.stdout:1/209: read d0/d6/de/d20/d2a/f31 [1464511,66472] 0 2024-04-26T04:42:10.859 INFO:tasks.workunit.client.1.smithi104.stdout:1/210: chown d0/d6/d19/c1f 7 1 2024-04-26T04:42:10.859 INFO:tasks.workunit.client.1.smithi104.stdout:1/211: chown d0/d6/c2f 57377 1 2024-04-26T04:42:10.862 INFO:tasks.workunit.client.1.smithi104.stdout:5/180: mkdir d0/d11/d26/d32 0 2024-04-26T04:42:10.872 INFO:tasks.workunit.client.1.smithi104.stdout:8/70: link d0/d13/l14 d0/l15 0 2024-04-26T04:42:10.873 INFO:tasks.workunit.client.1.smithi104.stdout:1/212: mkdir d0/d6/de/d29/d3f 0 2024-04-26T04:42:10.874 INFO:tasks.workunit.client.1.smithi104.stdout:5/181: mkdir d0/d1/d4/d2a/d33 0 2024-04-26T04:42:10.875 INFO:tasks.workunit.client.1.smithi104.stdout:5/182: read d0/d1/d4/f12 [4350848,100379] 0 2024-04-26T04:42:10.875 INFO:tasks.workunit.client.1.smithi104.stdout:5/183: write d0/d11/f31 [868745,125690] 0 2024-04-26T04:42:10.875 INFO:tasks.workunit.client.1.smithi104.stdout:5/184: write d0/f5 [4592097,28333] 0 2024-04-26T04:42:10.875 INFO:tasks.workunit.client.1.smithi104.stdout:5/185: chown d0/d1/d1e/f30 727306864 1 2024-04-26T04:42:10.876 INFO:tasks.workunit.client.1.smithi104.stdout:3/134: unlink d5/d9/f2c 0 2024-04-26T04:42:10.877 INFO:tasks.workunit.client.1.smithi104.stdout:8/71: mknod d0/d12/c16 0 2024-04-26T04:42:10.879 INFO:tasks.workunit.client.1.smithi104.stdout:1/213: unlink d0/d6/d19/c1c 0 2024-04-26T04:42:10.879 INFO:tasks.workunit.client.1.smithi104.stdout:1/214: dread - d0/d6/de/d29/f39 zero size 2024-04-26T04:42:10.879 INFO:tasks.workunit.client.1.smithi104.stdout:1/215: chown d0/d6 20614990 1 2024-04-26T04:42:10.880 INFO:tasks.workunit.client.1.smithi104.stdout:4/132: dwrite d4/d7/d1d/f1c [0,4194304] 0 2024-04-26T04:42:10.880 INFO:tasks.workunit.client.1.smithi104.stdout:4/133: write d4/d7/d1d/f23 [1017385,62244] 0 2024-04-26T04:42:10.881 INFO:tasks.workunit.client.0.smithi090.stdout:0/45: dwrite d0/f5 [4194304,4194304] 0 2024-04-26T04:42:10.882 INFO:tasks.workunit.client.0.smithi090.stdout:0/46: mkdir d0/de 0 2024-04-26T04:42:10.883 INFO:tasks.workunit.client.0.smithi090.stdout:5/54: dwrite d4/fe [4194304,4194304] 0 2024-04-26T04:42:10.884 INFO:tasks.workunit.client.0.smithi090.stdout:0/47: mkdir d0/df 0 2024-04-26T04:42:10.885 INFO:tasks.workunit.client.0.smithi090.stdout:5/55: truncate d4/d5/fc 1251212 0 2024-04-26T04:42:10.887 INFO:tasks.workunit.client.0.smithi090.stdout:0/48: symlink d0/d9/l10 0 2024-04-26T04:42:10.887 INFO:tasks.workunit.client.1.smithi104.stdout:3/135: mkdir d5/d9/dc/d29/d2d 0 2024-04-26T04:42:10.888 INFO:tasks.workunit.client.1.smithi104.stdout:3/136: chown d5/d9/dc/f16 130952126 1 2024-04-26T04:42:10.888 INFO:tasks.workunit.client.1.smithi104.stdout:3/137: stat d5/d9/fe 0 2024-04-26T04:42:10.889 INFO:tasks.workunit.client.0.smithi090.stdout:0/49: link d0/cd d0/de/c11 0 2024-04-26T04:42:10.889 INFO:tasks.workunit.client.0.smithi090.stdout:0/50: dread - d0/fc zero size 2024-04-26T04:42:10.889 INFO:tasks.workunit.client.0.smithi090.stdout:0/51: write d0/fc [735596,32138] 0 2024-04-26T04:42:10.889 INFO:tasks.workunit.client.1.smithi104.stdout:1/216: creat d0/d6/d19/d1b/d35/f40 x:0 0 0 2024-04-26T04:42:10.890 INFO:tasks.workunit.client.1.smithi104.stdout:1/217: creat d0/d6/de/f41 x:0 0 0 2024-04-26T04:42:10.893 INFO:tasks.workunit.client.1.smithi104.stdout:4/134: mkdir d4/d7/d1d/d2e 0 2024-04-26T04:42:10.894 INFO:tasks.workunit.client.1.smithi104.stdout:5/186: mknod d0/d1/c34 0 2024-04-26T04:42:10.894 INFO:tasks.workunit.client.1.smithi104.stdout:5/187: write d0/f18 [5538558,95936] 0 2024-04-26T04:42:10.896 INFO:tasks.workunit.client.1.smithi104.stdout:2/246: rename d6/d8/d3e/d21 to d6/d8/d5a 0 2024-04-26T04:42:10.896 INFO:tasks.workunit.client.1.smithi104.stdout:2/247: write d6/d8/f39 [563141,51226] 0 2024-04-26T04:42:10.896 INFO:tasks.workunit.client.1.smithi104.stdout:2/248: stat d6/d8/d5a/f59 0 2024-04-26T04:42:10.897 INFO:tasks.workunit.client.1.smithi104.stdout:3/138: mknod d5/d9/dc/d29/c2e 0 2024-04-26T04:42:10.899 INFO:tasks.workunit.client.1.smithi104.stdout:1/218: symlink d0/d6/de/d20/d2a/l42 0 2024-04-26T04:42:10.900 INFO:tasks.workunit.client.1.smithi104.stdout:8/72: dwrite d0/fb [0,4194304] 0 2024-04-26T04:42:10.900 INFO:tasks.workunit.client.1.smithi104.stdout:8/73: chown d0/d13 3986 1 2024-04-26T04:42:10.900 INFO:tasks.workunit.client.0.smithi090.stdout:6/28: dwrite f3 [4194304,4194304] 0 2024-04-26T04:42:10.902 INFO:tasks.workunit.client.1.smithi104.stdout:4/135: mknod d4/db/c2f 0 2024-04-26T04:42:10.903 INFO:tasks.workunit.client.1.smithi104.stdout:5/188: creat d0/d1/d4/d2a/d2b/f35 x:0 0 0 2024-04-26T04:42:10.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/52: dread d0/fc [0,4194304] 0 2024-04-26T04:42:10.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/53: chown d0/d9 72 1 2024-04-26T04:42:10.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/54: chown d0/l3 5 1 2024-04-26T04:42:10.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/55: chown d0 11 1 2024-04-26T04:42:10.904 INFO:tasks.workunit.client.0.smithi090.stdout:6/29: mknod d9/ca 0 2024-04-26T04:42:10.904 INFO:tasks.workunit.client.0.smithi090.stdout:0/56: dread d0/fc [0,4194304] 0 2024-04-26T04:42:10.906 INFO:tasks.workunit.client.0.smithi090.stdout:0/57: dread d0/fc [0,4194304] 0 2024-04-26T04:42:10.906 INFO:tasks.workunit.client.0.smithi090.stdout:0/58: chown d0/c8 205 1 2024-04-26T04:42:10.907 INFO:tasks.workunit.client.1.smithi104.stdout:2/249: write d6/d8/fb [1618629,77306] 0 2024-04-26T04:42:10.910 INFO:tasks.workunit.client.1.smithi104.stdout:8/74: dread d0/fe [0,4194304] 0 2024-04-26T04:42:10.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/56: write d4/fe [2641925,38599] 0 2024-04-26T04:42:10.912 INFO:tasks.workunit.client.0.smithi090.stdout:5/57: mkdir d4/d5/df 0 2024-04-26T04:42:10.915 INFO:tasks.workunit.client.1.smithi104.stdout:5/189: mkdir d0/d11/d26/d36 0 2024-04-26T04:42:10.915 INFO:tasks.workunit.client.1.smithi104.stdout:5/190: truncate d0/d11/f23 316804 0 2024-04-26T04:42:10.917 INFO:tasks.workunit.client.1.smithi104.stdout:2/250: creat d6/d8/f5b x:0 0 0 2024-04-26T04:42:10.917 INFO:tasks.workunit.client.1.smithi104.stdout:4/136: dread d4/f11 [0,4194304] 0 2024-04-26T04:42:10.917 INFO:tasks.workunit.client.1.smithi104.stdout:4/137: truncate d4/f17 284200 0 2024-04-26T04:42:10.917 INFO:tasks.workunit.client.0.smithi090.stdout:6/30: getdents d9 0 2024-04-26T04:42:10.917 INFO:tasks.workunit.client.1.smithi104.stdout:8/75: mknod d0/d13/c17 0 2024-04-26T04:42:10.919 INFO:tasks.workunit.client.0.smithi090.stdout:6/31: rename c0 to d9/cb 0 2024-04-26T04:42:10.919 INFO:tasks.workunit.client.1.smithi104.stdout:5/191: rename d0/d11/d26/d32 to d0/d1/d4/d2a/d2b/d37 0 2024-04-26T04:42:10.919 INFO:tasks.workunit.client.1.smithi104.stdout:8/76: rename d0/d12 to d0/d12/d18 22 2024-04-26T04:42:10.920 INFO:tasks.workunit.client.1.smithi104.stdout:5/192: read d0/f18 [566636,106633] 0 2024-04-26T04:42:10.920 INFO:tasks.workunit.client.1.smithi104.stdout:5/193: chown d0/d11/d26 8368745 1 2024-04-26T04:42:10.921 INFO:tasks.workunit.client.0.smithi090.stdout:6/32: unlink f7 0 2024-04-26T04:42:10.921 INFO:tasks.workunit.client.0.smithi090.stdout:6/33: rmdir d9 39 2024-04-26T04:42:10.922 INFO:tasks.workunit.client.1.smithi104.stdout:5/194: rmdir d0/d1/d4/d2a/d33 0 2024-04-26T04:42:10.924 INFO:tasks.workunit.client.1.smithi104.stdout:5/195: creat d0/d1/f38 x:0 0 0 2024-04-26T04:42:10.924 INFO:tasks.workunit.client.1.smithi104.stdout:5/196: write d0/d1/d4/f17 [3457672,64400] 0 2024-04-26T04:42:10.924 INFO:tasks.workunit.client.1.smithi104.stdout:5/197: fdatasync d0/d11/f19 0 2024-04-26T04:42:10.925 INFO:tasks.workunit.client.1.smithi104.stdout:5/198: symlink d0/d1/d25/l39 0 2024-04-26T04:42:10.929 INFO:tasks.workunit.client.1.smithi104.stdout:3/139: dwrite f0 [0,4194304] 0 2024-04-26T04:42:10.933 INFO:tasks.workunit.client.1.smithi104.stdout:8/77: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:10.933 INFO:tasks.workunit.client.1.smithi104.stdout:5/199: dread d0/d1/d1e/f21 [0,4194304] 0 2024-04-26T04:42:10.935 INFO:tasks.workunit.client.1.smithi104.stdout:5/200: link d0/d1/c15 d0/c3a 0 2024-04-26T04:42:10.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/34: write f3 [6886852,37765] 0 2024-04-26T04:42:10.936 INFO:tasks.workunit.client.1.smithi104.stdout:8/78: dread d0/fe [0,4194304] 0 2024-04-26T04:42:10.936 INFO:tasks.workunit.client.1.smithi104.stdout:5/201: symlink d0/d1/d4/l3b 0 2024-04-26T04:42:10.936 INFO:tasks.workunit.client.1.smithi104.stdout:1/219: dwrite d0/d6/de/f41 [0,4194304] 0 2024-04-26T04:42:10.937 INFO:tasks.workunit.client.0.smithi090.stdout:6/35: creat d9/fc x:0 0 0 2024-04-26T04:42:10.938 INFO:tasks.workunit.client.0.smithi090.stdout:6/36: stat d9/cb 0 2024-04-26T04:42:10.939 INFO:tasks.workunit.client.1.smithi104.stdout:5/202: chown d0/d1/c1b 15394508 1 2024-04-26T04:42:10.941 INFO:tasks.workunit.client.0.smithi090.stdout:6/37: rename f5 to d9/fd 0 2024-04-26T04:42:10.941 INFO:tasks.workunit.client.0.smithi090.stdout:6/38: stat d9/ca 0 2024-04-26T04:42:10.941 INFO:tasks.workunit.client.0.smithi090.stdout:6/39: truncate d9/fc 494009 0 2024-04-26T04:42:10.941 INFO:tasks.workunit.client.0.smithi090.stdout:6/40: write d9/fc [1317403,105144] 0 2024-04-26T04:42:10.941 INFO:tasks.workunit.client.0.smithi090.stdout:6/41: chown d9/fc 35 1 2024-04-26T04:42:10.941 INFO:tasks.workunit.client.0.smithi090.stdout:6/42: rename d9 to d9/de 22 2024-04-26T04:42:10.942 INFO:tasks.workunit.client.1.smithi104.stdout:2/251: dwrite f2 [0,4194304] 0 2024-04-26T04:42:10.942 INFO:tasks.workunit.client.1.smithi104.stdout:1/220: dread d0/d6/de/d20/d2a/f2c [0,4194304] 0 2024-04-26T04:42:10.943 INFO:tasks.workunit.client.0.smithi090.stdout:0/59: dwrite d0/f5 [4194304,4194304] 0 2024-04-26T04:42:10.944 INFO:tasks.workunit.client.0.smithi090.stdout:5/58: truncate d4/fe 6107160 0 2024-04-26T04:42:10.944 INFO:tasks.workunit.client.1.smithi104.stdout:5/203: creat d0/d1/d25/f3c x:0 0 0 2024-04-26T04:42:10.945 INFO:tasks.workunit.client.1.smithi104.stdout:1/221: mknod d0/d6/de/c43 0 2024-04-26T04:42:10.946 INFO:tasks.workunit.client.0.smithi090.stdout:0/60: unlink d0/d9/cb 0 2024-04-26T04:42:10.946 INFO:tasks.workunit.client.1.smithi104.stdout:5/204: creat d0/d1/f3d x:0 0 0 2024-04-26T04:42:10.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/222: truncate d0/d6/ff 7456758 0 2024-04-26T04:42:10.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/223: chown d0/d6/de/l10 3968 1 2024-04-26T04:42:10.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/224: chown d0/d6/de/l3d 1 1 2024-04-26T04:42:10.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/225: rename d0/d6/d2e to d0/d6/d2e/d44 22 2024-04-26T04:42:10.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/226: dread - d0/d6/d19/d21/d28/f36 zero size 2024-04-26T04:42:10.948 INFO:tasks.workunit.client.0.smithi090.stdout:5/59: unlink d4/d5/fc 0 2024-04-26T04:42:10.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/61: creat d0/df/f12 x:0 0 0 2024-04-26T04:42:10.950 INFO:tasks.workunit.client.1.smithi104.stdout:5/205: mkdir d0/d1/d3e 0 2024-04-26T04:42:10.952 INFO:tasks.workunit.client.0.smithi090.stdout:6/43: dread d9/fc [0,4194304] 0 2024-04-26T04:42:10.953 INFO:tasks.workunit.client.0.smithi090.stdout:5/60: rename d4/d9/fa to d4/d5/df/f10 0 2024-04-26T04:42:10.956 INFO:tasks.workunit.client.0.smithi090.stdout:6/44: mknod d9/cf 0 2024-04-26T04:42:10.963 INFO:tasks.workunit.client.0.smithi090.stdout:0/62: dread d0/f5 [4194304,4194304] 0 2024-04-26T04:42:10.964 INFO:tasks.workunit.client.1.smithi104.stdout:3/140: dwrite d5/f22 [0,4194304] 0 2024-04-26T04:42:10.964 INFO:tasks.workunit.client.0.smithi090.stdout:0/63: creat d0/d9/f13 x:0 0 0 2024-04-26T04:42:10.966 INFO:tasks.workunit.client.0.smithi090.stdout:0/64: dread d0/fc [0,4194304] 0 2024-04-26T04:42:10.967 INFO:tasks.workunit.client.0.smithi090.stdout:0/65: creat d0/de/f14 x:0 0 0 2024-04-26T04:42:10.969 INFO:tasks.workunit.client.0.smithi090.stdout:0/66: symlink d0/l15 0 2024-04-26T04:42:10.977 INFO:tasks.workunit.client.1.smithi104.stdout:8/79: dwrite d0/fe [0,4194304] 0 2024-04-26T04:42:10.980 INFO:tasks.workunit.client.1.smithi104.stdout:8/80: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:10.980 INFO:tasks.workunit.client.1.smithi104.stdout:2/252: dwrite d6/d8/d5a/f2e [0,4194304] 0 2024-04-26T04:42:10.980 INFO:tasks.workunit.client.1.smithi104.stdout:2/253: chown d6/d8 7051 1 2024-04-26T04:42:10.981 INFO:tasks.workunit.client.1.smithi104.stdout:8/81: chown d0/l10 31002015 1 2024-04-26T04:42:10.983 INFO:tasks.workunit.client.1.smithi104.stdout:8/82: symlink d0/l19 0 2024-04-26T04:42:10.984 INFO:tasks.workunit.client.1.smithi104.stdout:2/254: rmdir d6/d8 39 2024-04-26T04:42:10.984 INFO:tasks.workunit.client.1.smithi104.stdout:2/255: read - d6/d1b/f38 zero size 2024-04-26T04:42:10.985 INFO:tasks.workunit.client.1.smithi104.stdout:2/256: dread d6/de/f40 [0,4194304] 0 2024-04-26T04:42:10.985 INFO:tasks.workunit.client.1.smithi104.stdout:2/257: chown d6/d8/d3e/c50 339956159 1 2024-04-26T04:42:10.985 INFO:tasks.workunit.client.1.smithi104.stdout:2/258: write d6/d8/f46 [935377,37021] 0 2024-04-26T04:42:10.985 INFO:tasks.workunit.client.1.smithi104.stdout:2/259: write d6/d8/f56 [1468695,31225] 0 2024-04-26T04:42:10.986 INFO:tasks.workunit.client.1.smithi104.stdout:8/83: dread d0/f9 [0,4194304] 0 2024-04-26T04:42:10.986 INFO:tasks.workunit.client.1.smithi104.stdout:8/84: fsync d0/f1 0 2024-04-26T04:42:10.986 INFO:tasks.workunit.client.1.smithi104.stdout:2/260: mknod d6/de/c5c 0 2024-04-26T04:42:10.987 INFO:tasks.workunit.client.1.smithi104.stdout:2/261: truncate d6/d8/f37 699246 0 2024-04-26T04:42:10.988 INFO:tasks.workunit.client.1.smithi104.stdout:2/262: symlink d6/d8/d3e/l5d 0 2024-04-26T04:42:10.988 INFO:tasks.workunit.client.1.smithi104.stdout:1/227: dwrite d0/d6/d19/d21/d28/f36 [0,4194304] 0 2024-04-26T04:42:10.991 INFO:tasks.workunit.client.1.smithi104.stdout:5/206: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:10.991 INFO:tasks.workunit.client.1.smithi104.stdout:5/207: fsync d0/d1/f38 0 2024-04-26T04:42:10.991 INFO:tasks.workunit.client.1.smithi104.stdout:1/228: rmdir d0/d6 39 2024-04-26T04:42:10.991 INFO:tasks.workunit.client.1.smithi104.stdout:1/229: readlink d0/d6/l27 0 2024-04-26T04:42:10.993 INFO:tasks.workunit.client.1.smithi104.stdout:3/141: dwrite d5/d9/dc/f19 [0,4194304] 0 2024-04-26T04:42:10.993 INFO:tasks.workunit.client.1.smithi104.stdout:3/142: write d5/d9/dc/f21 [549593,38825] 0 2024-04-26T04:42:10.994 INFO:tasks.workunit.client.0.smithi090.stdout:0/67: getdents d0/df 0 2024-04-26T04:42:10.994 INFO:tasks.workunit.client.0.smithi090.stdout:6/45: dwrite d9/fd [8388608,4194304] 0 2024-04-26T04:42:10.997 INFO:tasks.workunit.client.0.smithi090.stdout:0/68: mkdir d0/df/d16 0 2024-04-26T04:42:10.997 INFO:tasks.workunit.client.0.smithi090.stdout:0/69: truncate d0/d9/f13 528578 0 2024-04-26T04:42:11.007 INFO:tasks.workunit.client.1.smithi104.stdout:5/208: rename d0/d1/f2f to d0/d1/d4/d2a/d2b/d37/f3f 0 2024-04-26T04:42:11.007 INFO:tasks.workunit.client.1.smithi104.stdout:5/209: truncate d0/f18 6102502 0 2024-04-26T04:42:11.008 INFO:tasks.workunit.client.1.smithi104.stdout:3/143: mknod d5/c2f 0 2024-04-26T04:42:11.008 INFO:tasks.workunit.client.1.smithi104.stdout:3/144: truncate d5/d9/d17/f1b 684024 0 2024-04-26T04:42:11.009 INFO:tasks.workunit.client.1.smithi104.stdout:2/263: rmdir d6 39 2024-04-26T04:42:11.009 INFO:tasks.workunit.client.1.smithi104.stdout:2/264: write d6/d8/d3e/f2a [3066839,26290] 0 2024-04-26T04:42:11.009 INFO:tasks.workunit.client.1.smithi104.stdout:3/145: read d5/f23 [256473,44821] 0 2024-04-26T04:42:11.010 INFO:tasks.workunit.client.1.smithi104.stdout:3/146: write d5/d9/dc/f16 [95122,101716] 0 2024-04-26T04:42:11.010 INFO:tasks.workunit.client.1.smithi104.stdout:3/147: dread - d5/f24 zero size 2024-04-26T04:42:11.011 INFO:tasks.workunit.client.1.smithi104.stdout:1/230: rename d0/d6/d19/d21/l2d to d0/d6/de/d29/d3f/l45 0 2024-04-26T04:42:11.011 INFO:tasks.workunit.client.1.smithi104.stdout:1/231: write d0/d6/de/d29/f39 [277563,24274] 0 2024-04-26T04:42:11.013 INFO:tasks.workunit.client.1.smithi104.stdout:5/210: creat d0/d11/d26/d36/f40 x:0 0 0 2024-04-26T04:42:11.013 INFO:tasks.workunit.client.1.smithi104.stdout:2/265: getdents d6/de 0 2024-04-26T04:42:11.014 INFO:tasks.workunit.client.1.smithi104.stdout:8/85: dwrite d0/f9 [0,4194304] 0 2024-04-26T04:42:11.015 INFO:tasks.workunit.client.1.smithi104.stdout:8/86: creat d0/f1a x:0 0 0 2024-04-26T04:42:11.015 INFO:tasks.workunit.client.1.smithi104.stdout:3/148: mknod d5/d9/dc/d29/d2d/c30 0 2024-04-26T04:42:11.015 INFO:tasks.workunit.client.1.smithi104.stdout:3/149: fdatasync d5/d9/f1d 0 2024-04-26T04:42:11.015 INFO:tasks.workunit.client.1.smithi104.stdout:1/232: creat d0/d6/d2e/f46 x:0 0 0 2024-04-26T04:42:11.016 INFO:tasks.workunit.client.1.smithi104.stdout:2/266: symlink d6/d8/d5a/d29/l5e 0 2024-04-26T04:42:11.018 INFO:tasks.workunit.client.1.smithi104.stdout:8/87: rename d0/l11 to d0/d12/l1b 0 2024-04-26T04:42:11.019 INFO:tasks.workunit.client.1.smithi104.stdout:1/233: rmdir d0/d6/d19/d21 39 2024-04-26T04:42:11.020 INFO:tasks.workunit.client.1.smithi104.stdout:2/267: rmdir d6/d8/d5a 39 2024-04-26T04:42:11.020 INFO:tasks.workunit.client.1.smithi104.stdout:2/268: chown d6/d8/f46 0 1 2024-04-26T04:42:11.020 INFO:tasks.workunit.client.1.smithi104.stdout:2/269: readlink d6/d1b/l4d 0 2024-04-26T04:42:11.020 INFO:tasks.workunit.client.1.smithi104.stdout:2/270: read - d6/f33 zero size 2024-04-26T04:42:11.020 INFO:tasks.workunit.client.1.smithi104.stdout:8/88: mkdir d0/d1c 0 2024-04-26T04:42:11.020 INFO:tasks.workunit.client.1.smithi104.stdout:8/89: chown d0/d12/l1b 4974810 1 2024-04-26T04:42:11.021 INFO:tasks.workunit.client.1.smithi104.stdout:2/271: creat d6/de/d1d/f5f x:0 0 0 2024-04-26T04:42:11.021 INFO:tasks.workunit.client.1.smithi104.stdout:2/272: write d6/d8/d3e/f2a [3167461,61600] 0 2024-04-26T04:42:11.023 INFO:tasks.workunit.client.0.smithi090.stdout:6/46: dwrite f2 [0,4194304] 0 2024-04-26T04:42:11.024 INFO:tasks.workunit.client.1.smithi104.stdout:2/273: write d6/de/f36 [117562,19349] 0 2024-04-26T04:42:11.024 INFO:tasks.workunit.client.1.smithi104.stdout:2/274: getdents d6/d1b/d51 0 2024-04-26T04:42:11.026 INFO:tasks.workunit.client.0.smithi090.stdout:5/61: dwrite d4/fe [0,4194304] 0 2024-04-26T04:42:11.026 INFO:tasks.workunit.client.0.smithi090.stdout:0/70: dwrite d0/f5 [4194304,4194304] 0 2024-04-26T04:42:11.026 INFO:tasks.workunit.client.0.smithi090.stdout:0/71: fdatasync d0/fc 0 2024-04-26T04:42:11.026 INFO:tasks.workunit.client.1.smithi104.stdout:2/275: rename d6/de/f52 to d6/d8/d5a/d29/f60 0 2024-04-26T04:42:11.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/72: dread - d0/df/f12 zero size 2024-04-26T04:42:11.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/73: creat d0/d9/f17 x:0 0 0 2024-04-26T04:42:11.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/74: write d0/d9/f17 [522534,59426] 0 2024-04-26T04:42:11.028 INFO:tasks.workunit.client.1.smithi104.stdout:0/141: sync 2024-04-26T04:42:11.029 INFO:tasks.workunit.client.1.smithi104.stdout:6/297: sync 2024-04-26T04:42:11.029 INFO:tasks.workunit.client.0.smithi090.stdout:0/75: symlink d0/de/l18 0 2024-04-26T04:42:11.029 INFO:tasks.workunit.client.0.smithi090.stdout:0/76: symlink d0/l19 0 2024-04-26T04:42:11.030 INFO:tasks.workunit.client.0.smithi090.stdout:0/77: chown d0/de/l18 3024766 1 2024-04-26T04:42:11.030 INFO:tasks.workunit.client.1.smithi104.stdout:2/276: mknod d6/c61 0 2024-04-26T04:42:11.030 INFO:tasks.workunit.client.1.smithi104.stdout:2/277: write d6/d8/fb [4556635,80970] 0 2024-04-26T04:42:11.031 INFO:tasks.workunit.client.1.smithi104.stdout:2/278: truncate d6/d1b/f38 458181 0 2024-04-26T04:42:11.032 INFO:tasks.workunit.client.0.smithi090.stdout:0/78: symlink d0/df/l1a 0 2024-04-26T04:42:11.032 INFO:tasks.workunit.client.0.smithi090.stdout:0/79: truncate d0/de/f14 315281 0 2024-04-26T04:42:11.033 INFO:tasks.workunit.client.1.smithi104.stdout:6/298: creat d0/d9/d25/f68 x:0 0 0 2024-04-26T04:42:11.034 INFO:tasks.workunit.client.1.smithi104.stdout:2/279: truncate d6/fd 1928846 0 2024-04-26T04:42:11.034 INFO:tasks.workunit.client.0.smithi090.stdout:0/80: mknod d0/df/d16/c1b 0 2024-04-26T04:42:11.034 INFO:tasks.workunit.client.0.smithi090.stdout:0/81: write d0/fc [950353,118889] 0 2024-04-26T04:42:11.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/299: rename d0/d9/d25/f34 to d0/d9/d5c/f69 0 2024-04-26T04:42:11.039 INFO:tasks.workunit.client.1.smithi104.stdout:2/280: creat d6/f62 x:0 0 0 2024-04-26T04:42:11.040 INFO:tasks.workunit.client.1.smithi104.stdout:6/300: mknod d0/d9/d52/c6a 0 2024-04-26T04:42:11.040 INFO:tasks.workunit.client.1.smithi104.stdout:6/301: chown d0/d9/d25/c4f 5570731 1 2024-04-26T04:42:11.042 INFO:tasks.workunit.client.0.smithi090.stdout:5/62: read d4/fe [4481190,79049] 0 2024-04-26T04:42:11.042 INFO:tasks.workunit.client.0.smithi090.stdout:0/82: dread d0/d9/f13 [0,4194304] 0 2024-04-26T04:42:11.043 INFO:tasks.workunit.client.1.smithi104.stdout:6/302: mknod d0/d9/d52/c6b 0 2024-04-26T04:42:11.043 INFO:tasks.workunit.client.1.smithi104.stdout:6/303: chown d0 714418 1 2024-04-26T04:42:11.043 INFO:tasks.workunit.client.1.smithi104.stdout:6/304: dread - d0/d9/d5c/d3f/f5d zero size 2024-04-26T04:42:11.043 INFO:tasks.workunit.client.1.smithi104.stdout:6/305: write d0/d9/d13/d44/f58 [1036171,37087] 0 2024-04-26T04:42:11.043 INFO:tasks.workunit.client.1.smithi104.stdout:6/306: readlink d0/d5/l59 0 2024-04-26T04:42:11.044 INFO:tasks.workunit.client.1.smithi104.stdout:1/234: dwrite d0/d6/d19/f3a [0,4194304] 0 2024-04-26T04:42:11.044 INFO:tasks.workunit.client.1.smithi104.stdout:1/235: chown d0/d6/d19/c1e 6443621 1 2024-04-26T04:42:11.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/63: rmdir d4/d5 39 2024-04-26T04:42:11.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/64: chown d4/d5/df 338101546 1 2024-04-26T04:42:11.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/65: rename d4 to d4/d9/d11 22 2024-04-26T04:42:11.046 INFO:tasks.workunit.client.0.smithi090.stdout:6/47: write d9/fd [11018821,14143] 0 2024-04-26T04:42:11.047 INFO:tasks.workunit.client.1.smithi104.stdout:8/90: dwrite d0/fb [4194304,4194304] 0 2024-04-26T04:42:11.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/48: mknod d9/c10 0 2024-04-26T04:42:11.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/49: write d9/fd [12313411,26823] 0 2024-04-26T04:42:11.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/50: creat d9/f11 x:0 0 0 2024-04-26T04:42:11.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/51: write d9/fc [1431707,67008] 0 2024-04-26T04:42:11.050 INFO:tasks.workunit.client.0.smithi090.stdout:6/52: write d9/f11 [416137,5490] 0 2024-04-26T04:42:11.050 INFO:tasks.workunit.client.0.smithi090.stdout:6/53: creat d9/f12 x:0 0 0 2024-04-26T04:42:11.055 INFO:tasks.workunit.client.1.smithi104.stdout:0/142: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:11.058 INFO:tasks.workunit.client.0.smithi090.stdout:0/83: getdents d0 0 2024-04-26T04:42:11.058 INFO:tasks.workunit.client.0.smithi090.stdout:0/84: chown d0/d9/l10 0 1 2024-04-26T04:42:11.059 INFO:tasks.workunit.client.0.smithi090.stdout:0/85: symlink d0/df/d16/l1c 0 2024-04-26T04:42:11.060 INFO:tasks.workunit.client.0.smithi090.stdout:0/86: symlink d0/df/d16/l1d 0 2024-04-26T04:42:11.060 INFO:tasks.workunit.client.1.smithi104.stdout:2/281: dwrite d6/d8/d5a/f2e [0,4194304] 0 2024-04-26T04:42:11.060 INFO:tasks.workunit.client.1.smithi104.stdout:2/282: chown d6/d1b/f38 230805 1 2024-04-26T04:42:11.060 INFO:tasks.workunit.client.1.smithi104.stdout:2/283: write d6/de/d1d/f42 [671451,9395] 0 2024-04-26T04:42:11.061 INFO:tasks.workunit.client.1.smithi104.stdout:1/236: mknod d0/d6/de/d29/c47 0 2024-04-26T04:42:11.064 INFO:tasks.workunit.client.0.smithi090.stdout:0/87: mknod d0/de/c1e 0 2024-04-26T04:42:11.064 INFO:tasks.workunit.client.0.smithi090.stdout:0/88: fsync d0/d9/f13 0 2024-04-26T04:42:11.066 INFO:tasks.workunit.client.0.smithi090.stdout:6/54: truncate d9/fd 10383791 0 2024-04-26T04:42:11.068 INFO:tasks.workunit.client.1.smithi104.stdout:8/91: creat d0/d13/f1d x:0 0 0 2024-04-26T04:42:11.068 INFO:tasks.workunit.client.1.smithi104.stdout:8/92: chown d0/l3 615595 1 2024-04-26T04:42:11.068 INFO:tasks.workunit.client.1.smithi104.stdout:8/93: creat d0/f1e x:0 0 0 2024-04-26T04:42:11.068 INFO:tasks.workunit.client.1.smithi104.stdout:8/94: chown d0/d12 466329 1 2024-04-26T04:42:11.068 INFO:tasks.workunit.client.1.smithi104.stdout:8/95: readlink d0/l6 0 2024-04-26T04:42:11.068 INFO:tasks.workunit.client.0.smithi090.stdout:0/89: symlink d0/df/d16/l1f 0 2024-04-26T04:42:11.071 INFO:tasks.workunit.client.1.smithi104.stdout:0/143: creat d0/d2/d21/d27/f30 x:0 0 0 2024-04-26T04:42:11.072 INFO:tasks.workunit.client.1.smithi104.stdout:0/144: truncate d0/d4/f11 265731 0 2024-04-26T04:42:11.073 INFO:tasks.workunit.client.1.smithi104.stdout:2/284: link d6/d8/d3e/f2a d6/f63 0 2024-04-26T04:42:11.075 INFO:tasks.workunit.client.1.smithi104.stdout:1/237: mkdir d0/d6/d2e/d32/d48 0 2024-04-26T04:42:11.076 INFO:tasks.workunit.client.0.smithi090.stdout:6/55: dread f2 [0,4194304] 0 2024-04-26T04:42:11.077 INFO:tasks.workunit.client.1.smithi104.stdout:8/96: creat d0/d12/f1f x:0 0 0 2024-04-26T04:42:11.078 INFO:tasks.workunit.client.1.smithi104.stdout:0/145: link d0/d4/d15/c16 d0/d2/d21/d27/d1e/c31 0 2024-04-26T04:42:11.079 INFO:tasks.workunit.client.0.smithi090.stdout:6/56: symlink d9/l13 0 2024-04-26T04:42:11.080 INFO:tasks.workunit.client.1.smithi104.stdout:2/285: link d6/d8/d5a/l48 d6/d1b/l64 0 2024-04-26T04:42:11.081 INFO:tasks.workunit.client.0.smithi090.stdout:6/57: link f1 d9/f14 0 2024-04-26T04:42:11.083 INFO:tasks.workunit.client.1.smithi104.stdout:8/97: truncate d0/f9 276324 0 2024-04-26T04:42:11.083 INFO:tasks.workunit.client.1.smithi104.stdout:8/98: write d0/d12/f1f [281705,46340] 0 2024-04-26T04:42:11.083 INFO:tasks.workunit.client.1.smithi104.stdout:8/99: readlink d0/l3 0 2024-04-26T04:42:11.083 INFO:tasks.workunit.client.1.smithi104.stdout:2/286: dread d6/f10 [0,4194304] 0 2024-04-26T04:42:11.084 INFO:tasks.workunit.client.1.smithi104.stdout:2/287: write d6/d1b/f38 [45669,61277] 0 2024-04-26T04:42:11.085 INFO:tasks.workunit.client.0.smithi090.stdout:0/90: dwrite d0/d9/f13 [0,4194304] 0 2024-04-26T04:42:11.087 INFO:tasks.workunit.client.1.smithi104.stdout:0/146: unlink d0/d2/f26 0 2024-04-26T04:42:11.091 INFO:tasks.workunit.client.1.smithi104.stdout:2/288: rename d6/de/l32 to d6/d1b/d51/l65 0 2024-04-26T04:42:11.092 INFO:tasks.workunit.client.1.smithi104.stdout:0/147: creat d0/d2/d10/f32 x:0 0 0 2024-04-26T04:42:11.092 INFO:tasks.workunit.client.1.smithi104.stdout:0/148: chown d0/d2/d21/d27/d1e/f2c 697 1 2024-04-26T04:42:11.092 INFO:tasks.workunit.client.1.smithi104.stdout:0/149: truncate d0/d2/d21/d27/d1e/f2d 999330 0 2024-04-26T04:42:11.094 INFO:tasks.workunit.client.1.smithi104.stdout:2/289: symlink d6/d8/l66 0 2024-04-26T04:42:11.096 INFO:tasks.workunit.client.1.smithi104.stdout:0/150: mknod d0/d2/d21/c33 0 2024-04-26T04:42:11.096 INFO:tasks.workunit.client.1.smithi104.stdout:0/151: truncate d0/d4/f1b 573010 0 2024-04-26T04:42:11.097 INFO:tasks.workunit.client.1.smithi104.stdout:2/290: symlink d6/d8/d3e/l67 0 2024-04-26T04:42:11.102 INFO:tasks.workunit.client.1.smithi104.stdout:1/238: truncate d0/d6/d19/f3a 1160588 0 2024-04-26T04:42:11.104 INFO:tasks.workunit.client.1.smithi104.stdout:1/239: mkdir d0/d6/d49 0 2024-04-26T04:42:11.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/58: dwrite f2 [0,4194304] 0 2024-04-26T04:42:11.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/59: creat d9/f15 x:0 0 0 2024-04-26T04:42:11.109 INFO:tasks.workunit.client.1.smithi104.stdout:8/100: fsync d0/f1e 0 2024-04-26T04:42:11.110 INFO:tasks.workunit.client.1.smithi104.stdout:8/101: write d0/f1a [161494,5209] 0 2024-04-26T04:42:11.110 INFO:tasks.workunit.client.1.smithi104.stdout:8/102: chown d0/ld 20909 1 2024-04-26T04:42:11.110 INFO:tasks.workunit.client.0.smithi090.stdout:6/60: read d9/fc [1143458,2370] 0 2024-04-26T04:42:11.119 INFO:tasks.workunit.client.0.smithi090.stdout:0/91: dwrite d0/de/f14 [0,4194304] 0 2024-04-26T04:42:11.122 INFO:tasks.workunit.client.0.smithi090.stdout:0/92: dread d0/fc [0,4194304] 0 2024-04-26T04:42:11.123 INFO:tasks.workunit.client.0.smithi090.stdout:0/93: symlink d0/d9/l20 0 2024-04-26T04:42:11.124 INFO:tasks.workunit.client.1.smithi104.stdout:0/152: truncate d0/d2/d21/d27/d1e/f2d 494255 0 2024-04-26T04:42:11.124 INFO:tasks.workunit.client.0.smithi090.stdout:0/94: mknod d0/d9/c21 0 2024-04-26T04:42:11.124 INFO:tasks.workunit.client.0.smithi090.stdout:0/95: stat d0/d9/la 0 2024-04-26T04:42:11.125 INFO:tasks.workunit.client.0.smithi090.stdout:0/96: symlink d0/de/l22 0 2024-04-26T04:42:11.126 INFO:tasks.workunit.client.0.smithi090.stdout:0/97: write d0/fc [9477,46944] 0 2024-04-26T04:42:11.126 INFO:tasks.workunit.client.1.smithi104.stdout:2/291: dwrite d6/f10 [0,4194304] 0 2024-04-26T04:42:11.126 INFO:tasks.workunit.client.1.smithi104.stdout:2/292: readlink d6/d8/d3e/l67 0 2024-04-26T04:42:11.127 INFO:tasks.workunit.client.1.smithi104.stdout:8/103: read d0/f9 [256423,112682] 0 2024-04-26T04:42:11.127 INFO:tasks.workunit.client.1.smithi104.stdout:1/240: dwrite d0/d6/de/d20/d2a/f2c [0,4194304] 0 2024-04-26T04:42:11.127 INFO:tasks.workunit.client.1.smithi104.stdout:8/104: chown d0/l3 24993218 1 2024-04-26T04:42:11.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/61: dwrite d9/f12 [0,4194304] 0 2024-04-26T04:42:11.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/62: chown d9/fd 4683862 1 2024-04-26T04:42:11.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/63: creat d9/f16 x:0 0 0 2024-04-26T04:42:11.131 INFO:tasks.workunit.client.1.smithi104.stdout:0/153: dread d0/d4/f29 [0,4194304] 0 2024-04-26T04:42:11.134 INFO:tasks.workunit.client.1.smithi104.stdout:1/241: rmdir d0/d6/d2e/d32/d48 0 2024-04-26T04:42:11.135 INFO:tasks.workunit.client.1.smithi104.stdout:1/242: creat d0/d6/d49/f4a x:0 0 0 2024-04-26T04:42:11.136 INFO:tasks.workunit.client.0.smithi090.stdout:6/64: rename f2 to d9/f17 0 2024-04-26T04:42:11.136 INFO:tasks.workunit.client.0.smithi090.stdout:6/65: readlink d9/l13 0 2024-04-26T04:42:11.137 INFO:tasks.workunit.client.1.smithi104.stdout:1/243: getdents d0/d6/de/d20 0 2024-04-26T04:42:11.138 INFO:tasks.workunit.client.0.smithi090.stdout:6/66: mkdir d9/d18 0 2024-04-26T04:42:11.140 INFO:tasks.workunit.client.1.smithi104.stdout:8/105: dread d0/fe [0,4194304] 0 2024-04-26T04:42:11.141 INFO:tasks.workunit.client.1.smithi104.stdout:1/244: rename d0/d6/de/d3c to d0/d6/d19/d21/d28/d4b 0 2024-04-26T04:42:11.141 INFO:tasks.workunit.client.1.smithi104.stdout:1/245: chown d0/d6/d19/d21/d28 12042228 1 2024-04-26T04:42:11.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/98: write d0/d9/f13 [2817825,106678] 0 2024-04-26T04:42:11.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/99: chown d0/df/l1a 19408 1 2024-04-26T04:42:11.143 INFO:tasks.workunit.client.1.smithi104.stdout:8/106: mknod d0/d12/c20 0 2024-04-26T04:42:11.145 INFO:tasks.workunit.client.1.smithi104.stdout:8/107: symlink d0/d12/l21 0 2024-04-26T04:42:11.145 INFO:tasks.workunit.client.1.smithi104.stdout:8/108: fdatasync d0/fe 0 2024-04-26T04:42:11.150 INFO:tasks.workunit.client.1.smithi104.stdout:8/109: dread d0/d12/f1f [0,4194304] 0 2024-04-26T04:42:11.151 INFO:tasks.workunit.client.1.smithi104.stdout:8/110: unlink d0/d12/c16 0 2024-04-26T04:42:11.151 INFO:tasks.workunit.client.1.smithi104.stdout:8/111: readlink d0/l4 0 2024-04-26T04:42:11.152 INFO:tasks.workunit.client.1.smithi104.stdout:0/154: dwrite d0/f2f [0,4194304] 0 2024-04-26T04:42:11.153 INFO:tasks.workunit.client.1.smithi104.stdout:8/112: rename d0/ca to d0/d12/c22 0 2024-04-26T04:42:11.154 INFO:tasks.workunit.client.1.smithi104.stdout:1/246: dread d0/d6/d19/d21/d28/f36 [0,4194304] 0 2024-04-26T04:42:11.155 INFO:tasks.workunit.client.1.smithi104.stdout:2/293: dwrite d6/de/d1d/f42 [0,4194304] 0 2024-04-26T04:42:11.155 INFO:tasks.workunit.client.1.smithi104.stdout:2/294: chown d6/de/c16 1 1 2024-04-26T04:42:11.157 INFO:tasks.workunit.client.1.smithi104.stdout:0/155: creat d0/d2/d28/f34 x:0 0 0 2024-04-26T04:42:11.157 INFO:tasks.workunit.client.1.smithi104.stdout:0/156: read d0/d4/f7 [392573,123484] 0 2024-04-26T04:42:11.157 INFO:tasks.workunit.client.1.smithi104.stdout:1/247: mkdir d0/d6/d2e/d32/d4c 0 2024-04-26T04:42:11.159 INFO:tasks.workunit.client.0.smithi090.stdout:0/100: truncate d0/d9/f13 3002789 0 2024-04-26T04:42:11.159 INFO:tasks.workunit.client.0.smithi090.stdout:0/101: dread - d0/df/f12 zero size 2024-04-26T04:42:11.160 INFO:tasks.workunit.client.0.smithi090.stdout:0/102: chown d0/d9/l10 0 1 2024-04-26T04:42:11.160 INFO:tasks.workunit.client.1.smithi104.stdout:7/47: sync 2024-04-26T04:42:11.161 INFO:tasks.workunit.client.1.smithi104.stdout:2/295: mkdir d6/d1b/d68 0 2024-04-26T04:42:11.163 INFO:tasks.workunit.client.0.smithi090.stdout:6/67: dwrite f1 [0,4194304] 0 2024-04-26T04:42:11.163 INFO:tasks.workunit.client.1.smithi104.stdout:7/48: link d2/l5 d2/le 0 2024-04-26T04:42:11.164 INFO:tasks.workunit.client.0.smithi090.stdout:0/103: dread d0/d9/f17 [0,4194304] 0 2024-04-26T04:42:11.164 INFO:tasks.workunit.client.0.smithi090.stdout:0/104: readlink d0/df/d16/l1c 0 2024-04-26T04:42:11.164 INFO:tasks.workunit.client.0.smithi090.stdout:0/105: chown d0/d9 1922908 1 2024-04-26T04:42:11.164 INFO:tasks.workunit.client.1.smithi104.stdout:2/296: rename d6/d8/d5a/f59 to d6/d8/d5a/d29/f69 0 2024-04-26T04:42:11.164 INFO:tasks.workunit.client.1.smithi104.stdout:2/297: write d6/d8/d5a/d29/f60 [457575,62394] 0 2024-04-26T04:42:11.166 INFO:tasks.workunit.client.0.smithi090.stdout:6/68: rmdir d9/d18 0 2024-04-26T04:42:11.167 INFO:tasks.workunit.client.0.smithi090.stdout:0/106: unlink d0/f5 0 2024-04-26T04:42:11.169 INFO:tasks.workunit.client.0.smithi090.stdout:6/69: link d9/f12 d9/f19 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:2/298: mkdir d6/d1b/d51/d6a 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:7/49: link d2/le d2/d4/lf 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:7/50: readlink d2/l5 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:7/51: creat d2/d4/f10 x:0 0 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:7/52: creat d2/d4/f11 x:0 0 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:7/53: dread - d2/d4/f10 zero size 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:7/54: write d2/d4/f10 [214397,20822] 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:0/157: rename d0/d4/lc to d0/d4/l35 0 2024-04-26T04:42:11.171 INFO:tasks.workunit.client.1.smithi104.stdout:0/158: chown d0/d2/d21/d27/c2a 91323 1 2024-04-26T04:42:11.172 INFO:tasks.workunit.client.0.smithi090.stdout:0/107: mkdir d0/d23 0 2024-04-26T04:42:11.172 INFO:tasks.workunit.client.0.smithi090.stdout:6/70: mknod d9/c1a 0 2024-04-26T04:42:11.172 INFO:tasks.workunit.client.0.smithi090.stdout:6/71: read d9/fc [598643,56065] 0 2024-04-26T04:42:11.172 INFO:tasks.workunit.client.0.smithi090.stdout:0/108: stat d0/de/c11 0 2024-04-26T04:42:11.172 INFO:tasks.workunit.client.0.smithi090.stdout:0/109: creat d0/df/d16/f24 x:0 0 0 2024-04-26T04:42:11.173 INFO:tasks.workunit.client.1.smithi104.stdout:2/299: unlink d6/d8/f56 0 2024-04-26T04:42:11.174 INFO:tasks.workunit.client.0.smithi090.stdout:0/110: creat d0/df/f25 x:0 0 0 2024-04-26T04:42:11.177 INFO:tasks.workunit.client.1.smithi104.stdout:7/55: dread d2/f3 [0,4194304] 0 2024-04-26T04:42:11.178 INFO:tasks.workunit.client.1.smithi104.stdout:7/56: chown c1 64664258 1 2024-04-26T04:42:11.184 INFO:tasks.workunit.client.1.smithi104.stdout:8/113: dwrite d0/d12/f1f [0,4194304] 0 2024-04-26T04:42:11.184 INFO:tasks.workunit.client.1.smithi104.stdout:8/114: readlink d0/ld 0 2024-04-26T04:42:11.187 INFO:tasks.workunit.client.0.smithi090.stdout:6/72: dread d9/f12 [0,4194304] 0 2024-04-26T04:42:11.188 INFO:tasks.workunit.client.0.smithi090.stdout:0/111: rename d0/df/d16 to d0/d26 0 2024-04-26T04:42:11.188 INFO:tasks.workunit.client.0.smithi090.stdout:0/112: write d0/d9/f17 [563136,39220] 0 2024-04-26T04:42:11.188 INFO:tasks.workunit.client.1.smithi104.stdout:8/115: rename d0/d12/l21 to d0/d12/l23 0 2024-04-26T04:42:11.189 INFO:tasks.workunit.client.0.smithi090.stdout:6/73: mknod d9/c1b 0 2024-04-26T04:42:11.189 INFO:tasks.workunit.client.0.smithi090.stdout:0/113: creat d0/d26/f27 x:0 0 0 2024-04-26T04:42:11.191 INFO:tasks.workunit.client.1.smithi104.stdout:7/57: rmdir d2/d4 39 2024-04-26T04:42:11.192 INFO:tasks.workunit.client.1.smithi104.stdout:7/58: unlink d2/d4/fd 0 2024-04-26T04:42:11.192 INFO:tasks.workunit.client.1.smithi104.stdout:8/116: dread d0/fe [0,4194304] 0 2024-04-26T04:42:11.193 INFO:tasks.workunit.client.1.smithi104.stdout:7/59: link c1 d2/d4/c12 0 2024-04-26T04:42:11.193 INFO:tasks.workunit.client.1.smithi104.stdout:7/60: fdatasync f0 0 2024-04-26T04:42:11.193 INFO:tasks.workunit.client.1.smithi104.stdout:0/159: dwrite d0/d2/d28/f34 [0,4194304] 0 2024-04-26T04:42:11.194 INFO:tasks.workunit.client.1.smithi104.stdout:0/160: write d0/d2/d21/d27/f30 [394998,40245] 0 2024-04-26T04:42:11.194 INFO:tasks.workunit.client.0.smithi090.stdout:0/114: rename d0/d9/l20 to d0/df/l28 0 2024-04-26T04:42:11.195 INFO:tasks.workunit.client.1.smithi104.stdout:2/300: dwrite d6/f35 [0,4194304] 0 2024-04-26T04:42:11.195 INFO:tasks.workunit.client.1.smithi104.stdout:2/301: write d6/d8/d3e/f49 [26090,90188] 0 2024-04-26T04:42:11.195 INFO:tasks.workunit.client.1.smithi104.stdout:8/117: creat d0/d1c/f24 x:0 0 0 2024-04-26T04:42:11.195 INFO:tasks.workunit.client.1.smithi104.stdout:7/61: rmdir d2 39 2024-04-26T04:42:11.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/115: rmdir d0/d23 0 2024-04-26T04:42:11.196 INFO:tasks.workunit.client.0.smithi090.stdout:0/116: truncate d0/d26/f24 938614 0 2024-04-26T04:42:11.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/74: getdents d9 0 2024-04-26T04:42:11.199 INFO:tasks.workunit.client.0.smithi090.stdout:0/117: mknod d0/df/c29 0 2024-04-26T04:42:11.200 INFO:tasks.workunit.client.0.smithi090.stdout:6/75: mkdir d9/d1c 0 2024-04-26T04:42:11.202 INFO:tasks.workunit.client.1.smithi104.stdout:7/62: unlink d2/d4/c6 0 2024-04-26T04:42:11.202 INFO:tasks.workunit.client.1.smithi104.stdout:7/63: fsync d2/f3 0 2024-04-26T04:42:11.204 INFO:tasks.workunit.client.1.smithi104.stdout:7/64: creat d2/d4/f13 x:0 0 0 2024-04-26T04:42:11.204 INFO:tasks.workunit.client.1.smithi104.stdout:7/65: readlink d2/l5 0 2024-04-26T04:42:11.204 INFO:tasks.workunit.client.1.smithi104.stdout:7/66: dread - d2/d4/f11 zero size 2024-04-26T04:42:11.205 INFO:tasks.workunit.client.1.smithi104.stdout:0/161: write d0/fa [3988385,13480] 0 2024-04-26T04:42:11.206 INFO:tasks.workunit.client.1.smithi104.stdout:7/67: link d2/d4/f11 d2/d4/f14 0 2024-04-26T04:42:11.208 INFO:tasks.workunit.client.0.smithi090.stdout:0/118: unlink d0/df/l28 0 2024-04-26T04:42:11.210 INFO:tasks.workunit.client.0.smithi090.stdout:6/76: write d9/f17 [1900850,102476] 0 2024-04-26T04:42:11.213 INFO:tasks.workunit.client.1.smithi104.stdout:0/162: write d0/f2f [2382977,44982] 0 2024-04-26T04:42:11.214 INFO:tasks.workunit.client.1.smithi104.stdout:7/68: dread d2/f3 [0,4194304] 0 2024-04-26T04:42:11.214 INFO:tasks.workunit.client.1.smithi104.stdout:7/69: chown d2/d4/lf 206301 1 2024-04-26T04:42:11.214 INFO:tasks.workunit.client.1.smithi104.stdout:7/70: creat d2/d4/f15 x:0 0 0 2024-04-26T04:42:11.214 INFO:tasks.workunit.client.1.smithi104.stdout:0/163: dread d0/d4/f29 [0,4194304] 0 2024-04-26T04:42:11.217 INFO:tasks.workunit.client.1.smithi104.stdout:7/71: rename d2/d4/f14 to d2/f16 0 2024-04-26T04:42:11.218 INFO:tasks.workunit.client.1.smithi104.stdout:2/302: dwrite d6/fd [0,4194304] 0 2024-04-26T04:42:11.218 INFO:tasks.workunit.client.1.smithi104.stdout:0/164: stat d0/d4/cf 0 2024-04-26T04:42:11.220 INFO:tasks.workunit.client.1.smithi104.stdout:7/72: symlink d2/d4/l17 0 2024-04-26T04:42:11.220 INFO:tasks.workunit.client.1.smithi104.stdout:7/73: chown d2/d4/f11 703551 1 2024-04-26T04:42:11.220 INFO:tasks.workunit.client.1.smithi104.stdout:7/74: truncate d2/f16 809156 0 2024-04-26T04:42:11.223 INFO:tasks.workunit.client.1.smithi104.stdout:2/303: rename d6/c34 to d6/d1b/d68/c6b 0 2024-04-26T04:42:11.229 INFO:tasks.workunit.client.1.smithi104.stdout:7/75: fsync d2/d4/f13 0 2024-04-26T04:42:11.229 INFO:tasks.workunit.client.1.smithi104.stdout:7/76: chown d2/cb 19 1 2024-04-26T04:42:11.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/77: dwrite d9/f17 [4194304,4194304] 0 2024-04-26T04:42:11.230 INFO:tasks.workunit.client.1.smithi104.stdout:7/77: unlink d2/cb 0 2024-04-26T04:42:11.231 INFO:tasks.workunit.client.1.smithi104.stdout:7/78: mknod d2/d4/c18 0 2024-04-26T04:42:11.231 INFO:tasks.workunit.client.1.smithi104.stdout:7/79: stat d2 0 2024-04-26T04:42:11.232 INFO:tasks.workunit.client.1.smithi104.stdout:7/80: creat d2/f19 x:0 0 0 2024-04-26T04:42:11.236 INFO:tasks.workunit.client.1.smithi104.stdout:0/165: truncate d0/fa 3264801 0 2024-04-26T04:42:11.236 INFO:tasks.workunit.client.1.smithi104.stdout:0/166: fdatasync d0/f17 0 2024-04-26T04:42:11.236 INFO:tasks.workunit.client.1.smithi104.stdout:0/167: chown d0/d2/d21/d27/c1f 38125 1 2024-04-26T04:42:11.236 INFO:tasks.workunit.client.1.smithi104.stdout:0/168: creat d0/d2/d28/f36 x:0 0 0 2024-04-26T04:42:11.237 INFO:tasks.workunit.client.1.smithi104.stdout:7/81: getdents d2/d4 0 2024-04-26T04:42:11.237 INFO:tasks.workunit.client.1.smithi104.stdout:7/82: write d2/f16 [631956,2689] 0 2024-04-26T04:42:11.239 INFO:tasks.workunit.client.1.smithi104.stdout:7/83: mknod d2/c1a 0 2024-04-26T04:42:11.240 INFO:tasks.workunit.client.1.smithi104.stdout:9/143: sync 2024-04-26T04:42:11.240 INFO:tasks.workunit.client.1.smithi104.stdout:4/138: sync 2024-04-26T04:42:11.240 INFO:tasks.workunit.client.1.smithi104.stdout:5/211: sync 2024-04-26T04:42:11.240 INFO:tasks.workunit.client.1.smithi104.stdout:5/212: chown d0/d1/d1e 17866637 1 2024-04-26T04:42:11.242 INFO:tasks.workunit.client.1.smithi104.stdout:4/139: mkdir d4/db/d30 0 2024-04-26T04:42:11.243 INFO:tasks.workunit.client.1.smithi104.stdout:4/140: creat d4/db/d30/f31 x:0 0 0 2024-04-26T04:42:11.243 INFO:tasks.workunit.client.1.smithi104.stdout:4/141: write d4/f11 [55423,107149] 0 2024-04-26T04:42:11.244 INFO:tasks.workunit.client.1.smithi104.stdout:0/169: write d0/d2/fd [3412535,110592] 0 2024-04-26T04:42:11.246 INFO:tasks.workunit.client.1.smithi104.stdout:0/170: symlink d0/d2/d10/l37 0 2024-04-26T04:42:11.246 INFO:tasks.workunit.client.1.smithi104.stdout:0/171: truncate d0/d2/d10/f32 125611 0 2024-04-26T04:42:11.248 INFO:tasks.workunit.client.1.smithi104.stdout:5/213: read d0/f5 [888243,98370] 0 2024-04-26T04:42:11.248 INFO:tasks.workunit.client.1.smithi104.stdout:5/214: chown d0/f22 1335220 1 2024-04-26T04:42:11.251 INFO:tasks.workunit.client.1.smithi104.stdout:5/215: unlink d0/f20 0 2024-04-26T04:42:11.251 INFO:tasks.workunit.client.1.smithi104.stdout:5/216: getdents d0/d1/d3e 0 2024-04-26T04:42:11.251 INFO:tasks.workunit.client.1.smithi104.stdout:5/217: write d0/d11/f31 [1512497,10025] 0 2024-04-26T04:42:11.252 INFO:tasks.workunit.client.1.smithi104.stdout:4/142: dread f3 [0,4194304] 0 2024-04-26T04:42:11.252 INFO:tasks.workunit.client.1.smithi104.stdout:4/143: read d4/f11 [1055704,130965] 0 2024-04-26T04:42:11.252 INFO:tasks.workunit.client.1.smithi104.stdout:4/144: fsync d4/d7/d1d/d1e/d15/f1a 0 2024-04-26T04:42:11.253 INFO:tasks.workunit.client.0.smithi090.stdout:6/78: creat d9/f1d x:0 0 0 2024-04-26T04:42:11.253 INFO:tasks.workunit.client.1.smithi104.stdout:0/172: rename d0/d2/d28 to d0/d2/d21/d38 0 2024-04-26T04:42:11.254 INFO:tasks.workunit.client.0.smithi090.stdout:0/119: link d0/de/l18 d0/df/l2a 0 2024-04-26T04:42:11.254 INFO:tasks.workunit.client.0.smithi090.stdout:6/79: rename d9 to d9/d1e 22 2024-04-26T04:42:11.256 INFO:tasks.workunit.client.1.smithi104.stdout:4/145: symlink d4/d20/l32 0 2024-04-26T04:42:11.256 INFO:tasks.workunit.client.1.smithi104.stdout:4/146: chown f3 2 1 2024-04-26T04:42:11.258 INFO:tasks.workunit.client.1.smithi104.stdout:9/144: write d11/d15/f18 [6211165,80484] 0 2024-04-26T04:42:11.259 INFO:tasks.workunit.client.1.smithi104.stdout:0/173: symlink d0/d2/d10/l39 0 2024-04-26T04:42:11.259 INFO:tasks.workunit.client.1.smithi104.stdout:5/218: rmdir d0/d1/d25 39 2024-04-26T04:42:11.259 INFO:tasks.workunit.client.1.smithi104.stdout:4/147: rename d4/d7/d1d/d1e/d15/c1b to d4/d7/d1d/d1e/d15/c33 0 2024-04-26T04:42:11.260 INFO:tasks.workunit.client.1.smithi104.stdout:0/174: symlink d0/d2/d21/d27/l3a 0 2024-04-26T04:42:11.267 INFO:tasks.workunit.client.0.smithi090.stdout:6/80: dread d9/f17 [4194304,4194304] 0 2024-04-26T04:42:11.267 INFO:tasks.workunit.client.1.smithi104.stdout:5/219: write d0/d1/d4/f12 [3921450,26710] 0 2024-04-26T04:42:11.268 INFO:tasks.workunit.client.1.smithi104.stdout:7/84: dwrite d2/f16 [0,4194304] 0 2024-04-26T04:42:11.268 INFO:tasks.workunit.client.1.smithi104.stdout:7/85: chown d2/f3 0 1 2024-04-26T04:42:11.268 INFO:tasks.workunit.client.1.smithi104.stdout:0/175: unlink d0/d2/l24 0 2024-04-26T04:42:11.271 INFO:tasks.workunit.client.1.smithi104.stdout:5/220: mkdir d0/d41 0 2024-04-26T04:42:11.271 INFO:tasks.workunit.client.1.smithi104.stdout:5/221: stat d0/d1/f3d 0 2024-04-26T04:42:11.271 INFO:tasks.workunit.client.1.smithi104.stdout:5/222: truncate d0/d1/f38 750743 0 2024-04-26T04:42:11.271 INFO:tasks.workunit.client.1.smithi104.stdout:7/86: symlink d2/d4/l1b 0 2024-04-26T04:42:11.272 INFO:tasks.workunit.client.1.smithi104.stdout:7/87: creat d2/f1c x:0 0 0 2024-04-26T04:42:11.273 INFO:tasks.workunit.client.1.smithi104.stdout:4/148: write d4/d7/d1d/f1c [1434143,38672] 0 2024-04-26T04:42:11.274 INFO:tasks.workunit.client.1.smithi104.stdout:5/223: getdents d0/d1/d1e 0 2024-04-26T04:42:11.274 INFO:tasks.workunit.client.1.smithi104.stdout:5/224: creat d0/d1/d4/d2a/d2b/f42 x:0 0 0 2024-04-26T04:42:11.275 INFO:tasks.workunit.client.1.smithi104.stdout:0/176: mkdir d0/d4/d3b 0 2024-04-26T04:42:11.275 INFO:tasks.workunit.client.1.smithi104.stdout:0/177: read d0/fa [2517873,29195] 0 2024-04-26T04:42:11.276 INFO:tasks.workunit.client.1.smithi104.stdout:0/178: chown d0/d2/d21/d27/d1e/f2d 25157990 1 2024-04-26T04:42:11.276 INFO:tasks.workunit.client.1.smithi104.stdout:0/179: write d0/fe [5820028,105834] 0 2024-04-26T04:42:11.276 INFO:tasks.workunit.client.1.smithi104.stdout:0/180: write d0/d2/d21/d38/f36 [139266,112891] 0 2024-04-26T04:42:11.276 INFO:tasks.workunit.client.1.smithi104.stdout:0/181: getdents d0/d4/d3b 0 2024-04-26T04:42:11.277 INFO:tasks.workunit.client.1.smithi104.stdout:4/149: symlink d4/db/d30/l34 0 2024-04-26T04:42:11.277 INFO:tasks.workunit.client.1.smithi104.stdout:9/145: dwrite d11/f17 [0,4194304] 0 2024-04-26T04:42:11.281 INFO:tasks.workunit.client.1.smithi104.stdout:0/182: unlink d0/l1a 0 2024-04-26T04:42:11.283 INFO:tasks.workunit.client.1.smithi104.stdout:4/150: mknod d4/c35 0 2024-04-26T04:42:11.284 INFO:tasks.workunit.client.1.smithi104.stdout:7/88: write d2/d4/f11 [1516215,101046] 0 2024-04-26T04:42:11.284 INFO:tasks.workunit.client.1.smithi104.stdout:7/89: write d2/d4/f15 [417313,86412] 0 2024-04-26T04:42:11.286 INFO:tasks.workunit.client.1.smithi104.stdout:4/151: creat d4/d20/f36 x:0 0 0 2024-04-26T04:42:11.287 INFO:tasks.workunit.client.1.smithi104.stdout:5/225: dwrite d0/d11/f23 [0,4194304] 0 2024-04-26T04:42:11.288 INFO:tasks.workunit.client.1.smithi104.stdout:4/152: truncate d4/db/f12 84484 0 2024-04-26T04:42:11.293 INFO:tasks.workunit.client.1.smithi104.stdout:4/153: unlink d4/f17 0 2024-04-26T04:42:11.294 INFO:tasks.workunit.client.1.smithi104.stdout:4/154: creat d4/f37 x:0 0 0 2024-04-26T04:42:11.294 INFO:tasks.workunit.client.1.smithi104.stdout:4/155: truncate d4/d20/f36 838012 0 2024-04-26T04:42:11.294 INFO:tasks.workunit.client.1.smithi104.stdout:4/156: readlink d4/db/d30/l34 0 2024-04-26T04:42:11.294 INFO:tasks.workunit.client.1.smithi104.stdout:4/157: dread - d4/db/f28 zero size 2024-04-26T04:42:11.294 INFO:tasks.workunit.client.1.smithi104.stdout:7/90: fsync d2/f1c 0 2024-04-26T04:42:11.295 INFO:tasks.workunit.client.1.smithi104.stdout:0/183: getdents d0/d4 0 2024-04-26T04:42:11.296 INFO:tasks.workunit.client.1.smithi104.stdout:0/184: chown d0/d2/d21/c33 10635258 1 2024-04-26T04:42:11.296 INFO:tasks.workunit.client.1.smithi104.stdout:0/185: stat d0/d2/d21 0 2024-04-26T04:42:11.296 INFO:tasks.workunit.client.1.smithi104.stdout:0/186: write d0/d2/d21/d27/d1e/f2d [985723,90139] 0 2024-04-26T04:42:11.296 INFO:tasks.workunit.client.1.smithi104.stdout:0/187: chown d0/d4/d15/c16 802505088 1 2024-04-26T04:42:11.298 INFO:tasks.workunit.client.0.smithi090.stdout:6/81: write d9/fc [91446,93324] 0 2024-04-26T04:42:11.300 INFO:tasks.workunit.client.0.smithi090.stdout:6/82: symlink d9/d1c/l1f 0 2024-04-26T04:42:11.300 INFO:tasks.workunit.client.1.smithi104.stdout:4/158: mknod d4/d7/d1d/d1e/d15/c38 0 2024-04-26T04:42:11.301 INFO:tasks.workunit.client.1.smithi104.stdout:4/159: chown d4/d7/d1d/d2e 11 1 2024-04-26T04:42:11.302 INFO:tasks.workunit.client.1.smithi104.stdout:7/91: dread d2/f3 [0,4194304] 0 2024-04-26T04:42:11.303 INFO:tasks.workunit.client.0.smithi090.stdout:6/83: mknod d9/c20 0 2024-04-26T04:42:11.303 INFO:tasks.workunit.client.0.smithi090.stdout:6/84: dread - d9/f1d zero size 2024-04-26T04:42:11.306 INFO:tasks.workunit.client.0.smithi090.stdout:6/85: mkdir d9/d21 0 2024-04-26T04:42:11.308 INFO:tasks.workunit.client.1.smithi104.stdout:4/160: rmdir d4/db/d30 39 2024-04-26T04:42:11.309 INFO:tasks.workunit.client.0.smithi090.stdout:6/86: mkdir d9/d22 0 2024-04-26T04:42:11.310 INFO:tasks.workunit.client.0.smithi090.stdout:6/87: creat d9/f23 x:0 0 0 2024-04-26T04:42:11.312 INFO:tasks.workunit.client.0.smithi090.stdout:6/88: fdatasync d9/f19 0 2024-04-26T04:42:11.313 INFO:tasks.workunit.client.1.smithi104.stdout:7/92: getdents d2 0 2024-04-26T04:42:11.313 INFO:tasks.workunit.client.1.smithi104.stdout:9/146: dwrite fc [4194304,4194304] 0 2024-04-26T04:42:11.314 INFO:tasks.workunit.client.0.smithi090.stdout:6/89: creat d9/f24 x:0 0 0 2024-04-26T04:42:11.314 INFO:tasks.workunit.client.0.smithi090.stdout:6/90: stat d9/f17 0 2024-04-26T04:42:11.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/91: truncate d9/f11 476655 0 2024-04-26T04:42:11.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/92: truncate d9/f11 897523 0 2024-04-26T04:42:11.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/93: write d9/f1d [548633,66890] 0 2024-04-26T04:42:11.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/94: truncate d9/f19 5077893 0 2024-04-26T04:42:11.316 INFO:tasks.workunit.client.1.smithi104.stdout:9/147: truncate d11/d15/f1f 26568 0 2024-04-26T04:42:11.317 INFO:tasks.workunit.client.0.smithi090.stdout:6/95: truncate d9/fd 2477047 0 2024-04-26T04:42:11.317 INFO:tasks.workunit.client.0.smithi090.stdout:6/96: write f1 [5028356,70035] 0 2024-04-26T04:42:11.319 INFO:tasks.workunit.client.0.smithi090.stdout:6/97: rename d9/f19 to d9/d22/f25 0 2024-04-26T04:42:11.320 INFO:tasks.workunit.client.0.smithi090.stdout:6/98: creat d9/d22/f26 x:0 0 0 2024-04-26T04:42:11.321 INFO:tasks.workunit.client.0.smithi090.stdout:6/99: symlink d9/d22/l27 0 2024-04-26T04:42:11.323 INFO:tasks.workunit.client.1.smithi104.stdout:0/188: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:11.323 INFO:tasks.workunit.client.1.smithi104.stdout:3/150: sync 2024-04-26T04:42:11.324 INFO:tasks.workunit.client.1.smithi104.stdout:6/307: sync 2024-04-26T04:42:11.324 INFO:tasks.workunit.client.1.smithi104.stdout:6/308: readlink d0/d5/l5b 0 2024-04-26T04:42:11.324 INFO:tasks.workunit.client.1.smithi104.stdout:6/309: truncate d0/d5/f67 415727 0 2024-04-26T04:42:11.324 INFO:tasks.workunit.client.1.smithi104.stdout:6/310: creat d0/d9/d52/f6c x:0 0 0 2024-04-26T04:42:11.325 INFO:tasks.workunit.client.1.smithi104.stdout:4/161: dwrite d4/d20/f36 [0,4194304] 0 2024-04-26T04:42:11.327 INFO:tasks.workunit.client.1.smithi104.stdout:0/189: dread d0/d2/d21/d38/f36 [0,4194304] 0 2024-04-26T04:42:11.328 INFO:tasks.workunit.client.1.smithi104.stdout:3/151: link d5/d9/dc/f21 d5/d9/dc/d29/f31 0 2024-04-26T04:42:11.328 INFO:tasks.workunit.client.1.smithi104.stdout:3/152: write d5/d9/d17/f1b [708603,25367] 0 2024-04-26T04:42:11.328 INFO:tasks.workunit.client.1.smithi104.stdout:3/153: write d5/d9/dc/f16 [541565,130229] 0 2024-04-26T04:42:11.328 INFO:tasks.workunit.client.1.smithi104.stdout:3/154: chown d5/d9/d17/f1b 246 1 2024-04-26T04:42:11.328 INFO:tasks.workunit.client.1.smithi104.stdout:3/155: chown d5/d9/d17/l27 1 1 2024-04-26T04:42:11.328 INFO:tasks.workunit.client.1.smithi104.stdout:3/156: read - d5/d9/f26 zero size 2024-04-26T04:42:11.329 INFO:tasks.workunit.client.1.smithi104.stdout:3/157: write d5/d9/f14 [276343,67225] 0 2024-04-26T04:42:11.329 INFO:tasks.workunit.client.1.smithi104.stdout:3/158: truncate d5/d9/f12 9310792 0 2024-04-26T04:42:11.332 INFO:tasks.workunit.client.1.smithi104.stdout:6/311: symlink d0/d9/d13/d44/d47/l6d 0 2024-04-26T04:42:11.332 INFO:tasks.workunit.client.1.smithi104.stdout:6/312: write d0/d9/d13/d15/f2b [4507105,25728] 0 2024-04-26T04:42:11.332 INFO:tasks.workunit.client.1.smithi104.stdout:6/313: write d0/d9/d13/d15/f60 [820396,64501] 0 2024-04-26T04:42:11.334 INFO:tasks.workunit.client.1.smithi104.stdout:6/314: read d0/d9/d13/d44/f58 [429178,57532] 0 2024-04-26T04:42:11.335 INFO:tasks.workunit.client.1.smithi104.stdout:4/162: creat d4/d7/d1d/d1e/f39 x:0 0 0 2024-04-26T04:42:11.335 INFO:tasks.workunit.client.1.smithi104.stdout:0/190: mkdir d0/d4/d3c 0 2024-04-26T04:42:11.337 INFO:tasks.workunit.client.1.smithi104.stdout:9/148: unlink d11/d15/f1f 0 2024-04-26T04:42:11.340 INFO:tasks.workunit.client.1.smithi104.stdout:6/315: truncate d0/f29 834974 0 2024-04-26T04:42:11.340 INFO:tasks.workunit.client.1.smithi104.stdout:0/191: mkdir d0/d2/d10/d3d 0 2024-04-26T04:42:11.342 INFO:tasks.workunit.client.1.smithi104.stdout:3/159: write d5/d9/f12 [1962477,48375] 0 2024-04-26T04:42:11.342 INFO:tasks.workunit.client.1.smithi104.stdout:3/160: readlink d5/d9/dc/l11 0 2024-04-26T04:42:11.342 INFO:tasks.workunit.client.1.smithi104.stdout:3/161: write f3 [4119206,103704] 0 2024-04-26T04:42:11.343 INFO:tasks.workunit.client.1.smithi104.stdout:7/93: dwrite d2/f16 [0,4194304] 0 2024-04-26T04:42:11.343 INFO:tasks.workunit.client.1.smithi104.stdout:7/94: stat d2/d4 0 2024-04-26T04:42:11.347 INFO:tasks.workunit.client.0.smithi090.stdout:6/100: dwrite d9/f15 [0,4194304] 0 2024-04-26T04:42:11.359 INFO:tasks.workunit.client.1.smithi104.stdout:3/162: getdents d5/d9/d17 0 2024-04-26T04:42:11.359 INFO:tasks.workunit.client.1.smithi104.stdout:3/163: fdatasync d5/d9/dc/f15 0 2024-04-26T04:42:11.359 INFO:tasks.workunit.client.1.smithi104.stdout:4/163: dwrite d4/f37 [0,4194304] 0 2024-04-26T04:42:11.359 INFO:tasks.workunit.client.1.smithi104.stdout:4/164: chown d4/d7 1 1 2024-04-26T04:42:11.360 INFO:tasks.workunit.client.1.smithi104.stdout:3/164: unlink d5/d9/dc/l13 0 2024-04-26T04:42:11.363 INFO:tasks.workunit.client.1.smithi104.stdout:6/316: rename d0/f29 to d0/f6e 0 2024-04-26T04:42:11.364 INFO:tasks.workunit.client.1.smithi104.stdout:3/165: unlink d5/d9/dc/d29/f31 0 2024-04-26T04:42:11.364 INFO:tasks.workunit.client.1.smithi104.stdout:3/166: chown d5 20 1 2024-04-26T04:42:11.365 INFO:tasks.workunit.client.1.smithi104.stdout:4/165: link d4/d7/d1d/d1e/d24/l29 d4/d7/l3a 0 2024-04-26T04:42:11.366 INFO:tasks.workunit.client.1.smithi104.stdout:4/166: truncate d4/db/f28 438245 0 2024-04-26T04:42:11.366 INFO:tasks.workunit.client.1.smithi104.stdout:4/167: chown f3 58147 1 2024-04-26T04:42:11.366 INFO:tasks.workunit.client.1.smithi104.stdout:3/167: rmdir d5/d9/dc/d29/d2d 39 2024-04-26T04:42:11.366 INFO:tasks.workunit.client.1.smithi104.stdout:4/168: mknod d4/d7/d1d/d1e/c3b 0 2024-04-26T04:42:11.367 INFO:tasks.workunit.client.1.smithi104.stdout:3/168: mkdir d5/d9/dc/d32 0 2024-04-26T04:42:11.369 INFO:tasks.workunit.client.1.smithi104.stdout:4/169: rename d4/db/d30 to d4/d20/d3c 0 2024-04-26T04:42:11.370 INFO:tasks.workunit.client.1.smithi104.stdout:3/169: symlink d5/d9/dc/d29/d2d/l33 0 2024-04-26T04:42:11.373 INFO:tasks.workunit.client.1.smithi104.stdout:7/95: read d2/d4/f11 [2605142,98484] 0 2024-04-26T04:42:11.375 INFO:tasks.workunit.client.1.smithi104.stdout:7/96: unlink f0 0 2024-04-26T04:42:11.375 INFO:tasks.workunit.client.1.smithi104.stdout:7/97: mkdir d2/d4/d1d 0 2024-04-26T04:42:11.375 INFO:tasks.workunit.client.1.smithi104.stdout:7/98: fdatasync d2/f1c 0 2024-04-26T04:42:11.376 INFO:tasks.workunit.client.1.smithi104.stdout:7/99: rmdir d2 39 2024-04-26T04:42:11.381 INFO:tasks.workunit.client.0.smithi090.stdout:6/101: dwrite d9/d22/f26 [0,4194304] 0 2024-04-26T04:42:11.381 INFO:tasks.workunit.client.0.smithi090.stdout:6/102: write d9/f11 [1344165,128691] 0 2024-04-26T04:42:11.381 INFO:tasks.workunit.client.1.smithi104.stdout:9/149: dwrite d11/f1a [0,4194304] 0 2024-04-26T04:42:11.381 INFO:tasks.workunit.client.1.smithi104.stdout:0/192: dwrite d0/d2/d21/d27/d1e/f2c [0,4194304] 0 2024-04-26T04:42:11.381 INFO:tasks.workunit.client.1.smithi104.stdout:0/193: write d0/d4/f7 [1839579,10882] 0 2024-04-26T04:42:11.381 INFO:tasks.workunit.client.1.smithi104.stdout:0/194: write d0/d4/f11 [446900,77370] 0 2024-04-26T04:42:11.387 INFO:tasks.workunit.client.1.smithi104.stdout:0/195: getdents d0/d4/d15 0 2024-04-26T04:42:11.389 INFO:tasks.workunit.client.1.smithi104.stdout:0/196: mkdir d0/d4/d15/d3e 0 2024-04-26T04:42:11.389 INFO:tasks.workunit.client.1.smithi104.stdout:6/317: dread d0/d9/d13/d15/f2b [0,4194304] 0 2024-04-26T04:42:11.391 INFO:tasks.workunit.client.1.smithi104.stdout:0/197: rename d0/cb to d0/d2/c3f 0 2024-04-26T04:42:11.392 INFO:tasks.workunit.client.1.smithi104.stdout:0/198: readlink d0/d4/d15/l25 0 2024-04-26T04:42:11.392 INFO:tasks.workunit.client.1.smithi104.stdout:0/199: chown d0/d4/d15/d3e 21 1 2024-04-26T04:42:11.393 INFO:tasks.workunit.client.1.smithi104.stdout:0/200: rename d0/c5 to d0/d2/d21/c40 0 2024-04-26T04:42:11.394 INFO:tasks.workunit.client.1.smithi104.stdout:0/201: stat d0/d4/d15/l20 0 2024-04-26T04:42:11.396 INFO:tasks.workunit.client.1.smithi104.stdout:0/202: creat d0/f41 x:0 0 0 2024-04-26T04:42:11.396 INFO:tasks.workunit.client.1.smithi104.stdout:0/203: chown d0/d2/d21 1359745 1 2024-04-26T04:42:11.399 INFO:tasks.workunit.client.1.smithi104.stdout:7/100: dwrite d2/f19 [0,4194304] 0 2024-04-26T04:42:11.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/103: dwrite f3 [4194304,4194304] 0 2024-04-26T04:42:11.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/104: fsync d9/f15 0 2024-04-26T04:42:11.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/105: creat d9/d22/f28 x:0 0 0 2024-04-26T04:42:11.404 INFO:tasks.workunit.client.1.smithi104.stdout:3/170: rmdir d5/d9 39 2024-04-26T04:42:11.409 INFO:tasks.workunit.client.1.smithi104.stdout:9/150: truncate d11/f1a 6811684 0 2024-04-26T04:42:11.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/106: write d9/f12 [1981150,101027] 0 2024-04-26T04:42:11.410 INFO:tasks.workunit.client.1.smithi104.stdout:3/171: creat d5/d9/d17/f34 x:0 0 0 2024-04-26T04:42:11.411 INFO:tasks.workunit.client.0.smithi090.stdout:6/107: unlink d9/fc 0 2024-04-26T04:42:11.412 INFO:tasks.workunit.client.0.smithi090.stdout:6/108: truncate d9/f14 6025446 0 2024-04-26T04:42:11.413 INFO:tasks.workunit.client.0.smithi090.stdout:6/109: link d9/f12 d9/d22/f29 0 2024-04-26T04:42:11.414 INFO:tasks.workunit.client.0.smithi090.stdout:6/110: mknod d9/d21/c2a 0 2024-04-26T04:42:11.414 INFO:tasks.workunit.client.1.smithi104.stdout:7/101: unlink d2/f19 0 2024-04-26T04:42:11.415 INFO:tasks.workunit.client.0.smithi090.stdout:6/111: creat d9/d1c/f2b x:0 0 0 2024-04-26T04:42:11.415 INFO:tasks.workunit.client.0.smithi090.stdout:6/112: write d9/d1c/f2b [409149,29092] 0 2024-04-26T04:42:11.415 INFO:tasks.workunit.client.1.smithi104.stdout:9/151: mknod d11/d1d/c25 0 2024-04-26T04:42:11.416 INFO:tasks.workunit.client.1.smithi104.stdout:3/172: mknod d5/d9/c35 0 2024-04-26T04:42:11.416 INFO:tasks.workunit.client.1.smithi104.stdout:7/102: chown d2/d4/d1d 3557469 1 2024-04-26T04:42:11.417 INFO:tasks.workunit.client.1.smithi104.stdout:9/152: creat d11/d1d/f26 x:0 0 0 2024-04-26T04:42:11.418 INFO:tasks.workunit.client.1.smithi104.stdout:9/153: creat d11/f27 x:0 0 0 2024-04-26T04:42:11.419 INFO:tasks.workunit.client.1.smithi104.stdout:3/173: creat d5/d9/dc/f36 x:0 0 0 2024-04-26T04:42:11.419 INFO:tasks.workunit.client.1.smithi104.stdout:3/174: write d5/d9/dc/f1c [596946,104097] 0 2024-04-26T04:42:11.419 INFO:tasks.workunit.client.1.smithi104.stdout:3/175: dread - d5/f25 zero size 2024-04-26T04:42:11.419 INFO:tasks.workunit.client.1.smithi104.stdout:6/318: dwrite d0/d9/d13/d15/f42 [0,4194304] 0 2024-04-26T04:42:11.419 INFO:tasks.workunit.client.1.smithi104.stdout:6/319: dread - d0/d9/f49 zero size 2024-04-26T04:42:11.419 INFO:tasks.workunit.client.1.smithi104.stdout:0/204: dwrite d0/f17 [0,4194304] 0 2024-04-26T04:42:11.419 INFO:tasks.workunit.client.1.smithi104.stdout:4/170: dwrite d4/f37 [4194304,4194304] 0 2024-04-26T04:42:11.420 INFO:tasks.workunit.client.1.smithi104.stdout:7/103: creat d2/d4/d1d/f1e x:0 0 0 2024-04-26T04:42:11.424 INFO:tasks.workunit.client.1.smithi104.stdout:3/176: symlink d5/l37 0 2024-04-26T04:42:11.424 INFO:tasks.workunit.client.1.smithi104.stdout:3/177: readlink d5/d9/dc/d29/d2d/l33 0 2024-04-26T04:42:11.425 INFO:tasks.workunit.client.1.smithi104.stdout:9/154: read fb [4682542,94151] 0 2024-04-26T04:42:11.426 INFO:tasks.workunit.client.0.smithi090.stdout:6/113: rmdir d9 39 2024-04-26T04:42:11.427 INFO:tasks.workunit.client.1.smithi104.stdout:4/171: write f3 [1384157,52938] 0 2024-04-26T04:42:11.427 INFO:tasks.workunit.client.1.smithi104.stdout:7/104: dread d2/d4/f10 [0,4194304] 0 2024-04-26T04:42:11.427 INFO:tasks.workunit.client.1.smithi104.stdout:7/105: truncate d2/d4/f10 663497 0 2024-04-26T04:42:11.428 INFO:tasks.workunit.client.1.smithi104.stdout:3/178: mknod d5/d9/d17/c38 0 2024-04-26T04:42:11.429 INFO:tasks.workunit.client.0.smithi090.stdout:6/114: dread d9/fd [0,4194304] 0 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:4/172: write d4/d7/f1f [2217622,55457] 0 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:4/173: chown d4/d7/d1d/d2e 1853 1 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:5/226: sync 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:1/248: sync 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:8/118: sync 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:2/304: sync 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:8/119: creat d0/d13/f25 x:0 0 0 2024-04-26T04:42:11.430 INFO:tasks.workunit.client.1.smithi104.stdout:8/120: creat d0/d1c/f26 x:0 0 0 2024-04-26T04:42:11.431 INFO:tasks.workunit.client.0.smithi090.stdout:6/115: chown d9/ca 204226 1 2024-04-26T04:42:11.432 INFO:tasks.workunit.client.0.smithi090.stdout:6/116: mknod d9/d1c/c2c 0 2024-04-26T04:42:11.432 INFO:tasks.workunit.client.1.smithi104.stdout:0/205: dread d0/d2/d21/d38/f34 [0,4194304] 0 2024-04-26T04:42:11.432 INFO:tasks.workunit.client.1.smithi104.stdout:3/179: creat d5/d9/dc/d32/f39 x:0 0 0 2024-04-26T04:42:11.433 INFO:tasks.workunit.client.1.smithi104.stdout:5/227: dread d0/d1/f38 [0,4194304] 0 2024-04-26T04:42:11.433 INFO:tasks.workunit.client.1.smithi104.stdout:5/228: fdatasync d0/f5 0 2024-04-26T04:42:11.433 INFO:tasks.workunit.client.0.smithi090.stdout:6/117: mknod d9/d21/c2d 0 2024-04-26T04:42:11.433 INFO:tasks.workunit.client.0.smithi090.stdout:6/118: readlink d9/d22/l27 0 2024-04-26T04:42:11.435 INFO:tasks.workunit.client.1.smithi104.stdout:4/174: dread - d4/d7/f2a zero size 2024-04-26T04:42:11.435 INFO:tasks.workunit.client.1.smithi104.stdout:4/175: fsync d4/f37 0 2024-04-26T04:42:11.436 INFO:tasks.workunit.client.0.smithi090.stdout:6/119: symlink d9/d22/l2e 0 2024-04-26T04:42:11.439 INFO:tasks.workunit.client.1.smithi104.stdout:1/249: rename d0/d6/d19/d21/d28/d4b to d0/d6/d2e/d32/d4d 0 2024-04-26T04:42:11.440 INFO:tasks.workunit.client.1.smithi104.stdout:2/305: symlink d6/d8/d5a/d29/l6c 0 2024-04-26T04:42:11.440 INFO:tasks.workunit.client.1.smithi104.stdout:2/306: fdatasync d6/d8/f37 0 2024-04-26T04:42:11.443 INFO:tasks.workunit.client.1.smithi104.stdout:9/155: dwrite d11/f16 [0,4194304] 0 2024-04-26T04:42:11.445 INFO:tasks.workunit.client.1.smithi104.stdout:7/106: mknod d2/c1f 0 2024-04-26T04:42:11.447 INFO:tasks.workunit.client.1.smithi104.stdout:3/180: mkdir d5/d9/dc/d3a 0 2024-04-26T04:42:11.448 INFO:tasks.workunit.client.1.smithi104.stdout:5/229: getdents d0/d1/d25 0 2024-04-26T04:42:11.448 INFO:tasks.workunit.client.1.smithi104.stdout:5/230: creat d0/d11/d26/d36/f43 x:0 0 0 2024-04-26T04:42:11.448 INFO:tasks.workunit.client.1.smithi104.stdout:5/231: truncate d0/d1/f3d 488007 0 2024-04-26T04:42:11.452 INFO:tasks.workunit.client.1.smithi104.stdout:4/176: rename d4/d7/d1d/d1e/d15/c38 to d4/d20/c3d 0 2024-04-26T04:42:11.452 INFO:tasks.workunit.client.1.smithi104.stdout:4/177: dread - d4/d7/f2a zero size 2024-04-26T04:42:11.452 INFO:tasks.workunit.client.1.smithi104.stdout:4/178: readlink d4/d7/l3a 0 2024-04-26T04:42:11.452 INFO:tasks.workunit.client.1.smithi104.stdout:4/179: write d4/d7/d1d/d1e/f14 [426379,56996] 0 2024-04-26T04:42:11.452 INFO:tasks.workunit.client.1.smithi104.stdout:4/180: dread - d4/d7/d1d/d1e/d15/f22 zero size 2024-04-26T04:42:11.453 INFO:tasks.workunit.client.0.smithi090.stdout:6/120: dread f1 [0,4194304] 0 2024-04-26T04:42:11.454 INFO:tasks.workunit.client.1.smithi104.stdout:1/250: link d0/d6/d19/d1b/l26 d0/d6/d2e/d32/d4c/l4e 0 2024-04-26T04:42:11.455 INFO:tasks.workunit.client.0.smithi090.stdout:6/121: mknod d9/d22/c2f 0 2024-04-26T04:42:11.459 INFO:tasks.workunit.client.1.smithi104.stdout:5/232: creat d0/d11/d26/f44 x:0 0 0 2024-04-26T04:42:11.460 INFO:tasks.workunit.client.1.smithi104.stdout:9/156: write d11/f16 [2762171,44731] 0 2024-04-26T04:42:11.461 INFO:tasks.workunit.client.1.smithi104.stdout:3/181: rename d5/d9/dc/d32/f39 to d5/d9/d17/f3b 0 2024-04-26T04:42:11.461 INFO:tasks.workunit.client.1.smithi104.stdout:3/182: fsync d5/d9/d17/f34 0 2024-04-26T04:42:11.462 INFO:tasks.workunit.client.1.smithi104.stdout:8/121: dwrite d0/fb [4194304,4194304] 0 2024-04-26T04:42:11.462 INFO:tasks.workunit.client.1.smithi104.stdout:8/122: dread - d0/d13/f25 zero size 2024-04-26T04:42:11.462 INFO:tasks.workunit.client.1.smithi104.stdout:4/181: creat d4/d20/d3c/f3e x:0 0 0 2024-04-26T04:42:11.463 INFO:tasks.workunit.client.1.smithi104.stdout:2/307: dwrite d6/de/f36 [0,4194304] 0 2024-04-26T04:42:11.464 INFO:tasks.workunit.client.1.smithi104.stdout:5/233: mkdir d0/d11/d26/d36/d45 0 2024-04-26T04:42:11.466 INFO:tasks.workunit.client.1.smithi104.stdout:1/251: rename d0/d6/de/d20/d2a/l42 to d0/d6/d2e/d32/l4f 0 2024-04-26T04:42:11.468 INFO:tasks.workunit.client.1.smithi104.stdout:4/182: mknod d4/d7/d1d/d1e/c3f 0 2024-04-26T04:42:11.470 INFO:tasks.workunit.client.1.smithi104.stdout:7/107: link d2/f16 d2/f20 0 2024-04-26T04:42:11.476 INFO:tasks.workunit.client.1.smithi104.stdout:8/123: write d0/f1 [1157255,28980] 0 2024-04-26T04:42:11.476 INFO:tasks.workunit.client.1.smithi104.stdout:8/124: dread - d0/f1e zero size 2024-04-26T04:42:11.476 INFO:tasks.workunit.client.1.smithi104.stdout:2/308: rename d6/fd to d6/de/d1d/f6d 0 2024-04-26T04:42:11.480 INFO:tasks.workunit.client.1.smithi104.stdout:7/108: mknod d2/d4/d1d/c21 0 2024-04-26T04:42:11.481 INFO:tasks.workunit.client.1.smithi104.stdout:2/309: link d6/d1b/l64 d6/l6e 0 2024-04-26T04:42:11.482 INFO:tasks.workunit.client.1.smithi104.stdout:5/234: write d0/d1/d1e/f2e [3250543,48191] 0 2024-04-26T04:42:11.482 INFO:tasks.workunit.client.1.smithi104.stdout:5/235: chown d0/d41 3 1 2024-04-26T04:42:11.484 INFO:tasks.workunit.client.0.smithi090.stdout:6/122: dwrite f1 [4194304,4194304] 0 2024-04-26T04:42:11.488 INFO:tasks.workunit.client.1.smithi104.stdout:8/125: write d0/fb [1071581,99175] 0 2024-04-26T04:42:11.489 INFO:tasks.workunit.client.1.smithi104.stdout:9/157: dwrite f6 [0,4194304] 0 2024-04-26T04:42:11.490 INFO:tasks.workunit.client.1.smithi104.stdout:1/252: dwrite d0/d6/ff [4194304,4194304] 0 2024-04-26T04:42:11.490 INFO:tasks.workunit.client.1.smithi104.stdout:1/253: write d0/d6/d2e/f46 [131083,52076] 0 2024-04-26T04:42:11.490 INFO:tasks.workunit.client.1.smithi104.stdout:2/310: creat d6/d1b/d51/d6a/f6f x:0 0 0 2024-04-26T04:42:11.492 INFO:tasks.workunit.client.1.smithi104.stdout:9/158: dread f10 [0,4194304] 0 2024-04-26T04:42:11.492 INFO:tasks.workunit.client.1.smithi104.stdout:5/236: symlink d0/d11/d26/l46 0 2024-04-26T04:42:11.492 INFO:tasks.workunit.client.1.smithi104.stdout:5/237: write d0/d11/d26/d36/f40 [159992,8412] 0 2024-04-26T04:42:11.494 INFO:tasks.workunit.client.1.smithi104.stdout:2/311: unlink d6/de/l2f 0 2024-04-26T04:42:11.495 INFO:tasks.workunit.client.1.smithi104.stdout:2/312: fdatasync d6/f63 0 2024-04-26T04:42:11.495 INFO:tasks.workunit.client.1.smithi104.stdout:3/183: rmdir d5/d9/d17 39 2024-04-26T04:42:11.497 INFO:tasks.workunit.client.1.smithi104.stdout:9/159: truncate d11/f12 772325 0 2024-04-26T04:42:11.500 INFO:tasks.workunit.client.1.smithi104.stdout:3/184: creat d5/d9/d17/f3c x:0 0 0 2024-04-26T04:42:11.501 INFO:tasks.workunit.client.1.smithi104.stdout:7/109: dwrite d2/d4/f15 [0,4194304] 0 2024-04-26T04:42:11.501 INFO:tasks.workunit.client.1.smithi104.stdout:7/110: dread - d2/d4/f13 zero size 2024-04-26T04:42:11.501 INFO:tasks.workunit.client.1.smithi104.stdout:7/111: chown d2/d4/l17 12688 1 2024-04-26T04:42:11.501 INFO:tasks.workunit.client.1.smithi104.stdout:3/185: write d5/d9/fe [5108034,92470] 0 2024-04-26T04:42:11.501 INFO:tasks.workunit.client.1.smithi104.stdout:3/186: read - d5/d9/d17/f3b zero size 2024-04-26T04:42:11.502 INFO:tasks.workunit.client.1.smithi104.stdout:4/183: rmdir d4 39 2024-04-26T04:42:11.502 INFO:tasks.workunit.client.1.smithi104.stdout:1/254: dread d0/d6/de/d20/d2a/f2c [0,4194304] 0 2024-04-26T04:42:11.503 INFO:tasks.workunit.client.1.smithi104.stdout:1/255: chown d0/d6/de/f41 2 1 2024-04-26T04:42:11.503 INFO:tasks.workunit.client.1.smithi104.stdout:1/256: write d0/d6/de/d29/f39 [1044431,118463] 0 2024-04-26T04:42:11.503 INFO:tasks.workunit.client.1.smithi104.stdout:1/257: write d0/d6/de/d20/d2a/f31 [2507307,56982] 0 2024-04-26T04:42:11.503 INFO:tasks.workunit.client.1.smithi104.stdout:9/160: link d11/l1c d11/d1d/l28 0 2024-04-26T04:42:11.506 INFO:tasks.workunit.client.1.smithi104.stdout:1/258: write d0/d6/de/d20/d2a/f2c [1854143,21138] 0 2024-04-26T04:42:11.506 INFO:tasks.workunit.client.1.smithi104.stdout:1/259: truncate d0/d6/d19/d1b/d35/f40 526890 0 2024-04-26T04:42:11.506 INFO:tasks.workunit.client.1.smithi104.stdout:4/184: dread d4/d7/d1d/f23 [0,4194304] 0 2024-04-26T04:42:11.506 INFO:tasks.workunit.client.1.smithi104.stdout:2/313: rmdir d6/de/d1d 39 2024-04-26T04:42:11.511 INFO:tasks.workunit.client.1.smithi104.stdout:7/112: symlink d2/d4/l22 0 2024-04-26T04:42:11.512 INFO:tasks.workunit.client.1.smithi104.stdout:3/187: symlink d5/d9/dc/d29/l3d 0 2024-04-26T04:42:11.514 INFO:tasks.workunit.client.1.smithi104.stdout:9/161: creat d11/d15/f29 x:0 0 0 2024-04-26T04:42:11.514 INFO:tasks.workunit.client.1.smithi104.stdout:9/162: fsync fc 0 2024-04-26T04:42:11.515 INFO:tasks.workunit.client.1.smithi104.stdout:2/314: creat d6/d8/f70 x:0 0 0 2024-04-26T04:42:11.516 INFO:tasks.workunit.client.1.smithi104.stdout:1/260: write d0/d6/de/f41 [1601427,8158] 0 2024-04-26T04:42:11.516 INFO:tasks.workunit.client.1.smithi104.stdout:1/261: write d0/d6/d19/d1b/d35/f40 [792815,35912] 0 2024-04-26T04:42:11.518 INFO:tasks.workunit.client.1.smithi104.stdout:3/188: creat d5/d9/dc/d29/d2d/f3e x:0 0 0 2024-04-26T04:42:11.518 INFO:tasks.workunit.client.1.smithi104.stdout:3/189: creat d5/f3f x:0 0 0 2024-04-26T04:42:11.518 INFO:tasks.workunit.client.1.smithi104.stdout:3/190: creat d5/d9/dc/f40 x:0 0 0 2024-04-26T04:42:11.518 INFO:tasks.workunit.client.1.smithi104.stdout:3/191: creat d5/d9/dc/f41 x:0 0 0 2024-04-26T04:42:11.518 INFO:tasks.workunit.client.1.smithi104.stdout:3/192: write d5/d9/f1d [4380182,88954] 0 2024-04-26T04:42:11.519 INFO:tasks.workunit.client.1.smithi104.stdout:3/193: dread - d5/f25 zero size 2024-04-26T04:42:11.519 INFO:tasks.workunit.client.1.smithi104.stdout:3/194: creat d5/d9/dc/f42 x:0 0 0 2024-04-26T04:42:11.519 INFO:tasks.workunit.client.1.smithi104.stdout:3/195: write d5/f24 [866845,81553] 0 2024-04-26T04:42:11.519 INFO:tasks.workunit.client.1.smithi104.stdout:7/113: dread d2/f16 [0,4194304] 0 2024-04-26T04:42:11.519 INFO:tasks.workunit.client.1.smithi104.stdout:7/114: write d2/f20 [5000318,6392] 0 2024-04-26T04:42:11.519 INFO:tasks.workunit.client.1.smithi104.stdout:9/163: truncate fc 5747817 0 2024-04-26T04:42:11.520 INFO:tasks.workunit.client.1.smithi104.stdout:5/238: dwrite d0/d11/d26/d36/f43 [0,4194304] 0 2024-04-26T04:42:11.524 INFO:tasks.workunit.client.1.smithi104.stdout:4/185: dwrite d4/d7/d1d/d1e/f39 [0,4194304] 0 2024-04-26T04:42:11.528 INFO:tasks.workunit.client.1.smithi104.stdout:5/239: rename d0/d1/d4/lb to d0/d11/d26/l47 0 2024-04-26T04:42:11.529 INFO:tasks.workunit.client.1.smithi104.stdout:3/196: dread d5/f22 [0,4194304] 0 2024-04-26T04:42:11.529 INFO:tasks.workunit.client.1.smithi104.stdout:3/197: readlink d5/d9/d17/l27 0 2024-04-26T04:42:11.532 INFO:tasks.workunit.client.0.smithi090.stdout:6/123: dwrite d9/d1c/f2b [0,4194304] 0 2024-04-26T04:42:11.534 INFO:tasks.workunit.client.1.smithi104.stdout:5/240: rmdir d0/d11/d26/d36/d45 0 2024-04-26T04:42:11.534 INFO:tasks.workunit.client.1.smithi104.stdout:5/241: chown d0/d1/d4 21333 1 2024-04-26T04:42:11.534 INFO:tasks.workunit.client.0.smithi090.stdout:6/124: symlink d9/d1c/l30 0 2024-04-26T04:42:11.536 INFO:tasks.workunit.client.1.smithi104.stdout:3/198: link d5/d9/d17/f1b d5/f43 0 2024-04-26T04:42:11.538 INFO:tasks.workunit.client.1.smithi104.stdout:5/242: mkdir d0/d1/d4/d2a/d2b/d37/d48 0 2024-04-26T04:42:11.539 INFO:tasks.workunit.client.1.smithi104.stdout:5/243: symlink d0/d1/d4/d2a/l49 0 2024-04-26T04:42:11.540 INFO:tasks.workunit.client.1.smithi104.stdout:5/244: mknod d0/d41/c4a 0 2024-04-26T04:42:11.540 INFO:tasks.workunit.client.1.smithi104.stdout:5/245: fdatasync d0/d11/f23 0 2024-04-26T04:42:11.541 INFO:tasks.workunit.client.1.smithi104.stdout:5/246: creat d0/d1/d4/d2a/d2b/d37/f4b x:0 0 0 2024-04-26T04:42:11.542 INFO:tasks.workunit.client.1.smithi104.stdout:5/247: symlink d0/d1/d3e/l4c 0 2024-04-26T04:42:11.548 INFO:tasks.workunit.client.1.smithi104.stdout:2/315: dwrite d6/de/d1d/f43 [0,4194304] 0 2024-04-26T04:42:11.548 INFO:tasks.workunit.client.1.smithi104.stdout:1/262: dwrite d0/d6/ff [8388608,4194304] 0 2024-04-26T04:42:11.548 INFO:tasks.workunit.client.1.smithi104.stdout:2/316: dread - d6/d8/f5b zero size 2024-04-26T04:42:11.548 INFO:tasks.workunit.client.1.smithi104.stdout:2/317: chown d6/d8/d3e/f2a 147575458 1 2024-04-26T04:42:11.553 INFO:tasks.workunit.client.1.smithi104.stdout:2/318: creat d6/d1b/f71 x:0 0 0 2024-04-26T04:42:11.553 INFO:tasks.workunit.client.1.smithi104.stdout:2/319: write d6/d8/d5a/d29/f60 [1503443,41870] 0 2024-04-26T04:42:11.555 INFO:tasks.workunit.client.1.smithi104.stdout:1/263: mkdir d0/d6/de/d20/d2a/d50 0 2024-04-26T04:42:11.555 INFO:tasks.workunit.client.1.smithi104.stdout:1/264: write d0/d6/f1a [399714,80674] 0 2024-04-26T04:42:11.555 INFO:tasks.workunit.client.1.smithi104.stdout:4/186: dwrite d4/d7/d1d/d1e/d15/f1a [0,4194304] 0 2024-04-26T04:42:11.557 INFO:tasks.workunit.client.1.smithi104.stdout:2/320: truncate d6/de/d1d/f42 3608063 0 2024-04-26T04:42:11.566 INFO:tasks.workunit.client.0.smithi090.stdout:6/125: dwrite d9/f14 [8388608,4194304] 0 2024-04-26T04:42:11.567 INFO:tasks.workunit.client.1.smithi104.stdout:4/187: read d4/d7/d1d/d1e/d15/f1a [3293288,54667] 0 2024-04-26T04:42:11.567 INFO:tasks.workunit.client.1.smithi104.stdout:7/115: truncate d2/f20 1421847 0 2024-04-26T04:42:11.567 INFO:tasks.workunit.client.1.smithi104.stdout:5/248: dwrite d0/f1d [4194304,4194304] 0 2024-04-26T04:42:11.570 INFO:tasks.workunit.client.0.smithi090.stdout:6/126: mkdir d9/d1c/d31 0 2024-04-26T04:42:11.570 INFO:tasks.workunit.client.1.smithi104.stdout:9/164: dwrite d11/d15/f18 [4194304,4194304] 0 2024-04-26T04:42:11.570 INFO:tasks.workunit.client.1.smithi104.stdout:4/188: mknod d4/d7/d1d/d1e/d24/c40 0 2024-04-26T04:42:11.571 INFO:tasks.workunit.client.0.smithi090.stdout:6/127: mknod d9/d22/c32 0 2024-04-26T04:42:11.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/321: dread d6/fc [0,4194304] 0 2024-04-26T04:42:11.573 INFO:tasks.workunit.client.1.smithi104.stdout:2/322: fsync d6/f35 0 2024-04-26T04:42:11.573 INFO:tasks.workunit.client.0.smithi090.stdout:6/128: symlink d9/l33 0 2024-04-26T04:42:11.574 INFO:tasks.workunit.client.1.smithi104.stdout:3/199: dread d5/d9/f1d [0,4194304] 0 2024-04-26T04:42:11.574 INFO:tasks.workunit.client.1.smithi104.stdout:3/200: fsync d5/d9/dc/f16 0 2024-04-26T04:42:11.576 INFO:tasks.workunit.client.1.smithi104.stdout:7/116: mknod d2/c23 0 2024-04-26T04:42:11.576 INFO:tasks.workunit.client.1.smithi104.stdout:7/117: readlink d2/d4/l1b 0 2024-04-26T04:42:11.577 INFO:tasks.workunit.client.1.smithi104.stdout:2/323: mknod d6/d1b/d68/c72 0 2024-04-26T04:42:11.590 INFO:tasks.workunit.client.1.smithi104.stdout:1/265: truncate d0/d6/ff 11213073 0 2024-04-26T04:42:11.592 INFO:tasks.workunit.client.1.smithi104.stdout:3/201: link d5/d9/f12 d5/d9/dc/d29/f44 0 2024-04-26T04:42:11.595 INFO:tasks.workunit.client.1.smithi104.stdout:7/118: link d2/d4/l22 d2/d4/l24 0 2024-04-26T04:42:11.595 INFO:tasks.workunit.client.1.smithi104.stdout:7/119: write d2/d4/f11 [1188006,89963] 0 2024-04-26T04:42:11.596 INFO:tasks.workunit.client.1.smithi104.stdout:7/120: write d2/f20 [723696,96828] 0 2024-04-26T04:42:11.597 INFO:tasks.workunit.client.1.smithi104.stdout:3/202: rename d5/l20 to d5/d9/dc/d29/d2d/l45 0 2024-04-26T04:42:11.597 INFO:tasks.workunit.client.1.smithi104.stdout:3/203: truncate d5/f3f 884532 0 2024-04-26T04:42:11.598 INFO:tasks.workunit.client.1.smithi104.stdout:7/121: link d2/d4/l24 d2/d4/l25 0 2024-04-26T04:42:11.599 INFO:tasks.workunit.client.1.smithi104.stdout:3/204: mkdir d5/d9/d17/d46 0 2024-04-26T04:42:11.600 INFO:tasks.workunit.client.1.smithi104.stdout:7/122: rename d2/d4/f15 to d2/d4/f26 0 2024-04-26T04:42:11.601 INFO:tasks.workunit.client.1.smithi104.stdout:3/205: mkdir d5/d9/dc/d32/d47 0 2024-04-26T04:42:11.602 INFO:tasks.workunit.client.1.smithi104.stdout:7/123: rename d2/d4/lf to d2/d4/l27 0 2024-04-26T04:42:11.621 INFO:tasks.workunit.client.1.smithi104.stdout:4/189: dwrite d4/f37 [0,4194304] 0 2024-04-26T04:42:11.621 INFO:tasks.workunit.client.1.smithi104.stdout:4/190: readlink d4/d7/l3a 0 2024-04-26T04:42:11.621 INFO:tasks.workunit.client.1.smithi104.stdout:7/124: write d2/d4/f26 [847171,31589] 0 2024-04-26T04:42:11.621 INFO:tasks.workunit.client.1.smithi104.stdout:4/191: fdatasync d4/d7/f8 0 2024-04-26T04:42:11.622 INFO:tasks.workunit.client.1.smithi104.stdout:7/125: stat d2/d4/l22 0 2024-04-26T04:42:11.622 INFO:tasks.workunit.client.1.smithi104.stdout:4/192: truncate d4/d7/d1d/d1e/f39 2590699 0 2024-04-26T04:42:11.622 INFO:tasks.workunit.client.1.smithi104.stdout:4/193: truncate d4/d7/f2a 513325 0 2024-04-26T04:42:11.626 INFO:tasks.workunit.client.1.smithi104.stdout:5/249: dwrite d0/d1/d4/d2a/d2b/f35 [0,4194304] 0 2024-04-26T04:42:11.626 INFO:tasks.workunit.client.1.smithi104.stdout:2/324: dwrite d6/d8/d3e/f26 [4194304,4194304] 0 2024-04-26T04:42:11.626 INFO:tasks.workunit.client.1.smithi104.stdout:9/165: dwrite d11/f16 [0,4194304] 0 2024-04-26T04:42:11.627 INFO:tasks.workunit.client.0.smithi090.stdout:6/129: dwrite d9/d22/f29 [0,4194304] 0 2024-04-26T04:42:11.627 INFO:tasks.workunit.client.1.smithi104.stdout:1/266: dwrite d0/d6/d19/f3a [0,4194304] 0 2024-04-26T04:42:11.627 INFO:tasks.workunit.client.1.smithi104.stdout:3/206: dwrite d5/d9/fa [0,4194304] 0 2024-04-26T04:42:11.627 INFO:tasks.workunit.client.0.smithi090.stdout:6/130: write d9/f16 [7024,32627] 0 2024-04-26T04:42:11.628 INFO:tasks.workunit.client.0.smithi090.stdout:6/131: symlink d9/d1c/l34 0 2024-04-26T04:42:11.630 INFO:tasks.workunit.client.1.smithi104.stdout:5/250: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:11.631 INFO:tasks.workunit.client.1.smithi104.stdout:2/325: truncate d6/de/d1d/f42 2146601 0 2024-04-26T04:42:11.631 INFO:tasks.workunit.client.1.smithi104.stdout:2/326: chown d6/d1b 980537 1 2024-04-26T04:42:11.638 INFO:tasks.workunit.client.1.smithi104.stdout:9/166: symlink d11/d15/d1b/l2a 0 2024-04-26T04:42:11.639 INFO:tasks.workunit.client.1.smithi104.stdout:1/267: mknod d0/d6/de/d20/c51 0 2024-04-26T04:42:11.639 INFO:tasks.workunit.client.1.smithi104.stdout:1/268: read d0/d6/f1a [2244626,85281] 0 2024-04-26T04:42:11.640 INFO:tasks.workunit.client.1.smithi104.stdout:1/269: readlink d0/d6/de/d29/d3f/l45 0 2024-04-26T04:42:11.640 INFO:tasks.workunit.client.0.smithi090.stdout:6/132: dread d9/f12 [0,4194304] 0 2024-04-26T04:42:11.641 INFO:tasks.workunit.client.0.smithi090.stdout:6/133: mkdir d9/d35 0 2024-04-26T04:42:11.641 INFO:tasks.workunit.client.0.smithi090.stdout:6/134: dread - d9/d22/f28 zero size 2024-04-26T04:42:11.642 INFO:tasks.workunit.client.0.smithi090.stdout:6/135: creat d9/d1c/f36 x:0 0 0 2024-04-26T04:42:11.642 INFO:tasks.workunit.client.0.smithi090.stdout:6/136: write d9/f1d [696291,45557] 0 2024-04-26T04:42:11.646 INFO:tasks.workunit.client.1.smithi104.stdout:3/207: mkdir d5/d9/dc/d32/d48 0 2024-04-26T04:42:11.647 INFO:tasks.workunit.client.1.smithi104.stdout:4/194: dwrite f3 [0,4194304] 0 2024-04-26T04:42:11.648 INFO:tasks.workunit.client.1.smithi104.stdout:5/251: creat d0/d41/f4d x:0 0 0 2024-04-26T04:42:11.648 INFO:tasks.workunit.client.1.smithi104.stdout:5/252: creat d0/f4e x:0 0 0 2024-04-26T04:42:11.648 INFO:tasks.workunit.client.1.smithi104.stdout:5/253: dread - d0/f4e zero size 2024-04-26T04:42:11.650 INFO:tasks.workunit.client.1.smithi104.stdout:2/327: rename d6/d8/d5a/c3c to d6/d8/d3e/c73 0 2024-04-26T04:42:11.650 INFO:tasks.workunit.client.1.smithi104.stdout:1/270: rename d0 to d0/d6/de/d20/d2a/d50/d52 22 2024-04-26T04:42:11.651 INFO:tasks.workunit.client.1.smithi104.stdout:9/167: symlink d11/d15/l2b 0 2024-04-26T04:42:11.655 INFO:tasks.workunit.client.1.smithi104.stdout:5/254: unlink d0/d1/d4/l3b 0 2024-04-26T04:42:11.655 INFO:tasks.workunit.client.1.smithi104.stdout:3/208: dread d5/d9/dc/f15 [0,4194304] 0 2024-04-26T04:42:11.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/209: stat d5/d9/d17/f3b 0 2024-04-26T04:42:11.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/210: fdatasync f4 0 2024-04-26T04:42:11.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/211: chown d5/d9/dc/f42 247140 1 2024-04-26T04:42:11.657 INFO:tasks.workunit.client.1.smithi104.stdout:1/271: symlink d0/d6/d19/d1b/l53 0 2024-04-26T04:42:11.658 INFO:tasks.workunit.client.1.smithi104.stdout:9/168: mknod d11/c2c 0 2024-04-26T04:42:11.658 INFO:tasks.workunit.client.1.smithi104.stdout:9/169: creat d11/d1d/f2d x:0 0 0 2024-04-26T04:42:11.659 INFO:tasks.workunit.client.1.smithi104.stdout:2/328: dread d6/d8/d5a/d29/f69 [0,4194304] 0 2024-04-26T04:42:11.659 INFO:tasks.workunit.client.1.smithi104.stdout:5/255: mkdir d0/d41/d4f 0 2024-04-26T04:42:11.659 INFO:tasks.workunit.client.1.smithi104.stdout:4/195: dread d4/f37 [4194304,4194304] 0 2024-04-26T04:42:11.661 INFO:tasks.workunit.client.1.smithi104.stdout:3/212: mkdir d5/d9/dc/d29/d49 0 2024-04-26T04:42:11.661 INFO:tasks.workunit.client.1.smithi104.stdout:3/213: write d5/d9/d17/f34 [153015,6724] 0 2024-04-26T04:42:11.661 INFO:tasks.workunit.client.1.smithi104.stdout:3/214: getdents d5/d9/dc/d32/d48 0 2024-04-26T04:42:11.661 INFO:tasks.workunit.client.1.smithi104.stdout:3/215: chown d5/d9/dc/d32 1652726 1 2024-04-26T04:42:11.661 INFO:tasks.workunit.client.1.smithi104.stdout:3/216: write d5/f23 [10159,10834] 0 2024-04-26T04:42:11.664 INFO:tasks.workunit.client.1.smithi104.stdout:9/170: mknod d11/d15/d1b/c2e 0 2024-04-26T04:42:11.664 INFO:tasks.workunit.client.1.smithi104.stdout:9/171: truncate d11/d1d/f26 441204 0 2024-04-26T04:42:11.666 INFO:tasks.workunit.client.1.smithi104.stdout:2/329: creat d6/d8/d5a/f74 x:0 0 0 2024-04-26T04:42:11.667 INFO:tasks.workunit.client.1.smithi104.stdout:1/272: write d0/f14 [1256407,48412] 0 2024-04-26T04:42:11.667 INFO:tasks.workunit.client.1.smithi104.stdout:1/273: fsync d0/d6/de/d29/f39 0 2024-04-26T04:42:11.669 INFO:tasks.workunit.client.1.smithi104.stdout:3/217: creat d5/d9/f4a x:0 0 0 2024-04-26T04:42:11.674 INFO:tasks.workunit.client.1.smithi104.stdout:2/330: mkdir d6/d1b/d75 0 2024-04-26T04:42:11.674 INFO:tasks.workunit.client.1.smithi104.stdout:2/331: fdatasync d6/d1b/d51/d6a/f6f 0 2024-04-26T04:42:11.674 INFO:tasks.workunit.client.1.smithi104.stdout:6/320: sync 2024-04-26T04:42:11.675 INFO:tasks.workunit.client.1.smithi104.stdout:0/206: sync 2024-04-26T04:42:11.675 INFO:tasks.workunit.client.1.smithi104.stdout:2/332: write d6/de/f40 [78674,80905] 0 2024-04-26T04:42:11.675 INFO:tasks.workunit.client.1.smithi104.stdout:4/196: rename d4/db/c2c to d4/d7/c41 0 2024-04-26T04:42:11.675 INFO:tasks.workunit.client.1.smithi104.stdout:6/321: mknod d0/d9/d13/d44/d47/c6f 0 2024-04-26T04:42:11.675 INFO:tasks.workunit.client.1.smithi104.stdout:6/322: fsync d0/d9/d5c/f69 0 2024-04-26T04:42:11.677 INFO:tasks.workunit.client.1.smithi104.stdout:2/333: link d6/d8/d3e/c4e d6/d1b/d51/c76 0 2024-04-26T04:42:11.680 INFO:tasks.workunit.client.1.smithi104.stdout:1/274: rename d0/d6/d19/d21/l25 to d0/d6/d19/d1b/d35/l54 0 2024-04-26T04:42:11.683 INFO:tasks.workunit.client.1.smithi104.stdout:2/334: mknod d6/d8/d5a/d29/c77 0 2024-04-26T04:42:11.683 INFO:tasks.workunit.client.1.smithi104.stdout:2/335: dread - d6/f33 zero size 2024-04-26T04:42:11.683 INFO:tasks.workunit.client.1.smithi104.stdout:2/336: fdatasync d6/d8/d3e/f2a 0 2024-04-26T04:42:11.683 INFO:tasks.workunit.client.1.smithi104.stdout:2/337: creat d6/d8/f78 x:0 0 0 2024-04-26T04:42:11.684 INFO:tasks.workunit.client.1.smithi104.stdout:0/207: rename d0/d4/f1b to d0/f42 0 2024-04-26T04:42:11.684 INFO:tasks.workunit.client.1.smithi104.stdout:0/208: chown d0/d2/c13 588233 1 2024-04-26T04:42:11.684 INFO:tasks.workunit.client.1.smithi104.stdout:0/209: read - d0/f41 zero size 2024-04-26T04:42:11.685 INFO:tasks.workunit.client.1.smithi104.stdout:2/338: dread d6/d1b/f38 [0,4194304] 0 2024-04-26T04:42:11.685 INFO:tasks.workunit.client.1.smithi104.stdout:2/339: write d6/d1b/d51/d6a/f6f [917522,82926] 0 2024-04-26T04:42:11.685 INFO:tasks.workunit.client.1.smithi104.stdout:2/340: readlink d6/d8/d5a/d29/l6c 0 2024-04-26T04:42:11.685 INFO:tasks.workunit.client.1.smithi104.stdout:2/341: readlink d6/d8/d5a/l48 0 2024-04-26T04:42:11.685 INFO:tasks.workunit.client.1.smithi104.stdout:2/342: truncate d6/de/f44 519835 0 2024-04-26T04:42:11.685 INFO:tasks.workunit.client.1.smithi104.stdout:2/343: fsync d6/d8/f17 0 2024-04-26T04:42:11.686 INFO:tasks.workunit.client.1.smithi104.stdout:1/275: link d0/d6/de/l10 d0/d6/de/l55 0 2024-04-26T04:42:11.686 INFO:tasks.workunit.client.1.smithi104.stdout:1/276: chown d0/d6/d19/d21 1 1 2024-04-26T04:42:11.687 INFO:tasks.workunit.client.1.smithi104.stdout:1/277: write d0/d6/d19/d1b/d35/f40 [330969,103432] 0 2024-04-26T04:42:11.689 INFO:tasks.workunit.client.1.smithi104.stdout:5/256: dwrite d0/d1/d1e/f30 [0,4194304] 0 2024-04-26T04:42:11.694 INFO:tasks.workunit.client.1.smithi104.stdout:0/210: symlink d0/d4/l43 0 2024-04-26T04:42:11.695 INFO:tasks.workunit.client.1.smithi104.stdout:2/344: symlink d6/d8/d5a/l79 0 2024-04-26T04:42:11.695 INFO:tasks.workunit.client.1.smithi104.stdout:2/345: chown d6/d8/d3e/c73 12979 1 2024-04-26T04:42:11.695 INFO:tasks.workunit.client.1.smithi104.stdout:8/126: sync 2024-04-26T04:42:11.695 INFO:tasks.workunit.client.1.smithi104.stdout:3/218: dwrite d5/d9/dc/f41 [0,4194304] 0 2024-04-26T04:42:11.696 INFO:tasks.workunit.client.1.smithi104.stdout:9/172: dwrite d11/f17 [4194304,4194304] 0 2024-04-26T04:42:11.696 INFO:tasks.workunit.client.1.smithi104.stdout:9/173: chown d11/d15/d1b/c2e 6 1 2024-04-26T04:42:11.696 INFO:tasks.workunit.client.1.smithi104.stdout:4/197: dwrite d4/d20/d3c/f3e [0,4194304] 0 2024-04-26T04:42:11.699 INFO:tasks.workunit.client.1.smithi104.stdout:5/257: creat d0/d1/d4/d2a/f50 x:0 0 0 2024-04-26T04:42:11.701 INFO:tasks.workunit.client.1.smithi104.stdout:0/211: mknod d0/d2/d21/d27/c44 0 2024-04-26T04:42:11.702 INFO:tasks.workunit.client.1.smithi104.stdout:2/346: creat d6/d8/d3e/f7a x:0 0 0 2024-04-26T04:42:11.705 INFO:tasks.workunit.client.1.smithi104.stdout:3/219: mknod d5/d9/dc/c4b 0 2024-04-26T04:42:11.705 INFO:tasks.workunit.client.1.smithi104.stdout:0/212: dread d0/d2/d21/d27/d1e/f2d [0,4194304] 0 2024-04-26T04:42:11.707 INFO:tasks.workunit.client.1.smithi104.stdout:4/198: dread d4/d7/f1f [0,4194304] 0 2024-04-26T04:42:11.708 INFO:tasks.workunit.client.1.smithi104.stdout:2/347: write d6/de/d1d/f42 [427962,95716] 0 2024-04-26T04:42:11.708 INFO:tasks.workunit.client.1.smithi104.stdout:6/323: rmdir d0/d9 39 2024-04-26T04:42:11.708 INFO:tasks.workunit.client.1.smithi104.stdout:6/324: chown d0/d5 47405082 1 2024-04-26T04:42:11.711 INFO:tasks.workunit.client.1.smithi104.stdout:5/258: dread d0/d1/d4/f12 [0,4194304] 0 2024-04-26T04:42:11.712 INFO:tasks.workunit.client.1.smithi104.stdout:9/174: rename d11/d15/d1b/c22 to d11/d1d/c2f 0 2024-04-26T04:42:11.714 INFO:tasks.workunit.client.1.smithi104.stdout:1/278: rmdir d0/d6/d19 39 2024-04-26T04:42:11.714 INFO:tasks.workunit.client.1.smithi104.stdout:1/279: stat d0/d6/d19/d21 0 2024-04-26T04:42:11.715 INFO:tasks.workunit.client.1.smithi104.stdout:8/127: write d0/d12/f1f [2731414,46714] 0 2024-04-26T04:42:11.715 INFO:tasks.workunit.client.1.smithi104.stdout:8/128: stat d0/f1a 0 2024-04-26T04:42:11.716 INFO:tasks.workunit.client.1.smithi104.stdout:0/213: read d0/d4/d15/f18 [1311089,85753] 0 2024-04-26T04:42:11.718 INFO:tasks.workunit.client.1.smithi104.stdout:4/199: creat d4/d7/f42 x:0 0 0 2024-04-26T04:42:11.718 INFO:tasks.workunit.client.1.smithi104.stdout:4/200: write d4/d20/d3c/f31 [664322,78833] 0 2024-04-26T04:42:11.718 INFO:tasks.workunit.client.1.smithi104.stdout:4/201: chown d4/d7/f1f 1 1 2024-04-26T04:42:11.723 INFO:tasks.workunit.client.1.smithi104.stdout:2/348: mknod d6/de/c7b 0 2024-04-26T04:42:11.723 INFO:tasks.workunit.client.1.smithi104.stdout:2/349: write d6/d8/f70 [333162,15817] 0 2024-04-26T04:42:11.723 INFO:tasks.workunit.client.1.smithi104.stdout:2/350: fsync d6/d8/d3e/f49 0 2024-04-26T04:42:11.726 INFO:tasks.workunit.client.1.smithi104.stdout:9/175: rename d11/d15/f29 to d11/d15/f30 0 2024-04-26T04:42:11.726 INFO:tasks.workunit.client.1.smithi104.stdout:9/176: creat d11/d1d/f31 x:0 0 0 2024-04-26T04:42:11.726 INFO:tasks.workunit.client.1.smithi104.stdout:1/280: truncate d0/d6/de/d20/d2a/f31 1698954 0 2024-04-26T04:42:11.728 INFO:tasks.workunit.client.1.smithi104.stdout:4/202: getdents d4/d20/d3c 0 2024-04-26T04:42:11.730 INFO:tasks.workunit.client.1.smithi104.stdout:2/351: creat d6/d1b/d75/f7c x:0 0 0 2024-04-26T04:42:11.732 INFO:tasks.workunit.client.1.smithi104.stdout:9/177: mkdir d11/d15/d32 0 2024-04-26T04:42:11.734 INFO:tasks.workunit.client.1.smithi104.stdout:1/281: mkdir d0/d6/de/d20/d2a/d56 0 2024-04-26T04:42:11.735 INFO:tasks.workunit.client.1.smithi104.stdout:8/129: dread d0/fb [4194304,4194304] 0 2024-04-26T04:42:11.735 INFO:tasks.workunit.client.1.smithi104.stdout:8/130: readlink d0/l2 0 2024-04-26T04:42:11.735 INFO:tasks.workunit.client.1.smithi104.stdout:8/131: write d0/d1c/f26 [162349,101980] 0 2024-04-26T04:42:11.735 INFO:tasks.workunit.client.1.smithi104.stdout:4/203: creat d4/d7/d1d/f43 x:0 0 0 2024-04-26T04:42:11.736 INFO:tasks.workunit.client.1.smithi104.stdout:4/204: readlink d4/d20/l2d 0 2024-04-26T04:42:11.736 INFO:tasks.workunit.client.1.smithi104.stdout:9/178: symlink d11/d1d/l33 0 2024-04-26T04:42:11.737 INFO:tasks.workunit.client.1.smithi104.stdout:9/179: truncate d11/f27 707869 0 2024-04-26T04:42:11.737 INFO:tasks.workunit.client.1.smithi104.stdout:9/180: write d11/f20 [186707,43442] 0 2024-04-26T04:42:11.737 INFO:tasks.workunit.client.1.smithi104.stdout:4/205: symlink d4/d7/d1d/d1e/d24/l44 0 2024-04-26T04:42:11.737 INFO:tasks.workunit.client.1.smithi104.stdout:4/206: truncate d4/d7/d1d/d1e/d15/f1a 4597993 0 2024-04-26T04:42:11.738 INFO:tasks.workunit.client.1.smithi104.stdout:9/181: mkdir d11/d1d/d34 0 2024-04-26T04:42:11.739 INFO:tasks.workunit.client.1.smithi104.stdout:9/182: truncate d11/f20 539237 0 2024-04-26T04:42:11.739 INFO:tasks.workunit.client.1.smithi104.stdout:9/183: chown d11/d15/d32 12126535 1 2024-04-26T04:42:11.744 INFO:tasks.workunit.client.1.smithi104.stdout:1/282: dread d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:11.748 INFO:tasks.workunit.client.1.smithi104.stdout:3/220: dwrite d5/d9/dc/f42 [0,4194304] 0 2024-04-26T04:42:11.748 INFO:tasks.workunit.client.1.smithi104.stdout:3/221: write d5/d9/f26 [53517,19412] 0 2024-04-26T04:42:11.751 INFO:tasks.workunit.client.1.smithi104.stdout:3/222: mknod d5/d9/dc/d32/c4c 0 2024-04-26T04:42:11.751 INFO:tasks.workunit.client.1.smithi104.stdout:3/223: dread - d5/d9/dc/f40 zero size 2024-04-26T04:42:11.752 INFO:tasks.workunit.client.1.smithi104.stdout:4/207: dread d4/d20/d3c/f3e [0,4194304] 0 2024-04-26T04:42:11.752 INFO:tasks.workunit.client.1.smithi104.stdout:8/132: fsync d0/d12/f1f 0 2024-04-26T04:42:11.752 INFO:tasks.workunit.client.1.smithi104.stdout:6/325: dwrite d0/d9/f2a [4194304,4194304] 0 2024-04-26T04:42:11.752 INFO:tasks.workunit.client.1.smithi104.stdout:5/259: dwrite d0/d1/d4/f12 [4194304,4194304] 0 2024-04-26T04:42:11.752 INFO:tasks.workunit.client.1.smithi104.stdout:6/326: fdatasync d0/d9/d25/f68 0 2024-04-26T04:42:11.752 INFO:tasks.workunit.client.1.smithi104.stdout:6/327: truncate d0/d1b/f2c 213469 0 2024-04-26T04:42:11.753 INFO:tasks.workunit.client.1.smithi104.stdout:3/224: creat d5/d9/d17/d46/f4d x:0 0 0 2024-04-26T04:42:11.753 INFO:tasks.workunit.client.1.smithi104.stdout:3/225: stat d5/d9/dc/d29/l3d 0 2024-04-26T04:42:11.753 INFO:tasks.workunit.client.1.smithi104.stdout:3/226: fdatasync f0 0 2024-04-26T04:42:11.754 INFO:tasks.workunit.client.1.smithi104.stdout:4/208: truncate d4/d7/d1d/d1e/f14 84521 0 2024-04-26T04:42:11.754 INFO:tasks.workunit.client.1.smithi104.stdout:4/209: write d4/d7/d1d/d1e/d15/f22 [588842,70296] 0 2024-04-26T04:42:11.755 INFO:tasks.workunit.client.1.smithi104.stdout:5/260: creat d0/d1/d3e/f51 x:0 0 0 2024-04-26T04:42:11.757 INFO:tasks.workunit.client.1.smithi104.stdout:6/328: rename d0/d5/c55 to d0/d9/d5c/c70 0 2024-04-26T04:42:11.759 INFO:tasks.workunit.client.1.smithi104.stdout:2/352: dwrite d6/f10 [0,4194304] 0 2024-04-26T04:42:11.759 INFO:tasks.workunit.client.1.smithi104.stdout:2/353: creat d6/d1b/d51/d6a/f7d x:0 0 0 2024-04-26T04:42:11.759 INFO:tasks.workunit.client.1.smithi104.stdout:2/354: fsync d6/d8/d5a/d29/f69 0 2024-04-26T04:42:11.759 INFO:tasks.workunit.client.1.smithi104.stdout:3/227: link d5/d9/dc/c4b d5/d9/dc/d29/d49/c4e 0 2024-04-26T04:42:11.760 INFO:tasks.workunit.client.1.smithi104.stdout:4/210: getdents d4/d20 0 2024-04-26T04:42:11.761 INFO:tasks.workunit.client.1.smithi104.stdout:4/211: chown d4/d7/d1d/d1e/d15/c33 215211840 1 2024-04-26T04:42:11.761 INFO:tasks.workunit.client.1.smithi104.stdout:4/212: creat d4/d7/d1d/f45 x:0 0 0 2024-04-26T04:42:11.761 INFO:tasks.workunit.client.1.smithi104.stdout:4/213: write d4/d7/f2a [1288552,20362] 0 2024-04-26T04:42:11.761 INFO:tasks.workunit.client.1.smithi104.stdout:5/261: creat d0/d1/f52 x:0 0 0 2024-04-26T04:42:11.764 INFO:tasks.workunit.client.1.smithi104.stdout:3/228: symlink d5/d9/dc/d3a/l4f 0 2024-04-26T04:42:11.764 INFO:tasks.workunit.client.1.smithi104.stdout:3/229: write d5/d9/d17/f3c [639068,68641] 0 2024-04-26T04:42:11.764 INFO:tasks.workunit.client.1.smithi104.stdout:3/230: creat d5/d9/d17/f50 x:0 0 0 2024-04-26T04:42:11.764 INFO:tasks.workunit.client.1.smithi104.stdout:3/231: write d5/d9/dc/f40 [954203,87013] 0 2024-04-26T04:42:11.765 INFO:tasks.workunit.client.1.smithi104.stdout:4/214: creat d4/d7/d1d/d1e/d15/f46 x:0 0 0 2024-04-26T04:42:11.766 INFO:tasks.workunit.client.1.smithi104.stdout:5/262: mknod d0/d1/d4/d2a/d2b/d37/d48/c53 0 2024-04-26T04:42:11.766 INFO:tasks.workunit.client.1.smithi104.stdout:0/214: dwrite d0/d2/d21/d27/d1e/f2d [0,4194304] 0 2024-04-26T04:42:11.772 INFO:tasks.workunit.client.1.smithi104.stdout:3/232: dread d5/d9/f14 [0,4194304] 0 2024-04-26T04:42:11.774 INFO:tasks.workunit.client.1.smithi104.stdout:9/184: dwrite d11/f1a [0,4194304] 0 2024-04-26T04:42:11.774 INFO:tasks.workunit.client.1.smithi104.stdout:2/355: write d6/de/d1d/f43 [73228,28509] 0 2024-04-26T04:42:11.774 INFO:tasks.workunit.client.1.smithi104.stdout:2/356: write d6/d1b/d75/f7c [853782,114767] 0 2024-04-26T04:42:11.780 INFO:tasks.workunit.client.1.smithi104.stdout:5/263: dread d0/d11/f23 [0,4194304] 0 2024-04-26T04:42:11.783 INFO:tasks.workunit.client.1.smithi104.stdout:1/283: dwrite d0/d6/d19/d21/d28/f36 [0,4194304] 0 2024-04-26T04:42:11.783 INFO:tasks.workunit.client.1.smithi104.stdout:1/284: fdatasync d0/d6/ff 0 2024-04-26T04:42:11.783 INFO:tasks.workunit.client.1.smithi104.stdout:9/185: truncate d11/f16 1002966 0 2024-04-26T04:42:11.784 INFO:tasks.workunit.client.1.smithi104.stdout:2/357: mknod d6/de/c7e 0 2024-04-26T04:42:11.785 INFO:tasks.workunit.client.1.smithi104.stdout:5/264: mknod d0/d1/d4/d2a/d2b/c54 0 2024-04-26T04:42:11.785 INFO:tasks.workunit.client.1.smithi104.stdout:8/133: dwrite d0/d12/f1f [0,4194304] 0 2024-04-26T04:42:11.787 INFO:tasks.workunit.client.1.smithi104.stdout:2/358: rmdir d6/d1b/d68 39 2024-04-26T04:42:11.787 INFO:tasks.workunit.client.1.smithi104.stdout:2/359: readlink d6/d1b/l3a 0 2024-04-26T04:42:11.792 INFO:tasks.workunit.client.1.smithi104.stdout:2/360: mknod d6/de/d1d/c7f 0 2024-04-26T04:42:11.793 INFO:tasks.workunit.client.1.smithi104.stdout:2/361: write d6/de/f44 [1125981,125167] 0 2024-04-26T04:42:11.794 INFO:tasks.workunit.client.1.smithi104.stdout:9/186: write d11/d15/f18 [5832709,53562] 0 2024-04-26T04:42:11.795 INFO:tasks.workunit.client.1.smithi104.stdout:9/187: truncate f10 1056057 0 2024-04-26T04:42:11.795 INFO:tasks.workunit.client.1.smithi104.stdout:2/362: creat d6/de/d1d/f80 x:0 0 0 2024-04-26T04:42:11.801 INFO:tasks.workunit.client.1.smithi104.stdout:6/329: rename d0/d9/d5c to d0/d9/d13/d44/d71 0 2024-04-26T04:42:11.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/285: dwrite d0/d6/d2e/f46 [0,4194304] 0 2024-04-26T04:42:11.805 INFO:tasks.workunit.client.1.smithi104.stdout:6/330: rename d0/d9/d52/c6b to d0/d9/c72 0 2024-04-26T04:42:11.805 INFO:tasks.workunit.client.1.smithi104.stdout:6/331: write d0/d5/f67 [637797,102417] 0 2024-04-26T04:42:11.809 INFO:tasks.workunit.client.1.smithi104.stdout:1/286: rename d0/d6/l8 to d0/d6/de/d30/l57 0 2024-04-26T04:42:11.810 INFO:tasks.workunit.client.1.smithi104.stdout:6/332: creat d0/d9/d13/d41/f73 x:0 0 0 2024-04-26T04:42:11.811 INFO:tasks.workunit.client.1.smithi104.stdout:6/333: creat d0/d9/d13/d44/d71/f74 x:0 0 0 2024-04-26T04:42:11.812 INFO:tasks.workunit.client.1.smithi104.stdout:6/334: truncate d0/d9/d13/d15/f2b 514319 0 2024-04-26T04:42:11.813 INFO:tasks.workunit.client.1.smithi104.stdout:6/335: creat d0/d9/d52/f75 x:0 0 0 2024-04-26T04:42:11.814 INFO:tasks.workunit.client.1.smithi104.stdout:3/233: getdents d5/d9/d17 0 2024-04-26T04:42:11.815 INFO:tasks.workunit.client.1.smithi104.stdout:8/134: dwrite d0/d12/f1f [0,4194304] 0 2024-04-26T04:42:11.815 INFO:tasks.workunit.client.1.smithi104.stdout:1/287: dread d0/d6/de/d29/f39 [0,4194304] 0 2024-04-26T04:42:11.815 INFO:tasks.workunit.client.1.smithi104.stdout:8/135: truncate d0/d13/f25 743077 0 2024-04-26T04:42:11.815 INFO:tasks.workunit.client.1.smithi104.stdout:1/288: readlink d0/d6/de/d29/l3b 0 2024-04-26T04:42:11.818 INFO:tasks.workunit.client.1.smithi104.stdout:8/136: mkdir d0/d1c/d27 0 2024-04-26T04:42:11.818 INFO:tasks.workunit.client.1.smithi104.stdout:8/137: readlink d0/d13/l14 0 2024-04-26T04:42:11.818 INFO:tasks.workunit.client.1.smithi104.stdout:8/138: creat d0/d12/f28 x:0 0 0 2024-04-26T04:42:11.818 INFO:tasks.workunit.client.1.smithi104.stdout:8/139: fdatasync d0/f1e 0 2024-04-26T04:42:11.818 INFO:tasks.workunit.client.1.smithi104.stdout:8/140: stat d0/d12 0 2024-04-26T04:42:11.818 INFO:tasks.workunit.client.1.smithi104.stdout:2/363: dwrite d6/d1b/d51/d6a/f7d [0,4194304] 0 2024-04-26T04:42:11.818 INFO:tasks.workunit.client.1.smithi104.stdout:2/364: read d6/d8/d3e/f2a [3066132,49665] 0 2024-04-26T04:42:11.819 INFO:tasks.workunit.client.1.smithi104.stdout:5/265: truncate d0/d1/d4/f12 3913731 0 2024-04-26T04:42:11.822 INFO:tasks.workunit.client.1.smithi104.stdout:3/234: read f4 [943987,23021] 0 2024-04-26T04:42:11.823 INFO:tasks.workunit.client.1.smithi104.stdout:3/235: getdents d5/d9/dc/d29 0 2024-04-26T04:42:11.824 INFO:tasks.workunit.client.1.smithi104.stdout:3/236: fsync d5/d9/d17/f34 0 2024-04-26T04:42:11.824 INFO:tasks.workunit.client.1.smithi104.stdout:3/237: mknod d5/c51 0 2024-04-26T04:42:11.825 INFO:tasks.workunit.client.1.smithi104.stdout:8/141: dread d0/f1 [0,4194304] 0 2024-04-26T04:42:11.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/142: mknod d0/d1c/c29 0 2024-04-26T04:42:11.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/143: readlink d0/d13/l14 0 2024-04-26T04:42:11.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/144: chown d0/f1e 2960742 1 2024-04-26T04:42:11.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/145: truncate d0/f1e 545670 0 2024-04-26T04:42:11.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/146: fdatasync d0/d13/f25 0 2024-04-26T04:42:11.828 INFO:tasks.workunit.client.1.smithi104.stdout:3/238: rmdir d5/d9/d17/d46 39 2024-04-26T04:42:11.835 INFO:tasks.workunit.client.1.smithi104.stdout:7/126: sync 2024-04-26T04:42:11.837 INFO:tasks.workunit.client.1.smithi104.stdout:7/127: truncate d2/f20 111248 0 2024-04-26T04:42:11.845 INFO:tasks.workunit.client.1.smithi104.stdout:2/365: dwrite d6/d8/f17 [0,4194304] 0 2024-04-26T04:42:11.845 INFO:tasks.workunit.client.1.smithi104.stdout:2/366: truncate d6/d8/d5a/d29/f60 2337490 0 2024-04-26T04:42:11.847 INFO:tasks.workunit.client.1.smithi104.stdout:2/367: mkdir d6/d1b/d68/d81 0 2024-04-26T04:42:11.849 INFO:tasks.workunit.client.1.smithi104.stdout:5/266: dwrite d0/d1/f2 [0,4194304] 0 2024-04-26T04:42:11.850 INFO:tasks.workunit.client.1.smithi104.stdout:2/368: symlink d6/d8/d5a/d29/l82 0 2024-04-26T04:42:11.850 INFO:tasks.workunit.client.1.smithi104.stdout:2/369: fdatasync d6/d1b/d75/f7c 0 2024-04-26T04:42:11.853 INFO:tasks.workunit.client.1.smithi104.stdout:5/267: unlink d0/d1/d25/l39 0 2024-04-26T04:42:11.853 INFO:tasks.workunit.client.1.smithi104.stdout:5/268: creat d0/d41/f55 x:0 0 0 2024-04-26T04:42:11.853 INFO:tasks.workunit.client.1.smithi104.stdout:3/239: dwrite d5/d9/f26 [0,4194304] 0 2024-04-26T04:42:11.857 INFO:tasks.workunit.client.1.smithi104.stdout:3/240: link d5/l1a d5/d9/d17/d46/l52 0 2024-04-26T04:42:11.859 INFO:tasks.workunit.client.1.smithi104.stdout:3/241: rmdir d5/d9/d17/d46 39 2024-04-26T04:42:11.859 INFO:tasks.workunit.client.1.smithi104.stdout:3/242: chown d5/d9/c35 66463128 1 2024-04-26T04:42:11.859 INFO:tasks.workunit.client.1.smithi104.stdout:3/243: creat d5/d9/dc/d32/f53 x:0 0 0 2024-04-26T04:42:11.861 INFO:tasks.workunit.client.1.smithi104.stdout:3/244: rename d5/d9/dc/f1c to d5/f54 0 2024-04-26T04:42:11.861 INFO:tasks.workunit.client.1.smithi104.stdout:3/245: chown d5/f23 2150588 1 2024-04-26T04:42:11.862 INFO:tasks.workunit.client.1.smithi104.stdout:3/246: truncate d5/f3f 1477005 0 2024-04-26T04:42:11.862 INFO:tasks.workunit.client.1.smithi104.stdout:3/247: stat d5/d9/fa 0 2024-04-26T04:42:11.868 INFO:tasks.workunit.client.1.smithi104.stdout:5/269: dread d0/d1/d1e/f21 [0,4194304] 0 2024-04-26T04:42:11.868 INFO:tasks.workunit.client.1.smithi104.stdout:5/270: fdatasync d0/d11/d26/d36/f43 0 2024-04-26T04:42:11.868 INFO:tasks.workunit.client.1.smithi104.stdout:5/271: fdatasync d0/f6 0 2024-04-26T04:42:11.869 INFO:tasks.workunit.client.1.smithi104.stdout:5/272: creat d0/d1/d1e/f56 x:0 0 0 2024-04-26T04:42:11.878 INFO:tasks.workunit.client.1.smithi104.stdout:3/248: dread d5/d9/f12 [0,4194304] 0 2024-04-26T04:42:11.885 INFO:tasks.workunit.client.1.smithi104.stdout:3/249: read f4 [5781066,101890] 0 2024-04-26T04:42:11.887 INFO:tasks.workunit.client.1.smithi104.stdout:3/250: rename d5/d9/d17/f50 to d5/d9/dc/d29/d2d/f55 0 2024-04-26T04:42:11.896 INFO:tasks.workunit.client.1.smithi104.stdout:1/289: dwrite d0/d6/d2e/f46 [4194304,4194304] 0 2024-04-26T04:42:11.896 INFO:tasks.workunit.client.1.smithi104.stdout:7/128: dwrite d2/f1c [0,4194304] 0 2024-04-26T04:42:11.896 INFO:tasks.workunit.client.1.smithi104.stdout:8/147: dwrite d0/fe [4194304,4194304] 0 2024-04-26T04:42:11.896 INFO:tasks.workunit.client.1.smithi104.stdout:8/148: write d0/d1c/f26 [488122,5699] 0 2024-04-26T04:42:11.897 INFO:tasks.workunit.client.1.smithi104.stdout:6/336: dwrite d0/d9/d13/d15/f2b [0,4194304] 0 2024-04-26T04:42:11.897 INFO:tasks.workunit.client.1.smithi104.stdout:5/273: dwrite d0/d41/f4d [0,4194304] 0 2024-04-26T04:42:11.897 INFO:tasks.workunit.client.1.smithi104.stdout:2/370: dwrite d6/d8/f78 [0,4194304] 0 2024-04-26T04:42:11.897 INFO:tasks.workunit.client.1.smithi104.stdout:5/274: chown d0/d11/d26/l47 7536 1 2024-04-26T04:42:11.897 INFO:tasks.workunit.client.1.smithi104.stdout:2/371: creat d6/de/d1d/f83 x:0 0 0 2024-04-26T04:42:11.897 INFO:tasks.workunit.client.1.smithi104.stdout:2/372: write d6/d8/d3e/f7a [371886,63868] 0 2024-04-26T04:42:11.898 INFO:tasks.workunit.client.1.smithi104.stdout:1/290: mkdir d0/d6/d19/d21/d58 0 2024-04-26T04:42:11.898 INFO:tasks.workunit.client.1.smithi104.stdout:1/291: stat d0/d6 0 2024-04-26T04:42:11.899 INFO:tasks.workunit.client.1.smithi104.stdout:8/149: dread d0/f1a [0,4194304] 0 2024-04-26T04:42:11.899 INFO:tasks.workunit.client.1.smithi104.stdout:8/150: read - d0/d13/f1d zero size 2024-04-26T04:42:11.899 INFO:tasks.workunit.client.1.smithi104.stdout:8/151: write d0/f9 [666491,51228] 0 2024-04-26T04:42:11.906 INFO:tasks.workunit.client.1.smithi104.stdout:3/251: dwrite d5/f25 [0,4194304] 0 2024-04-26T04:42:11.906 INFO:tasks.workunit.client.1.smithi104.stdout:3/252: readlink d5/d9/dc/l18 0 2024-04-26T04:42:11.908 INFO:tasks.workunit.client.1.smithi104.stdout:2/373: dread d6/d8/d5a/f2e [0,4194304] 0 2024-04-26T04:42:11.908 INFO:tasks.workunit.client.1.smithi104.stdout:2/374: stat d6/d1b/d75/f7c 0 2024-04-26T04:42:11.908 INFO:tasks.workunit.client.1.smithi104.stdout:2/375: read - d6/f33 zero size 2024-04-26T04:42:11.918 INFO:tasks.workunit.client.1.smithi104.stdout:6/337: rename d0/d9/d13/d44/l56 to d0/l76 0 2024-04-26T04:42:11.920 INFO:tasks.workunit.client.1.smithi104.stdout:5/275: dwrite d0/d1/d4/d2a/d2b/f35 [0,4194304] 0 2024-04-26T04:42:11.923 INFO:tasks.workunit.client.1.smithi104.stdout:7/129: rename d2/f20 to d2/f28 0 2024-04-26T04:42:11.927 INFO:tasks.workunit.client.1.smithi104.stdout:5/276: write d0/f22 [1549198,9641] 0 2024-04-26T04:42:11.929 INFO:tasks.workunit.client.1.smithi104.stdout:3/253: creat d5/d9/dc/d29/f56 x:0 0 0 2024-04-26T04:42:11.929 INFO:tasks.workunit.client.1.smithi104.stdout:3/254: stat d5/d9/f14 0 2024-04-26T04:42:11.930 INFO:tasks.workunit.client.1.smithi104.stdout:5/277: dread d0/d11/f31 [0,4194304] 0 2024-04-26T04:42:11.931 INFO:tasks.workunit.client.1.smithi104.stdout:2/376: mknod d6/d8/d3e/c84 0 2024-04-26T04:42:11.931 INFO:tasks.workunit.client.1.smithi104.stdout:6/338: symlink d0/d9/d13/d44/d47/l77 0 2024-04-26T04:42:11.934 INFO:tasks.workunit.client.1.smithi104.stdout:3/255: creat d5/d9/dc/d3a/f57 x:0 0 0 2024-04-26T04:42:11.936 INFO:tasks.workunit.client.1.smithi104.stdout:5/278: link d0/d11/f23 d0/d1/d4/d2a/d2b/d37/f57 0 2024-04-26T04:42:11.938 INFO:tasks.workunit.client.1.smithi104.stdout:2/377: mkdir d6/de/d85 0 2024-04-26T04:42:11.938 INFO:tasks.workunit.client.1.smithi104.stdout:2/378: fsync d6/d8/d3e/f26 0 2024-04-26T04:42:11.938 INFO:tasks.workunit.client.1.smithi104.stdout:2/379: truncate d6/f63 5138070 0 2024-04-26T04:42:11.938 INFO:tasks.workunit.client.1.smithi104.stdout:2/380: write d6/f4b [862511,2751] 0 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:6/339: symlink d0/d9/d13/d44/d71/d3f/l78 0 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:6/340: fdatasync d0/d9/d13/d44/d71/f69 0 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:6/341: chown d0/d9/d13/d15/f5e 89441220 1 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:6/342: fsync d0/d5/f22 0 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:6/343: fsync d0/d9/d13/d15/f42 0 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:7/130: dwrite d2/f16 [0,4194304] 0 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:7/131: chown d2/d4/c12 707385679 1 2024-04-26T04:42:11.940 INFO:tasks.workunit.client.1.smithi104.stdout:6/344: write d0/d5/f67 [421803,105395] 0 2024-04-26T04:42:11.943 INFO:tasks.workunit.client.1.smithi104.stdout:5/279: truncate d0/d11/f31 157637 0 2024-04-26T04:42:11.946 INFO:tasks.workunit.client.1.smithi104.stdout:3/256: rmdir d5/d9/dc/d32 39 2024-04-26T04:42:11.946 INFO:tasks.workunit.client.1.smithi104.stdout:3/257: chown d5/d9/dc/d29/d49 482 1 2024-04-26T04:42:11.946 INFO:tasks.workunit.client.1.smithi104.stdout:3/258: chown d5/d9/dc/d29/d49/c4e 2 1 2024-04-26T04:42:11.946 INFO:tasks.workunit.client.1.smithi104.stdout:3/259: chown d5/d9/dc/d29/d49/c4e 22307 1 2024-04-26T04:42:11.946 INFO:tasks.workunit.client.1.smithi104.stdout:3/260: fdatasync f0 0 2024-04-26T04:42:11.947 INFO:tasks.workunit.client.1.smithi104.stdout:3/261: write d5/d9/d17/f3b [745054,99930] 0 2024-04-26T04:42:11.947 INFO:tasks.workunit.client.1.smithi104.stdout:3/262: chown d5/d9/d17/d46 13947 1 2024-04-26T04:42:11.947 INFO:tasks.workunit.client.1.smithi104.stdout:4/215: sync 2024-04-26T04:42:11.947 INFO:tasks.workunit.client.1.smithi104.stdout:0/215: sync 2024-04-26T04:42:11.948 INFO:tasks.workunit.client.1.smithi104.stdout:4/216: truncate d4/d7/f42 994188 0 2024-04-26T04:42:11.949 INFO:tasks.workunit.client.1.smithi104.stdout:3/263: link d5/d9/f4a d5/d9/dc/d29/f58 0 2024-04-26T04:42:11.949 INFO:tasks.workunit.client.1.smithi104.stdout:3/264: stat d5/d9/f4a 0 2024-04-26T04:42:11.951 INFO:tasks.workunit.client.1.smithi104.stdout:4/217: mknod d4/d7/d1d/d1e/d15/c47 0 2024-04-26T04:42:11.953 INFO:tasks.workunit.client.1.smithi104.stdout:6/345: rename d0/d9/d52 to d0/d9/d13/d44/d71/d3f/d79 0 2024-04-26T04:42:11.953 INFO:tasks.workunit.client.1.smithi104.stdout:6/346: write d0/d9/f2a [3419746,23977] 0 2024-04-26T04:42:11.955 INFO:tasks.workunit.client.1.smithi104.stdout:4/218: mkdir d4/d7/d1d/d1e/d24/d48 0 2024-04-26T04:42:11.957 INFO:tasks.workunit.client.1.smithi104.stdout:0/216: rename d0/d4/d15/f18 to d0/d2/d21/f45 0 2024-04-26T04:42:11.961 INFO:tasks.workunit.client.1.smithi104.stdout:4/219: link d4/d20/l2d d4/d7/d1d/d1e/d24/d48/l49 0 2024-04-26T04:42:11.964 INFO:tasks.workunit.client.1.smithi104.stdout:6/347: rmdir d0/d1b 39 2024-04-26T04:42:11.964 INFO:tasks.workunit.client.1.smithi104.stdout:6/348: readlink d0/d9/d25/l2f 0 2024-04-26T04:42:11.964 INFO:tasks.workunit.client.1.smithi104.stdout:4/220: write d4/d7/ff [909432,61335] 0 2024-04-26T04:42:11.965 INFO:tasks.workunit.client.1.smithi104.stdout:0/217: getdents d0/d2 0 2024-04-26T04:42:11.966 INFO:tasks.workunit.client.1.smithi104.stdout:7/132: dwrite d2/f3 [4194304,4194304] 0 2024-04-26T04:42:11.966 INFO:tasks.workunit.client.1.smithi104.stdout:2/381: dwrite d6/d8/d5a/d29/f60 [0,4194304] 0 2024-04-26T04:42:11.966 INFO:tasks.workunit.client.1.smithi104.stdout:3/265: dwrite d5/d9/d17/f34 [0,4194304] 0 2024-04-26T04:42:11.967 INFO:tasks.workunit.client.1.smithi104.stdout:2/382: dread d6/d1b/f38 [0,4194304] 0 2024-04-26T04:42:11.967 INFO:tasks.workunit.client.1.smithi104.stdout:7/133: chown d2/l5 0 1 2024-04-26T04:42:11.973 INFO:tasks.workunit.client.1.smithi104.stdout:0/218: dread d0/d2/d21/d38/f36 [0,4194304] 0 2024-04-26T04:42:11.976 INFO:tasks.workunit.client.1.smithi104.stdout:6/349: write d0/d9/d13/d15/f2b [2088372,104319] 0 2024-04-26T04:42:11.978 INFO:tasks.workunit.client.1.smithi104.stdout:4/221: dread d4/d20/f36 [0,4194304] 0 2024-04-26T04:42:11.980 INFO:tasks.workunit.client.1.smithi104.stdout:7/134: write d2/d4/f11 [1531518,63455] 0 2024-04-26T04:42:11.980 INFO:tasks.workunit.client.1.smithi104.stdout:7/135: chown d2/d4/c12 224 1 2024-04-26T04:42:11.981 INFO:tasks.workunit.client.1.smithi104.stdout:2/383: getdents d6/d1b/d68 0 2024-04-26T04:42:11.984 INFO:tasks.workunit.client.1.smithi104.stdout:2/384: dread d6/f4b [0,4194304] 0 2024-04-26T04:42:11.988 INFO:tasks.workunit.client.1.smithi104.stdout:4/222: write d4/d7/f8 [450059,104973] 0 2024-04-26T04:42:11.989 INFO:tasks.workunit.client.1.smithi104.stdout:4/223: truncate d4/d7/d1d/d1e/f39 3493143 0 2024-04-26T04:42:11.991 INFO:tasks.workunit.client.1.smithi104.stdout:3/266: dwrite f4 [8388608,4194304] 0 2024-04-26T04:42:11.993 INFO:tasks.workunit.client.1.smithi104.stdout:3/267: mkdir d5/d9/dc/d29/d49/d59 0 2024-04-26T04:42:11.993 INFO:tasks.workunit.client.1.smithi104.stdout:3/268: write d5/d9/dc/d29/f58 [217834,76365] 0 2024-04-26T04:42:11.993 INFO:tasks.workunit.client.1.smithi104.stdout:3/269: fdatasync d5/d9/ff 0 2024-04-26T04:42:11.993 INFO:tasks.workunit.client.1.smithi104.stdout:3/270: dread - d5/d9/dc/d29/f56 zero size 2024-04-26T04:42:11.994 INFO:tasks.workunit.client.1.smithi104.stdout:3/271: stat d5/d9/dc/f19 0 2024-04-26T04:42:11.995 INFO:tasks.workunit.client.1.smithi104.stdout:3/272: creat d5/d9/dc/d32/f5a x:0 0 0 2024-04-26T04:42:11.995 INFO:tasks.workunit.client.1.smithi104.stdout:3/273: write d5/d9/dc/d32/f53 [151317,25384] 0 2024-04-26T04:42:11.996 INFO:tasks.workunit.client.1.smithi104.stdout:3/274: mknod d5/c5b 0 2024-04-26T04:42:11.998 INFO:tasks.workunit.client.1.smithi104.stdout:3/275: link d5/d9/dc/d29/d2d/f55 d5/d9/d17/f5c 0 2024-04-26T04:42:11.998 INFO:tasks.workunit.client.1.smithi104.stdout:3/276: stat d5/d9/dc/d29 0 2024-04-26T04:42:11.998 INFO:tasks.workunit.client.1.smithi104.stdout:3/277: creat d5/d9/f5d x:0 0 0 2024-04-26T04:42:12.000 INFO:tasks.workunit.client.1.smithi104.stdout:3/278: link d5/d9/dc/l18 d5/d9/d17/d46/l5e 0 2024-04-26T04:42:12.001 INFO:tasks.workunit.client.1.smithi104.stdout:9/188: sync 2024-04-26T04:42:12.002 INFO:tasks.workunit.client.1.smithi104.stdout:9/189: truncate d11/d15/f18 4476413 0 2024-04-26T04:42:12.005 INFO:tasks.workunit.client.1.smithi104.stdout:0/219: dwrite d0/d4/f11 [0,4194304] 0 2024-04-26T04:42:12.005 INFO:tasks.workunit.client.1.smithi104.stdout:0/220: read d0/f42 [5279,64332] 0 2024-04-26T04:42:12.007 INFO:tasks.workunit.client.1.smithi104.stdout:0/221: rename d0/d2/fd to d0/d4/d3c/f46 0 2024-04-26T04:42:12.010 INFO:tasks.workunit.client.1.smithi104.stdout:0/222: dread d0/d2/d21/d27/f30 [0,4194304] 0 2024-04-26T04:42:12.011 INFO:tasks.workunit.client.1.smithi104.stdout:0/223: mknod d0/d4/d15/c47 0 2024-04-26T04:42:12.012 INFO:tasks.workunit.client.1.smithi104.stdout:6/350: dwrite d0/d9/d13/d44/d71/d3f/d79/f75 [0,4194304] 0 2024-04-26T04:42:12.013 INFO:tasks.workunit.client.1.smithi104.stdout:4/224: dwrite d4/f37 [8388608,4194304] 0 2024-04-26T04:42:12.015 INFO:tasks.workunit.client.1.smithi104.stdout:3/279: fsync d5/d9/dc/d32/f5a 0 2024-04-26T04:42:12.015 INFO:tasks.workunit.client.1.smithi104.stdout:3/280: readlink d5/d9/dc/d29/l3d 0 2024-04-26T04:42:12.017 INFO:tasks.workunit.client.1.smithi104.stdout:7/136: dread d2/d4/f11 [0,4194304] 0 2024-04-26T04:42:12.018 INFO:tasks.workunit.client.1.smithi104.stdout:3/281: creat d5/d9/dc/d32/d48/f5f x:0 0 0 2024-04-26T04:42:12.022 INFO:tasks.workunit.client.1.smithi104.stdout:3/282: mkdir d5/d9/d17/d60 0 2024-04-26T04:42:12.022 INFO:tasks.workunit.client.1.smithi104.stdout:3/283: truncate d5/d9/d17/f3c 1602924 0 2024-04-26T04:42:12.022 INFO:tasks.workunit.client.1.smithi104.stdout:6/351: dread d0/d9/d25/f3e [0,4194304] 0 2024-04-26T04:42:12.022 INFO:tasks.workunit.client.1.smithi104.stdout:4/225: read d4/d7/f42 [668855,93265] 0 2024-04-26T04:42:12.022 INFO:tasks.workunit.client.1.smithi104.stdout:6/352: write d0/d9/d13/d15/f5e [797936,85598] 0 2024-04-26T04:42:12.023 INFO:tasks.workunit.client.1.smithi104.stdout:4/226: mkdir d4/d7/d1d/d1e/d24/d4a 0 2024-04-26T04:42:12.023 INFO:tasks.workunit.client.1.smithi104.stdout:6/353: read d0/d9/d13/d44/d71/f69 [43199,52036] 0 2024-04-26T04:42:12.024 INFO:tasks.workunit.client.1.smithi104.stdout:3/284: dread d5/d9/d17/f1b [0,4194304] 0 2024-04-26T04:42:12.024 INFO:tasks.workunit.client.1.smithi104.stdout:0/224: dread d0/fe [0,4194304] 0 2024-04-26T04:42:12.025 INFO:tasks.workunit.client.1.smithi104.stdout:4/227: rename d4/f11 to d4/db/f4b 0 2024-04-26T04:42:12.028 INFO:tasks.workunit.client.1.smithi104.stdout:4/228: rmdir d4/d7/d1d/d1e/d24 39 2024-04-26T04:42:12.033 INFO:tasks.workunit.client.1.smithi104.stdout:3/285: rmdir d5/d9/dc 39 2024-04-26T04:42:12.033 INFO:tasks.workunit.client.1.smithi104.stdout:3/286: write d5/d9/dc/d29/d2d/f3e [952261,96630] 0 2024-04-26T04:42:12.034 INFO:tasks.workunit.client.1.smithi104.stdout:0/225: write d0/d2/d21/d38/f34 [3235417,54716] 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:3/287: symlink d5/d9/d17/d60/l61 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/354: dread d0/d5/f22 [0,4194304] 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/355: read - d0/d9/d13/d44/d71/d3f/f5d zero size 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/356: creat d0/d9/d13/d41/f7a x:0 0 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/357: write d0/f6e [1110075,41541] 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/358: stat d0/d9/d25/f3e 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/359: fdatasync d0/d9/d25/f3e 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/360: truncate d0/d9/d13/f43 1043428 0 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/361: chown d0/d9/d13/d44/d71/f19 140910576 1 2024-04-26T04:42:12.036 INFO:tasks.workunit.client.1.smithi104.stdout:6/362: write d0/d9/d13/d44/f58 [607584,80159] 0 2024-04-26T04:42:12.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/363: write d0/d9/d13/d44/f58 [341830,62304] 0 2024-04-26T04:42:12.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/364: stat d0/d9/d13/d15/f42 0 2024-04-26T04:42:12.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/365: chown d0/f6e 5 1 2024-04-26T04:42:12.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/366: readlink d0/l76 0 2024-04-26T04:42:12.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/367: readlink d0/d9/d13/d15/l3c 0 2024-04-26T04:42:12.037 INFO:tasks.workunit.client.1.smithi104.stdout:4/229: rename d4/d20/d3c to d4/d7/d4c 0 2024-04-26T04:42:12.040 INFO:tasks.workunit.client.1.smithi104.stdout:6/368: mkdir d0/d7b 0 2024-04-26T04:42:12.041 INFO:tasks.workunit.client.1.smithi104.stdout:6/369: read d0/d9/d13/f3d [4336628,42072] 0 2024-04-26T04:42:12.041 INFO:tasks.workunit.client.1.smithi104.stdout:0/226: rename d0/d4/l43 to d0/d2/d21/d27/l48 0 2024-04-26T04:42:12.042 INFO:tasks.workunit.client.1.smithi104.stdout:4/230: unlink d4/d7/d1d/d1e/d15/c2b 0 2024-04-26T04:42:12.043 INFO:tasks.workunit.client.1.smithi104.stdout:0/227: rename d0/d4/d15/d3e to d0/d2/d21/d49 0 2024-04-26T04:42:12.043 INFO:tasks.workunit.client.1.smithi104.stdout:7/137: dwrite d2/f3 [4194304,4194304] 0 2024-04-26T04:42:12.043 INFO:tasks.workunit.client.1.smithi104.stdout:7/138: write d2/f16 [4675939,31355] 0 2024-04-26T04:42:12.045 INFO:tasks.workunit.client.1.smithi104.stdout:4/231: mknod d4/d7/d1d/d2e/c4d 0 2024-04-26T04:42:12.048 INFO:tasks.workunit.client.1.smithi104.stdout:4/232: creat d4/d7/d1d/d1e/d24/d48/f4e x:0 0 0 2024-04-26T04:42:12.049 INFO:tasks.workunit.client.1.smithi104.stdout:3/288: dread d5/d9/fa [0,4194304] 0 2024-04-26T04:42:12.049 INFO:tasks.workunit.client.1.smithi104.stdout:3/289: write d5/d9/dc/d29/d2d/f55 [16035,32271] 0 2024-04-26T04:42:12.049 INFO:tasks.workunit.client.1.smithi104.stdout:3/290: fdatasync d5/d9/d17/f2a 0 2024-04-26T04:42:12.050 INFO:tasks.workunit.client.1.smithi104.stdout:3/291: dread d5/d9/d17/f1b [0,4194304] 0 2024-04-26T04:42:12.050 INFO:tasks.workunit.client.1.smithi104.stdout:4/233: dread d4/d7/d4c/f31 [0,4194304] 0 2024-04-26T04:42:12.052 INFO:tasks.workunit.client.1.smithi104.stdout:4/234: symlink d4/d7/d1d/d1e/d24/l4f 0 2024-04-26T04:42:12.054 INFO:tasks.workunit.client.1.smithi104.stdout:0/228: dread d0/fa [0,4194304] 0 2024-04-26T04:42:12.056 INFO:tasks.workunit.client.1.smithi104.stdout:4/235: creat d4/d7/d1d/d1e/d24/d4a/f50 x:0 0 0 2024-04-26T04:42:12.056 INFO:tasks.workunit.client.1.smithi104.stdout:4/236: write d4/db/fe [674410,39156] 0 2024-04-26T04:42:12.058 INFO:tasks.workunit.client.1.smithi104.stdout:0/229: link d0/d2/c13 d0/c4a 0 2024-04-26T04:42:12.059 INFO:tasks.workunit.client.1.smithi104.stdout:4/237: rename d4/d7/d1d/d1e/f39 to d4/d20/f51 0 2024-04-26T04:42:12.060 INFO:tasks.workunit.client.1.smithi104.stdout:0/230: getdents d0/d4/d3c 0 2024-04-26T04:42:12.062 INFO:tasks.workunit.client.1.smithi104.stdout:4/238: stat d4/c5 0 2024-04-26T04:42:12.063 INFO:tasks.workunit.client.1.smithi104.stdout:4/239: mknod d4/d7/d1d/c52 0 2024-04-26T04:42:12.063 INFO:tasks.workunit.client.1.smithi104.stdout:4/240: chown d4/d7/f8 32 1 2024-04-26T04:42:12.063 INFO:tasks.workunit.client.1.smithi104.stdout:4/241: creat d4/d7/f53 x:0 0 0 2024-04-26T04:42:12.064 INFO:tasks.workunit.client.1.smithi104.stdout:4/242: fsync d4/d20/f36 0 2024-04-26T04:42:12.065 INFO:tasks.workunit.client.1.smithi104.stdout:6/370: dwrite d0/d1b/f2c [0,4194304] 0 2024-04-26T04:42:12.065 INFO:tasks.workunit.client.1.smithi104.stdout:4/243: mknod d4/d7/d1d/d1e/d15/c54 0 2024-04-26T04:42:12.065 INFO:tasks.workunit.client.1.smithi104.stdout:4/244: write d4/d7/d1d/d1e/d24/d48/f4e [485740,17265] 0 2024-04-26T04:42:12.070 INFO:tasks.workunit.client.1.smithi104.stdout:6/371: creat d0/d1b/f7c x:0 0 0 2024-04-26T04:42:12.071 INFO:tasks.workunit.client.1.smithi104.stdout:6/372: mknod d0/d9/d13/d44/d71/d3f/c7d 0 2024-04-26T04:42:12.074 INFO:tasks.workunit.client.1.smithi104.stdout:7/139: dwrite d2/d4/f10 [0,4194304] 0 2024-04-26T04:42:12.074 INFO:tasks.workunit.client.1.smithi104.stdout:7/140: creat d2/d4/d1d/f29 x:0 0 0 2024-04-26T04:42:12.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/231: write d0/d4/f11 [375999,61673] 0 2024-04-26T04:42:12.075 INFO:tasks.workunit.client.1.smithi104.stdout:3/292: dwrite d5/d9/dc/d3a/f57 [0,4194304] 0 2024-04-26T04:42:12.075 INFO:tasks.workunit.client.1.smithi104.stdout:1/292: sync 2024-04-26T04:42:12.082 INFO:tasks.workunit.client.1.smithi104.stdout:3/293: symlink d5/d9/d17/l62 0 2024-04-26T04:42:12.083 INFO:tasks.workunit.client.1.smithi104.stdout:7/141: symlink d2/l2a 0 2024-04-26T04:42:12.086 INFO:tasks.workunit.client.1.smithi104.stdout:3/294: symlink d5/d9/d17/d46/l63 0 2024-04-26T04:42:12.089 INFO:tasks.workunit.client.1.smithi104.stdout:7/142: rename d2/f3 to d2/f2b 0 2024-04-26T04:42:12.089 INFO:tasks.workunit.client.1.smithi104.stdout:7/143: read d2/f28 [505616,93636] 0 2024-04-26T04:42:12.090 INFO:tasks.workunit.client.1.smithi104.stdout:7/144: rename d2/d4/l24 to d2/d4/d1d/l2c 0 2024-04-26T04:42:12.092 INFO:tasks.workunit.client.1.smithi104.stdout:7/145: symlink d2/d4/l2d 0 2024-04-26T04:42:12.093 INFO:tasks.workunit.client.1.smithi104.stdout:4/245: dwrite d4/d20/f36 [0,4194304] 0 2024-04-26T04:42:12.094 INFO:tasks.workunit.client.1.smithi104.stdout:7/146: truncate d2/f2b 6711554 0 2024-04-26T04:42:12.098 INFO:tasks.workunit.client.1.smithi104.stdout:0/232: dwrite d0/f42 [0,4194304] 0 2024-04-26T04:42:12.099 INFO:tasks.workunit.client.1.smithi104.stdout:0/233: chown d0/d4/d15/l25 1479 1 2024-04-26T04:42:12.099 INFO:tasks.workunit.client.1.smithi104.stdout:3/295: dread d5/d9/dc/f28 [0,4194304] 0 2024-04-26T04:42:12.106 INFO:tasks.workunit.client.1.smithi104.stdout:7/147: write d2/d4/f10 [3488087,35722] 0 2024-04-26T04:42:12.106 INFO:tasks.workunit.client.1.smithi104.stdout:7/148: write d2/d4/d1d/f29 [339032,30565] 0 2024-04-26T04:42:12.107 INFO:tasks.workunit.client.1.smithi104.stdout:7/149: chown d2/d4/f11 83342180 1 2024-04-26T04:42:12.112 INFO:tasks.workunit.client.1.smithi104.stdout:7/150: mknod d2/c2e 0 2024-04-26T04:42:12.112 INFO:tasks.workunit.client.1.smithi104.stdout:7/151: chown d2/d4/c12 2086 1 2024-04-26T04:42:12.114 INFO:tasks.workunit.client.1.smithi104.stdout:7/152: link d2/d4/f10 d2/f2f 0 2024-04-26T04:42:12.114 INFO:tasks.workunit.client.1.smithi104.stdout:7/153: chown d2/c23 158 1 2024-04-26T04:42:12.118 INFO:tasks.workunit.client.1.smithi104.stdout:4/246: dwrite d4/d7/f53 [0,4194304] 0 2024-04-26T04:42:12.119 INFO:tasks.workunit.client.1.smithi104.stdout:3/296: write d5/f25 [973397,77775] 0 2024-04-26T04:42:12.120 INFO:tasks.workunit.client.1.smithi104.stdout:0/234: dwrite d0/fe [4194304,4194304] 0 2024-04-26T04:42:12.120 INFO:tasks.workunit.client.1.smithi104.stdout:0/235: fsync d0/f2f 0 2024-04-26T04:42:12.121 INFO:tasks.workunit.client.1.smithi104.stdout:4/247: mkdir d4/d20/d55 0 2024-04-26T04:42:12.121 INFO:tasks.workunit.client.1.smithi104.stdout:4/248: fdatasync d4/d7/d1d/f1c 0 2024-04-26T04:42:12.125 INFO:tasks.workunit.client.1.smithi104.stdout:3/297: link d5/d9/dc/d29/c2e d5/d9/dc/d29/d2d/c64 0 2024-04-26T04:42:12.127 INFO:tasks.workunit.client.1.smithi104.stdout:0/236: creat d0/d4/d3c/f4b x:0 0 0 2024-04-26T04:42:12.128 INFO:tasks.workunit.client.1.smithi104.stdout:3/298: mknod d5/d9/dc/c65 0 2024-04-26T04:42:12.129 INFO:tasks.workunit.client.1.smithi104.stdout:3/299: dread - d5/d9/f5d zero size 2024-04-26T04:42:12.129 INFO:tasks.workunit.client.1.smithi104.stdout:0/237: creat d0/d2/d10/d3d/f4c x:0 0 0 2024-04-26T04:42:12.130 INFO:tasks.workunit.client.1.smithi104.stdout:0/238: dread - d0/f41 zero size 2024-04-26T04:42:12.130 INFO:tasks.workunit.client.1.smithi104.stdout:3/300: dread d5/d9/dc/d32/f53 [0,4194304] 0 2024-04-26T04:42:12.131 INFO:tasks.workunit.client.1.smithi104.stdout:3/301: mknod d5/d9/d17/c66 0 2024-04-26T04:42:12.132 INFO:tasks.workunit.client.1.smithi104.stdout:3/302: fsync d5/f24 0 2024-04-26T04:42:12.132 INFO:tasks.workunit.client.1.smithi104.stdout:3/303: dread - d5/d9/dc/f36 zero size 2024-04-26T04:42:12.139 INFO:tasks.workunit.client.1.smithi104.stdout:0/239: dread d0/fe [4194304,4194304] 0 2024-04-26T04:42:12.139 INFO:tasks.workunit.client.1.smithi104.stdout:0/240: readlink d0/d4/d15/l25 0 2024-04-26T04:42:12.141 INFO:tasks.workunit.client.1.smithi104.stdout:7/154: dwrite d2/d4/d1d/f29 [0,4194304] 0 2024-04-26T04:42:12.141 INFO:tasks.workunit.client.1.smithi104.stdout:7/155: creat d2/d4/d1d/f30 x:0 0 0 2024-04-26T04:42:12.141 INFO:tasks.workunit.client.1.smithi104.stdout:0/241: rmdir d0/d2/d21/d27 39 2024-04-26T04:42:12.142 INFO:tasks.workunit.client.1.smithi104.stdout:4/249: dwrite d4/db/f12 [0,4194304] 0 2024-04-26T04:42:12.145 INFO:tasks.workunit.client.1.smithi104.stdout:7/156: unlink d2/f16 0 2024-04-26T04:42:12.152 INFO:tasks.workunit.client.1.smithi104.stdout:4/250: rename d4/d7/d4c to d4/d20/d55/d56 0 2024-04-26T04:42:12.153 INFO:tasks.workunit.client.1.smithi104.stdout:4/251: symlink d4/d20/d55/l57 0 2024-04-26T04:42:12.154 INFO:tasks.workunit.client.1.smithi104.stdout:4/252: creat d4/d20/d55/d56/f58 x:0 0 0 2024-04-26T04:42:12.157 INFO:tasks.workunit.client.1.smithi104.stdout:2/385: sync 2024-04-26T04:42:12.157 INFO:tasks.workunit.client.1.smithi104.stdout:8/152: sync 2024-04-26T04:42:12.157 INFO:tasks.workunit.client.1.smithi104.stdout:5/280: sync 2024-04-26T04:42:12.158 INFO:tasks.workunit.client.1.smithi104.stdout:8/153: mknod d0/d1c/d27/c2a 0 2024-04-26T04:42:12.158 INFO:tasks.workunit.client.1.smithi104.stdout:8/154: readlink d0/l10 0 2024-04-26T04:42:12.159 INFO:tasks.workunit.client.1.smithi104.stdout:8/155: creat d0/d12/f2b x:0 0 0 2024-04-26T04:42:12.159 INFO:tasks.workunit.client.1.smithi104.stdout:8/156: creat d0/d12/f2c x:0 0 0 2024-04-26T04:42:12.159 INFO:tasks.workunit.client.1.smithi104.stdout:2/386: symlink d6/d1b/d68/d81/l86 0 2024-04-26T04:42:12.160 INFO:tasks.workunit.client.1.smithi104.stdout:0/242: dwrite d0/d2/d21/d27/d1e/f2d [0,4194304] 0 2024-04-26T04:42:12.160 INFO:tasks.workunit.client.1.smithi104.stdout:0/243: stat d0/d2/d21/d27/d1e 0 2024-04-26T04:42:12.165 INFO:tasks.workunit.client.1.smithi104.stdout:8/157: mkdir d0/d1c/d2d 0 2024-04-26T04:42:12.165 INFO:tasks.workunit.client.1.smithi104.stdout:8/158: dread - d0/d12/f28 zero size 2024-04-26T04:42:12.165 INFO:tasks.workunit.client.1.smithi104.stdout:8/159: creat d0/f2e x:0 0 0 2024-04-26T04:42:12.165 INFO:tasks.workunit.client.1.smithi104.stdout:8/160: creat d0/f2f x:0 0 0 2024-04-26T04:42:12.166 INFO:tasks.workunit.client.1.smithi104.stdout:8/161: chown d0/d13/f1d 65693128 1 2024-04-26T04:42:12.168 INFO:tasks.workunit.client.1.smithi104.stdout:5/281: write d0/d41/f4d [2035073,88771] 0 2024-04-26T04:42:12.168 INFO:tasks.workunit.client.1.smithi104.stdout:8/162: mknod d0/d1c/d2d/c30 0 2024-04-26T04:42:12.171 INFO:tasks.workunit.client.1.smithi104.stdout:2/387: read d6/f35 [1940957,42416] 0 2024-04-26T04:42:12.172 INFO:tasks.workunit.client.1.smithi104.stdout:5/282: rename d0/d1/d4/d2a/d2b/d37/f3f to d0/f58 0 2024-04-26T04:42:12.173 INFO:tasks.workunit.client.1.smithi104.stdout:2/388: dread d6/d8/f70 [0,4194304] 0 2024-04-26T04:42:12.174 INFO:tasks.workunit.client.1.smithi104.stdout:8/163: rename d0/f1 to d0/d12/f31 0 2024-04-26T04:42:12.175 INFO:tasks.workunit.client.1.smithi104.stdout:2/389: getdents d6/d1b/d51 0 2024-04-26T04:42:12.175 INFO:tasks.workunit.client.1.smithi104.stdout:8/164: creat d0/d1c/d2d/f32 x:0 0 0 2024-04-26T04:42:12.176 INFO:tasks.workunit.client.1.smithi104.stdout:2/390: stat d6/d1b/d51/l65 0 2024-04-26T04:42:12.184 INFO:tasks.workunit.client.1.smithi104.stdout:8/165: rename d0/d12 to d0/d33 0 2024-04-26T04:42:12.184 INFO:tasks.workunit.client.1.smithi104.stdout:8/166: chown d0/f2f 109 1 2024-04-26T04:42:12.184 INFO:tasks.workunit.client.1.smithi104.stdout:8/167: chown d0/d33/c20 16948 1 2024-04-26T04:42:12.185 INFO:tasks.workunit.client.1.smithi104.stdout:8/168: fsync d0/d33/f2b 0 2024-04-26T04:42:12.185 INFO:tasks.workunit.client.1.smithi104.stdout:8/169: creat d0/f34 x:0 0 0 2024-04-26T04:42:12.185 INFO:tasks.workunit.client.1.smithi104.stdout:5/283: write d0/f1d [5423183,23699] 0 2024-04-26T04:42:12.185 INFO:tasks.workunit.client.1.smithi104.stdout:5/284: creat d0/d1/d3e/f59 x:0 0 0 2024-04-26T04:42:12.185 INFO:tasks.workunit.client.1.smithi104.stdout:0/244: dwrite d0/d2/d21/d27/f30 [0,4194304] 0 2024-04-26T04:42:12.185 INFO:tasks.workunit.client.1.smithi104.stdout:0/245: creat d0/f4d x:0 0 0 2024-04-26T04:42:12.186 INFO:tasks.workunit.client.1.smithi104.stdout:0/246: fsync d0/d2/d21/d38/f34 0 2024-04-26T04:42:12.186 INFO:tasks.workunit.client.1.smithi104.stdout:0/247: truncate d0/f41 660627 0 2024-04-26T04:42:12.192 INFO:tasks.workunit.client.1.smithi104.stdout:2/391: dwrite d6/d1b/f38 [0,4194304] 0 2024-04-26T04:42:12.192 INFO:tasks.workunit.client.1.smithi104.stdout:2/392: readlink d6/d8/d3e/l67 0 2024-04-26T04:42:12.192 INFO:tasks.workunit.client.1.smithi104.stdout:2/393: fsync d6/d8/d3e/f26 0 2024-04-26T04:42:12.193 INFO:tasks.workunit.client.1.smithi104.stdout:0/248: rename d0/d2/d21/d27/d1e/f2d to d0/d2/d10/f4e 0 2024-04-26T04:42:12.193 INFO:tasks.workunit.client.1.smithi104.stdout:0/249: write d0/f4d [72693,122854] 0 2024-04-26T04:42:12.194 INFO:tasks.workunit.client.1.smithi104.stdout:8/170: rmdir d0/d1c/d2d 39 2024-04-26T04:42:12.194 INFO:tasks.workunit.client.1.smithi104.stdout:0/250: dread d0/d2/d21/d38/f36 [0,4194304] 0 2024-04-26T04:42:12.195 INFO:tasks.workunit.client.1.smithi104.stdout:5/285: rmdir d0/d1/d1e 39 2024-04-26T04:42:12.197 INFO:tasks.workunit.client.1.smithi104.stdout:2/394: creat d6/d1b/d51/f87 x:0 0 0 2024-04-26T04:42:12.197 INFO:tasks.workunit.client.1.smithi104.stdout:2/395: chown d6/de/d1d/f83 461016 1 2024-04-26T04:42:12.198 INFO:tasks.workunit.client.1.smithi104.stdout:8/171: mkdir d0/d33/d35 0 2024-04-26T04:42:12.201 INFO:tasks.workunit.client.1.smithi104.stdout:0/251: rename d0/f41 to d0/d2/d21/d38/f4f 0 2024-04-26T04:42:12.201 INFO:tasks.workunit.client.1.smithi104.stdout:0/252: creat d0/d2/d10/d3d/f50 x:0 0 0 2024-04-26T04:42:12.203 INFO:tasks.workunit.client.1.smithi104.stdout:8/172: creat d0/d33/d35/f36 x:0 0 0 2024-04-26T04:42:12.206 INFO:tasks.workunit.client.1.smithi104.stdout:8/173: truncate d0/fe 3708608 0 2024-04-26T04:42:12.207 INFO:tasks.workunit.client.1.smithi104.stdout:2/396: read d6/de/f36 [3627093,24907] 0 2024-04-26T04:42:12.208 INFO:tasks.workunit.client.1.smithi104.stdout:8/174: link d0/d1c/c29 d0/c37 0 2024-04-26T04:42:12.208 INFO:tasks.workunit.client.1.smithi104.stdout:8/175: chown d0/d1c/c29 811560 1 2024-04-26T04:42:12.208 INFO:tasks.workunit.client.1.smithi104.stdout:8/176: creat d0/d13/f38 x:0 0 0 2024-04-26T04:42:12.211 INFO:tasks.workunit.client.1.smithi104.stdout:8/177: link d0/l15 d0/d1c/l39 0 2024-04-26T04:42:12.211 INFO:tasks.workunit.client.1.smithi104.stdout:0/253: read d0/d2/d10/f4e [247752,31626] 0 2024-04-26T04:42:12.212 INFO:tasks.workunit.client.1.smithi104.stdout:8/178: truncate d0/d33/f1f 1474569 0 2024-04-26T04:42:12.214 INFO:tasks.workunit.client.1.smithi104.stdout:0/254: rmdir d0/d2/d21 39 2024-04-26T04:42:12.214 INFO:tasks.workunit.client.1.smithi104.stdout:2/397: dread d6/f10 [0,4194304] 0 2024-04-26T04:42:12.221 INFO:tasks.workunit.client.1.smithi104.stdout:8/179: rmdir d0 39 2024-04-26T04:42:12.222 INFO:tasks.workunit.client.1.smithi104.stdout:8/180: symlink d0/d1c/d2d/l3a 0 2024-04-26T04:42:12.223 INFO:tasks.workunit.client.1.smithi104.stdout:8/181: creat d0/f3b x:0 0 0 2024-04-26T04:42:12.226 INFO:tasks.workunit.client.1.smithi104.stdout:0/255: write d0/f17 [2524557,30923] 0 2024-04-26T04:42:12.226 INFO:tasks.workunit.client.1.smithi104.stdout:2/398: dread d6/d8/f78 [0,4194304] 0 2024-04-26T04:42:12.227 INFO:tasks.workunit.client.1.smithi104.stdout:0/256: mkdir d0/d4/d3c/d51 0 2024-04-26T04:42:12.229 INFO:tasks.workunit.client.1.smithi104.stdout:2/399: unlink d6/f35 0 2024-04-26T04:42:12.230 INFO:tasks.workunit.client.1.smithi104.stdout:0/257: creat d0/d2/d21/f52 x:0 0 0 2024-04-26T04:42:12.230 INFO:tasks.workunit.client.1.smithi104.stdout:0/258: write d0/d2/d21/d38/f36 [1272731,111135] 0 2024-04-26T04:42:12.231 INFO:tasks.workunit.client.1.smithi104.stdout:2/400: dread d6/d8/f70 [0,4194304] 0 2024-04-26T04:42:12.232 INFO:tasks.workunit.client.1.smithi104.stdout:2/401: dread d6/d8/d3e/f49 [0,4194304] 0 2024-04-26T04:42:12.233 INFO:tasks.workunit.client.1.smithi104.stdout:2/402: symlink d6/d8/l88 0 2024-04-26T04:42:12.248 INFO:tasks.workunit.client.1.smithi104.stdout:6/373: sync 2024-04-26T04:42:12.248 INFO:tasks.workunit.client.1.smithi104.stdout:9/190: sync 2024-04-26T04:42:12.249 INFO:tasks.workunit.client.1.smithi104.stdout:9/191: write d11/d15/d1b/f24 [479112,39501] 0 2024-04-26T04:42:12.249 INFO:tasks.workunit.client.1.smithi104.stdout:2/403: dwrite d6/d8/d5a/f2e [0,4194304] 0 2024-04-26T04:42:12.270 INFO:tasks.workunit.client.1.smithi104.stdout:9/192: dwrite fb [4194304,4194304] 0 2024-04-26T04:42:12.270 INFO:tasks.workunit.client.1.smithi104.stdout:2/404: dwrite d6/d8/d5a/f54 [0,4194304] 0 2024-04-26T04:42:12.271 INFO:tasks.workunit.client.1.smithi104.stdout:9/193: mkdir d11/d15/d32/d35 0 2024-04-26T04:42:12.274 INFO:tasks.workunit.client.1.smithi104.stdout:2/405: link d6/d8/d3e/c4e d6/d1b/c89 0 2024-04-26T04:42:12.274 INFO:tasks.workunit.client.1.smithi104.stdout:2/406: chown d6/d8/d5a/l79 240090 1 2024-04-26T04:42:12.274 INFO:tasks.workunit.client.1.smithi104.stdout:2/407: truncate d6/d8/d5a/d29/f4c 690587 0 2024-04-26T04:42:12.274 INFO:tasks.workunit.client.1.smithi104.stdout:2/408: truncate d6/d8/f70 1094690 0 2024-04-26T04:42:12.275 INFO:tasks.workunit.client.1.smithi104.stdout:2/409: mkdir d6/d1b/d51/d6a/d8a 0 2024-04-26T04:42:12.275 INFO:tasks.workunit.client.1.smithi104.stdout:2/410: creat d6/f8b x:0 0 0 2024-04-26T04:42:12.278 INFO:tasks.workunit.client.1.smithi104.stdout:2/411: link d6/d1b/d68/c6b d6/d1b/d75/c8c 0 2024-04-26T04:42:12.280 INFO:tasks.workunit.client.1.smithi104.stdout:2/412: mknod d6/d1b/d68/c8d 0 2024-04-26T04:42:12.280 INFO:tasks.workunit.client.1.smithi104.stdout:2/413: fsync d6/d8/f5b 0 2024-04-26T04:42:12.283 INFO:tasks.workunit.client.1.smithi104.stdout:2/414: dread d6/d8/f37 [0,4194304] 0 2024-04-26T04:42:12.285 INFO:tasks.workunit.client.1.smithi104.stdout:2/415: rename d6/d8/d5a/d29 to d6/de/d8e 0 2024-04-26T04:42:12.285 INFO:tasks.workunit.client.1.smithi104.stdout:2/416: fdatasync d6/d1b/f71 0 2024-04-26T04:42:12.285 INFO:tasks.workunit.client.1.smithi104.stdout:2/417: chown d6/d1b/d68/d81/l86 389 1 2024-04-26T04:42:12.285 INFO:tasks.workunit.client.1.smithi104.stdout:2/418: fdatasync d6/d8/d5a/f2e 0 2024-04-26T04:42:12.289 INFO:tasks.workunit.client.1.smithi104.stdout:9/194: dwrite d11/d15/f18 [0,4194304] 0 2024-04-26T04:42:12.291 INFO:tasks.workunit.client.1.smithi104.stdout:9/195: creat d11/d1d/d34/f36 x:0 0 0 2024-04-26T04:42:12.291 INFO:tasks.workunit.client.1.smithi104.stdout:9/196: creat d11/d15/d1b/f37 x:0 0 0 2024-04-26T04:42:12.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/419: dwrite d6/d8/f39 [0,4194304] 0 2024-04-26T04:42:12.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/420: dread - d6/d1b/f71 zero size 2024-04-26T04:42:12.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/421: stat d6/d8/d3e/f49 0 2024-04-26T04:42:12.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/422: readlink d6/d8/d3e/l3b 0 2024-04-26T04:42:12.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/423: write d6/d8/d3e/f7a [1039482,55293] 0 2024-04-26T04:42:12.307 INFO:tasks.workunit.client.1.smithi104.stdout:2/424: truncate d6/de/f44 503150 0 2024-04-26T04:42:12.307 INFO:tasks.workunit.client.1.smithi104.stdout:2/425: read d6/d8/d3e/f49 [24644,79596] 0 2024-04-26T04:42:12.307 INFO:tasks.workunit.client.1.smithi104.stdout:2/426: chown d6/d1b/d51/c76 157808 1 2024-04-26T04:42:12.307 INFO:tasks.workunit.client.1.smithi104.stdout:2/427: creat d6/de/d1d/f8f x:0 0 0 2024-04-26T04:42:12.308 INFO:tasks.workunit.client.1.smithi104.stdout:2/428: truncate d6/de/f40 815447 0 2024-04-26T04:42:12.308 INFO:tasks.workunit.client.1.smithi104.stdout:2/429: chown d6/de/c7b 31972 1 2024-04-26T04:42:12.308 INFO:tasks.workunit.client.1.smithi104.stdout:2/430: write d6/d8/f70 [1064299,60752] 0 2024-04-26T04:42:12.310 INFO:tasks.workunit.client.1.smithi104.stdout:2/431: symlink d6/l90 0 2024-04-26T04:42:12.313 INFO:tasks.workunit.client.1.smithi104.stdout:2/432: rename d6/f4b to d6/de/d85/f91 0 2024-04-26T04:42:12.313 INFO:tasks.workunit.client.1.smithi104.stdout:2/433: stat d6/d8/f46 0 2024-04-26T04:42:12.318 INFO:tasks.workunit.client.1.smithi104.stdout:2/434: fsync d6/de/f44 0 2024-04-26T04:42:12.318 INFO:tasks.workunit.client.1.smithi104.stdout:2/435: chown d6/d1b/f38 20508717 1 2024-04-26T04:42:12.371 INFO:tasks.workunit.client.1.smithi104.stdout:1/293: sync 2024-04-26T04:42:12.371 INFO:tasks.workunit.client.1.smithi104.stdout:1/294: mkdir d0/d6/d19/d21/d59 0 2024-04-26T04:42:12.373 INFO:tasks.workunit.client.1.smithi104.stdout:1/295: symlink d0/d6/d19/l5a 0 2024-04-26T04:42:12.373 INFO:tasks.workunit.client.1.smithi104.stdout:1/296: creat d0/d6/de/d29/d3f/f5b x:0 0 0 2024-04-26T04:42:12.375 INFO:tasks.workunit.client.1.smithi104.stdout:1/297: rmdir d0/d6/de/d20/d2a/d50 0 2024-04-26T04:42:12.389 INFO:tasks.workunit.client.1.smithi104.stdout:1/298: dwrite d0/d6/d2e/f46 [0,4194304] 0 2024-04-26T04:42:12.390 INFO:tasks.workunit.client.1.smithi104.stdout:1/299: mknod d0/d6/de/d20/d2a/d56/c5c 0 2024-04-26T04:42:12.390 INFO:tasks.workunit.client.1.smithi104.stdout:1/300: chown d0/d6/de/f41 6096549 1 2024-04-26T04:42:12.407 INFO:tasks.workunit.client.1.smithi104.stdout:1/301: dwrite d0/d6/d19/f3a [0,4194304] 0 2024-04-26T04:42:12.407 INFO:tasks.workunit.client.1.smithi104.stdout:1/302: chown d0/d6/de 6 1 2024-04-26T04:42:12.411 INFO:tasks.workunit.client.1.smithi104.stdout:1/303: dread d0/d6/d19/d1b/d35/f40 [0,4194304] 0 2024-04-26T04:42:12.522 INFO:tasks.workunit.client.1.smithi104.stdout:3/304: sync 2024-04-26T04:42:12.524 INFO:tasks.workunit.client.1.smithi104.stdout:3/305: creat d5/d9/dc/d32/d48/f67 x:0 0 0 2024-04-26T04:42:12.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:12 smithi090 ceph-mon[26648]: pgmap v248: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-04-26T04:42:12.728 INFO:tasks.workunit.client.1.smithi104.stdout:4/253: sync 2024-04-26T04:42:12.728 INFO:tasks.workunit.client.1.smithi104.stdout:7/157: sync 2024-04-26T04:42:12.729 INFO:tasks.workunit.client.1.smithi104.stdout:4/254: chown d4/d7/d1d/d1e/d15/c47 16395 1 2024-04-26T04:42:12.730 INFO:tasks.workunit.client.1.smithi104.stdout:7/158: mknod d2/c31 0 2024-04-26T04:42:12.740 INFO:tasks.workunit.client.1.smithi104.stdout:7/159: dread d2/d4/d1d/f29 [0,4194304] 0 2024-04-26T04:42:12.741 INFO:tasks.workunit.client.1.smithi104.stdout:7/160: mkdir d2/d4/d32 0 2024-04-26T04:42:12.741 INFO:tasks.workunit.client.1.smithi104.stdout:7/161: fdatasync d2/f1c 0 2024-04-26T04:42:12.742 INFO:tasks.workunit.client.1.smithi104.stdout:7/162: chown d2/d4/l17 625599 1 2024-04-26T04:42:12.745 INFO:tasks.workunit.client.1.smithi104.stdout:4/255: dwrite d4/d7/ff [0,4194304] 0 2024-04-26T04:42:12.748 INFO:tasks.workunit.client.1.smithi104.stdout:4/256: mkdir d4/d7/d1d/d1e/d24/d59 0 2024-04-26T04:42:12.748 INFO:tasks.workunit.client.1.smithi104.stdout:4/257: chown d4/d20/f36 1597 1 2024-04-26T04:42:12.748 INFO:tasks.workunit.client.1.smithi104.stdout:4/258: readlink d4/d7/d1d/d1e/d24/l29 0 2024-04-26T04:42:12.750 INFO:tasks.workunit.client.1.smithi104.stdout:4/259: creat d4/d7/d1d/d1e/f5a x:0 0 0 2024-04-26T04:42:12.757 INFO:tasks.workunit.client.1.smithi104.stdout:4/260: dread d4/db/f12 [0,4194304] 0 2024-04-26T04:42:12.758 INFO:tasks.workunit.client.1.smithi104.stdout:4/261: stat d4/d7/d1d/d1e/d24/l4f 0 2024-04-26T04:42:12.761 INFO:tasks.workunit.client.1.smithi104.stdout:7/163: dwrite d2/d4/f11 [0,4194304] 0 2024-04-26T04:42:12.761 INFO:tasks.workunit.client.1.smithi104.stdout:7/164: dread - d2/d4/d1d/f30 zero size 2024-04-26T04:42:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:12 smithi104 ceph-mon[32369]: pgmap v248: 65 pgs: 65 active+clean; 317 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 6.8 MiB/s rd, 19 MiB/s wr, 1.02k op/s 2024-04-26T04:42:12.763 INFO:tasks.workunit.client.1.smithi104.stdout:7/165: creat d2/d4/f33 x:0 0 0 2024-04-26T04:42:12.763 INFO:tasks.workunit.client.1.smithi104.stdout:7/166: chown d2/f2f 0 1 2024-04-26T04:42:12.764 INFO:tasks.workunit.client.1.smithi104.stdout:7/167: mkdir d2/d4/d1d/d34 0 2024-04-26T04:42:12.765 INFO:tasks.workunit.client.1.smithi104.stdout:7/168: unlink d2/f1c 0 2024-04-26T04:42:12.766 INFO:tasks.workunit.client.1.smithi104.stdout:7/169: unlink d2/d4/d1d/f1e 0 2024-04-26T04:42:12.768 INFO:tasks.workunit.client.1.smithi104.stdout:4/262: dread d4/f37 [0,4194304] 0 2024-04-26T04:42:12.782 INFO:tasks.workunit.client.1.smithi104.stdout:4/263: dread f3 [0,4194304] 0 2024-04-26T04:42:12.783 INFO:tasks.workunit.client.1.smithi104.stdout:4/264: creat d4/d7/d1d/d1e/f5b x:0 0 0 2024-04-26T04:42:12.784 INFO:tasks.workunit.client.1.smithi104.stdout:7/170: dwrite d2/d4/f10 [0,4194304] 0 2024-04-26T04:42:12.784 INFO:tasks.workunit.client.1.smithi104.stdout:4/265: rename d4/d7/d1d/f1c to d4/d7/d1d/d1e/d24/d48/f5c 0 2024-04-26T04:42:12.785 INFO:tasks.workunit.client.1.smithi104.stdout:7/171: mkdir d2/d4/d32/d35 0 2024-04-26T04:42:12.786 INFO:tasks.workunit.client.1.smithi104.stdout:7/172: symlink d2/d4/d32/d35/l36 0 2024-04-26T04:42:12.786 INFO:tasks.workunit.client.1.smithi104.stdout:4/266: dread d4/d7/d1d/d1e/f14 [0,4194304] 0 2024-04-26T04:42:12.786 INFO:tasks.workunit.client.1.smithi104.stdout:4/267: write d4/d7/d1d/f45 [198713,86493] 0 2024-04-26T04:42:12.787 INFO:tasks.workunit.client.1.smithi104.stdout:4/268: unlink f3 0 2024-04-26T04:42:12.787 INFO:tasks.workunit.client.1.smithi104.stdout:4/269: fsync d4/d20/d55/d56/f31 0 2024-04-26T04:42:12.792 INFO:tasks.workunit.client.1.smithi104.stdout:4/270: getdents d4/d7/d1d/d1e 0 2024-04-26T04:42:12.794 INFO:tasks.workunit.client.1.smithi104.stdout:4/271: getdents d4/d7/d1d/d1e/d24/d48 0 2024-04-26T04:42:12.795 INFO:tasks.workunit.client.1.smithi104.stdout:4/272: creat d4/d7/d1d/d1e/d24/d59/f5d x:0 0 0 2024-04-26T04:42:12.796 INFO:tasks.workunit.client.1.smithi104.stdout:7/173: write d2/f2f [1976717,113085] 0 2024-04-26T04:42:12.796 INFO:tasks.workunit.client.1.smithi104.stdout:7/174: write d2/d4/d1d/f30 [829537,128632] 0 2024-04-26T04:42:12.796 INFO:tasks.workunit.client.1.smithi104.stdout:7/175: getdents d2/d4 0 2024-04-26T04:42:12.798 INFO:tasks.workunit.client.1.smithi104.stdout:4/273: dread d4/d7/d1d/d1e/d15/f22 [0,4194304] 0 2024-04-26T04:42:12.798 INFO:tasks.workunit.client.1.smithi104.stdout:7/176: mkdir d2/d4/d32/d35/d37 0 2024-04-26T04:42:12.798 INFO:tasks.workunit.client.1.smithi104.stdout:4/274: fsync f2 0 2024-04-26T04:42:12.798 INFO:tasks.workunit.client.1.smithi104.stdout:7/177: stat d2/d4 0 2024-04-26T04:42:12.798 INFO:tasks.workunit.client.1.smithi104.stdout:7/178: symlink d2/d4/d1d/l38 0 2024-04-26T04:42:12.804 INFO:tasks.workunit.client.1.smithi104.stdout:4/275: dread d4/d7/ff [0,4194304] 0 2024-04-26T04:42:12.805 INFO:tasks.workunit.client.1.smithi104.stdout:4/276: symlink d4/d7/d1d/d2e/l5e 0 2024-04-26T04:42:12.806 INFO:tasks.workunit.client.1.smithi104.stdout:4/277: dread d4/d7/f42 [0,4194304] 0 2024-04-26T04:42:12.808 INFO:tasks.workunit.client.1.smithi104.stdout:4/278: link d4/d7/d1d/d1e/d15/c54 d4/d7/d1d/c5f 0 2024-04-26T04:42:12.810 INFO:tasks.workunit.client.1.smithi104.stdout:4/279: unlink d4/d7/d1d/f23 0 2024-04-26T04:42:12.811 INFO:tasks.workunit.client.1.smithi104.stdout:4/280: creat d4/d7/d1d/d1e/d24/d48/f60 x:0 0 0 2024-04-26T04:42:12.811 INFO:tasks.workunit.client.1.smithi104.stdout:4/281: chown d4/d7/f2a 79733 1 2024-04-26T04:42:12.812 INFO:tasks.workunit.client.1.smithi104.stdout:4/282: creat d4/d7/d1d/d1e/f61 x:0 0 0 2024-04-26T04:42:12.812 INFO:tasks.workunit.client.1.smithi104.stdout:4/283: chown d4/d7/d1d/d1e/d15/c54 219 1 2024-04-26T04:42:12.813 INFO:tasks.workunit.client.1.smithi104.stdout:4/284: symlink d4/d7/d1d/d1e/d24/d59/l62 0 2024-04-26T04:42:12.813 INFO:tasks.workunit.client.1.smithi104.stdout:4/285: write d4/db/f28 [773074,64989] 0 2024-04-26T04:42:12.817 INFO:tasks.workunit.client.1.smithi104.stdout:7/179: dwrite d2/d4/d1d/f30 [0,4194304] 0 2024-04-26T04:42:12.822 INFO:tasks.workunit.client.1.smithi104.stdout:4/286: fsync d4/d7/d1d/d1e/f61 0 2024-04-26T04:42:12.822 INFO:tasks.workunit.client.1.smithi104.stdout:4/287: read d4/d7/d1d/d1e/d24/d48/f5c [2975017,50047] 0 2024-04-26T04:42:12.822 INFO:tasks.workunit.client.1.smithi104.stdout:4/288: dread - d4/d7/d1d/f43 zero size 2024-04-26T04:42:12.822 INFO:tasks.workunit.client.1.smithi104.stdout:4/289: stat d4/c5 0 2024-04-26T04:42:12.822 INFO:tasks.workunit.client.1.smithi104.stdout:4/290: creat d4/d7/f63 x:0 0 0 2024-04-26T04:42:12.823 INFO:tasks.workunit.client.1.smithi104.stdout:4/291: mknod d4/c64 0 2024-04-26T04:42:12.824 INFO:tasks.workunit.client.1.smithi104.stdout:4/292: truncate d4/d20/f51 3216073 0 2024-04-26T04:42:12.825 INFO:tasks.workunit.client.1.smithi104.stdout:4/293: mknod d4/d20/c65 0 2024-04-26T04:42:12.827 INFO:tasks.workunit.client.1.smithi104.stdout:7/180: dread d2/d4/f10 [0,4194304] 0 2024-04-26T04:42:12.828 INFO:tasks.workunit.client.1.smithi104.stdout:7/181: rmdir d2/d4/d32/d35 39 2024-04-26T04:42:12.828 INFO:tasks.workunit.client.1.smithi104.stdout:7/182: chown d2/c2e 0 1 2024-04-26T04:42:12.828 INFO:tasks.workunit.client.1.smithi104.stdout:4/294: dread d4/d7/f2a [0,4194304] 0 2024-04-26T04:42:12.828 INFO:tasks.workunit.client.1.smithi104.stdout:7/183: write d2/d4/f33 [977261,61695] 0 2024-04-26T04:42:12.828 INFO:tasks.workunit.client.1.smithi104.stdout:4/295: getdents d4/d20/d55/d56 0 2024-04-26T04:42:12.830 INFO:tasks.workunit.client.1.smithi104.stdout:4/296: link d4/d20/d55/d56/f3e d4/d7/d1d/d1e/d15/f66 0 2024-04-26T04:42:12.831 INFO:tasks.workunit.client.1.smithi104.stdout:4/297: read d4/db/f28 [337759,86642] 0 2024-04-26T04:42:12.831 INFO:tasks.workunit.client.1.smithi104.stdout:4/298: truncate d4/d7/ff 4846073 0 2024-04-26T04:42:12.833 INFO:tasks.workunit.client.1.smithi104.stdout:7/184: mkdir d2/d39 0 2024-04-26T04:42:12.835 INFO:tasks.workunit.client.1.smithi104.stdout:7/185: creat d2/d4/d32/d35/d37/f3a x:0 0 0 2024-04-26T04:42:12.835 INFO:tasks.workunit.client.1.smithi104.stdout:7/186: dread - d2/d4/f13 zero size 2024-04-26T04:42:12.836 INFO:tasks.workunit.client.1.smithi104.stdout:4/299: write d4/d20/f51 [2692270,108167] 0 2024-04-26T04:42:12.836 INFO:tasks.workunit.client.1.smithi104.stdout:7/187: symlink d2/d4/d32/d35/l3b 0 2024-04-26T04:42:12.836 INFO:tasks.workunit.client.1.smithi104.stdout:7/188: readlink d2/d4/l25 0 2024-04-26T04:42:12.839 INFO:tasks.workunit.client.1.smithi104.stdout:7/189: truncate d2/f28 3194592 0 2024-04-26T04:42:12.840 INFO:tasks.workunit.client.1.smithi104.stdout:7/190: chown d2/le 2093310 1 2024-04-26T04:42:12.840 INFO:tasks.workunit.client.1.smithi104.stdout:7/191: read - d2/d4/d32/d35/d37/f3a zero size 2024-04-26T04:42:12.840 INFO:tasks.workunit.client.1.smithi104.stdout:7/192: write d2/d4/f26 [1716031,56097] 0 2024-04-26T04:42:12.840 INFO:tasks.workunit.client.1.smithi104.stdout:5/286: sync 2024-04-26T04:42:12.840 INFO:tasks.workunit.client.1.smithi104.stdout:7/193: rmdir d2/d4/d32 39 2024-04-26T04:42:12.846 INFO:tasks.workunit.client.1.smithi104.stdout:7/194: write d2/f2b [4630923,18550] 0 2024-04-26T04:42:12.848 INFO:tasks.workunit.client.1.smithi104.stdout:7/195: rmdir d2/d39 0 2024-04-26T04:42:12.849 INFO:tasks.workunit.client.1.smithi104.stdout:7/196: symlink d2/d4/l3c 0 2024-04-26T04:42:12.850 INFO:tasks.workunit.client.1.smithi104.stdout:7/197: mkdir d2/d4/d32/d3d 0 2024-04-26T04:42:12.851 INFO:tasks.workunit.client.1.smithi104.stdout:7/198: chown d2/c1f 33359 1 2024-04-26T04:42:12.851 INFO:tasks.workunit.client.1.smithi104.stdout:7/199: read - d2/d4/d32/d35/d37/f3a zero size 2024-04-26T04:42:12.851 INFO:tasks.workunit.client.1.smithi104.stdout:7/200: readlink d2/d4/d1d/l2c 0 2024-04-26T04:42:12.851 INFO:tasks.workunit.client.1.smithi104.stdout:7/201: fsync d2/d4/d1d/f29 0 2024-04-26T04:42:12.851 INFO:tasks.workunit.client.1.smithi104.stdout:7/202: creat d2/d4/d1d/f3e x:0 0 0 2024-04-26T04:42:12.853 INFO:tasks.workunit.client.1.smithi104.stdout:7/203: rename d2/c2e to d2/c3f 0 2024-04-26T04:42:12.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/287: dwrite d0/d11/f19 [0,4194304] 0 2024-04-26T04:42:12.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/288: readlink d0/d11/d26/l2c 0 2024-04-26T04:42:12.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/289: chown d0/f5 21 1 2024-04-26T04:42:12.856 INFO:tasks.workunit.client.1.smithi104.stdout:5/290: fdatasync d0/d1/f3d 0 2024-04-26T04:42:12.857 INFO:tasks.workunit.client.1.smithi104.stdout:5/291: symlink d0/d1/d4/l5a 0 2024-04-26T04:42:12.857 INFO:tasks.workunit.client.1.smithi104.stdout:5/292: creat d0/d11/f5b x:0 0 0 2024-04-26T04:42:12.858 INFO:tasks.workunit.client.1.smithi104.stdout:5/293: mkdir d0/d1/d4/d2a/d2b/d5c 0 2024-04-26T04:42:12.859 INFO:tasks.workunit.client.1.smithi104.stdout:5/294: link d0/d11/d26/l2c d0/d11/d26/d36/l5d 0 2024-04-26T04:42:12.860 INFO:tasks.workunit.client.1.smithi104.stdout:5/295: creat d0/d1/d4/d2a/d2b/d5c/f5e x:0 0 0 2024-04-26T04:42:12.860 INFO:tasks.workunit.client.1.smithi104.stdout:5/296: readlink d0/d11/d26/l2c 0 2024-04-26T04:42:12.867 INFO:tasks.workunit.client.1.smithi104.stdout:7/204: write d2/d4/d1d/f30 [187680,81504] 0 2024-04-26T04:42:12.875 INFO:tasks.workunit.client.1.smithi104.stdout:5/297: dwrite d0/d11/f31 [0,4194304] 0 2024-04-26T04:42:12.953 INFO:tasks.workunit.client.1.smithi104.stdout:5/298: mknod d0/d1/d4/d2a/d2b/d37/d48/c5f 0 2024-04-26T04:42:12.996 INFO:tasks.workunit.client.1.smithi104.stdout:7/205: truncate d2/d4/f26 3235730 0 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:6/374: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:9/197: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:8/182: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:4/300: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:0/259: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:3/306: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:1/304: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:2/436: sync 2024-04-26T04:42:13.000 INFO:tasks.workunit.client.1.smithi104.stdout:2/437: truncate d6/d1b/d51/f87 512345 0 2024-04-26T04:42:13.001 INFO:tasks.workunit.client.1.smithi104.stdout:3/307: write d5/d9/d17/f3c [1884443,130845] 0 2024-04-26T04:42:13.001 INFO:tasks.workunit.client.1.smithi104.stdout:8/183: read d0/d1c/f26 [339529,79942] 0 2024-04-26T04:42:13.001 INFO:tasks.workunit.client.1.smithi104.stdout:8/184: fdatasync d0/d33/f1f 0 2024-04-26T04:42:13.002 INFO:tasks.workunit.client.1.smithi104.stdout:0/260: mkdir d0/d4/d3c/d53 0 2024-04-26T04:42:13.002 INFO:tasks.workunit.client.1.smithi104.stdout:0/261: stat d0/d2/d10/f32 0 2024-04-26T04:42:13.002 INFO:tasks.workunit.client.1.smithi104.stdout:0/262: truncate d0/d4/f7 2743205 0 2024-04-26T04:42:13.002 INFO:tasks.workunit.client.1.smithi104.stdout:0/263: read d0/f4d [49128,28221] 0 2024-04-26T04:42:13.002 INFO:tasks.workunit.client.1.smithi104.stdout:1/305: symlink d0/d6/l5d 0 2024-04-26T04:42:13.004 INFO:tasks.workunit.client.1.smithi104.stdout:3/308: truncate d5/d9/fa 4149790 0 2024-04-26T04:42:13.004 INFO:tasks.workunit.client.1.smithi104.stdout:3/309: rename d5/d9/dc to d5/d9/dc/d68 22 2024-04-26T04:42:13.004 INFO:tasks.workunit.client.1.smithi104.stdout:0/264: truncate d0/f17 1150501 0 2024-04-26T04:42:13.007 INFO:tasks.workunit.client.1.smithi104.stdout:3/310: rename d5/d9/dc/d29/d2d to d5/d9/dc/d69 0 2024-04-26T04:42:13.019 INFO:tasks.workunit.client.1.smithi104.stdout:7/206: dwrite d2/d4/f10 [4194304,4194304] 0 2024-04-26T04:42:13.022 INFO:tasks.workunit.client.1.smithi104.stdout:7/207: creat d2/d4/d1d/d34/f40 x:0 0 0 2024-04-26T04:42:13.022 INFO:tasks.workunit.client.1.smithi104.stdout:7/208: read - d2/d4/f13 zero size 2024-04-26T04:42:13.022 INFO:tasks.workunit.client.1.smithi104.stdout:1/306: dread d0/f14 [8388608,4194304] 0 2024-04-26T04:42:13.023 INFO:tasks.workunit.client.1.smithi104.stdout:7/209: mkdir d2/d4/d32/d3d/d41 0 2024-04-26T04:42:13.023 INFO:tasks.workunit.client.1.smithi104.stdout:7/210: chown d2/d4/d32/d35/d37/f3a 923 1 2024-04-26T04:42:13.023 INFO:tasks.workunit.client.1.smithi104.stdout:7/211: write d2/d4/d32/d35/d37/f3a [323575,27522] 0 2024-04-26T04:42:13.024 INFO:tasks.workunit.client.1.smithi104.stdout:1/307: truncate d0/d6/de/d29/f39 351876 0 2024-04-26T04:42:13.024 INFO:tasks.workunit.client.1.smithi104.stdout:1/308: write d0/d6/d49/f4a [995720,37946] 0 2024-04-26T04:42:13.025 INFO:tasks.workunit.client.1.smithi104.stdout:1/309: unlink d0/d6/d19/c1f 0 2024-04-26T04:42:13.035 INFO:tasks.workunit.client.1.smithi104.stdout:9/198: dwrite d11/d1d/f26 [0,4194304] 0 2024-04-26T04:42:13.039 INFO:tasks.workunit.client.1.smithi104.stdout:0/265: dwrite d0/d4/f11 [0,4194304] 0 2024-04-26T04:42:13.039 INFO:tasks.workunit.client.1.smithi104.stdout:6/375: dwrite d0/d9/d13/d15/f60 [0,4194304] 0 2024-04-26T04:42:13.040 INFO:tasks.workunit.client.1.smithi104.stdout:9/199: rename d11/d1d/l28 to d11/d15/d32/d35/l38 0 2024-04-26T04:42:13.040 INFO:tasks.workunit.client.1.smithi104.stdout:4/301: dwrite d4/d7/f2a [0,4194304] 0 2024-04-26T04:42:13.042 INFO:tasks.workunit.client.1.smithi104.stdout:6/376: rename d0/d9/c72 to d0/d9/d13/d44/d71/d3f/c7e 0 2024-04-26T04:42:13.044 INFO:tasks.workunit.client.1.smithi104.stdout:4/302: creat d4/d7/d1d/d2e/f67 x:0 0 0 2024-04-26T04:42:13.044 INFO:tasks.workunit.client.1.smithi104.stdout:4/303: truncate d4/f37 4782946 0 2024-04-26T04:42:13.046 INFO:tasks.workunit.client.1.smithi104.stdout:4/304: creat d4/d7/d1d/d1e/d24/f68 x:0 0 0 2024-04-26T04:42:13.047 INFO:tasks.workunit.client.1.smithi104.stdout:4/305: read d4/d7/d1d/d1e/d15/f1a [4268711,11668] 0 2024-04-26T04:42:13.047 INFO:tasks.workunit.client.1.smithi104.stdout:4/306: stat d4/d20/f51 0 2024-04-26T04:42:13.047 INFO:tasks.workunit.client.1.smithi104.stdout:4/307: readlink d4/d7/d1d/d1e/d24/d59/l62 0 2024-04-26T04:42:13.047 INFO:tasks.workunit.client.1.smithi104.stdout:4/308: fdatasync d4/d7/d1d/d1e/d15/f46 0 2024-04-26T04:42:13.056 INFO:tasks.workunit.client.1.smithi104.stdout:4/309: write d4/d7/f53 [3077929,21559] 0 2024-04-26T04:42:13.058 INFO:tasks.workunit.client.1.smithi104.stdout:4/310: creat d4/f69 x:0 0 0 2024-04-26T04:42:13.059 INFO:tasks.workunit.client.1.smithi104.stdout:4/311: write d4/d7/d1d/d1e/d24/d48/f60 [325017,37194] 0 2024-04-26T04:42:13.059 INFO:tasks.workunit.client.1.smithi104.stdout:4/312: creat d4/d7/d1d/d1e/d24/d59/f6a x:0 0 0 2024-04-26T04:42:13.068 INFO:tasks.workunit.client.1.smithi104.stdout:6/377: dwrite d0/d1b/f2c [0,4194304] 0 2024-04-26T04:42:13.069 INFO:tasks.workunit.client.1.smithi104.stdout:6/378: rmdir d0/d7b 0 2024-04-26T04:42:13.070 INFO:tasks.workunit.client.1.smithi104.stdout:6/379: creat d0/d9/d13/d41/f7f x:0 0 0 2024-04-26T04:42:13.072 INFO:tasks.workunit.client.1.smithi104.stdout:6/380: rename d0/d9/d13/d44/d71/d3f/d79 to d0/d9/d13/d15/d80 0 2024-04-26T04:42:13.072 INFO:tasks.workunit.client.1.smithi104.stdout:6/381: stat d0/d5/f22 0 2024-04-26T04:42:13.072 INFO:tasks.workunit.client.1.smithi104.stdout:6/382: read - d0/d9/f4a zero size 2024-04-26T04:42:13.072 INFO:tasks.workunit.client.1.smithi104.stdout:6/383: dread - d0/d9/d13/d44/d71/f74 zero size 2024-04-26T04:42:13.074 INFO:tasks.workunit.client.1.smithi104.stdout:6/384: dread d0/d9/d13/d44/d47/f4c [0,4194304] 0 2024-04-26T04:42:13.231 INFO:tasks.workunit.client.1.smithi104.stdout:7/212: dread d2/d4/f10 [0,4194304] 0 2024-04-26T04:42:13.235 INFO:tasks.workunit.client.1.smithi104.stdout:9/200: dwrite d11/d1d/f2d [0,4194304] 0 2024-04-26T04:42:13.240 INFO:tasks.workunit.client.1.smithi104.stdout:9/201: link d11/c2c d11/d1d/d34/c39 0 2024-04-26T04:42:13.248 INFO:tasks.workunit.client.1.smithi104.stdout:9/202: rename d11/d1d/f26 to d11/d15/d32/d35/f3a 0 2024-04-26T04:42:13.248 INFO:tasks.workunit.client.1.smithi104.stdout:9/203: stat d11/d15/d32/d35/f3a 0 2024-04-26T04:42:13.248 INFO:tasks.workunit.client.1.smithi104.stdout:9/204: chown d11/l1c 4855672 1 2024-04-26T04:42:13.249 INFO:tasks.workunit.client.1.smithi104.stdout:9/205: mknod d11/d15/d32/c3b 0 2024-04-26T04:42:13.249 INFO:tasks.workunit.client.1.smithi104.stdout:9/206: chown f6 1234909 1 2024-04-26T04:42:13.250 INFO:tasks.workunit.client.1.smithi104.stdout:9/207: creat d11/d1d/d34/f3c x:0 0 0 2024-04-26T04:42:13.250 INFO:tasks.workunit.client.1.smithi104.stdout:9/208: creat d11/d15/f3d x:0 0 0 2024-04-26T04:42:13.258 INFO:tasks.workunit.client.1.smithi104.stdout:0/266: dwrite d0/f42 [0,4194304] 0 2024-04-26T04:42:13.258 INFO:tasks.workunit.client.1.smithi104.stdout:0/267: stat d0/d4/f11 0 2024-04-26T04:42:13.258 INFO:tasks.workunit.client.1.smithi104.stdout:0/268: write d0/f4d [201228,26951] 0 2024-04-26T04:42:13.260 INFO:tasks.workunit.client.1.smithi104.stdout:0/269: creat d0/d2/d21/d38/f54 x:0 0 0 2024-04-26T04:42:13.260 INFO:tasks.workunit.client.1.smithi104.stdout:0/270: fsync d0/d2/d21/d27/d1e/f2c 0 2024-04-26T04:42:13.265 INFO:tasks.workunit.client.1.smithi104.stdout:7/213: dwrite d2/d4/f13 [0,4194304] 0 2024-04-26T04:42:13.265 INFO:tasks.workunit.client.1.smithi104.stdout:7/214: write d2/f2b [4876858,31201] 0 2024-04-26T04:42:13.265 INFO:tasks.workunit.client.1.smithi104.stdout:6/385: dwrite d0/d9/f3b [0,4194304] 0 2024-04-26T04:42:13.267 INFO:tasks.workunit.client.1.smithi104.stdout:4/313: dwrite d4/d7/f63 [0,4194304] 0 2024-04-26T04:42:13.268 INFO:tasks.workunit.client.1.smithi104.stdout:3/311: dwrite d5/d9/fa [0,4194304] 0 2024-04-26T04:42:13.275 INFO:tasks.workunit.client.1.smithi104.stdout:7/215: write d2/d4/f11 [2630467,42939] 0 2024-04-26T04:42:13.275 INFO:tasks.workunit.client.1.smithi104.stdout:9/209: dwrite d11/f12 [0,4194304] 0 2024-04-26T04:42:13.275 INFO:tasks.workunit.client.1.smithi104.stdout:9/210: write d11/d15/f30 [455822,110928] 0 2024-04-26T04:42:13.275 INFO:tasks.workunit.client.1.smithi104.stdout:6/386: unlink d0/d9/d25/l57 0 2024-04-26T04:42:13.275 INFO:tasks.workunit.client.1.smithi104.stdout:4/314: mkdir d4/db/d6b 0 2024-04-26T04:42:13.276 INFO:tasks.workunit.client.1.smithi104.stdout:3/312: mknod d5/d9/dc/d32/d48/c6a 0 2024-04-26T04:42:13.281 INFO:tasks.workunit.client.1.smithi104.stdout:7/216: creat d2/f42 x:0 0 0 2024-04-26T04:42:13.282 INFO:tasks.workunit.client.1.smithi104.stdout:9/211: symlink d11/l3e 0 2024-04-26T04:42:13.282 INFO:tasks.workunit.client.1.smithi104.stdout:9/212: fdatasync d11/f17 0 2024-04-26T04:42:13.282 INFO:tasks.workunit.client.1.smithi104.stdout:9/213: fsync d11/f12 0 2024-04-26T04:42:13.283 INFO:tasks.workunit.client.1.smithi104.stdout:9/214: write d11/f20 [21719,121864] 0 2024-04-26T04:42:13.283 INFO:tasks.workunit.client.1.smithi104.stdout:9/215: chown f4 321494 1 2024-04-26T04:42:13.283 INFO:tasks.workunit.client.1.smithi104.stdout:6/387: link d0/d9/l4d d0/d9/d13/d15/l81 0 2024-04-26T04:42:13.283 INFO:tasks.workunit.client.1.smithi104.stdout:6/388: truncate d0/d9/d13/d44/d71/f69 752144 0 2024-04-26T04:42:13.284 INFO:tasks.workunit.client.1.smithi104.stdout:4/315: creat d4/db/d6b/f6c x:0 0 0 2024-04-26T04:42:13.285 INFO:tasks.workunit.client.1.smithi104.stdout:3/313: symlink d5/d9/dc/d32/d47/l6b 0 2024-04-26T04:42:13.285 INFO:tasks.workunit.client.1.smithi104.stdout:5/299: sync 2024-04-26T04:42:13.285 INFO:tasks.workunit.client.1.smithi104.stdout:5/300: write d0/f18 [5506684,90541] 0 2024-04-26T04:42:13.285 INFO:tasks.workunit.client.1.smithi104.stdout:5/301: fdatasync d0/d1/d4/f24 0 2024-04-26T04:42:13.285 INFO:tasks.workunit.client.1.smithi104.stdout:5/302: write d0/d1/d1e/f21 [2476408,127606] 0 2024-04-26T04:42:13.286 INFO:tasks.workunit.client.1.smithi104.stdout:7/217: truncate d2/d4/f26 201852 0 2024-04-26T04:42:13.288 INFO:tasks.workunit.client.1.smithi104.stdout:9/216: symlink d11/d15/d32/l3f 0 2024-04-26T04:42:13.289 INFO:tasks.workunit.client.1.smithi104.stdout:6/389: mkdir d0/d9/d13/d41/d82 0 2024-04-26T04:42:13.290 INFO:tasks.workunit.client.1.smithi104.stdout:4/316: mkdir d4/d7/d1d/d1e/d6d 0 2024-04-26T04:42:13.291 INFO:tasks.workunit.client.1.smithi104.stdout:5/303: mknod d0/d1/d4/d2a/d2b/d37/d48/c60 0 2024-04-26T04:42:13.292 INFO:tasks.workunit.client.1.smithi104.stdout:3/314: dread d5/d9/dc/f42 [0,4194304] 0 2024-04-26T04:42:13.292 INFO:tasks.workunit.client.1.smithi104.stdout:7/218: chown d2/c1a 616 1 2024-04-26T04:42:13.292 INFO:tasks.workunit.client.1.smithi104.stdout:3/315: dread - d5/d9/dc/f36 zero size 2024-04-26T04:42:13.292 INFO:tasks.workunit.client.1.smithi104.stdout:7/219: write d2/d4/d32/d35/d37/f3a [1042879,23867] 0 2024-04-26T04:42:13.293 INFO:tasks.workunit.client.1.smithi104.stdout:6/390: truncate d0/d5/f22 59835 0 2024-04-26T04:42:13.293 INFO:tasks.workunit.client.1.smithi104.stdout:4/317: stat d4/d7/d1d/d1e/d15/c54 0 2024-04-26T04:42:13.294 INFO:tasks.workunit.client.1.smithi104.stdout:7/220: rmdir d2/d4 39 2024-04-26T04:42:13.296 INFO:tasks.workunit.client.1.smithi104.stdout:5/304: dread d0/d1/f2 [4194304,4194304] 0 2024-04-26T04:42:13.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/391: unlink d0/d9/d13/d44/f58 0 2024-04-26T04:42:13.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/392: write d0/d9/d13/d44/d47/f4c [685966,82217] 0 2024-04-26T04:42:13.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/393: chown d0/d9/d13/c5a 612 1 2024-04-26T04:42:13.297 INFO:tasks.workunit.client.1.smithi104.stdout:9/217: read fc [395111,34834] 0 2024-04-26T04:42:13.298 INFO:tasks.workunit.client.1.smithi104.stdout:4/318: link d4/d7/d1d/c5f d4/d7/d1d/d1e/d24/d59/c6e 0 2024-04-26T04:42:13.298 INFO:tasks.workunit.client.1.smithi104.stdout:9/218: write d11/d15/f30 [310275,94743] 0 2024-04-26T04:42:13.302 INFO:tasks.workunit.client.1.smithi104.stdout:7/221: mknod d2/d4/d32/d35/c43 0 2024-04-26T04:42:13.307 INFO:tasks.workunit.client.1.smithi104.stdout:9/219: write f6 [2313466,1405] 0 2024-04-26T04:42:13.307 INFO:tasks.workunit.client.1.smithi104.stdout:9/220: write d11/f16 [1404558,95342] 0 2024-04-26T04:42:13.309 INFO:tasks.workunit.client.1.smithi104.stdout:6/394: dread d0/d9/d13/d15/f42 [0,4194304] 0 2024-04-26T04:42:13.309 INFO:tasks.workunit.client.1.smithi104.stdout:6/395: fdatasync d0/d9/d13/d44/d71/f74 0 2024-04-26T04:42:13.309 INFO:tasks.workunit.client.1.smithi104.stdout:2/438: sync 2024-04-26T04:42:13.309 INFO:tasks.workunit.client.1.smithi104.stdout:1/310: sync 2024-04-26T04:42:13.309 INFO:tasks.workunit.client.1.smithi104.stdout:8/185: sync 2024-04-26T04:42:13.309 INFO:tasks.workunit.client.1.smithi104.stdout:8/186: chown d0/d13/f25 6720 1 2024-04-26T04:42:13.309 INFO:tasks.workunit.client.1.smithi104.stdout:8/187: dread - d0/d33/f2c zero size 2024-04-26T04:42:13.311 INFO:tasks.workunit.client.1.smithi104.stdout:8/188: dread d0/f1a [0,4194304] 0 2024-04-26T04:42:13.311 INFO:tasks.workunit.client.1.smithi104.stdout:8/189: chown d0/c37 131339842 1 2024-04-26T04:42:13.312 INFO:tasks.workunit.client.1.smithi104.stdout:8/190: write d0/d13/f25 [1396126,101557] 0 2024-04-26T04:42:13.316 INFO:tasks.workunit.client.1.smithi104.stdout:1/311: dread d0/d6/d49/f4a [0,4194304] 0 2024-04-26T04:42:13.316 INFO:tasks.workunit.client.1.smithi104.stdout:3/316: dwrite d5/d9/dc/f28 [0,4194304] 0 2024-04-26T04:42:13.319 INFO:tasks.workunit.client.1.smithi104.stdout:9/221: symlink d11/d1d/d34/l40 0 2024-04-26T04:42:13.319 INFO:tasks.workunit.client.1.smithi104.stdout:9/222: creat d11/d1d/d34/f41 x:0 0 0 2024-04-26T04:42:13.322 INFO:tasks.workunit.client.1.smithi104.stdout:6/396: mknod d0/d9/d25/c83 0 2024-04-26T04:42:13.325 INFO:tasks.workunit.client.1.smithi104.stdout:2/439: mknod d6/d8/d5a/c92 0 2024-04-26T04:42:13.325 INFO:tasks.workunit.client.1.smithi104.stdout:2/440: readlink d6/de/d8e/l57 0 2024-04-26T04:42:13.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/319: rmdir d4/d7 39 2024-04-26T04:42:13.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/320: write d4/d7/d1d/d1e/d15/f46 [816819,22694] 0 2024-04-26T04:42:13.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/321: write d4/d7/d1d/d2e/f67 [81476,24387] 0 2024-04-26T04:42:13.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/322: chown d4/d7/l3a 134379 1 2024-04-26T04:42:13.326 INFO:tasks.workunit.client.1.smithi104.stdout:1/312: mknod d0/d6/d19/d21/d59/c5e 0 2024-04-26T04:42:13.328 INFO:tasks.workunit.client.1.smithi104.stdout:5/305: truncate d0/f18 5764714 0 2024-04-26T04:42:13.329 INFO:tasks.workunit.client.1.smithi104.stdout:3/317: getdents d5/d9/dc 0 2024-04-26T04:42:13.330 INFO:tasks.workunit.client.1.smithi104.stdout:4/323: dread d4/d20/f51 [0,4194304] 0 2024-04-26T04:42:13.330 INFO:tasks.workunit.client.1.smithi104.stdout:4/324: chown d4/f69 3340164 1 2024-04-26T04:42:13.330 INFO:tasks.workunit.client.1.smithi104.stdout:4/325: stat d4/d7/d1d/d2e/l5e 0 2024-04-26T04:42:13.332 INFO:tasks.workunit.client.1.smithi104.stdout:9/223: symlink d11/d1d/d34/l42 0 2024-04-26T04:42:13.334 INFO:tasks.workunit.client.1.smithi104.stdout:8/191: dwrite d0/d33/f2b [0,4194304] 0 2024-04-26T04:42:13.334 INFO:tasks.workunit.client.1.smithi104.stdout:8/192: stat d0/d33/f2c 0 2024-04-26T04:42:13.334 INFO:tasks.workunit.client.1.smithi104.stdout:6/397: creat d0/d9/d13/d15/f84 x:0 0 0 2024-04-26T04:42:13.335 INFO:tasks.workunit.client.1.smithi104.stdout:1/313: symlink d0/d6/d19/d1b/d35/l5f 0 2024-04-26T04:42:13.337 INFO:tasks.workunit.client.1.smithi104.stdout:9/224: dread d11/d15/f30 [0,4194304] 0 2024-04-26T04:42:13.339 INFO:tasks.workunit.client.1.smithi104.stdout:3/318: unlink d5/d9/dc/f19 0 2024-04-26T04:42:13.340 INFO:tasks.workunit.client.1.smithi104.stdout:4/326: creat d4/d7/d1d/f6f x:0 0 0 2024-04-26T04:42:13.340 INFO:tasks.workunit.client.1.smithi104.stdout:4/327: write d4/d7/d1d/d1e/d24/d59/f5d [455931,46589] 0 2024-04-26T04:42:13.341 INFO:tasks.workunit.client.1.smithi104.stdout:4/328: readlink d4/d20/d55/l57 0 2024-04-26T04:42:13.341 INFO:tasks.workunit.client.1.smithi104.stdout:4/329: truncate d4/d7/d1d/d1e/d24/d59/f5d 897784 0 2024-04-26T04:42:13.341 INFO:tasks.workunit.client.1.smithi104.stdout:4/330: readlink d4/d7/d1d/d1e/d24/l4f 0 2024-04-26T04:42:13.342 INFO:tasks.workunit.client.1.smithi104.stdout:4/331: truncate d4/d7/f2a 4391558 0 2024-04-26T04:42:13.342 INFO:tasks.workunit.client.1.smithi104.stdout:4/332: fdatasync d4/d7/d1d/d1e/d24/d48/f4e 0 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:2/441: dwrite d6/f8b [0,4194304] 0 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:2/442: fsync d6/de/d1d/f83 0 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:2/443: truncate d6/d8/f37 887688 0 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:6/398: rmdir d0/d9/d13 39 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:6/399: fsync d0/d9/d13/d41/f7f 0 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:6/400: fdatasync d0/d9/d13/d15/d80/f6c 0 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:6/401: chown d0/d9/d13/d15/l81 38093 1 2024-04-26T04:42:13.343 INFO:tasks.workunit.client.1.smithi104.stdout:1/314: mknod d0/d6/d19/d1b/d35/c60 0 2024-04-26T04:42:13.345 INFO:tasks.workunit.client.1.smithi104.stdout:9/225: mknod d11/d15/d32/d35/c43 0 2024-04-26T04:42:13.346 INFO:tasks.workunit.client.1.smithi104.stdout:9/226: creat d11/f44 x:0 0 0 2024-04-26T04:42:13.348 INFO:tasks.workunit.client.1.smithi104.stdout:5/306: unlink d0/d1/c14 0 2024-04-26T04:42:13.349 INFO:tasks.workunit.client.1.smithi104.stdout:4/333: mknod d4/d20/c70 0 2024-04-26T04:42:13.349 INFO:tasks.workunit.client.1.smithi104.stdout:4/334: creat d4/f71 x:0 0 0 2024-04-26T04:42:13.350 INFO:tasks.workunit.client.1.smithi104.stdout:3/319: dread d5/d9/fa [0,4194304] 0 2024-04-26T04:42:13.351 INFO:tasks.workunit.client.1.smithi104.stdout:6/402: write d0/d9/d13/d15/d80/f75 [4007059,127651] 0 2024-04-26T04:42:13.352 INFO:tasks.workunit.client.1.smithi104.stdout:1/315: write d0/d6/d19/f3a [235400,89586] 0 2024-04-26T04:42:13.353 INFO:tasks.workunit.client.1.smithi104.stdout:2/444: mkdir d6/d1b/d93 0 2024-04-26T04:42:13.358 INFO:tasks.workunit.client.1.smithi104.stdout:9/227: unlink d11/l19 0 2024-04-26T04:42:13.358 INFO:tasks.workunit.client.1.smithi104.stdout:9/228: write d11/d1d/f2d [4645346,1807] 0 2024-04-26T04:42:13.360 INFO:tasks.workunit.client.1.smithi104.stdout:4/335: mkdir d4/d20/d55/d72 0 2024-04-26T04:42:13.361 INFO:tasks.workunit.client.1.smithi104.stdout:4/336: truncate d4/d7/d1d/d1e/d24/d59/f6a 502778 0 2024-04-26T04:42:13.361 INFO:tasks.workunit.client.1.smithi104.stdout:4/337: readlink d4/d20/l2d 0 2024-04-26T04:42:13.362 INFO:tasks.workunit.client.1.smithi104.stdout:3/320: creat d5/d9/d17/f6c x:0 0 0 2024-04-26T04:42:13.364 INFO:tasks.workunit.client.1.smithi104.stdout:6/403: creat d0/d9/d25/f85 x:0 0 0 2024-04-26T04:42:13.364 INFO:tasks.workunit.client.1.smithi104.stdout:2/445: dread d6/de/d1d/f25 [0,4194304] 0 2024-04-26T04:42:13.365 INFO:tasks.workunit.client.1.smithi104.stdout:1/316: link d0/d6/ff d0/d6/de/d20/d2a/f61 0 2024-04-26T04:42:13.367 INFO:tasks.workunit.client.1.smithi104.stdout:5/307: dwrite d0/d1/d1e/f2e [0,4194304] 0 2024-04-26T04:42:13.367 INFO:tasks.workunit.client.1.smithi104.stdout:5/308: fdatasync d0/d1/d25/f3c 0 2024-04-26T04:42:13.371 INFO:tasks.workunit.client.1.smithi104.stdout:3/321: rename d5/d9/d17/l62 to d5/d9/d17/d46/l6d 0 2024-04-26T04:42:13.372 INFO:tasks.workunit.client.1.smithi104.stdout:6/404: rmdir d0/d9/d13/d15/d80 39 2024-04-26T04:42:13.372 INFO:tasks.workunit.client.1.smithi104.stdout:6/405: write d0/d9/d13/d44/d71/f74 [866478,91731] 0 2024-04-26T04:42:13.373 INFO:tasks.workunit.client.1.smithi104.stdout:1/317: creat d0/d6/d2e/d32/f62 x:0 0 0 2024-04-26T04:42:13.374 INFO:tasks.workunit.client.1.smithi104.stdout:5/309: mknod d0/d1/d4/d2a/d2b/d37/d48/c61 0 2024-04-26T04:42:13.377 INFO:tasks.workunit.client.1.smithi104.stdout:4/338: rmdir d4 39 2024-04-26T04:42:13.378 INFO:tasks.workunit.client.1.smithi104.stdout:3/322: dread - d5/d9/dc/d32/f5a zero size 2024-04-26T04:42:13.379 INFO:tasks.workunit.client.1.smithi104.stdout:6/406: symlink d0/d9/l86 0 2024-04-26T04:42:13.379 INFO:tasks.workunit.client.1.smithi104.stdout:6/407: chown d0/d5/l5b 8790 1 2024-04-26T04:42:13.380 INFO:tasks.workunit.client.1.smithi104.stdout:1/318: mknod d0/d6/d19/d21/c63 0 2024-04-26T04:42:13.381 INFO:tasks.workunit.client.1.smithi104.stdout:5/310: mknod d0/d41/c62 0 2024-04-26T04:42:13.381 INFO:tasks.workunit.client.1.smithi104.stdout:5/311: creat d0/d1/d4/f63 x:0 0 0 2024-04-26T04:42:13.383 INFO:tasks.workunit.client.1.smithi104.stdout:6/408: truncate d0/d9/f3b 173689 0 2024-04-26T04:42:13.385 INFO:tasks.workunit.client.1.smithi104.stdout:1/319: mknod d0/d6/d2e/d32/d4d/c64 0 2024-04-26T04:42:13.387 INFO:tasks.workunit.client.1.smithi104.stdout:3/323: creat d5/d9/dc/d69/f6e x:0 0 0 2024-04-26T04:42:13.388 INFO:tasks.workunit.client.1.smithi104.stdout:6/409: creat d0/d9/f87 x:0 0 0 2024-04-26T04:42:13.395 INFO:tasks.workunit.client.1.smithi104.stdout:3/324: link d5/d9/f5d d5/d9/dc/d29/f6f 0 2024-04-26T04:42:13.396 INFO:tasks.workunit.client.1.smithi104.stdout:5/312: rename d0/d11 to d0/d1/d64 0 2024-04-26T04:42:13.397 INFO:tasks.workunit.client.1.smithi104.stdout:3/325: fdatasync d5/d9/dc/d3a/f57 0 2024-04-26T04:42:13.397 INFO:tasks.workunit.client.1.smithi104.stdout:3/326: chown d5/d9/dc/d69/f6e 2459211 1 2024-04-26T04:42:13.404 INFO:tasks.workunit.client.1.smithi104.stdout:1/320: dwrite d0/d6/de/d29/d3f/f5b [0,4194304] 0 2024-04-26T04:42:13.405 INFO:tasks.workunit.client.1.smithi104.stdout:5/313: write d0/d1/f2 [385561,102537] 0 2024-04-26T04:42:13.405 INFO:tasks.workunit.client.1.smithi104.stdout:4/339: dwrite d4/d7/d1d/d1e/d24/d48/f4e [0,4194304] 0 2024-04-26T04:42:13.405 INFO:tasks.workunit.client.1.smithi104.stdout:4/340: write d4/d7/f2a [4374796,87931] 0 2024-04-26T04:42:13.405 INFO:tasks.workunit.client.1.smithi104.stdout:4/341: truncate d4/d7/f2a 4724550 0 2024-04-26T04:42:13.406 INFO:tasks.workunit.client.1.smithi104.stdout:1/321: dread d0/d6/d19/d1b/d35/f40 [0,4194304] 0 2024-04-26T04:42:13.406 INFO:tasks.workunit.client.1.smithi104.stdout:1/322: fdatasync d0/d6/de/f41 0 2024-04-26T04:42:13.406 INFO:tasks.workunit.client.1.smithi104.stdout:1/323: chown d0/d6/d2e/d32/d4d/l3e 7779 1 2024-04-26T04:42:13.408 INFO:tasks.workunit.client.1.smithi104.stdout:5/314: truncate d0/d1/d64/f19 561538 0 2024-04-26T04:42:13.412 INFO:tasks.workunit.client.1.smithi104.stdout:4/342: rmdir d4/d20/d55 39 2024-04-26T04:42:13.414 INFO:tasks.workunit.client.1.smithi104.stdout:5/315: unlink d0/d1/d4/f12 0 2024-04-26T04:42:13.416 INFO:tasks.workunit.client.1.smithi104.stdout:5/316: creat d0/d1/f65 x:0 0 0 2024-04-26T04:42:13.416 INFO:tasks.workunit.client.1.smithi104.stdout:5/317: write d0/f5 [2662521,91810] 0 2024-04-26T04:42:13.416 INFO:tasks.workunit.client.1.smithi104.stdout:5/318: fsync d0/d1/d1e/f21 0 2024-04-26T04:42:13.416 INFO:tasks.workunit.client.1.smithi104.stdout:5/319: stat d0/d1/c1b 0 2024-04-26T04:42:13.416 INFO:tasks.workunit.client.1.smithi104.stdout:5/320: truncate d0/d1/d1e/f27 303325 0 2024-04-26T04:42:13.416 INFO:tasks.workunit.client.1.smithi104.stdout:5/321: dread - d0/d1/f52 zero size 2024-04-26T04:42:13.417 INFO:tasks.workunit.client.1.smithi104.stdout:5/322: dread - d0/d41/f55 zero size 2024-04-26T04:42:13.417 INFO:tasks.workunit.client.1.smithi104.stdout:4/343: mknod d4/d7/c73 0 2024-04-26T04:42:13.418 INFO:tasks.workunit.client.1.smithi104.stdout:3/327: rmdir d5 39 2024-04-26T04:42:13.419 INFO:tasks.workunit.client.1.smithi104.stdout:3/328: chown d5/d9/d17/d46/l63 135030 1 2024-04-26T04:42:13.419 INFO:tasks.workunit.client.1.smithi104.stdout:3/329: write d5/d9/d17/f3b [820166,422] 0 2024-04-26T04:42:13.420 INFO:tasks.workunit.client.1.smithi104.stdout:3/330: read d5/d9/dc/d69/f3e [171295,77243] 0 2024-04-26T04:42:13.421 INFO:tasks.workunit.client.1.smithi104.stdout:3/331: rename d5/d9/dc/d29/f6f to d5/d9/d17/d46/f70 0 2024-04-26T04:42:13.422 INFO:tasks.workunit.client.1.smithi104.stdout:3/332: symlink d5/d9/dc/d29/l71 0 2024-04-26T04:42:13.423 INFO:tasks.workunit.client.1.smithi104.stdout:1/324: dwrite d0/d6/f1a [4194304,4194304] 0 2024-04-26T04:42:13.424 INFO:tasks.workunit.client.1.smithi104.stdout:5/323: dread d0/d1/d64/d26/d36/f43 [0,4194304] 0 2024-04-26T04:42:13.424 INFO:tasks.workunit.client.1.smithi104.stdout:5/324: creat d0/f66 x:0 0 0 2024-04-26T04:42:13.424 INFO:tasks.workunit.client.1.smithi104.stdout:0/271: sync 2024-04-26T04:42:13.426 INFO:tasks.workunit.client.1.smithi104.stdout:3/333: dread d5/d9/dc/f40 [0,4194304] 0 2024-04-26T04:42:13.430 INFO:tasks.workunit.client.1.smithi104.stdout:4/344: read d4/d7/f63 [549413,41699] 0 2024-04-26T04:42:13.430 INFO:tasks.workunit.client.1.smithi104.stdout:3/334: getdents d5/d9/dc/d32/d47 0 2024-04-26T04:42:13.430 INFO:tasks.workunit.client.1.smithi104.stdout:3/335: fdatasync d5/d9/dc/d69/f55 0 2024-04-26T04:42:13.432 INFO:tasks.workunit.client.1.smithi104.stdout:5/325: mknod d0/d1/c67 0 2024-04-26T04:42:13.433 INFO:tasks.workunit.client.1.smithi104.stdout:1/325: write d0/d6/d2e/f46 [3166342,82563] 0 2024-04-26T04:42:13.434 INFO:tasks.workunit.client.1.smithi104.stdout:5/326: rename d0/d1/d25/f3c to d0/d1/d3e/f68 0 2024-04-26T04:42:13.436 INFO:tasks.workunit.client.1.smithi104.stdout:5/327: symlink d0/d1/d4/d2a/d2b/d37/l69 0 2024-04-26T04:42:13.436 INFO:tasks.workunit.client.1.smithi104.stdout:1/326: rmdir d0/d6/de/d20/d2a 39 2024-04-26T04:42:13.439 INFO:tasks.workunit.client.1.smithi104.stdout:3/336: read d5/d9/dc/f28 [3516476,69900] 0 2024-04-26T04:42:13.612 INFO:tasks.workunit.client.1.smithi104.stdout:5/328: mkdir d0/d1/d4/d2a/d2b/d6a 0 2024-04-26T04:42:13.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/327: mknod d0/d6/d19/d1b/d35/c65 0 2024-04-26T04:42:13.612 INFO:tasks.workunit.client.1.smithi104.stdout:3/337: mkdir d5/d9/d17/d72 0 2024-04-26T04:42:13.615 INFO:tasks.workunit.client.1.smithi104.stdout:5/329: mkdir d0/d1/d4/d2a/d6b 0 2024-04-26T04:42:13.617 INFO:tasks.workunit.client.1.smithi104.stdout:5/330: creat d0/d1/d64/f6c x:0 0 0 2024-04-26T04:42:13.618 INFO:tasks.workunit.client.1.smithi104.stdout:5/331: mknod d0/d1/d64/d26/c6d 0 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:0/272: sync 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:2/446: sync 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:9/229: sync 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:4/345: sync 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:4/346: fdatasync d4/db/f28 0 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:4/347: write d4/d7/d1d/d2e/f67 [258265,49408] 0 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:4/348: chown d4/d7/d1d/d1e/d24 9 1 2024-04-26T04:42:13.626 INFO:tasks.workunit.client.1.smithi104.stdout:8/193: sync 2024-04-26T04:42:13.627 INFO:tasks.workunit.client.1.smithi104.stdout:6/410: sync 2024-04-26T04:42:13.627 INFO:tasks.workunit.client.1.smithi104.stdout:6/411: readlink d0/d9/l4d 0 2024-04-26T04:42:13.627 INFO:tasks.workunit.client.1.smithi104.stdout:6/412: write d0/d9/d13/d44/d71/f4b [1279643,50480] 0 2024-04-26T04:42:13.627 INFO:tasks.workunit.client.1.smithi104.stdout:6/413: chown d0/d5/c66 4538 1 2024-04-26T04:42:13.627 INFO:tasks.workunit.client.1.smithi104.stdout:6/414: dread - d0/d9/d13/d15/d80/f6c zero size 2024-04-26T04:42:13.627 INFO:tasks.workunit.client.1.smithi104.stdout:7/222: sync 2024-04-26T04:42:13.627 INFO:tasks.workunit.client.1.smithi104.stdout:0/273: creat d0/d2/d21/d49/f55 x:0 0 0 2024-04-26T04:42:13.628 INFO:tasks.workunit.client.1.smithi104.stdout:0/274: stat d0/l2e 0 2024-04-26T04:42:13.628 INFO:tasks.workunit.client.1.smithi104.stdout:8/194: symlink d0/d33/l3c 0 2024-04-26T04:42:13.628 INFO:tasks.workunit.client.1.smithi104.stdout:4/349: getdents d4/d7 0 2024-04-26T04:42:13.630 INFO:tasks.workunit.client.1.smithi104.stdout:4/350: rename d4/d7/d1d/d1e/d6d to d4/d7/d1d/d2e/d74 0 2024-04-26T04:42:13.634 INFO:tasks.workunit.client.1.smithi104.stdout:4/351: link d4/d7/d1d/d1e/d24/d59/c6e d4/d7/d1d/d2e/c75 0 2024-04-26T04:42:13.637 INFO:tasks.workunit.client.1.smithi104.stdout:4/352: creat d4/d7/d1d/d1e/d24/f76 x:0 0 0 2024-04-26T04:42:13.640 INFO:tasks.workunit.client.1.smithi104.stdout:4/353: symlink d4/d20/d55/d72/l77 0 2024-04-26T04:42:13.644 INFO:tasks.workunit.client.1.smithi104.stdout:6/415: dread d0/d9/d13/d15/d80/f75 [0,4194304] 0 2024-04-26T04:42:13.644 INFO:tasks.workunit.client.1.smithi104.stdout:6/416: dread - d0/d9/d25/f68 zero size 2024-04-26T04:42:13.645 INFO:tasks.workunit.client.1.smithi104.stdout:6/417: symlink d0/d1b/l88 0 2024-04-26T04:42:13.654 INFO:tasks.workunit.client.1.smithi104.stdout:3/338: dwrite d5/d9/dc/f16 [0,4194304] 0 2024-04-26T04:42:13.657 INFO:tasks.workunit.client.1.smithi104.stdout:1/328: dwrite d0/d6/d49/f4a [0,4194304] 0 2024-04-26T04:42:13.663 INFO:tasks.workunit.client.1.smithi104.stdout:1/329: rename d0/d6/de to d0/d6/d19/d21/d66 0 2024-04-26T04:42:13.664 INFO:tasks.workunit.client.1.smithi104.stdout:1/330: mkdir d0/d6/d19/d21/d66/d20/d2a/d56/d67 0 2024-04-26T04:42:13.679 INFO:tasks.workunit.client.1.smithi104.stdout:7/223: dwrite d2/d4/d1d/f29 [0,4194304] 0 2024-04-26T04:42:13.679 INFO:tasks.workunit.client.1.smithi104.stdout:7/224: dread - d2/d4/d1d/f3e zero size 2024-04-26T04:42:13.679 INFO:tasks.workunit.client.1.smithi104.stdout:7/225: write d2/d4/f11 [2353084,51988] 0 2024-04-26T04:42:13.680 INFO:tasks.workunit.client.1.smithi104.stdout:7/226: rmdir d2/d4/d32 39 2024-04-26T04:42:13.680 INFO:tasks.workunit.client.1.smithi104.stdout:7/227: write d2/f42 [852459,63126] 0 2024-04-26T04:42:13.680 INFO:tasks.workunit.client.1.smithi104.stdout:7/228: mkdir d2/d4/d32/d35/d44 0 2024-04-26T04:42:13.681 INFO:tasks.workunit.client.1.smithi104.stdout:7/229: symlink d2/d4/d1d/l45 0 2024-04-26T04:42:13.682 INFO:tasks.workunit.client.1.smithi104.stdout:7/230: unlink d2/l2a 0 2024-04-26T04:42:13.683 INFO:tasks.workunit.client.1.smithi104.stdout:7/231: link d2/d4/l17 d2/d4/d32/d35/l46 0 2024-04-26T04:42:13.683 INFO:tasks.workunit.client.1.smithi104.stdout:7/232: write d2/d4/f26 [860818,110135] 0 2024-04-26T04:42:13.684 INFO:tasks.workunit.client.1.smithi104.stdout:7/233: mkdir d2/d4/d32/d3d/d47 0 2024-04-26T04:42:13.689 INFO:tasks.workunit.client.1.smithi104.stdout:7/234: dread d2/f2b [4194304,4194304] 0 2024-04-26T04:42:13.690 INFO:tasks.workunit.client.1.smithi104.stdout:7/235: getdents d2/d4/d32 0 2024-04-26T04:42:13.690 INFO:tasks.workunit.client.1.smithi104.stdout:7/236: stat d2/f42 0 2024-04-26T04:42:13.693 INFO:tasks.workunit.client.1.smithi104.stdout:9/230: dwrite d11/d1d/d34/f36 [0,4194304] 0 2024-04-26T04:42:13.693 INFO:tasks.workunit.client.1.smithi104.stdout:2/447: dwrite d6/d8/f37 [0,4194304] 0 2024-04-26T04:42:13.693 INFO:tasks.workunit.client.1.smithi104.stdout:2/448: chown d6/de/c7b 201678 1 2024-04-26T04:42:13.695 INFO:tasks.workunit.client.1.smithi104.stdout:9/231: mknod d11/d1d/d34/c45 0 2024-04-26T04:42:13.695 INFO:tasks.workunit.client.1.smithi104.stdout:9/232: truncate d11/d15/f18 4827313 0 2024-04-26T04:42:13.697 INFO:tasks.workunit.client.1.smithi104.stdout:0/275: dwrite d0/d4/f11 [0,4194304] 0 2024-04-26T04:42:13.697 INFO:tasks.workunit.client.1.smithi104.stdout:0/276: write d0/fe [3176432,124286] 0 2024-04-26T04:42:13.698 INFO:tasks.workunit.client.1.smithi104.stdout:5/332: dwrite d0/d1/f2 [8388608,4194304] 0 2024-04-26T04:42:13.698 INFO:tasks.workunit.client.1.smithi104.stdout:6/418: dwrite d0/d9/f87 [0,4194304] 0 2024-04-26T04:42:13.698 INFO:tasks.workunit.client.1.smithi104.stdout:4/354: dwrite d4/d7/d1d/d1e/d15/f66 [0,4194304] 0 2024-04-26T04:42:13.698 INFO:tasks.workunit.client.1.smithi104.stdout:4/355: truncate d4/d7/d1d/d1e/f5b 281675 0 2024-04-26T04:42:13.698 INFO:tasks.workunit.client.1.smithi104.stdout:6/419: write d0/d9/d13/d44/d71/d3f/f5d [92465,44787] 0 2024-04-26T04:42:13.698 INFO:tasks.workunit.client.1.smithi104.stdout:6/420: fdatasync d0/d9/d13/d15/d80/f61 0 2024-04-26T04:42:13.698 INFO:tasks.workunit.client.1.smithi104.stdout:3/339: dwrite d5/d9/d17/f2a [0,4194304] 0 2024-04-26T04:42:13.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/331: dwrite d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:13.701 INFO:tasks.workunit.client.1.smithi104.stdout:4/356: dread d4/db/f28 [0,4194304] 0 2024-04-26T04:42:13.703 INFO:tasks.workunit.client.1.smithi104.stdout:0/277: creat d0/d4/d15/f56 x:0 0 0 2024-04-26T04:42:13.704 INFO:tasks.workunit.client.1.smithi104.stdout:0/278: chown d0/d4/d15/l20 31576 1 2024-04-26T04:42:13.707 INFO:tasks.workunit.client.1.smithi104.stdout:5/333: mknod d0/d1/d64/c6e 0 2024-04-26T04:42:13.707 INFO:tasks.workunit.client.1.smithi104.stdout:5/334: write d0/d1/d4/d2a/d2b/f42 [245772,97257] 0 2024-04-26T04:42:13.710 INFO:tasks.workunit.client.1.smithi104.stdout:2/449: dwrite d6/de/d1d/f80 [0,4194304] 0 2024-04-26T04:42:13.710 INFO:tasks.workunit.client.1.smithi104.stdout:2/450: write d6/d8/fb [1658939,4816] 0 2024-04-26T04:42:13.710 INFO:tasks.workunit.client.1.smithi104.stdout:2/451: chown d6/de/d1d/f43 13245318 1 2024-04-26T04:42:13.713 INFO:tasks.workunit.client.1.smithi104.stdout:6/421: rename d0/d9/d13/c63 to d0/d1b/c89 0 2024-04-26T04:42:13.719 INFO:tasks.workunit.client.1.smithi104.stdout:3/340: readlink d5/d9/dc/l18 0 2024-04-26T04:42:13.719 INFO:tasks.workunit.client.1.smithi104.stdout:9/233: getdents d11/d15/d1b 0 2024-04-26T04:42:13.719 INFO:tasks.workunit.client.1.smithi104.stdout:4/357: getdents d4/db 0 2024-04-26T04:42:13.719 INFO:tasks.workunit.client.1.smithi104.stdout:4/358: chown d4/d7/d1d 15224238 1 2024-04-26T04:42:13.720 INFO:tasks.workunit.client.1.smithi104.stdout:5/335: truncate d0/d1/d4/d2a/d2b/d37/f57 1189606 0 2024-04-26T04:42:13.721 INFO:tasks.workunit.client.1.smithi104.stdout:7/237: dwrite d2/d4/f11 [4194304,4194304] 0 2024-04-26T04:42:13.721 INFO:tasks.workunit.client.1.smithi104.stdout:9/234: dread d11/f20 [0,4194304] 0 2024-04-26T04:42:13.723 INFO:tasks.workunit.client.1.smithi104.stdout:2/452: rmdir d6/de/d8e 39 2024-04-26T04:42:13.726 INFO:tasks.workunit.client.1.smithi104.stdout:3/341: creat d5/d9/d17/f73 x:0 0 0 2024-04-26T04:42:13.727 INFO:tasks.workunit.client.1.smithi104.stdout:4/359: getdents d4/d7/d1d/d1e/d24/d48 0 2024-04-26T04:42:13.727 INFO:tasks.workunit.client.1.smithi104.stdout:4/360: dread - d4/d7/d1d/f6f zero size 2024-04-26T04:42:13.727 INFO:tasks.workunit.client.1.smithi104.stdout:0/279: rmdir d0/d2/d21/d38 39 2024-04-26T04:42:13.728 INFO:tasks.workunit.client.1.smithi104.stdout:0/280: read - d0/d2/d10/d3d/f4c zero size 2024-04-26T04:42:13.728 INFO:tasks.workunit.client.1.smithi104.stdout:3/342: dread d5/d9/dc/f40 [0,4194304] 0 2024-04-26T04:42:13.728 INFO:tasks.workunit.client.1.smithi104.stdout:3/343: chown d5/f22 14603401 1 2024-04-26T04:42:13.728 INFO:tasks.workunit.client.1.smithi104.stdout:3/344: chown d5/d9/d17/f2a 493 1 2024-04-26T04:42:13.729 INFO:tasks.workunit.client.1.smithi104.stdout:9/235: rename d11/d1d/f2d to d11/d15/d1b/f46 0 2024-04-26T04:42:13.730 INFO:tasks.workunit.client.1.smithi104.stdout:9/236: truncate d11/d1d/d34/f36 5182667 0 2024-04-26T04:42:13.730 INFO:tasks.workunit.client.1.smithi104.stdout:1/332: dwrite d0/d6/d19/d21/d66/d29/f39 [0,4194304] 0 2024-04-26T04:42:13.730 INFO:tasks.workunit.client.1.smithi104.stdout:7/238: symlink d2/d4/d32/l48 0 2024-04-26T04:42:13.731 INFO:tasks.workunit.client.1.smithi104.stdout:4/361: mknod d4/db/c78 0 2024-04-26T04:42:13.732 INFO:tasks.workunit.client.1.smithi104.stdout:2/453: dread d6/de/d1d/f42 [0,4194304] 0 2024-04-26T04:42:13.733 INFO:tasks.workunit.client.1.smithi104.stdout:3/345: creat d5/d9/dc/d29/d49/d59/f74 x:0 0 0 2024-04-26T04:42:13.734 INFO:tasks.workunit.client.1.smithi104.stdout:3/346: getdents d5/d9/d17/d72 0 2024-04-26T04:42:13.736 INFO:tasks.workunit.client.1.smithi104.stdout:7/239: mknod d2/d4/d32/c49 0 2024-04-26T04:42:13.737 INFO:tasks.workunit.client.1.smithi104.stdout:4/362: symlink d4/d7/d1d/d2e/l79 0 2024-04-26T04:42:13.738 INFO:tasks.workunit.client.1.smithi104.stdout:2/454: symlink d6/d8/l94 0 2024-04-26T04:42:13.740 INFO:tasks.workunit.client.1.smithi104.stdout:2/455: rmdir d6/d1b/d51/d6a/d8a 0 2024-04-26T04:42:13.741 INFO:tasks.workunit.client.1.smithi104.stdout:3/347: symlink d5/d9/dc/l75 0 2024-04-26T04:42:13.742 INFO:tasks.workunit.client.1.smithi104.stdout:2/456: creat d6/d1b/d68/f95 x:0 0 0 2024-04-26T04:42:13.742 INFO:tasks.workunit.client.1.smithi104.stdout:6/422: getdents d0/d9/d13 0 2024-04-26T04:42:13.747 INFO:tasks.workunit.client.1.smithi104.stdout:6/423: symlink d0/l8a 0 2024-04-26T04:42:13.747 INFO:tasks.workunit.client.1.smithi104.stdout:9/237: dread fb [4194304,4194304] 0 2024-04-26T04:42:13.747 INFO:tasks.workunit.client.1.smithi104.stdout:3/348: dread d5/d9/fa [0,4194304] 0 2024-04-26T04:42:13.747 INFO:tasks.workunit.client.1.smithi104.stdout:9/238: dread d11/f27 [0,4194304] 0 2024-04-26T04:42:13.748 INFO:tasks.workunit.client.1.smithi104.stdout:2/457: dread d6/de/d1d/f25 [0,4194304] 0 2024-04-26T04:42:13.749 INFO:tasks.workunit.client.1.smithi104.stdout:6/424: dread d0/d5/f22 [0,4194304] 0 2024-04-26T04:42:13.749 INFO:tasks.workunit.client.1.smithi104.stdout:6/425: write d0/d9/d13/d44/d71/f4b [1399907,80336] 0 2024-04-26T04:42:13.749 INFO:tasks.workunit.client.1.smithi104.stdout:6/426: chown d0/d9/d13/d44/d47/l6d 1798415984 1 2024-04-26T04:42:13.750 INFO:tasks.workunit.client.1.smithi104.stdout:3/349: mknod d5/d9/d17/d60/c76 0 2024-04-26T04:42:13.750 INFO:tasks.workunit.client.1.smithi104.stdout:3/350: truncate f0 5254171 0 2024-04-26T04:42:13.752 INFO:tasks.workunit.client.1.smithi104.stdout:9/239: link d11/d15/d1b/f37 d11/d15/d1b/f47 0 2024-04-26T04:42:13.752 INFO:tasks.workunit.client.1.smithi104.stdout:9/240: chown fb 51 1 2024-04-26T04:42:13.752 INFO:tasks.workunit.client.1.smithi104.stdout:2/458: mknod d6/d1b/d68/d81/c96 0 2024-04-26T04:42:13.752 INFO:tasks.workunit.client.1.smithi104.stdout:1/333: dread d0/d6/d19/d21/d28/f36 [0,4194304] 0 2024-04-26T04:42:13.752 INFO:tasks.workunit.client.1.smithi104.stdout:1/334: chown d0/d6/d19/d21/d28 10034 1 2024-04-26T04:42:13.753 INFO:tasks.workunit.client.1.smithi104.stdout:4/363: dread d4/d7/d1d/d1e/d24/d48/f4e [0,4194304] 0 2024-04-26T04:42:13.753 INFO:tasks.workunit.client.1.smithi104.stdout:6/427: dread d0/d9/f3b [0,4194304] 0 2024-04-26T04:42:13.753 INFO:tasks.workunit.client.1.smithi104.stdout:6/428: write d0/d9/d13/f20 [3858439,8007] 0 2024-04-26T04:42:13.753 INFO:tasks.workunit.client.1.smithi104.stdout:2/459: mkdir d6/d1b/d68/d81/d97 0 2024-04-26T04:42:13.754 INFO:tasks.workunit.client.1.smithi104.stdout:7/240: dread d2/f2f [4194304,4194304] 0 2024-04-26T04:42:13.754 INFO:tasks.workunit.client.1.smithi104.stdout:7/241: chown d2/d4/d32/l48 37255 1 2024-04-26T04:42:13.755 INFO:tasks.workunit.client.1.smithi104.stdout:6/429: mkdir d0/d9/d13/d41/d8b 0 2024-04-26T04:42:13.755 INFO:tasks.workunit.client.1.smithi104.stdout:6/430: chown d0/d9/d13 81056670 1 2024-04-26T04:42:13.755 INFO:tasks.workunit.client.1.smithi104.stdout:6/431: fsync d0/d9/d25/f85 0 2024-04-26T04:42:13.755 INFO:tasks.workunit.client.1.smithi104.stdout:2/460: truncate d6/de/d1d/f6d 770789 0 2024-04-26T04:42:13.756 INFO:tasks.workunit.client.1.smithi104.stdout:5/336: dwrite d0/f22 [0,4194304] 0 2024-04-26T04:42:13.759 INFO:tasks.workunit.client.1.smithi104.stdout:2/461: mknod d6/de/c98 0 2024-04-26T04:42:13.759 INFO:tasks.workunit.client.1.smithi104.stdout:7/242: link d2/d4/d32/c49 d2/d4/c4a 0 2024-04-26T04:42:13.759 INFO:tasks.workunit.client.1.smithi104.stdout:2/462: write d6/d1b/f71 [841273,27709] 0 2024-04-26T04:42:13.759 INFO:tasks.workunit.client.1.smithi104.stdout:0/281: dwrite d0/d2/d21/d38/f36 [0,4194304] 0 2024-04-26T04:42:13.759 INFO:tasks.workunit.client.1.smithi104.stdout:6/432: dread d0/f6e [0,4194304] 0 2024-04-26T04:42:13.761 INFO:tasks.workunit.client.1.smithi104.stdout:7/243: rename d2/d4/d32/d35/l36 to d2/d4/d32/d3d/d41/l4b 0 2024-04-26T04:42:13.771 INFO:tasks.workunit.client.1.smithi104.stdout:0/282: truncate d0/fa 2245615 0 2024-04-26T04:42:13.774 INFO:tasks.workunit.client.1.smithi104.stdout:7/244: stat d2/d4/l17 0 2024-04-26T04:42:13.775 INFO:tasks.workunit.client.1.smithi104.stdout:0/283: mkdir d0/d2/d21/d27/d1e/d57 0 2024-04-26T04:42:13.775 INFO:tasks.workunit.client.1.smithi104.stdout:0/284: truncate d0/d4/f29 1275960 0 2024-04-26T04:42:13.777 INFO:tasks.workunit.client.1.smithi104.stdout:0/285: mknod d0/d4/d3b/c58 0 2024-04-26T04:42:13.778 INFO:tasks.workunit.client.1.smithi104.stdout:5/337: dread d0/d1/d64/f31 [0,4194304] 0 2024-04-26T04:42:13.779 INFO:tasks.workunit.client.1.smithi104.stdout:0/286: truncate d0/d2/d21/d27/d1e/f2c 109400 0 2024-04-26T04:42:13.780 INFO:tasks.workunit.client.1.smithi104.stdout:0/287: creat d0/d4/d15/f59 x:0 0 0 2024-04-26T04:42:13.783 INFO:tasks.workunit.client.1.smithi104.stdout:0/288: unlink d0/d4/f7 0 2024-04-26T04:42:13.784 INFO:tasks.workunit.client.1.smithi104.stdout:0/289: rmdir d0/d4/d3c/d51 0 2024-04-26T04:42:13.787 INFO:tasks.workunit.client.1.smithi104.stdout:9/241: dwrite d11/f16 [0,4194304] 0 2024-04-26T04:42:13.795 INFO:tasks.workunit.client.1.smithi104.stdout:5/338: write d0/d1/d1e/f30 [1799039,101687] 0 2024-04-26T04:42:13.796 INFO:tasks.workunit.client.1.smithi104.stdout:5/339: symlink d0/d1/d4/d2a/d2b/d37/d48/l6f 0 2024-04-26T04:42:13.800 INFO:tasks.workunit.client.1.smithi104.stdout:2/463: dwrite d6/f62 [0,4194304] 0 2024-04-26T04:42:13.800 INFO:tasks.workunit.client.1.smithi104.stdout:1/335: dwrite d0/d6/d2e/f46 [4194304,4194304] 0 2024-04-26T04:42:13.800 INFO:tasks.workunit.client.1.smithi104.stdout:1/336: fdatasync d0/d6/d19/f3a 0 2024-04-26T04:42:13.801 INFO:tasks.workunit.client.1.smithi104.stdout:1/337: readlink d0/d6/d19/d21/l37 0 2024-04-26T04:42:13.802 INFO:tasks.workunit.client.1.smithi104.stdout:8/195: sync 2024-04-26T04:42:13.803 INFO:tasks.workunit.client.1.smithi104.stdout:4/364: dwrite d4/f37 [0,4194304] 0 2024-04-26T04:42:13.803 INFO:tasks.workunit.client.1.smithi104.stdout:7/245: dwrite d2/d4/f33 [0,4194304] 0 2024-04-26T04:42:13.806 INFO:tasks.workunit.client.1.smithi104.stdout:4/365: creat d4/d7/d1d/f7a x:0 0 0 2024-04-26T04:42:13.806 INFO:tasks.workunit.client.1.smithi104.stdout:4/366: readlink d4/d20/d55/d56/l34 0 2024-04-26T04:42:13.808 INFO:tasks.workunit.client.1.smithi104.stdout:8/196: dread d0/d13/f25 [0,4194304] 0 2024-04-26T04:42:13.810 INFO:tasks.workunit.client.1.smithi104.stdout:0/290: dwrite d0/d2/d21/d38/f36 [0,4194304] 0 2024-04-26T04:42:13.811 INFO:tasks.workunit.client.1.smithi104.stdout:0/291: creat d0/d2/d10/f5a x:0 0 0 2024-04-26T04:42:13.811 INFO:tasks.workunit.client.1.smithi104.stdout:0/292: write d0/f2f [2469900,95967] 0 2024-04-26T04:42:13.811 INFO:tasks.workunit.client.1.smithi104.stdout:8/197: truncate d0/d13/f25 695943 0 2024-04-26T04:42:13.816 INFO:tasks.workunit.client.1.smithi104.stdout:8/198: symlink d0/d33/d35/l3d 0 2024-04-26T04:42:13.816 INFO:tasks.workunit.client.1.smithi104.stdout:8/199: fsync d0/fe 0 2024-04-26T04:42:13.819 INFO:tasks.workunit.client.1.smithi104.stdout:8/200: creat d0/d33/d35/f3e x:0 0 0 2024-04-26T04:42:13.820 INFO:tasks.workunit.client.1.smithi104.stdout:9/242: dwrite d11/d1d/d34/f36 [4194304,4194304] 0 2024-04-26T04:42:13.820 INFO:tasks.workunit.client.1.smithi104.stdout:8/201: rmdir d0/d1c 39 2024-04-26T04:42:13.824 INFO:tasks.workunit.client.1.smithi104.stdout:2/464: dread d6/f8b [0,4194304] 0 2024-04-26T04:42:13.825 INFO:tasks.workunit.client.1.smithi104.stdout:8/202: rename d0/d33/f31 to d0/f3f 0 2024-04-26T04:42:13.826 INFO:tasks.workunit.client.1.smithi104.stdout:8/203: creat d0/d1c/d27/f40 x:0 0 0 2024-04-26T04:42:13.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/204: mknod d0/c41 0 2024-04-26T04:42:13.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/205: chown d0/l4 196013214 1 2024-04-26T04:42:13.828 INFO:tasks.workunit.client.1.smithi104.stdout:8/206: creat d0/d13/f42 x:0 0 0 2024-04-26T04:42:13.842 INFO:tasks.workunit.client.1.smithi104.stdout:9/243: read d11/f12 [3522139,125592] 0 2024-04-26T04:42:13.842 INFO:tasks.workunit.client.1.smithi104.stdout:9/244: fdatasync d11/d1d/f31 0 2024-04-26T04:42:13.849 INFO:tasks.workunit.client.1.smithi104.stdout:1/338: dwrite d0/d6/f1a [4194304,4194304] 0 2024-04-26T04:42:13.849 INFO:tasks.workunit.client.1.smithi104.stdout:4/367: dwrite d4/d7/d1d/f7a [0,4194304] 0 2024-04-26T04:42:13.849 INFO:tasks.workunit.client.1.smithi104.stdout:5/340: dwrite d0/d1/d4/d2a/d2b/f35 [0,4194304] 0 2024-04-26T04:42:13.850 INFO:tasks.workunit.client.1.smithi104.stdout:1/339: truncate d0/d6/d19/d21/d66/d29/f39 4050414 0 2024-04-26T04:42:13.850 INFO:tasks.workunit.client.1.smithi104.stdout:9/245: write d11/d1d/d34/f36 [2100294,61331] 0 2024-04-26T04:42:13.850 INFO:tasks.workunit.client.1.smithi104.stdout:0/293: dwrite d0/d2/d10/f5a [0,4194304] 0 2024-04-26T04:42:13.856 INFO:tasks.workunit.client.1.smithi104.stdout:4/368: symlink d4/d20/l7b 0 2024-04-26T04:42:13.856 INFO:tasks.workunit.client.1.smithi104.stdout:4/369: dread - d4/d7/d1d/d1e/d24/d4a/f50 zero size 2024-04-26T04:42:13.859 INFO:tasks.workunit.client.1.smithi104.stdout:5/341: write d0/f6 [2222441,6464] 0 2024-04-26T04:42:13.859 INFO:tasks.workunit.client.1.smithi104.stdout:5/342: write d0/d1/f52 [344725,63314] 0 2024-04-26T04:42:13.860 INFO:tasks.workunit.client.1.smithi104.stdout:9/246: mknod d11/d1d/d34/c48 0 2024-04-26T04:42:13.861 INFO:tasks.workunit.client.1.smithi104.stdout:9/247: fdatasync d11/d15/d32/d35/f3a 0 2024-04-26T04:42:13.861 INFO:tasks.workunit.client.1.smithi104.stdout:9/248: stat d11/d15/d32/d35 0 2024-04-26T04:42:13.862 INFO:tasks.workunit.client.1.smithi104.stdout:8/207: fsync d0/d1c/d27/f40 0 2024-04-26T04:42:13.864 INFO:tasks.workunit.client.1.smithi104.stdout:4/370: rmdir d4/d7/d1d/d1e/d24/d48 39 2024-04-26T04:42:13.864 INFO:tasks.workunit.client.1.smithi104.stdout:4/371: chown d4/d7/d1d/d1e/d24/d59 132169398 1 2024-04-26T04:42:13.864 INFO:tasks.workunit.client.1.smithi104.stdout:4/372: write d4/d7/d1d/d1e/d24/d48/f60 [434282,79598] 0 2024-04-26T04:42:13.868 INFO:tasks.workunit.client.1.smithi104.stdout:5/343: creat d0/d1/d4/d2a/d2b/d37/d48/f70 x:0 0 0 2024-04-26T04:42:13.869 INFO:tasks.workunit.client.1.smithi104.stdout:4/373: dread d4/d7/d1d/d1e/d15/f46 [0,4194304] 0 2024-04-26T04:42:13.870 INFO:tasks.workunit.client.1.smithi104.stdout:4/374: rmdir d4/d20/d55 39 2024-04-26T04:42:13.873 INFO:tasks.workunit.client.1.smithi104.stdout:4/375: creat d4/d7/d1d/d1e/d24/d48/f7c x:0 0 0 2024-04-26T04:42:13.877 INFO:tasks.workunit.client.1.smithi104.stdout:5/344: dread d0/d1/d64/d26/d36/f43 [0,4194304] 0 2024-04-26T04:42:13.878 INFO:tasks.workunit.client.1.smithi104.stdout:0/294: dwrite d0/d4/d3c/f4b [0,4194304] 0 2024-04-26T04:42:13.878 INFO:tasks.workunit.client.1.smithi104.stdout:4/376: mkdir d4/d20/d55/d72/d7d 0 2024-04-26T04:42:13.881 INFO:tasks.workunit.client.1.smithi104.stdout:0/295: creat d0/d2/d21/d38/f5b x:0 0 0 2024-04-26T04:42:13.882 INFO:tasks.workunit.client.1.smithi104.stdout:4/377: link d4/db/f28 d4/d7/d1d/d2e/f7e 0 2024-04-26T04:42:13.883 INFO:tasks.workunit.client.1.smithi104.stdout:0/296: symlink d0/d2/d10/d3d/l5c 0 2024-04-26T04:42:13.885 INFO:tasks.workunit.client.1.smithi104.stdout:8/208: dwrite d0/d33/f28 [0,4194304] 0 2024-04-26T04:42:13.885 INFO:tasks.workunit.client.1.smithi104.stdout:1/340: dwrite d0/d6/d19/d1b/d35/f40 [0,4194304] 0 2024-04-26T04:42:13.885 INFO:tasks.workunit.client.1.smithi104.stdout:8/209: write d0/d13/f38 [239683,129468] 0 2024-04-26T04:42:13.885 INFO:tasks.workunit.client.1.smithi104.stdout:1/341: creat d0/d6/d49/f68 x:0 0 0 2024-04-26T04:42:13.885 INFO:tasks.workunit.client.1.smithi104.stdout:1/342: write d0/d6/d49/f68 [542676,20243] 0 2024-04-26T04:42:13.885 INFO:tasks.workunit.client.1.smithi104.stdout:1/343: chown d0/d6/d2e 186778 1 2024-04-26T04:42:13.893 INFO:tasks.workunit.client.1.smithi104.stdout:1/344: truncate d0/d6/d19/d21/d66/d29/d3f/f5b 1117000 0 2024-04-26T04:42:13.909 INFO:tasks.workunit.client.1.smithi104.stdout:4/378: dwrite d4/d7/d1d/d1e/d24/d59/f6a [0,4194304] 0 2024-04-26T04:42:13.909 INFO:tasks.workunit.client.1.smithi104.stdout:1/345: dread d0/d6/d19/d21/d66/f41 [0,4194304] 0 2024-04-26T04:42:13.909 INFO:tasks.workunit.client.1.smithi104.stdout:4/379: chown d4/d7/d1d/d1e/d24/l4f 63 1 2024-04-26T04:42:13.909 INFO:tasks.workunit.client.1.smithi104.stdout:1/346: write d0/d6/d49/f68 [1001280,87080] 0 2024-04-26T04:42:13.909 INFO:tasks.workunit.client.1.smithi104.stdout:1/347: write d0/d6/d19/d21/d66/d20/d2a/f31 [52633,43177] 0 2024-04-26T04:42:13.913 INFO:tasks.workunit.client.1.smithi104.stdout:5/345: dwrite d0/d1/d4/f63 [0,4194304] 0 2024-04-26T04:42:13.913 INFO:tasks.workunit.client.1.smithi104.stdout:5/346: chown d0/d1/f3d 4738242 1 2024-04-26T04:42:13.914 INFO:tasks.workunit.client.1.smithi104.stdout:5/347: dread - d0/d1/d3e/f68 zero size 2024-04-26T04:42:13.919 INFO:tasks.workunit.client.1.smithi104.stdout:5/348: truncate d0/d1/d4/d2a/d2b/d37/f57 482535 0 2024-04-26T04:42:13.919 INFO:tasks.workunit.client.1.smithi104.stdout:5/349: chown d0/d41/c4a 262019 1 2024-04-26T04:42:13.922 INFO:tasks.workunit.client.1.smithi104.stdout:5/350: link d0/d1/d64/c6e d0/d1/c71 0 2024-04-26T04:42:13.930 INFO:tasks.workunit.client.1.smithi104.stdout:0/297: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:13.930 INFO:tasks.workunit.client.1.smithi104.stdout:0/298: write d0/d2/d21/d27/d1e/f2c [443385,74709] 0 2024-04-26T04:42:13.930 INFO:tasks.workunit.client.1.smithi104.stdout:0/299: write d0/d2/d21/d38/f5b [510372,17952] 0 2024-04-26T04:42:13.931 INFO:tasks.workunit.client.1.smithi104.stdout:8/210: dwrite d0/f2f [0,4194304] 0 2024-04-26T04:42:13.934 INFO:tasks.workunit.client.1.smithi104.stdout:8/211: rename d0/d13/c17 to d0/d33/d35/c43 0 2024-04-26T04:42:13.934 INFO:tasks.workunit.client.1.smithi104.stdout:8/212: write d0/f34 [588875,26232] 0 2024-04-26T04:42:13.937 INFO:tasks.workunit.client.1.smithi104.stdout:5/351: dread d0/f22 [0,4194304] 0 2024-04-26T04:42:13.937 INFO:tasks.workunit.client.1.smithi104.stdout:1/348: dwrite d0/d6/d19/d21/d66/d29/f39 [0,4194304] 0 2024-04-26T04:42:13.938 INFO:tasks.workunit.client.1.smithi104.stdout:5/352: mkdir d0/d1/d64/d26/d36/d72 0 2024-04-26T04:42:13.943 INFO:tasks.workunit.client.1.smithi104.stdout:1/349: mknod d0/d6/d19/d21/d66/d20/d2a/d56/d67/c69 0 2024-04-26T04:42:13.943 INFO:tasks.workunit.client.1.smithi104.stdout:1/350: write d0/d6/d19/d21/d66/d20/d2a/f2c [624962,98505] 0 2024-04-26T04:42:13.945 INFO:tasks.workunit.client.1.smithi104.stdout:5/353: unlink d0/d1/d64/d26/d36/f40 0 2024-04-26T04:42:13.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/351: mkdir d0/d6/d2e/d32/d6a 0 2024-04-26T04:42:13.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/352: chown d0/d6/d2e/d32/d6a 168177539 1 2024-04-26T04:42:13.947 INFO:tasks.workunit.client.1.smithi104.stdout:1/353: write d0/d6/d19/d21/d66/d29/f39 [4883165,4695] 0 2024-04-26T04:42:13.948 INFO:tasks.workunit.client.1.smithi104.stdout:5/354: mkdir d0/d1/d4/d2a/d6b/d73 0 2024-04-26T04:42:13.948 INFO:tasks.workunit.client.1.smithi104.stdout:5/355: write d0/d1/d4/f17 [2808926,14352] 0 2024-04-26T04:42:13.949 INFO:tasks.workunit.client.1.smithi104.stdout:1/354: mknod d0/d6/d19/d21/d66/d20/c6b 0 2024-04-26T04:42:13.952 INFO:tasks.workunit.client.1.smithi104.stdout:5/356: rmdir d0/d1/d4/d2a/d2b/d6a 0 2024-04-26T04:42:13.952 INFO:tasks.workunit.client.1.smithi104.stdout:5/357: creat d0/d41/f74 x:0 0 0 2024-04-26T04:42:13.954 INFO:tasks.workunit.client.1.smithi104.stdout:8/213: dwrite d0/f1a [0,4194304] 0 2024-04-26T04:42:13.954 INFO:tasks.workunit.client.1.smithi104.stdout:8/214: write d0/d13/f25 [1676624,39111] 0 2024-04-26T04:42:13.954 INFO:tasks.workunit.client.1.smithi104.stdout:8/215: chown d0/f1e 194 1 2024-04-26T04:42:13.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/216: dread - d0/f2e zero size 2024-04-26T04:42:13.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/217: creat d0/d33/d35/f44 x:0 0 0 2024-04-26T04:42:13.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/218: chown d0/d33/l1b 643101 1 2024-04-26T04:42:13.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/219: readlink d0/l3 0 2024-04-26T04:42:13.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/220: write d0/f3b [545260,78937] 0 2024-04-26T04:42:13.955 INFO:tasks.workunit.client.1.smithi104.stdout:1/355: link d0/d6/d19/d1b/d35/l5f d0/d6/d19/d21/d66/d30/l6c 0 2024-04-26T04:42:13.961 INFO:tasks.workunit.client.1.smithi104.stdout:1/356: creat d0/d6/d19/d21/f6d x:0 0 0 2024-04-26T04:42:13.964 INFO:tasks.workunit.client.1.smithi104.stdout:1/357: creat d0/d6/d19/d21/d59/f6e x:0 0 0 2024-04-26T04:42:13.964 INFO:tasks.workunit.client.1.smithi104.stdout:1/358: chown d0/d6/d19/d21/d58 121983 1 2024-04-26T04:42:13.965 INFO:tasks.workunit.client.1.smithi104.stdout:1/359: chown d0/d6/d19/d21/d66/d20/d2a/d56/d67/c69 45529 1 2024-04-26T04:42:13.966 INFO:tasks.workunit.client.1.smithi104.stdout:1/360: creat d0/d6/d19/d21/d66/d20/d2a/f6f x:0 0 0 2024-04-26T04:42:13.968 INFO:tasks.workunit.client.1.smithi104.stdout:1/361: mknod d0/d6/d19/d1b/d35/c70 0 2024-04-26T04:42:13.968 INFO:tasks.workunit.client.1.smithi104.stdout:1/362: fsync d0/d6/d19/d1b/d35/f40 0 2024-04-26T04:42:13.968 INFO:tasks.workunit.client.1.smithi104.stdout:1/363: write d0/d6/d19/d21/d66/d20/d2a/f6f [386972,96725] 0 2024-04-26T04:42:13.973 INFO:tasks.workunit.client.1.smithi104.stdout:5/358: dwrite d0/d1/d4/d2a/d2b/d37/d48/f70 [0,4194304] 0 2024-04-26T04:42:13.973 INFO:tasks.workunit.client.1.smithi104.stdout:5/359: fdatasync d0/d1/d4/d2a/d2b/f35 0 2024-04-26T04:42:13.973 INFO:tasks.workunit.client.1.smithi104.stdout:5/360: fsync d0/d1/f38 0 2024-04-26T04:42:13.977 INFO:tasks.workunit.client.1.smithi104.stdout:1/364: write d0/d6/d19/d21/d66/d29/f39 [935129,15977] 0 2024-04-26T04:42:13.977 INFO:tasks.workunit.client.1.smithi104.stdout:1/365: write d0/d6/ff [11408266,92443] 0 2024-04-26T04:42:13.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/120: sync 2024-04-26T04:42:13.981 INFO:tasks.workunit.client.0.smithi090.stdout:1/16: sync 2024-04-26T04:42:13.981 INFO:tasks.workunit.client.0.smithi090.stdout:4/8: sync 2024-04-26T04:42:13.981 INFO:tasks.workunit.client.0.smithi090.stdout:3/10: sync 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:6/137: sync 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:1/17: chown d0/l6 656789 1 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:4/9: write f0 [698867,45025] 0 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:8/32: sync 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:9/3: sync 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:7/6: sync 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:7/7: write - no filename 2024-04-26T04:42:13.982 INFO:tasks.workunit.client.0.smithi090.stdout:2/6: sync 2024-04-26T04:42:13.983 INFO:tasks.workunit.client.0.smithi090.stdout:5/66: sync 2024-04-26T04:42:13.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/7: chown l0 1 1 2024-04-26T04:42:13.984 INFO:tasks.workunit.client.1.smithi104.stdout:1/366: dread d0/d6/d19/f3a [0,4194304] 0 2024-04-26T04:42:13.984 INFO:tasks.workunit.client.1.smithi104.stdout:8/221: rmdir d0 39 2024-04-26T04:42:13.984 INFO:tasks.workunit.client.0.smithi090.stdout:0/121: getdents d0/d26 0 2024-04-26T04:42:13.984 INFO:tasks.workunit.client.0.smithi090.stdout:1/18: mkdir d0/d4/d7 0 2024-04-26T04:42:13.984 INFO:tasks.workunit.client.0.smithi090.stdout:8/33: fdatasync f8 0 2024-04-26T04:42:13.984 INFO:tasks.workunit.client.0.smithi090.stdout:9/4: creat f0 x:0 0 0 2024-04-26T04:42:13.985 INFO:tasks.workunit.client.0.smithi090.stdout:7/8: mkdir d0/d1 0 2024-04-26T04:42:13.985 INFO:tasks.workunit.client.0.smithi090.stdout:7/9: write - no filename 2024-04-26T04:42:13.985 INFO:tasks.workunit.client.0.smithi090.stdout:7/10: dread - no filename 2024-04-26T04:42:13.985 INFO:tasks.workunit.client.0.smithi090.stdout:7/11: chown d0 0 1 2024-04-26T04:42:13.986 INFO:tasks.workunit.client.0.smithi090.stdout:2/8: mknod c1 0 2024-04-26T04:42:13.986 INFO:tasks.workunit.client.0.smithi090.stdout:2/9: truncate - no filename 2024-04-26T04:42:13.986 INFO:tasks.workunit.client.0.smithi090.stdout:4/10: dread f0 [0,4194304] 0 2024-04-26T04:42:13.986 INFO:tasks.workunit.client.0.smithi090.stdout:2/10: chown c1 1828888592 1 2024-04-26T04:42:13.986 INFO:tasks.workunit.client.0.smithi090.stdout:4/11: chown f0 5 1 2024-04-26T04:42:13.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/367: symlink d0/d6/d19/d21/l71 0 2024-04-26T04:42:13.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/368: dread - d0/d6/d19/d21/f6d zero size 2024-04-26T04:42:13.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/369: creat d0/d6/d2e/d32/f72 x:0 0 0 2024-04-26T04:42:13.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/370: fsync d0/d6/d49/f4a 0 2024-04-26T04:42:13.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/371: write d0/d6/d49/f68 [146317,115561] 0 2024-04-26T04:42:13.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/372: chown d0/d6/d19/d1b 31709 1 2024-04-26T04:42:13.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/373: creat d0/d6/d19/d21/d59/f73 x:0 0 0 2024-04-26T04:42:13.988 INFO:tasks.workunit.client.1.smithi104.stdout:8/222: creat d0/d33/d35/f45 x:0 0 0 2024-04-26T04:42:13.988 INFO:tasks.workunit.client.1.smithi104.stdout:8/223: read - d0/d33/f2c zero size 2024-04-26T04:42:13.988 INFO:tasks.workunit.client.1.smithi104.stdout:8/224: chown d0/d33/l23 764 1 2024-04-26T04:42:13.988 INFO:tasks.workunit.client.0.smithi090.stdout:5/67: symlink d4/d5/l12 0 2024-04-26T04:42:13.988 INFO:tasks.workunit.client.0.smithi090.stdout:5/68: fsync d4/d5/df/f10 0 2024-04-26T04:42:13.989 INFO:tasks.workunit.client.0.smithi090.stdout:3/11: getdents d1 0 2024-04-26T04:42:13.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/5: unlink f0 0 2024-04-26T04:42:13.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/6: dread - no filename 2024-04-26T04:42:13.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/7: readlink - no filename 2024-04-26T04:42:13.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/8: fsync - no filename 2024-04-26T04:42:13.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/9: creat f1 x:0 0 0 2024-04-26T04:42:13.992 INFO:tasks.workunit.client.0.smithi090.stdout:7/12: mkdir d0/d2 0 2024-04-26T04:42:13.992 INFO:tasks.workunit.client.1.smithi104.stdout:8/225: mkdir d0/d1c/d27/d46 0 2024-04-26T04:42:13.992 INFO:tasks.workunit.client.0.smithi090.stdout:2/11: creat f2 x:0 0 0 2024-04-26T04:42:13.993 INFO:tasks.workunit.client.0.smithi090.stdout:2/12: write f2 [63850,124463] 0 2024-04-26T04:42:13.993 INFO:tasks.workunit.client.0.smithi090.stdout:0/122: rmdir d0/df 39 2024-04-26T04:42:13.994 INFO:tasks.workunit.client.0.smithi090.stdout:1/19: getdents d0 0 2024-04-26T04:42:13.995 INFO:tasks.workunit.client.1.smithi104.stdout:8/226: rename d0/d13 to d0/d1c/d27/d47 0 2024-04-26T04:42:13.995 INFO:tasks.workunit.client.0.smithi090.stdout:3/12: unlink d1/l2 0 2024-04-26T04:42:13.997 INFO:tasks.workunit.client.0.smithi090.stdout:8/34: mkdir dc 0 2024-04-26T04:42:13.999 INFO:tasks.workunit.client.0.smithi090.stdout:7/13: mkdir d0/d3 0 2024-04-26T04:42:14.000 INFO:tasks.workunit.client.0.smithi090.stdout:0/123: link d0/l15 d0/d26/l2b 0 2024-04-26T04:42:14.000 INFO:tasks.workunit.client.0.smithi090.stdout:0/124: creat d0/de/f2c x:0 0 0 2024-04-26T04:42:14.002 INFO:tasks.workunit.client.0.smithi090.stdout:1/20: creat d0/d4/f8 x:0 0 0 2024-04-26T04:42:14.002 INFO:tasks.workunit.client.0.smithi090.stdout:1/21: read - d0/d4/f8 zero size 2024-04-26T04:42:14.002 INFO:tasks.workunit.client.0.smithi090.stdout:1/22: chown d0/l5 354649270 1 2024-04-26T04:42:14.002 INFO:tasks.workunit.client.0.smithi090.stdout:6/138: dwrite d9/f23 [0,4194304] 0 2024-04-26T04:42:14.002 INFO:tasks.workunit.client.0.smithi090.stdout:6/139: chown d9/d22/l27 3 1 2024-04-26T04:42:14.002 INFO:tasks.workunit.client.0.smithi090.stdout:6/140: fdatasync d9/d1c/f36 0 2024-04-26T04:42:14.003 INFO:tasks.workunit.client.0.smithi090.stdout:3/13: symlink d1/l4 0 2024-04-26T04:42:14.003 INFO:tasks.workunit.client.0.smithi090.stdout:8/35: chown l3 4224 1 2024-04-26T04:42:14.003 INFO:tasks.workunit.client.0.smithi090.stdout:7/14: mkdir d0/d1/d4 0 2024-04-26T04:42:14.003 INFO:tasks.workunit.client.0.smithi090.stdout:7/15: write - no filename 2024-04-26T04:42:14.006 INFO:tasks.workunit.client.0.smithi090.stdout:3/14: dread d1/f3 [0,4194304] 0 2024-04-26T04:42:14.011 INFO:tasks.workunit.client.0.smithi090.stdout:6/141: truncate f3 4780180 0 2024-04-26T04:42:14.015 INFO:tasks.workunit.client.0.smithi090.stdout:7/16: creat d0/d1/f5 x:0 0 0 2024-04-26T04:42:14.015 INFO:tasks.workunit.client.0.smithi090.stdout:4/12: dwrite f0 [0,4194304] 0 2024-04-26T04:42:14.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/17: rmdir d0/d3 0 2024-04-26T04:42:14.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/18: write d0/d1/f5 [553557,534] 0 2024-04-26T04:42:14.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/19: chown d0/d1 638 1 2024-04-26T04:42:14.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/20: getdents d0/d2 0 2024-04-26T04:42:14.019 INFO:tasks.workunit.client.1.smithi104.stdout:3/351: sync 2024-04-26T04:42:14.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/21: unlink d0/d1/f5 0 2024-04-26T04:42:14.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/22: getdents d0/d1/d4 0 2024-04-26T04:42:14.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/23: fdatasync - no filename 2024-04-26T04:42:14.021 INFO:tasks.workunit.client.0.smithi090.stdout:7/24: unlink - no file 2024-04-26T04:42:14.022 INFO:tasks.workunit.client.0.smithi090.stdout:7/25: read - no filename 2024-04-26T04:42:14.022 INFO:tasks.workunit.client.0.smithi090.stdout:7/26: rename d0/d1 to d0/d1/d4/d6 22 2024-04-26T04:42:14.022 INFO:tasks.workunit.client.0.smithi090.stdout:7/27: write - no filename 2024-04-26T04:42:14.022 INFO:tasks.workunit.client.1.smithi104.stdout:3/352: creat d5/d9/dc/d32/f77 x:0 0 0 2024-04-26T04:42:14.024 INFO:tasks.workunit.client.0.smithi090.stdout:7/28: mkdir d0/d1/d4/d7 0 2024-04-26T04:42:14.024 INFO:tasks.workunit.client.0.smithi090.stdout:7/29: dread - no filename 2024-04-26T04:42:14.024 INFO:tasks.workunit.client.1.smithi104.stdout:3/353: creat d5/d9/dc/d32/d48/f78 x:0 0 0 2024-04-26T04:42:14.024 INFO:tasks.workunit.client.1.smithi104.stdout:3/354: chown d5/d9/dc/d29/d49/d59/f74 210176 1 2024-04-26T04:42:14.026 INFO:tasks.workunit.client.0.smithi090.stdout:7/30: creat d0/f8 x:0 0 0 2024-04-26T04:42:14.028 INFO:tasks.workunit.client.0.smithi090.stdout:6/142: read d9/d22/f29 [4023245,22222] 0 2024-04-26T04:42:14.029 INFO:tasks.workunit.client.0.smithi090.stdout:7/31: rename d0/d1/d4/d7 to d0/d1/d4/d9 0 2024-04-26T04:42:14.030 INFO:tasks.workunit.client.0.smithi090.stdout:4/13: read f0 [1582783,102375] 0 2024-04-26T04:42:14.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/32: creat d0/d2/fa x:0 0 0 2024-04-26T04:42:14.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/33: fdatasync d0/f8 0 2024-04-26T04:42:14.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/143: mkdir d9/d37 0 2024-04-26T04:42:14.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/144: fsync d9/d22/f26 0 2024-04-26T04:42:14.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/145: write d9/f12 [3909117,45775] 0 2024-04-26T04:42:14.033 INFO:tasks.workunit.client.0.smithi090.stdout:6/146: chown d9/d22/f29 403340589 1 2024-04-26T04:42:14.034 INFO:tasks.workunit.client.1.smithi104.stdout:3/355: write d5/d9/fe [3965827,18918] 0 2024-04-26T04:42:14.034 INFO:tasks.workunit.client.1.smithi104.stdout:3/356: truncate d5/d9/f4a 1099137 0 2024-04-26T04:42:14.034 INFO:tasks.workunit.client.1.smithi104.stdout:3/357: chown d5/c2f 1672440951 1 2024-04-26T04:42:14.034 INFO:tasks.workunit.client.1.smithi104.stdout:3/358: stat d5/d9/dc/d69/c30 0 2024-04-26T04:42:14.035 INFO:tasks.workunit.client.0.smithi090.stdout:7/34: mkdir d0/db 0 2024-04-26T04:42:14.035 INFO:tasks.workunit.client.0.smithi090.stdout:4/14: dread f0 [0,4194304] 0 2024-04-26T04:42:14.035 INFO:tasks.workunit.client.0.smithi090.stdout:4/15: chown f0 2 1 2024-04-26T04:42:14.036 INFO:tasks.workunit.client.0.smithi090.stdout:4/16: creat f1 x:0 0 0 2024-04-26T04:42:14.036 INFO:tasks.workunit.client.0.smithi090.stdout:4/17: creat f2 x:0 0 0 2024-04-26T04:42:14.036 INFO:tasks.workunit.client.0.smithi090.stdout:6/147: symlink d9/d37/l38 0 2024-04-26T04:42:14.036 INFO:tasks.workunit.client.0.smithi090.stdout:7/35: link d0/d2/fa d0/d1/d4/d9/fc 0 2024-04-26T04:42:14.036 INFO:tasks.workunit.client.0.smithi090.stdout:7/36: readlink - no filename 2024-04-26T04:42:14.036 INFO:tasks.workunit.client.0.smithi090.stdout:7/37: dread - d0/d1/d4/d9/fc zero size 2024-04-26T04:42:14.037 INFO:tasks.workunit.client.0.smithi090.stdout:4/18: mkdir d3 0 2024-04-26T04:42:14.037 INFO:tasks.workunit.client.1.smithi104.stdout:3/359: getdents d5/d9/dc/d29/d49/d59 0 2024-04-26T04:42:14.037 INFO:tasks.workunit.client.1.smithi104.stdout:3/360: symlink d5/d9/dc/l79 0 2024-04-26T04:42:14.038 INFO:tasks.workunit.client.1.smithi104.stdout:3/361: symlink d5/d9/dc/d29/d49/d59/l7a 0 2024-04-26T04:42:14.038 INFO:tasks.workunit.client.1.smithi104.stdout:3/362: write d5/d9/dc/d32/f77 [362342,118593] 0 2024-04-26T04:42:14.041 INFO:tasks.workunit.client.1.smithi104.stdout:3/363: creat d5/d9/dc/d29/d49/d59/f7b x:0 0 0 2024-04-26T04:42:14.041 INFO:tasks.workunit.client.1.smithi104.stdout:3/364: fdatasync d5/d9/f14 0 2024-04-26T04:42:14.041 INFO:tasks.workunit.client.1.smithi104.stdout:3/365: readlink d5/d9/dc/l79 0 2024-04-26T04:42:14.041 INFO:tasks.workunit.client.1.smithi104.stdout:3/366: truncate d5/d9/dc/d32/d48/f5f 667750 0 2024-04-26T04:42:14.041 INFO:tasks.workunit.client.0.smithi090.stdout:1/23: fsync d0/d4/f8 0 2024-04-26T04:42:14.042 INFO:tasks.workunit.client.1.smithi104.stdout:3/367: link d5/d9/d17/d46/l63 d5/d9/d17/l7c 0 2024-04-26T04:42:14.044 INFO:tasks.workunit.client.0.smithi090.stdout:5/69: dwrite d4/d5/df/f10 [4194304,4194304] 0 2024-04-26T04:42:14.044 INFO:tasks.workunit.client.0.smithi090.stdout:5/70: fdatasync d4/fe 0 2024-04-26T04:42:14.044 INFO:tasks.workunit.client.0.smithi090.stdout:5/71: creat d4/f13 x:0 0 0 2024-04-26T04:42:14.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/72: creat d4/d5/f14 x:0 0 0 2024-04-26T04:42:14.048 INFO:tasks.workunit.client.1.smithi104.stdout:3/368: dread d5/f23 [0,4194304] 0 2024-04-26T04:42:14.049 INFO:tasks.workunit.client.0.smithi090.stdout:5/73: link d4/d5/c6 d4/d5/df/c15 0 2024-04-26T04:42:14.049 INFO:tasks.workunit.client.0.smithi090.stdout:5/74: write d4/d5/f14 [665272,127923] 0 2024-04-26T04:42:14.050 INFO:tasks.workunit.client.1.smithi104.stdout:3/369: mkdir d5/d9/dc/d29/d7d 0 2024-04-26T04:42:14.050 INFO:tasks.workunit.client.1.smithi104.stdout:3/370: getdents d5/d9/dc/d32/d47 0 2024-04-26T04:42:14.052 INFO:tasks.workunit.client.1.smithi104.stdout:3/371: creat d5/d9/d17/d60/f7e x:0 0 0 2024-04-26T04:42:14.052 INFO:tasks.workunit.client.1.smithi104.stdout:3/372: chown d5/d9/dc/d69/l45 15892 1 2024-04-26T04:42:14.053 INFO:tasks.workunit.client.0.smithi090.stdout:9/10: dwrite f1 [0,4194304] 0 2024-04-26T04:42:14.053 INFO:tasks.workunit.client.0.smithi090.stdout:9/11: creat f2 x:0 0 0 2024-04-26T04:42:14.053 INFO:tasks.workunit.client.0.smithi090.stdout:9/12: rmdir - no directory 2024-04-26T04:42:14.053 INFO:tasks.workunit.client.0.smithi090.stdout:9/13: creat f3 x:0 0 0 2024-04-26T04:42:14.054 INFO:tasks.workunit.client.1.smithi104.stdout:3/373: rename d5/d9/dc/d69/c64 to d5/d9/dc/d29/d7d/c7f 0 2024-04-26T04:42:14.054 INFO:tasks.workunit.client.1.smithi104.stdout:3/374: mkdir d5/d9/d80 0 2024-04-26T04:42:14.055 INFO:tasks.workunit.client.1.smithi104.stdout:3/375: symlink d5/d9/d17/d60/l81 0 2024-04-26T04:42:14.055 INFO:tasks.workunit.client.1.smithi104.stdout:3/376: truncate d5/d9/d17/d46/f70 992548 0 2024-04-26T04:42:14.055 INFO:tasks.workunit.client.1.smithi104.stdout:3/377: write d5/d9/dc/d29/d49/d59/f74 [580828,99761] 0 2024-04-26T04:42:14.056 INFO:tasks.workunit.client.1.smithi104.stdout:3/378: rename f0 to d5/d9/dc/d29/d49/f82 0 2024-04-26T04:42:14.056 INFO:tasks.workunit.client.0.smithi090.stdout:9/14: mknod c4 0 2024-04-26T04:42:14.057 INFO:tasks.workunit.client.0.smithi090.stdout:9/15: symlink l5 0 2024-04-26T04:42:14.060 INFO:tasks.workunit.client.1.smithi104.stdout:1/374: dwrite d0/d6/ff [4194304,4194304] 0 2024-04-26T04:42:14.060 INFO:tasks.workunit.client.1.smithi104.stdout:1/375: dread - d0/d6/d2e/d32/f62 zero size 2024-04-26T04:42:14.060 INFO:tasks.workunit.client.1.smithi104.stdout:8/227: dwrite d0/d33/f1f [0,4194304] 0 2024-04-26T04:42:14.060 INFO:tasks.workunit.client.1.smithi104.stdout:1/376: creat d0/d6/d19/d21/f74 x:0 0 0 2024-04-26T04:42:14.063 INFO:tasks.workunit.client.1.smithi104.stdout:1/377: creat d0/d6/d19/f75 x:0 0 0 2024-04-26T04:42:14.066 INFO:tasks.workunit.client.0.smithi090.stdout:5/75: dread d4/d5/df/f10 [4194304,4194304] 0 2024-04-26T04:42:14.069 INFO:tasks.workunit.client.1.smithi104.stdout:1/378: rename d0/d6/d2e/d32/f72 to d0/d6/d19/d21/d66/d20/d2a/d56/d67/f76 0 2024-04-26T04:42:14.069 INFO:tasks.workunit.client.1.smithi104.stdout:1/379: readlink d0/d6/d19/d21/d66/l3d 0 2024-04-26T04:42:14.071 INFO:tasks.workunit.client.0.smithi090.stdout:9/16: read f1 [1979206,97800] 0 2024-04-26T04:42:14.076 INFO:tasks.workunit.client.0.smithi090.stdout:9/17: link f3 f6 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:9/18: write f6 [532480,119772] 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/13: dwrite f2 [0,4194304] 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/14: readlink l0 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/15: creat f3 x:0 0 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:0/125: dwrite d0/de/f14 [0,4194304] 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:7/38: dwrite d0/d2/fa [0,4194304] 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:7/39: creat d0/d2/fd x:0 0 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:0/126: write d0/df/f25 [65846,125047] 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:1/24: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:14.077 INFO:tasks.workunit.client.0.smithi090.stdout:4/19: dwrite f0 [0,4194304] 0 2024-04-26T04:42:14.078 INFO:tasks.workunit.client.1.smithi104.stdout:1/380: rename d0/d6/d19/d21/d66/d29/c47 to d0/c77 0 2024-04-26T04:42:14.078 INFO:tasks.workunit.client.1.smithi104.stdout:1/381: write d0/d6/d19/d21/f6d [866765,36972] 0 2024-04-26T04:42:14.078 INFO:tasks.workunit.client.1.smithi104.stdout:1/382: creat d0/d6/d19/d21/d66/d20/d2a/f78 x:0 0 0 2024-04-26T04:42:14.078 INFO:tasks.workunit.client.1.smithi104.stdout:3/379: dwrite d5/d9/f14 [0,4194304] 0 2024-04-26T04:42:14.080 INFO:tasks.workunit.client.1.smithi104.stdout:1/383: link d0/d6/d19/d21/d66/d29/d3f/l45 d0/l79 0 2024-04-26T04:42:14.082 INFO:tasks.workunit.client.0.smithi090.stdout:7/40: symlink d0/d1/d4/d9/le 0 2024-04-26T04:42:14.083 INFO:tasks.workunit.client.0.smithi090.stdout:0/127: rename d0/df/f12 to d0/f2d 0 2024-04-26T04:42:14.086 INFO:tasks.workunit.client.0.smithi090.stdout:4/20: link f2 d3/f4 0 2024-04-26T04:42:14.086 INFO:tasks.workunit.client.0.smithi090.stdout:1/25: write d0/f2 [880587,110179] 0 2024-04-26T04:42:14.086 INFO:tasks.workunit.client.0.smithi090.stdout:1/26: read d0/f2 [2603984,80115] 0 2024-04-26T04:42:14.087 INFO:tasks.workunit.client.0.smithi090.stdout:1/27: chown d0/d4/f8 7 1 2024-04-26T04:42:14.087 INFO:tasks.workunit.client.0.smithi090.stdout:1/28: creat d0/d4/f9 x:0 0 0 2024-04-26T04:42:14.089 INFO:tasks.workunit.client.0.smithi090.stdout:7/41: mkdir d0/db/df 0 2024-04-26T04:42:14.093 INFO:tasks.workunit.client.0.smithi090.stdout:7/42: creat d0/d1/d4/f10 x:0 0 0 2024-04-26T04:42:14.095 INFO:tasks.workunit.client.0.smithi090.stdout:7/43: symlink d0/db/df/l11 0 2024-04-26T04:42:14.095 INFO:tasks.workunit.client.0.smithi090.stdout:7/44: dread - d0/d1/d4/f10 zero size 2024-04-26T04:42:14.098 INFO:tasks.workunit.client.0.smithi090.stdout:0/128: dread d0/de/f14 [0,4194304] 0 2024-04-26T04:42:14.099 INFO:tasks.workunit.client.0.smithi090.stdout:0/129: truncate d0/d26/f27 893478 0 2024-04-26T04:42:14.102 INFO:tasks.workunit.client.0.smithi090.stdout:0/130: rename d0/d9/c21 to d0/df/c2e 0 2024-04-26T04:42:14.103 INFO:tasks.workunit.client.0.smithi090.stdout:4/21: write f0 [4182039,4079] 0 2024-04-26T04:42:14.103 INFO:tasks.workunit.client.0.smithi090.stdout:4/22: symlink d3/l5 0 2024-04-26T04:42:14.103 INFO:tasks.workunit.client.0.smithi090.stdout:4/23: write f1 [316890,83696] 0 2024-04-26T04:42:14.104 INFO:tasks.workunit.client.0.smithi090.stdout:0/131: getdents d0/de 0 2024-04-26T04:42:14.104 INFO:tasks.workunit.client.0.smithi090.stdout:0/132: fsync d0/de/f14 0 2024-04-26T04:42:14.107 INFO:tasks.workunit.client.0.smithi090.stdout:0/133: creat d0/de/f2f x:0 0 0 2024-04-26T04:42:14.107 INFO:tasks.workunit.client.0.smithi090.stdout:0/134: dread - d0/f2d zero size 2024-04-26T04:42:14.108 INFO:tasks.workunit.client.0.smithi090.stdout:0/135: rename d0/d26/c1b to d0/c30 0 2024-04-26T04:42:14.110 INFO:tasks.workunit.client.0.smithi090.stdout:0/136: readlink d0/de/l18 0 2024-04-26T04:42:14.114 INFO:tasks.workunit.client.0.smithi090.stdout:9/19: fsync f3 0 2024-04-26T04:42:14.114 INFO:tasks.workunit.client.0.smithi090.stdout:9/20: symlink l7 0 2024-04-26T04:42:14.115 INFO:tasks.workunit.client.0.smithi090.stdout:9/21: rename f1 to f8 0 2024-04-26T04:42:14.117 INFO:tasks.workunit.client.0.smithi090.stdout:9/22: dread f6 [0,4194304] 0 2024-04-26T04:42:14.118 INFO:tasks.workunit.client.1.smithi104.stdout:3/380: dwrite d5/d9/fa [4194304,4194304] 0 2024-04-26T04:42:14.120 INFO:tasks.workunit.client.0.smithi090.stdout:5/76: dwrite d4/d5/df/f10 [4194304,4194304] 0 2024-04-26T04:42:14.120 INFO:tasks.workunit.client.0.smithi090.stdout:5/77: write d4/fe [5111873,11700] 0 2024-04-26T04:42:14.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/78: mknod d4/c16 0 2024-04-26T04:42:14.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/79: dread - d4/f13 zero size 2024-04-26T04:42:14.122 INFO:tasks.workunit.client.0.smithi090.stdout:5/80: chown d4/d5 1 1 2024-04-26T04:42:14.122 INFO:tasks.workunit.client.1.smithi104.stdout:1/384: dwrite d0/d6/d19/d21/d66/d20/d2a/f78 [0,4194304] 0 2024-04-26T04:42:14.127 INFO:tasks.workunit.client.0.smithi090.stdout:4/24: fsync f2 0 2024-04-26T04:42:14.127 INFO:tasks.workunit.client.0.smithi090.stdout:4/25: write f1 [859393,109102] 0 2024-04-26T04:42:14.128 INFO:tasks.workunit.client.1.smithi104.stdout:1/385: rename d0/d6/d2e/d32/d4d/c64 to d0/d6/c7a 0 2024-04-26T04:42:14.128 INFO:tasks.workunit.client.1.smithi104.stdout:1/386: chown d0/d6/l5d 46628 1 2024-04-26T04:42:14.128 INFO:tasks.workunit.client.1.smithi104.stdout:1/387: write d0/d6/d49/f68 [1658443,76619] 0 2024-04-26T04:42:14.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/26: creat d3/f6 x:0 0 0 2024-04-26T04:42:14.129 INFO:tasks.workunit.client.0.smithi090.stdout:2/16: dwrite f2 [0,4194304] 0 2024-04-26T04:42:14.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/27: dread - d3/f4 zero size 2024-04-26T04:42:14.129 INFO:tasks.workunit.client.0.smithi090.stdout:2/17: chown f3 7099 1 2024-04-26T04:42:14.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/28: truncate f2 49031 0 2024-04-26T04:42:14.129 INFO:tasks.workunit.client.0.smithi090.stdout:4/29: write f0 [1741203,120941] 0 2024-04-26T04:42:14.130 INFO:tasks.workunit.client.1.smithi104.stdout:1/388: unlink d0/d6/d19/d21/f74 0 2024-04-26T04:42:14.130 INFO:tasks.workunit.client.0.smithi090.stdout:7/45: getdents d0/db 0 2024-04-26T04:42:14.130 INFO:tasks.workunit.client.0.smithi090.stdout:7/46: write d0/d2/fd [27009,70896] 0 2024-04-26T04:42:14.130 INFO:tasks.workunit.client.1.smithi104.stdout:1/389: creat d0/d6/d19/d21/d66/d29/d3f/f7b x:0 0 0 2024-04-26T04:42:14.131 INFO:tasks.workunit.client.1.smithi104.stdout:3/381: dread d5/d9/f1d [0,4194304] 0 2024-04-26T04:42:14.131 INFO:tasks.workunit.client.0.smithi090.stdout:4/30: mknod d3/c7 0 2024-04-26T04:42:14.131 INFO:tasks.workunit.client.0.smithi090.stdout:4/31: creat d3/f8 x:0 0 0 2024-04-26T04:42:14.132 INFO:tasks.workunit.client.1.smithi104.stdout:1/390: truncate d0/d6/f1a 202788 0 2024-04-26T04:42:14.132 INFO:tasks.workunit.client.1.smithi104.stdout:1/391: chown d0/d6/ff 48841 1 2024-04-26T04:42:14.132 INFO:tasks.workunit.client.1.smithi104.stdout:1/392: creat d0/d6/d2e/f7c x:0 0 0 2024-04-26T04:42:14.132 INFO:tasks.workunit.client.1.smithi104.stdout:3/382: mknod d5/d9/dc/d3a/c83 0 2024-04-26T04:42:14.133 INFO:tasks.workunit.client.0.smithi090.stdout:7/47: symlink d0/db/df/l12 0 2024-04-26T04:42:14.134 INFO:tasks.workunit.client.1.smithi104.stdout:1/393: mknod d0/d6/d19/d1b/d35/c7d 0 2024-04-26T04:42:14.136 INFO:tasks.workunit.client.0.smithi090.stdout:7/48: symlink d0/l13 0 2024-04-26T04:42:14.137 INFO:tasks.workunit.client.1.smithi104.stdout:1/394: mkdir d0/d6/d19/d21/d66/d30/d7e 0 2024-04-26T04:42:14.137 INFO:tasks.workunit.client.1.smithi104.stdout:1/395: write d0/d6/d2e/f7c [912480,79343] 0 2024-04-26T04:42:14.137 INFO:tasks.workunit.client.1.smithi104.stdout:1/396: truncate d0/d6/d19/d21/d59/f73 131344 0 2024-04-26T04:42:14.137 INFO:tasks.workunit.client.0.smithi090.stdout:7/49: creat d0/d1/d4/d9/f14 x:0 0 0 2024-04-26T04:42:14.138 INFO:tasks.workunit.client.1.smithi104.stdout:6/433: sync 2024-04-26T04:42:14.139 INFO:tasks.workunit.client.1.smithi104.stdout:7/246: sync 2024-04-26T04:42:14.139 INFO:tasks.workunit.client.1.smithi104.stdout:6/434: write d0/d9/d13/f43 [957932,81314] 0 2024-04-26T04:42:14.141 INFO:tasks.workunit.client.0.smithi090.stdout:0/137: dwrite d0/de/f14 [4194304,4194304] 0 2024-04-26T04:42:14.141 INFO:tasks.workunit.client.0.smithi090.stdout:5/81: truncate d4/fe 1795185 0 2024-04-26T04:42:14.142 INFO:tasks.workunit.client.0.smithi090.stdout:5/82: write d4/f13 [939269,43965] 0 2024-04-26T04:42:14.142 INFO:tasks.workunit.client.0.smithi090.stdout:5/83: write d4/d5/df/f10 [3885070,76632] 0 2024-04-26T04:42:14.142 INFO:tasks.workunit.client.0.smithi090.stdout:5/84: creat d4/d5/f17 x:0 0 0 2024-04-26T04:42:14.142 INFO:tasks.workunit.client.0.smithi090.stdout:5/85: rename d4/d5 to d4/d5/df/d18 22 2024-04-26T04:42:14.143 INFO:tasks.workunit.client.0.smithi090.stdout:0/138: creat d0/df/f31 x:0 0 0 2024-04-26T04:42:14.143 INFO:tasks.workunit.client.0.smithi090.stdout:2/18: write f2 [267394,76675] 0 2024-04-26T04:42:14.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/139: rename d0/l2 to d0/d26/l32 0 2024-04-26T04:42:14.147 INFO:tasks.workunit.client.0.smithi090.stdout:0/140: write d0/f2d [890157,36331] 0 2024-04-26T04:42:14.148 INFO:tasks.workunit.client.0.smithi090.stdout:2/19: rename f3 to f4 0 2024-04-26T04:42:14.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/29: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:14.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/30: stat d0/d4 0 2024-04-26T04:42:14.155 INFO:tasks.workunit.client.0.smithi090.stdout:1/31: fdatasync d0/f2 0 2024-04-26T04:42:14.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/32: creat d0/d4/d7/fa x:0 0 0 2024-04-26T04:42:14.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/33: fsync d0/f2 0 2024-04-26T04:42:14.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/34: getdents d0 0 2024-04-26T04:42:14.160 INFO:tasks.workunit.client.1.smithi104.stdout:7/247: dread d2/d4/f11 [4194304,4194304] 0 2024-04-26T04:42:14.160 INFO:tasks.workunit.client.1.smithi104.stdout:7/248: dread - d2/d4/d1d/f3e zero size 2024-04-26T04:42:14.161 INFO:tasks.workunit.client.0.smithi090.stdout:1/35: creat d0/fb x:0 0 0 2024-04-26T04:42:14.163 INFO:tasks.workunit.client.1.smithi104.stdout:6/435: write d0/d1b/f24 [3599226,65622] 0 2024-04-26T04:42:14.163 INFO:tasks.workunit.client.0.smithi090.stdout:4/32: dread f0 [0,4194304] 0 2024-04-26T04:42:14.164 INFO:tasks.workunit.client.1.smithi104.stdout:6/436: mknod d0/d9/d13/d15/d80/c8c 0 2024-04-26T04:42:14.164 INFO:tasks.workunit.client.1.smithi104.stdout:6/437: fsync d0/d5/f22 0 2024-04-26T04:42:14.165 INFO:tasks.workunit.client.1.smithi104.stdout:7/249: dread d2/d4/d32/d35/d37/f3a [0,4194304] 0 2024-04-26T04:42:14.165 INFO:tasks.workunit.client.1.smithi104.stdout:6/438: rmdir d0/d9/d13/d41/d8b 0 2024-04-26T04:42:14.166 INFO:tasks.workunit.client.0.smithi090.stdout:4/33: rename f2 to d3/f9 0 2024-04-26T04:42:14.166 INFO:tasks.workunit.client.1.smithi104.stdout:7/250: rename d2/d4/d32/c49 to d2/d4/d32/d35/d37/c4c 0 2024-04-26T04:42:14.167 INFO:tasks.workunit.client.1.smithi104.stdout:6/439: truncate d0/d9/fc 7838200 0 2024-04-26T04:42:14.168 INFO:tasks.workunit.client.1.smithi104.stdout:7/251: creat d2/d4/d32/d35/f4d x:0 0 0 2024-04-26T04:42:14.170 INFO:tasks.workunit.client.1.smithi104.stdout:7/252: unlink d2/d4/d32/d35/l46 0 2024-04-26T04:42:14.171 INFO:tasks.workunit.client.1.smithi104.stdout:7/253: mknod d2/d4/d32/d3d/c4e 0 2024-04-26T04:42:14.172 INFO:tasks.workunit.client.1.smithi104.stdout:3/383: dwrite d5/d9/dc/d69/f55 [0,4194304] 0 2024-04-26T04:42:14.174 INFO:tasks.workunit.client.1.smithi104.stdout:7/254: mknod d2/d4/d1d/c4f 0 2024-04-26T04:42:14.174 INFO:tasks.workunit.client.1.smithi104.stdout:7/255: chown d2/d4/d1d/f3e 121 1 2024-04-26T04:42:14.174 INFO:tasks.workunit.client.1.smithi104.stdout:7/256: read - d2/d4/d32/d35/f4d zero size 2024-04-26T04:42:14.174 INFO:tasks.workunit.client.1.smithi104.stdout:7/257: chown d2/d4/d1d 3 1 2024-04-26T04:42:14.175 INFO:tasks.workunit.client.0.smithi090.stdout:9/23: dwrite f2 [0,4194304] 0 2024-04-26T04:42:14.175 INFO:tasks.workunit.client.0.smithi090.stdout:9/24: creat f9 x:0 0 0 2024-04-26T04:42:14.175 INFO:tasks.workunit.client.0.smithi090.stdout:9/25: fdatasync f2 0 2024-04-26T04:42:14.177 INFO:tasks.workunit.client.0.smithi090.stdout:1/36: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:14.180 INFO:tasks.workunit.client.0.smithi090.stdout:1/37: getdents d0/d4 0 2024-04-26T04:42:14.180 INFO:tasks.workunit.client.0.smithi090.stdout:1/38: write d0/d4/d7/fa [298340,30210] 0 2024-04-26T04:42:14.181 INFO:tasks.workunit.client.0.smithi090.stdout:1/39: stat d0/l6 0 2024-04-26T04:42:14.181 INFO:tasks.workunit.client.0.smithi090.stdout:1/40: chown d0 4652440 1 2024-04-26T04:42:14.182 INFO:tasks.workunit.client.1.smithi104.stdout:7/258: dread d2/f28 [4194304,4194304] 0 2024-04-26T04:42:14.182 INFO:tasks.workunit.client.1.smithi104.stdout:7/259: chown d2/d4/d1d/d34 107 1 2024-04-26T04:42:14.183 INFO:tasks.workunit.client.1.smithi104.stdout:6/440: dread d0/d1b/f2c [0,4194304] 0 2024-04-26T04:42:14.184 INFO:tasks.workunit.client.0.smithi090.stdout:1/41: mknod d0/cc 0 2024-04-26T04:42:14.186 INFO:tasks.workunit.client.0.smithi090.stdout:1/42: mknod d0/d4/cd 0 2024-04-26T04:42:14.187 INFO:tasks.workunit.client.0.smithi090.stdout:5/86: rmdir d4 39 2024-04-26T04:42:14.189 INFO:tasks.workunit.client.0.smithi090.stdout:1/43: mknod d0/d4/ce 0 2024-04-26T04:42:14.191 INFO:tasks.workunit.client.0.smithi090.stdout:9/26: write f2 [1085998,94679] 0 2024-04-26T04:42:14.192 INFO:tasks.workunit.client.0.smithi090.stdout:9/27: symlink la 0 2024-04-26T04:42:14.192 INFO:tasks.workunit.client.0.smithi090.stdout:9/28: write f6 [1050978,110890] 0 2024-04-26T04:42:14.192 INFO:tasks.workunit.client.0.smithi090.stdout:7/50: dwrite d0/d1/d4/d9/fc [0,4194304] 0 2024-04-26T04:42:14.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/51: rmdir d0/d1/d4 39 2024-04-26T04:42:14.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/52: write d0/d1/d4/f10 [859268,50676] 0 2024-04-26T04:42:14.199 INFO:tasks.workunit.client.0.smithi090.stdout:0/141: dwrite d0/de/f2c [0,4194304] 0 2024-04-26T04:42:14.199 INFO:tasks.workunit.client.0.smithi090.stdout:0/142: dread - d0/de/f2f zero size 2024-04-26T04:42:14.199 INFO:tasks.workunit.client.1.smithi104.stdout:6/441: dread d0/d9/d13/d44/d71/f19 [0,4194304] 0 2024-04-26T04:42:14.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/34: dwrite d3/f9 [0,4194304] 0 2024-04-26T04:42:14.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/20: dwrite f2 [0,4194304] 0 2024-04-26T04:42:14.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/21: truncate f4 641413 0 2024-04-26T04:42:14.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/22: chown f4 563786 1 2024-04-26T04:42:14.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/23: chown f4 3 1 2024-04-26T04:42:14.201 INFO:tasks.workunit.client.0.smithi090.stdout:7/53: rmdir d0/d1/d4/d9 39 2024-04-26T04:42:14.201 INFO:tasks.workunit.client.0.smithi090.stdout:7/54: write d0/d2/fa [5009906,65745] 0 2024-04-26T04:42:14.203 INFO:tasks.workunit.client.1.smithi104.stdout:6/442: rename d0/d9/d13/d15/d80/c8c to d0/d9/d13/c8d 0 2024-04-26T04:42:14.204 INFO:tasks.workunit.client.0.smithi090.stdout:0/143: dread d0/de/f14 [0,4194304] 0 2024-04-26T04:42:14.206 INFO:tasks.workunit.client.0.smithi090.stdout:2/24: symlink l5 0 2024-04-26T04:42:14.207 INFO:tasks.workunit.client.1.smithi104.stdout:6/443: dread d0/d9/d13/d44/d71/f4b [0,4194304] 0 2024-04-26T04:42:14.208 INFO:tasks.workunit.client.0.smithi090.stdout:7/55: mkdir d0/db/d15 0 2024-04-26T04:42:14.208 INFO:tasks.workunit.client.0.smithi090.stdout:7/56: creat d0/d2/f16 x:0 0 0 2024-04-26T04:42:14.210 INFO:tasks.workunit.client.0.smithi090.stdout:2/25: unlink c1 0 2024-04-26T04:42:14.211 INFO:tasks.workunit.client.0.smithi090.stdout:7/57: rename d0/d2/fd to d0/d2/f17 0 2024-04-26T04:42:14.212 INFO:tasks.workunit.client.0.smithi090.stdout:2/26: rename l0 to l6 0 2024-04-26T04:42:14.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/58: creat d0/d1/d4/d9/f18 x:0 0 0 2024-04-26T04:42:14.214 INFO:tasks.workunit.client.0.smithi090.stdout:5/87: dread d4/d5/df/f10 [0,4194304] 0 2024-04-26T04:42:14.216 INFO:tasks.workunit.client.0.smithi090.stdout:1/44: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:14.216 INFO:tasks.workunit.client.0.smithi090.stdout:1/45: read d0/d4/d7/fa [314974,103571] 0 2024-04-26T04:42:14.216 INFO:tasks.workunit.client.0.smithi090.stdout:1/46: stat d0/cc 0 2024-04-26T04:42:14.216 INFO:tasks.workunit.client.0.smithi090.stdout:5/88: write d4/d5/f14 [356183,118742] 0 2024-04-26T04:42:14.216 INFO:tasks.workunit.client.1.smithi104.stdout:7/260: dwrite d2/d4/f26 [0,4194304] 0 2024-04-26T04:42:14.217 INFO:tasks.workunit.client.1.smithi104.stdout:7/261: write d2/d4/d1d/f30 [846435,50909] 0 2024-04-26T04:42:14.217 INFO:tasks.workunit.client.1.smithi104.stdout:7/262: rmdir d2/d4/d32/d3d/d41 39 2024-04-26T04:42:14.218 INFO:tasks.workunit.client.1.smithi104.stdout:6/444: read d0/d9/d13/d44/d71/f21 [1882067,40875] 0 2024-04-26T04:42:14.219 INFO:tasks.workunit.client.1.smithi104.stdout:2/465: sync 2024-04-26T04:42:14.219 INFO:tasks.workunit.client.1.smithi104.stdout:6/445: mkdir d0/d5/d8e 0 2024-04-26T04:42:14.220 INFO:tasks.workunit.client.1.smithi104.stdout:2/466: mkdir d6/d1b/d68/d81/d97/d99 0 2024-04-26T04:42:14.222 INFO:tasks.workunit.client.1.smithi104.stdout:2/467: rename d6/de/f44 to d6/d1b/d68/d81/d97/f9a 0 2024-04-26T04:42:14.223 INFO:tasks.workunit.client.0.smithi090.stdout:7/59: link d0/d2/fa d0/db/f19 0 2024-04-26T04:42:14.225 INFO:tasks.workunit.client.0.smithi090.stdout:1/47: symlink d0/lf 0 2024-04-26T04:42:14.226 INFO:tasks.workunit.client.0.smithi090.stdout:1/48: write d0/fb [1020124,33738] 0 2024-04-26T04:42:14.226 INFO:tasks.workunit.client.1.smithi104.stdout:7/263: write d2/d4/d1d/f29 [3138139,120932] 0 2024-04-26T04:42:14.228 INFO:tasks.workunit.client.0.smithi090.stdout:5/89: creat d4/d5/f19 x:0 0 0 2024-04-26T04:42:14.231 INFO:tasks.workunit.client.0.smithi090.stdout:7/60: symlink d0/db/df/l1a 0 2024-04-26T04:42:14.233 INFO:tasks.workunit.client.0.smithi090.stdout:1/49: symlink d0/l10 0 2024-04-26T04:42:14.233 INFO:tasks.workunit.client.0.smithi090.stdout:1/50: chown d0/lf 510067 1 2024-04-26T04:42:14.233 INFO:tasks.workunit.client.0.smithi090.stdout:1/51: chown d0/d4/f8 116773385 1 2024-04-26T04:42:14.233 INFO:tasks.workunit.client.0.smithi090.stdout:1/52: chown d0/f2 1 1 2024-04-26T04:42:14.234 INFO:tasks.workunit.client.0.smithi090.stdout:7/61: mknod d0/db/d15/c1b 0 2024-04-26T04:42:14.235 INFO:tasks.workunit.client.0.smithi090.stdout:7/62: mkdir d0/db/df/d1c 0 2024-04-26T04:42:14.236 INFO:tasks.workunit.client.0.smithi090.stdout:5/90: rmdir d4/d9 0 2024-04-26T04:42:14.244 INFO:tasks.workunit.client.0.smithi090.stdout:5/91: write d4/fe [599885,5898] 0 2024-04-26T04:42:14.244 INFO:tasks.workunit.client.0.smithi090.stdout:5/92: truncate d4/d5/f17 700964 0 2024-04-26T04:42:14.250 INFO:tasks.workunit.client.1.smithi104.stdout:2/468: symlink d6/d1b/d68/l9b 0 2024-04-26T04:42:14.250 INFO:tasks.workunit.client.1.smithi104.stdout:2/469: read d6/de/f36 [843348,15622] 0 2024-04-26T04:42:14.253 INFO:tasks.workunit.client.1.smithi104.stdout:2/470: link f2 d6/d8/d5a/f9c 0 2024-04-26T04:42:14.253 INFO:tasks.workunit.client.1.smithi104.stdout:2/471: truncate d6/d8/d3e/f49 482690 0 2024-04-26T04:42:14.253 INFO:tasks.workunit.client.1.smithi104.stdout:2/472: readlink d6/de/d8e/l82 0 2024-04-26T04:42:14.253 INFO:tasks.workunit.client.1.smithi104.stdout:2/473: truncate d6/de/d85/f91 1513933 0 2024-04-26T04:42:14.255 INFO:tasks.workunit.client.1.smithi104.stdout:7/264: rmdir d2/d4/d1d 39 2024-04-26T04:42:14.257 INFO:tasks.workunit.client.1.smithi104.stdout:2/474: creat d6/d1b/d93/f9d x:0 0 0 2024-04-26T04:42:14.259 INFO:tasks.workunit.client.1.smithi104.stdout:7/265: creat d2/d4/d32/d3d/f50 x:0 0 0 2024-04-26T04:42:14.259 INFO:tasks.workunit.client.1.smithi104.stdout:7/266: fdatasync d2/f28 0 2024-04-26T04:42:14.260 INFO:tasks.workunit.client.1.smithi104.stdout:7/267: stat d2/d4/f10 0 2024-04-26T04:42:14.260 INFO:tasks.workunit.client.1.smithi104.stdout:7/268: write d2/d4/d1d/f29 [3480999,73997] 0 2024-04-26T04:42:14.264 INFO:tasks.workunit.client.1.smithi104.stdout:7/269: dread d2/f42 [0,4194304] 0 2024-04-26T04:42:14.264 INFO:tasks.workunit.client.1.smithi104.stdout:7/270: read d2/f2f [602283,23355] 0 2024-04-26T04:42:14.264 INFO:tasks.workunit.client.1.smithi104.stdout:7/271: creat d2/d4/d1d/d34/f51 x:0 0 0 2024-04-26T04:42:14.266 INFO:tasks.workunit.client.1.smithi104.stdout:7/272: rename d2/l5 to d2/d4/d32/l52 0 2024-04-26T04:42:14.269 INFO:tasks.workunit.client.1.smithi104.stdout:2/475: write d6/f62 [232350,36151] 0 2024-04-26T04:42:14.271 INFO:tasks.workunit.client.1.smithi104.stdout:2/476: rename d6/de/c7e to d6/d1b/d51/c9e 0 2024-04-26T04:42:14.271 INFO:tasks.workunit.client.1.smithi104.stdout:2/477: fsync d6/d8/f70 0 2024-04-26T04:42:14.272 INFO:tasks.workunit.client.1.smithi104.stdout:2/478: symlink d6/de/l9f 0 2024-04-26T04:42:14.278 INFO:tasks.workunit.client.0.smithi090.stdout:2/27: dwrite f2 [0,4194304] 0 2024-04-26T04:42:14.278 INFO:tasks.workunit.client.0.smithi090.stdout:2/28: chown l6 223433 1 2024-04-26T04:42:14.278 INFO:tasks.workunit.client.0.smithi090.stdout:2/29: stat f2 0 2024-04-26T04:42:14.278 INFO:tasks.workunit.client.0.smithi090.stdout:2/30: creat f7 x:0 0 0 2024-04-26T04:42:14.279 INFO:tasks.workunit.client.0.smithi090.stdout:2/31: chown l6 2506 1 2024-04-26T04:42:14.279 INFO:tasks.workunit.client.0.smithi090.stdout:2/32: chown f2 415204 1 2024-04-26T04:42:14.285 INFO:tasks.workunit.client.0.smithi090.stdout:4/35: dwrite d3/f9 [4194304,4194304] 0 2024-04-26T04:42:14.285 INFO:tasks.workunit.client.0.smithi090.stdout:1/53: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:14.285 INFO:tasks.workunit.client.0.smithi090.stdout:4/36: write d3/f6 [312896,122614] 0 2024-04-26T04:42:14.285 INFO:tasks.workunit.client.0.smithi090.stdout:7/63: dwrite d0/f8 [0,4194304] 0 2024-04-26T04:42:14.286 INFO:tasks.workunit.client.0.smithi090.stdout:7/64: getdents d0/db/df/d1c 0 2024-04-26T04:42:14.286 INFO:tasks.workunit.client.1.smithi104.stdout:6/446: dwrite d0/d9/d25/f85 [0,4194304] 0 2024-04-26T04:42:14.287 INFO:tasks.workunit.client.0.smithi090.stdout:1/54: link d0/cc d0/c11 0 2024-04-26T04:42:14.287 INFO:tasks.workunit.client.0.smithi090.stdout:1/55: write d0/d4/f8 [243342,89016] 0 2024-04-26T04:42:14.291 INFO:tasks.workunit.client.0.smithi090.stdout:7/65: link d0/db/d15/c1b d0/db/df/d1c/c1d 0 2024-04-26T04:42:14.292 INFO:tasks.workunit.client.0.smithi090.stdout:1/56: rename d0/fb to d0/f12 0 2024-04-26T04:42:14.292 INFO:tasks.workunit.client.0.smithi090.stdout:1/57: fsync d0/d4/d7/fa 0 2024-04-26T04:42:14.294 INFO:tasks.workunit.client.0.smithi090.stdout:7/66: symlink d0/db/df/l1e 0 2024-04-26T04:42:14.296 INFO:tasks.workunit.client.1.smithi104.stdout:2/479: dwrite d6/d8/f70 [0,4194304] 0 2024-04-26T04:42:14.296 INFO:tasks.workunit.client.1.smithi104.stdout:2/480: readlink d6/d1b/d68/l9b 0 2024-04-26T04:42:14.296 INFO:tasks.workunit.client.1.smithi104.stdout:7/273: dwrite d2/d4/f11 [0,4194304] 0 2024-04-26T04:42:14.297 INFO:tasks.workunit.client.0.smithi090.stdout:4/37: fsync d3/f6 0 2024-04-26T04:42:14.297 INFO:tasks.workunit.client.0.smithi090.stdout:4/38: stat d3/f4 0 2024-04-26T04:42:14.297 INFO:tasks.workunit.client.0.smithi090.stdout:4/39: creat d3/fa x:0 0 0 2024-04-26T04:42:14.297 INFO:tasks.workunit.client.0.smithi090.stdout:4/40: write d3/f6 [906608,46455] 0 2024-04-26T04:42:14.299 INFO:tasks.workunit.client.1.smithi104.stdout:7/274: symlink d2/d4/d32/d35/d44/l53 0 2024-04-26T04:42:14.299 INFO:tasks.workunit.client.1.smithi104.stdout:7/275: truncate d2/d4/d32/d35/f4d 458306 0 2024-04-26T04:42:14.299 INFO:tasks.workunit.client.1.smithi104.stdout:7/276: write d2/d4/d1d/f30 [2842074,100663] 0 2024-04-26T04:42:14.299 INFO:tasks.workunit.client.1.smithi104.stdout:7/277: stat d2/d4/d32/d35/c43 0 2024-04-26T04:42:14.300 INFO:tasks.workunit.client.1.smithi104.stdout:2/481: getdents d6 0 2024-04-26T04:42:14.300 INFO:tasks.workunit.client.1.smithi104.stdout:2/482: dread - d6/de/d1d/f4a zero size 2024-04-26T04:42:14.300 INFO:tasks.workunit.client.1.smithi104.stdout:2/483: stat d6/d8/d3e/f49 0 2024-04-26T04:42:14.300 INFO:tasks.workunit.client.1.smithi104.stdout:7/278: rmdir d2/d4 39 2024-04-26T04:42:14.301 INFO:tasks.workunit.client.1.smithi104.stdout:2/484: symlink d6/d8/d5a/la0 0 2024-04-26T04:42:14.301 INFO:tasks.workunit.client.1.smithi104.stdout:2/485: readlink d6/d8/d3e/l67 0 2024-04-26T04:42:14.302 INFO:tasks.workunit.client.1.smithi104.stdout:2/486: truncate d6/d8/d5a/f2e 907213 0 2024-04-26T04:42:14.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/487: rename d6/f63 to d6/d1b/d68/fa1 0 2024-04-26T04:42:14.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/488: readlink d6/d1b/d68/d81/l86 0 2024-04-26T04:42:14.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/489: fsync d6/d1b/f71 0 2024-04-26T04:42:14.307 INFO:tasks.workunit.client.1.smithi104.stdout:6/447: dwrite d0/d9/d13/d15/f42 [0,4194304] 0 2024-04-26T04:42:14.307 INFO:tasks.workunit.client.1.smithi104.stdout:2/490: creat d6/d1b/d68/d81/fa2 x:0 0 0 2024-04-26T04:42:14.310 INFO:tasks.workunit.client.0.smithi090.stdout:7/67: rename d0/d2 to d0/d1/d1f 0 2024-04-26T04:42:14.310 INFO:tasks.workunit.client.1.smithi104.stdout:2/491: dread d6/de/d1d/f42 [0,4194304] 0 2024-04-26T04:42:14.310 INFO:tasks.workunit.client.1.smithi104.stdout:2/492: creat d6/d8/d3e/fa3 x:0 0 0 2024-04-26T04:42:14.310 INFO:tasks.workunit.client.1.smithi104.stdout:6/448: rename d0/d9/d25/c4f to d0/d9/d13/d44/c8f 0 2024-04-26T04:42:14.310 INFO:tasks.workunit.client.1.smithi104.stdout:6/449: truncate d0/d9/d13/d41/f7f 963867 0 2024-04-26T04:42:14.313 INFO:tasks.workunit.client.0.smithi090.stdout:5/93: rename d4/d5 to d4/d1a 0 2024-04-26T04:42:14.314 INFO:tasks.workunit.client.1.smithi104.stdout:2/493: read d6/f10 [3124377,60020] 0 2024-04-26T04:42:14.314 INFO:tasks.workunit.client.1.smithi104.stdout:2/494: truncate d6/d8/d3e/f49 1235296 0 2024-04-26T04:42:14.316 INFO:tasks.workunit.client.0.smithi090.stdout:4/41: dwrite d3/f8 [0,4194304] 0 2024-04-26T04:42:14.316 INFO:tasks.workunit.client.0.smithi090.stdout:4/42: write d3/fa [515914,41946] 0 2024-04-26T04:42:14.316 INFO:tasks.workunit.client.0.smithi090.stdout:4/43: chown d3/fa 118353 1 2024-04-26T04:42:14.316 INFO:tasks.workunit.client.0.smithi090.stdout:4/44: read f0 [3459146,39755] 0 2024-04-26T04:42:14.316 INFO:tasks.workunit.client.1.smithi104.stdout:6/450: mknod d0/c90 0 2024-04-26T04:42:14.318 INFO:tasks.workunit.client.1.smithi104.stdout:6/451: link d0/d9/d13/d15/d80/f6c d0/d9/d13/f91 0 2024-04-26T04:42:14.318 INFO:tasks.workunit.client.1.smithi104.stdout:6/452: fdatasync d0/d9/d25/f3e 0 2024-04-26T04:42:14.319 INFO:tasks.workunit.client.1.smithi104.stdout:2/495: dread d6/d8/d3e/f49 [0,4194304] 0 2024-04-26T04:42:14.321 INFO:tasks.workunit.client.1.smithi104.stdout:2/496: mknod d6/de/d85/ca4 0 2024-04-26T04:42:14.322 INFO:tasks.workunit.client.0.smithi090.stdout:2/33: truncate f2 1587180 0 2024-04-26T04:42:14.322 INFO:tasks.workunit.client.1.smithi104.stdout:2/497: mknod d6/d1b/ca5 0 2024-04-26T04:42:14.323 INFO:tasks.workunit.client.1.smithi104.stdout:6/453: dread d0/d9/d13/d44/d47/f4c [0,4194304] 0 2024-04-26T04:42:14.323 INFO:tasks.workunit.client.1.smithi104.stdout:6/454: write d0/d9/d13/d41/f7f [1237092,116405] 0 2024-04-26T04:42:14.323 INFO:tasks.workunit.client.1.smithi104.stdout:2/498: creat d6/de/d85/fa6 x:0 0 0 2024-04-26T04:42:14.324 INFO:tasks.workunit.client.1.smithi104.stdout:6/455: mknod d0/d5/d8e/c92 0 2024-04-26T04:42:14.325 INFO:tasks.workunit.client.1.smithi104.stdout:2/499: link d6/d8/d3e/l67 d6/d1b/d51/d6a/la7 0 2024-04-26T04:42:14.326 INFO:tasks.workunit.client.0.smithi090.stdout:2/34: getdents . 0 2024-04-26T04:42:14.326 INFO:tasks.workunit.client.0.smithi090.stdout:2/35: fdatasync f4 0 2024-04-26T04:42:14.327 INFO:tasks.workunit.client.1.smithi104.stdout:2/500: creat d6/d1b/d68/d81/d97/d99/fa8 x:0 0 0 2024-04-26T04:42:14.329 INFO:tasks.workunit.client.0.smithi090.stdout:4/45: rmdir d3 39 2024-04-26T04:42:14.331 INFO:tasks.workunit.client.1.smithi104.stdout:2/501: symlink d6/de/d1d/la9 0 2024-04-26T04:42:14.332 INFO:tasks.workunit.client.0.smithi090.stdout:4/46: mkdir d3/db 0 2024-04-26T04:42:14.332 INFO:tasks.workunit.client.0.smithi090.stdout:4/47: write f0 [1330923,130241] 0 2024-04-26T04:42:14.338 INFO:tasks.workunit.client.0.smithi090.stdout:4/48: truncate d3/f8 1389414 0 2024-04-26T04:42:14.338 INFO:tasks.workunit.client.1.smithi104.stdout:6/456: write d0/d9/d25/f85 [311893,49685] 0 2024-04-26T04:42:14.339 INFO:tasks.workunit.client.0.smithi090.stdout:4/49: creat d3/db/fc x:0 0 0 2024-04-26T04:42:14.339 INFO:tasks.workunit.client.0.smithi090.stdout:7/68: dwrite d0/d1/d4/d9/fc [4194304,4194304] 0 2024-04-26T04:42:14.341 INFO:tasks.workunit.client.0.smithi090.stdout:4/50: mkdir d3/db/dd 0 2024-04-26T04:42:14.341 INFO:tasks.workunit.client.0.smithi090.stdout:4/51: readlink d3/l5 0 2024-04-26T04:42:14.341 INFO:tasks.workunit.client.0.smithi090.stdout:2/36: write f2 [1883045,89534] 0 2024-04-26T04:42:14.342 INFO:tasks.workunit.client.0.smithi090.stdout:5/94: dwrite d4/d1a/df/f10 [0,4194304] 0 2024-04-26T04:42:14.345 INFO:tasks.workunit.client.1.smithi104.stdout:2/502: dread d6/d8/f39 [0,4194304] 0 2024-04-26T04:42:14.346 INFO:tasks.workunit.client.0.smithi090.stdout:7/69: mknod d0/db/df/c20 0 2024-04-26T04:42:14.354 INFO:tasks.workunit.client.1.smithi104.stdout:4/380: sync 2024-04-26T04:42:14.355 INFO:tasks.workunit.client.1.smithi104.stdout:9/249: sync 2024-04-26T04:42:14.355 INFO:tasks.workunit.client.1.smithi104.stdout:0/300: sync 2024-04-26T04:42:14.355 INFO:tasks.workunit.client.1.smithi104.stdout:0/301: stat d0/d4/d3b/c58 0 2024-04-26T04:42:14.355 INFO:tasks.workunit.client.1.smithi104.stdout:2/503: rmdir d6/d1b 39 2024-04-26T04:42:14.355 INFO:tasks.workunit.client.1.smithi104.stdout:0/302: write d0/d2/d21/d38/f5b [133009,52720] 0 2024-04-26T04:42:14.356 INFO:tasks.workunit.client.1.smithi104.stdout:4/381: dread d4/d7/d1d/d2e/f7e [0,4194304] 0 2024-04-26T04:42:14.356 INFO:tasks.workunit.client.1.smithi104.stdout:4/382: readlink d4/d7/d1d/d1e/d24/l44 0 2024-04-26T04:42:14.356 INFO:tasks.workunit.client.1.smithi104.stdout:4/383: write d4/d7/d1d/f43 [502688,66663] 0 2024-04-26T04:42:14.357 INFO:tasks.workunit.client.0.smithi090.stdout:2/37: mknod c8 0 2024-04-26T04:42:14.357 INFO:tasks.workunit.client.0.smithi090.stdout:2/38: chown l5 0 1 2024-04-26T04:42:14.357 INFO:tasks.workunit.client.0.smithi090.stdout:2/39: chown f7 13 1 2024-04-26T04:42:14.358 INFO:tasks.workunit.client.0.smithi090.stdout:5/95: rename d4/c16 to d4/c1b 0 2024-04-26T04:42:14.358 INFO:tasks.workunit.client.0.smithi090.stdout:5/96: truncate d4/fe 2071324 0 2024-04-26T04:42:14.359 INFO:tasks.workunit.client.1.smithi104.stdout:2/504: link d6/f8b d6/d1b/d93/faa 0 2024-04-26T04:42:14.359 INFO:tasks.workunit.client.0.smithi090.stdout:2/40: mknod c9 0 2024-04-26T04:42:14.360 INFO:tasks.workunit.client.0.smithi090.stdout:2/41: chown l6 9308 1 2024-04-26T04:42:14.360 INFO:tasks.workunit.client.0.smithi090.stdout:2/42: rmdir - no directory 2024-04-26T04:42:14.360 INFO:tasks.workunit.client.1.smithi104.stdout:0/303: mkdir d0/d4/d3c/d53/d5d 0 2024-04-26T04:42:14.360 INFO:tasks.workunit.client.1.smithi104.stdout:9/250: getdents d11/d1d 0 2024-04-26T04:42:14.360 INFO:tasks.workunit.client.1.smithi104.stdout:9/251: fsync d11/d1d/f31 0 2024-04-26T04:42:14.361 INFO:tasks.workunit.client.0.smithi090.stdout:5/97: mknod d4/d1a/c1c 0 2024-04-26T04:42:14.362 INFO:tasks.workunit.client.1.smithi104.stdout:2/505: truncate d6/d8/fb 1950464 0 2024-04-26T04:42:14.362 INFO:tasks.workunit.client.1.smithi104.stdout:2/506: fsync d6/de/d8e/f4c 0 2024-04-26T04:42:14.362 INFO:tasks.workunit.client.1.smithi104.stdout:2/507: fdatasync d6/de/d1d/f25 0 2024-04-26T04:42:14.362 INFO:tasks.workunit.client.1.smithi104.stdout:0/304: mkdir d0/d2/d21/d49/d5e 0 2024-04-26T04:42:14.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/43: mknod ca 0 2024-04-26T04:42:14.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/44: truncate f7 61191 0 2024-04-26T04:42:14.363 INFO:tasks.workunit.client.0.smithi090.stdout:4/52: truncate d3/f6 203507 0 2024-04-26T04:42:14.364 INFO:tasks.workunit.client.1.smithi104.stdout:9/252: rename d11/d1d/d34/f36 to d11/d1d/f49 0 2024-04-26T04:42:14.364 INFO:tasks.workunit.client.1.smithi104.stdout:2/508: rmdir d6/d1b/d75 39 2024-04-26T04:42:14.365 INFO:tasks.workunit.client.0.smithi090.stdout:2/45: mkdir db 0 2024-04-26T04:42:14.366 INFO:tasks.workunit.client.1.smithi104.stdout:9/253: symlink d11/d15/l4a 0 2024-04-26T04:42:14.366 INFO:tasks.workunit.client.1.smithi104.stdout:6/457: dwrite d0/d9/d13/d44/d71/f21 [0,4194304] 0 2024-04-26T04:42:14.367 INFO:tasks.workunit.client.1.smithi104.stdout:6/458: creat d0/d9/f93 x:0 0 0 2024-04-26T04:42:14.367 INFO:tasks.workunit.client.1.smithi104.stdout:6/459: read d0/d9/d25/f3e [863940,120388] 0 2024-04-26T04:42:14.367 INFO:tasks.workunit.client.1.smithi104.stdout:6/460: write d0/d9/d13/d41/f7f [2321954,96229] 0 2024-04-26T04:42:14.367 INFO:tasks.workunit.client.0.smithi090.stdout:5/98: dread d4/fe [0,4194304] 0 2024-04-26T04:42:14.368 INFO:tasks.workunit.client.1.smithi104.stdout:2/509: symlink d6/d1b/d68/d81/d97/d99/lab 0 2024-04-26T04:42:14.368 INFO:tasks.workunit.client.1.smithi104.stdout:2/510: write d6/d1b/d75/f7c [1171627,70870] 0 2024-04-26T04:42:14.368 INFO:tasks.workunit.client.1.smithi104.stdout:2/511: fdatasync d6/d8/d5a/f74 0 2024-04-26T04:42:14.369 INFO:tasks.workunit.client.0.smithi090.stdout:4/53: symlink d3/le 0 2024-04-26T04:42:14.369 INFO:tasks.workunit.client.1.smithi104.stdout:2/512: write d6/d1b/d51/f87 [264820,9331] 0 2024-04-26T04:42:14.371 INFO:tasks.workunit.client.1.smithi104.stdout:9/254: link d11/d15/l2b d11/d1d/d34/l4b 0 2024-04-26T04:42:14.371 INFO:tasks.workunit.client.1.smithi104.stdout:9/255: chown d11/d15/d32/c3b 61023 1 2024-04-26T04:42:14.371 INFO:tasks.workunit.client.1.smithi104.stdout:4/384: read d4/d20/f36 [3788190,16225] 0 2024-04-26T04:42:14.371 INFO:tasks.workunit.client.1.smithi104.stdout:4/385: chown d4/d7/d1d/d1e/d24/d59/f6a 12 1 2024-04-26T04:42:14.371 INFO:tasks.workunit.client.1.smithi104.stdout:4/386: chown d4/d7/d1d/d1e/d24/d48/f60 4749619 1 2024-04-26T04:42:14.371 INFO:tasks.workunit.client.1.smithi104.stdout:4/387: dread - d4/f71 zero size 2024-04-26T04:42:14.371 INFO:tasks.workunit.client.1.smithi104.stdout:4/388: fsync d4/d7/d1d/d1e/f61 0 2024-04-26T04:42:14.372 INFO:tasks.workunit.client.0.smithi090.stdout:7/70: dwrite d0/d1/d4/f10 [0,4194304] 0 2024-04-26T04:42:14.372 INFO:tasks.workunit.client.0.smithi090.stdout:7/71: dread - d0/d1/d4/d9/f18 zero size 2024-04-26T04:42:14.372 INFO:tasks.workunit.client.1.smithi104.stdout:2/513: symlink d6/d1b/d68/lac 0 2024-04-26T04:42:14.373 INFO:tasks.workunit.client.0.smithi090.stdout:2/46: unlink f4 0 2024-04-26T04:42:14.376 INFO:tasks.workunit.client.1.smithi104.stdout:9/256: mkdir d11/d15/d32/d4c 0 2024-04-26T04:42:14.377 INFO:tasks.workunit.client.1.smithi104.stdout:9/257: write f4 [3928543,19901] 0 2024-04-26T04:42:14.377 INFO:tasks.workunit.client.1.smithi104.stdout:4/389: dread f2 [0,4194304] 0 2024-04-26T04:42:14.378 INFO:tasks.workunit.client.0.smithi090.stdout:7/72: link d0/db/df/l11 d0/db/l21 0 2024-04-26T04:42:14.378 INFO:tasks.workunit.client.0.smithi090.stdout:5/99: creat d4/d1a/f1d x:0 0 0 2024-04-26T04:42:14.378 INFO:tasks.workunit.client.0.smithi090.stdout:5/100: chown d4/f13 502 1 2024-04-26T04:42:14.378 INFO:tasks.workunit.client.0.smithi090.stdout:5/101: write d4/d1a/df/f10 [8966089,26901] 0 2024-04-26T04:42:14.378 INFO:tasks.workunit.client.0.smithi090.stdout:5/102: stat d4/d1a/f14 0 2024-04-26T04:42:14.379 INFO:tasks.workunit.client.1.smithi104.stdout:9/258: symlink d11/d1d/l4d 0 2024-04-26T04:42:14.379 INFO:tasks.workunit.client.1.smithi104.stdout:9/259: dread - d11/d15/d1b/f37 zero size 2024-04-26T04:42:14.380 INFO:tasks.workunit.client.0.smithi090.stdout:2/47: unlink f7 0 2024-04-26T04:42:14.380 INFO:tasks.workunit.client.1.smithi104.stdout:4/390: mkdir d4/db/d6b/d7f 0 2024-04-26T04:42:14.382 INFO:tasks.workunit.client.1.smithi104.stdout:9/260: rmdir d11/d15/d1b 39 2024-04-26T04:42:14.382 INFO:tasks.workunit.client.0.smithi090.stdout:5/103: link d4/d1a/f14 d4/f1e 0 2024-04-26T04:42:14.383 INFO:tasks.workunit.client.1.smithi104.stdout:2/514: dread f2 [0,4194304] 0 2024-04-26T04:42:14.383 INFO:tasks.workunit.client.0.smithi090.stdout:2/48: mkdir db/dc 0 2024-04-26T04:42:14.385 INFO:tasks.workunit.client.0.smithi090.stdout:5/104: mkdir d4/d1a/d1f 0 2024-04-26T04:42:14.385 INFO:tasks.workunit.client.1.smithi104.stdout:4/391: rename d4/db/c2f to d4/d7/d1d/d1e/c80 0 2024-04-26T04:42:14.386 INFO:tasks.workunit.client.1.smithi104.stdout:9/261: mkdir d11/d15/d32/d4e 0 2024-04-26T04:42:14.386 INFO:tasks.workunit.client.1.smithi104.stdout:9/262: chown d11/d15/l4a 0 1 2024-04-26T04:42:14.386 INFO:tasks.workunit.client.0.smithi090.stdout:5/105: dread d4/d1a/f17 [0,4194304] 0 2024-04-26T04:42:14.388 INFO:tasks.workunit.client.0.smithi090.stdout:5/106: creat d4/f20 x:0 0 0 2024-04-26T04:42:14.388 INFO:tasks.workunit.client.1.smithi104.stdout:9/263: rename d11/d15/d1b/f24 to d11/d1d/f4f 0 2024-04-26T04:42:14.388 INFO:tasks.workunit.client.1.smithi104.stdout:9/264: dread - d11/d1d/d34/f3c zero size 2024-04-26T04:42:14.389 INFO:tasks.workunit.client.0.smithi090.stdout:5/107: mkdir d4/d1a/df/d21 0 2024-04-26T04:42:14.390 INFO:tasks.workunit.client.0.smithi090.stdout:5/108: creat d4/d1a/df/f22 x:0 0 0 2024-04-26T04:42:14.393 INFO:tasks.workunit.client.0.smithi090.stdout:7/73: dread d0/f8 [0,4194304] 0 2024-04-26T04:42:14.394 INFO:tasks.workunit.client.0.smithi090.stdout:7/74: creat d0/d1/d1f/f22 x:0 0 0 2024-04-26T04:42:14.397 INFO:tasks.workunit.client.1.smithi104.stdout:0/305: dwrite d0/d2/d10/f5a [0,4194304] 0 2024-04-26T04:42:14.398 INFO:tasks.workunit.client.1.smithi104.stdout:0/306: dread d0/d2/d21/d38/f4f [0,4194304] 0 2024-04-26T04:42:14.416 INFO:tasks.workunit.client.1.smithi104.stdout:0/307: read d0/d2/d21/d27/f30 [179758,119389] 0 2024-04-26T04:42:14.417 INFO:tasks.workunit.client.1.smithi104.stdout:0/308: rmdir d0/d4/d3c/d53 39 2024-04-26T04:42:14.417 INFO:tasks.workunit.client.0.smithi090.stdout:2/49: dread f2 [0,4194304] 0 2024-04-26T04:42:14.419 INFO:tasks.workunit.client.0.smithi090.stdout:2/50: truncate f2 63007 0 2024-04-26T04:42:14.420 INFO:tasks.workunit.client.0.smithi090.stdout:4/54: dwrite d3/f4 [0,4194304] 0 2024-04-26T04:42:14.420 INFO:tasks.workunit.client.1.smithi104.stdout:6/461: dwrite d0/d9/f87 [0,4194304] 0 2024-04-26T04:42:14.421 INFO:tasks.workunit.client.1.smithi104.stdout:6/462: truncate d0/d9/d13/d44/d71/f4b 955011 0 2024-04-26T04:42:14.422 INFO:tasks.workunit.client.0.smithi090.stdout:4/55: creat d3/db/dd/ff x:0 0 0 2024-04-26T04:42:14.423 INFO:tasks.workunit.client.1.smithi104.stdout:6/463: getdents d0/d9/d13/d44 0 2024-04-26T04:42:14.423 INFO:tasks.workunit.client.1.smithi104.stdout:6/464: dread - d0/d9/f93 zero size 2024-04-26T04:42:14.423 INFO:tasks.workunit.client.1.smithi104.stdout:6/465: write d0/d9/d25/f3e [3378672,65462] 0 2024-04-26T04:42:14.423 INFO:tasks.workunit.client.1.smithi104.stdout:6/466: chown d0/d9/d13 23 1 2024-04-26T04:42:14.423 INFO:tasks.workunit.client.1.smithi104.stdout:6/467: creat d0/d9/d13/d15/f94 x:0 0 0 2024-04-26T04:42:14.425 INFO:tasks.workunit.client.1.smithi104.stdout:6/468: symlink d0/d9/d13/d44/d47/l95 0 2024-04-26T04:42:14.429 INFO:tasks.workunit.client.1.smithi104.stdout:2/515: dwrite d6/d1b/d68/fa1 [4194304,4194304] 0 2024-04-26T04:42:14.429 INFO:tasks.workunit.client.1.smithi104.stdout:2/516: chown d6/d8/d3e/c73 47061 1 2024-04-26T04:42:14.430 INFO:tasks.workunit.client.0.smithi090.stdout:5/109: dwrite d4/d1a/df/f22 [0,4194304] 0 2024-04-26T04:42:14.430 INFO:tasks.workunit.client.0.smithi090.stdout:7/75: dwrite d0/d1/d1f/f16 [0,4194304] 0 2024-04-26T04:42:14.430 INFO:tasks.workunit.client.0.smithi090.stdout:7/76: dread - d0/d1/d4/d9/f14 zero size 2024-04-26T04:42:14.432 INFO:tasks.workunit.client.1.smithi104.stdout:6/469: dread d0/d9/d13/d41/f7f [0,4194304] 0 2024-04-26T04:42:14.432 INFO:tasks.workunit.client.1.smithi104.stdout:9/265: dwrite d11/f12 [0,4194304] 0 2024-04-26T04:42:14.432 INFO:tasks.workunit.client.1.smithi104.stdout:9/266: chown d11/d1d/d34/l40 412007148 1 2024-04-26T04:42:14.433 INFO:tasks.workunit.client.1.smithi104.stdout:2/517: dread d6/d1b/d51/d6a/f6f [0,4194304] 0 2024-04-26T04:42:14.433 INFO:tasks.workunit.client.1.smithi104.stdout:6/470: read d0/d5/f22 [21255,64147] 0 2024-04-26T04:42:14.434 INFO:tasks.workunit.client.0.smithi090.stdout:7/77: mkdir d0/db/df/d1c/d23 0 2024-04-26T04:42:14.434 INFO:tasks.workunit.client.1.smithi104.stdout:9/267: mkdir d11/d15/d1b/d50 0 2024-04-26T04:42:14.440 INFO:tasks.workunit.client.1.smithi104.stdout:2/518: creat d6/d1b/d68/fad x:0 0 0 2024-04-26T04:42:14.440 INFO:tasks.workunit.client.1.smithi104.stdout:6/471: mkdir d0/d9/d13/d41/d96 0 2024-04-26T04:42:14.440 INFO:tasks.workunit.client.1.smithi104.stdout:9/268: symlink d11/d15/l51 0 2024-04-26T04:42:14.440 INFO:tasks.workunit.client.1.smithi104.stdout:0/309: dwrite d0/d4/d15/f59 [0,4194304] 0 2024-04-26T04:42:14.440 INFO:tasks.workunit.client.0.smithi090.stdout:4/56: dwrite d3/f6 [0,4194304] 0 2024-04-26T04:42:14.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/57: write f1 [1231401,29385] 0 2024-04-26T04:42:14.441 INFO:tasks.workunit.client.1.smithi104.stdout:6/472: creat d0/d9/d13/d44/f97 x:0 0 0 2024-04-26T04:42:14.442 INFO:tasks.workunit.client.1.smithi104.stdout:9/269: rename d11/d15/d1b/f47 to d11/f52 0 2024-04-26T04:42:14.443 INFO:tasks.workunit.client.1.smithi104.stdout:6/473: write d0/d9/d13/d44/d71/f4b [1158178,22313] 0 2024-04-26T04:42:14.444 INFO:tasks.workunit.client.1.smithi104.stdout:2/519: rmdir d6/de 39 2024-04-26T04:42:14.445 INFO:tasks.workunit.client.1.smithi104.stdout:6/474: dread d0/d9/d13/d44/d47/f4c [0,4194304] 0 2024-04-26T04:42:14.447 INFO:tasks.workunit.client.0.smithi090.stdout:5/110: getdents d4/d1a 0 2024-04-26T04:42:14.447 INFO:tasks.workunit.client.1.smithi104.stdout:0/310: link d0/l2e d0/l5f 0 2024-04-26T04:42:14.449 INFO:tasks.workunit.client.1.smithi104.stdout:2/520: truncate d6/de/d1d/f42 924621 0 2024-04-26T04:42:14.451 INFO:tasks.workunit.client.1.smithi104.stdout:6/475: mknod d0/d9/d13/c98 0 2024-04-26T04:42:14.452 INFO:tasks.workunit.client.0.smithi090.stdout:4/58: rmdir d3/db/dd 39 2024-04-26T04:42:14.452 INFO:tasks.workunit.client.0.smithi090.stdout:4/59: readlink d3/l5 0 2024-04-26T04:42:14.452 INFO:tasks.workunit.client.0.smithi090.stdout:4/60: chown f0 0 1 2024-04-26T04:42:14.452 INFO:tasks.workunit.client.0.smithi090.stdout:4/61: fdatasync d3/db/dd/ff 0 2024-04-26T04:42:14.452 INFO:tasks.workunit.client.0.smithi090.stdout:2/51: read f2 [31438,88384] 0 2024-04-26T04:42:14.453 INFO:tasks.workunit.client.1.smithi104.stdout:0/311: link d0/d2/d21/d38/f36 d0/d4/d3c/d53/f60 0 2024-04-26T04:42:14.453 INFO:tasks.workunit.client.1.smithi104.stdout:9/270: write d11/d15/d1b/f46 [2768050,111619] 0 2024-04-26T04:42:14.453 INFO:tasks.workunit.client.0.smithi090.stdout:4/62: creat d3/db/dd/f10 x:0 0 0 2024-04-26T04:42:14.454 INFO:tasks.workunit.client.1.smithi104.stdout:2/521: truncate d6/de/f36 316784 0 2024-04-26T04:42:14.454 INFO:tasks.workunit.client.1.smithi104.stdout:2/522: fdatasync d6/d8/fb 0 2024-04-26T04:42:14.455 INFO:tasks.workunit.client.0.smithi090.stdout:2/52: getdents db 0 2024-04-26T04:42:14.455 INFO:tasks.workunit.client.0.smithi090.stdout:2/53: write f2 [177748,46818] 0 2024-04-26T04:42:14.455 INFO:tasks.workunit.client.1.smithi104.stdout:0/312: creat d0/d2/d21/d27/d1e/f61 x:0 0 0 2024-04-26T04:42:14.456 INFO:tasks.workunit.client.1.smithi104.stdout:6/476: dread d0/d9/d13/f20 [0,4194304] 0 2024-04-26T04:42:14.457 INFO:tasks.workunit.client.0.smithi090.stdout:4/63: link d3/le d3/l11 0 2024-04-26T04:42:14.458 INFO:tasks.workunit.client.1.smithi104.stdout:9/271: rename d11/d15/d1b/d50 to d11/d53 0 2024-04-26T04:42:14.459 INFO:tasks.workunit.client.1.smithi104.stdout:9/272: write d11/d1d/d34/f41 [265696,70390] 0 2024-04-26T04:42:14.461 INFO:tasks.workunit.client.1.smithi104.stdout:6/477: symlink d0/d9/l99 0 2024-04-26T04:42:14.461 INFO:tasks.workunit.client.1.smithi104.stdout:6/478: chown d0/d5/c31 8338673 1 2024-04-26T04:42:14.462 INFO:tasks.workunit.client.1.smithi104.stdout:6/479: creat d0/d9/d13/d44/f9a x:0 0 0 2024-04-26T04:42:14.462 INFO:tasks.workunit.client.1.smithi104.stdout:6/480: chown d0/d9/d13/c5a 43 1 2024-04-26T04:42:14.462 INFO:tasks.workunit.client.1.smithi104.stdout:6/481: write d0/d9/d13/d15/f84 [224287,59972] 0 2024-04-26T04:42:14.462 INFO:tasks.workunit.client.1.smithi104.stdout:6/482: write d0/d9/d25/f68 [485446,79262] 0 2024-04-26T04:42:14.463 INFO:tasks.workunit.client.1.smithi104.stdout:2/523: rename d6/d1b/l4d to d6/d8/lae 0 2024-04-26T04:42:14.463 INFO:tasks.workunit.client.1.smithi104.stdout:2/524: chown d6/d1b/l41 3 1 2024-04-26T04:42:14.465 INFO:tasks.workunit.client.1.smithi104.stdout:0/313: unlink d0/d4/d15/c16 0 2024-04-26T04:42:14.465 INFO:tasks.workunit.client.1.smithi104.stdout:0/314: truncate d0/d2/d21/d27/d1e/f2c 1040290 0 2024-04-26T04:42:14.466 INFO:tasks.workunit.client.0.smithi090.stdout:5/111: dwrite d4/d1a/f14 [0,4194304] 0 2024-04-26T04:42:14.468 INFO:tasks.workunit.client.1.smithi104.stdout:2/525: symlink d6/d1b/d51/laf 0 2024-04-26T04:42:14.468 INFO:tasks.workunit.client.1.smithi104.stdout:2/526: fsync d6/d8/d3e/f49 0 2024-04-26T04:42:14.469 INFO:tasks.workunit.client.1.smithi104.stdout:2/527: dread - d6/d8/d5a/f74 zero size 2024-04-26T04:42:14.469 INFO:tasks.workunit.client.1.smithi104.stdout:0/315: mkdir d0/d2/d21/d49/d62 0 2024-04-26T04:42:14.470 INFO:tasks.workunit.client.1.smithi104.stdout:6/483: dread d0/d9/d13/d15/f84 [0,4194304] 0 2024-04-26T04:42:14.470 INFO:tasks.workunit.client.1.smithi104.stdout:2/528: rmdir d6/d1b 39 2024-04-26T04:42:14.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:14 smithi090 ceph-mon[26648]: pgmap v249: 65 pgs: 65 active+clean; 606 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 42 MiB/s rd, 63 MiB/s wr, 1.01k op/s 2024-04-26T04:42:14.472 INFO:tasks.workunit.client.0.smithi090.stdout:5/112: stat d4/d1a/df/c15 0 2024-04-26T04:42:14.472 INFO:tasks.workunit.client.1.smithi104.stdout:0/316: rename d0/d2/d21/c33 to d0/d2/d10/c63 0 2024-04-26T04:42:14.472 INFO:tasks.workunit.client.1.smithi104.stdout:0/317: chown d0/d4/d3c/f46 4 1 2024-04-26T04:42:14.473 INFO:tasks.workunit.client.1.smithi104.stdout:0/318: write d0/d4/f29 [1158032,79536] 0 2024-04-26T04:42:14.473 INFO:tasks.workunit.client.0.smithi090.stdout:4/64: rename d3/db/dd to d3/db/d12 0 2024-04-26T04:42:14.474 INFO:tasks.workunit.client.1.smithi104.stdout:6/484: creat d0/d5/d8e/f9b x:0 0 0 2024-04-26T04:42:14.474 INFO:tasks.workunit.client.1.smithi104.stdout:6/485: fsync d0/d9/f93 0 2024-04-26T04:42:14.475 INFO:tasks.workunit.client.1.smithi104.stdout:6/486: chown d0/d9/d25/c2e 6341629 1 2024-04-26T04:42:14.475 INFO:tasks.workunit.client.1.smithi104.stdout:2/529: rmdir d6/d1b/d93 39 2024-04-26T04:42:14.475 INFO:tasks.workunit.client.1.smithi104.stdout:2/530: creat d6/d8/d5a/fb0 x:0 0 0 2024-04-26T04:42:14.475 INFO:tasks.workunit.client.1.smithi104.stdout:2/531: chown d6/d8/d3e/l3b 510 1 2024-04-26T04:42:14.475 INFO:tasks.workunit.client.1.smithi104.stdout:2/532: creat d6/d8/fb1 x:0 0 0 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/113: creat d4/d1a/f23 x:0 0 0 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/114: dread - d4/d1a/f19 zero size 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/115: stat d4/d1a/f23 0 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/116: dread - d4/d1a/f19 zero size 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/117: fsync d4/d1a/df/f22 0 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/118: fdatasync d4/f1e 0 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:4/65: mkdir d3/d13 0 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:4/66: readlink d3/le 0 2024-04-26T04:42:14.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/119: symlink d4/d1a/d1f/l24 0 2024-04-26T04:42:14.477 INFO:tasks.workunit.client.1.smithi104.stdout:0/319: unlink d0/d2/d21/d27/d1e/f61 0 2024-04-26T04:42:14.477 INFO:tasks.workunit.client.1.smithi104.stdout:6/487: read d0/f6e [603915,49345] 0 2024-04-26T04:42:14.478 INFO:tasks.workunit.client.0.smithi090.stdout:4/67: creat d3/f14 x:0 0 0 2024-04-26T04:42:14.478 INFO:tasks.workunit.client.1.smithi104.stdout:2/533: truncate d6/de/d8e/f60 2897085 0 2024-04-26T04:42:14.479 INFO:tasks.workunit.client.1.smithi104.stdout:2/534: mknod d6/de/d8e/cb2 0 2024-04-26T04:42:14.480 INFO:tasks.workunit.client.0.smithi090.stdout:4/68: mknod d3/db/d12/c15 0 2024-04-26T04:42:14.485 INFO:tasks.workunit.client.1.smithi104.stdout:2/535: unlink d6/d8/d3e/f26 0 2024-04-26T04:42:14.485 INFO:tasks.workunit.client.1.smithi104.stdout:2/536: creat d6/d8/d5a/fb3 x:0 0 0 2024-04-26T04:42:14.485 INFO:tasks.workunit.client.1.smithi104.stdout:9/273: dwrite fc [0,4194304] 0 2024-04-26T04:42:14.485 INFO:tasks.workunit.client.1.smithi104.stdout:9/274: chown d11/f27 28485 1 2024-04-26T04:42:14.486 INFO:tasks.workunit.client.0.smithi090.stdout:2/54: dwrite f2 [0,4194304] 0 2024-04-26T04:42:14.489 INFO:tasks.workunit.client.1.smithi104.stdout:2/537: symlink d6/de/d1d/lb4 0 2024-04-26T04:42:14.493 INFO:tasks.workunit.client.1.smithi104.stdout:6/488: dwrite d0/d9/d13/f91 [0,4194304] 0 2024-04-26T04:42:14.494 INFO:tasks.workunit.client.1.smithi104.stdout:6/489: creat d0/f9c x:0 0 0 2024-04-26T04:42:14.494 INFO:tasks.workunit.client.0.smithi090.stdout:5/120: truncate d4/d1a/f14 3052324 0 2024-04-26T04:42:14.497 INFO:tasks.workunit.client.0.smithi090.stdout:5/121: truncate d4/d1a/df/f10 3504181 0 2024-04-26T04:42:14.501 INFO:tasks.workunit.client.1.smithi104.stdout:2/538: unlink d6/de/d8e/l6c 0 2024-04-26T04:42:14.501 INFO:tasks.workunit.client.0.smithi090.stdout:5/122: rename d4/d1a/c8 to d4/d1a/df/d21/c25 0 2024-04-26T04:42:14.503 INFO:tasks.workunit.client.0.smithi090.stdout:5/123: link d4/d1a/c6 d4/d1a/d1f/c26 0 2024-04-26T04:42:14.504 INFO:tasks.workunit.client.0.smithi090.stdout:5/124: write d4/d1a/f1d [582316,121175] 0 2024-04-26T04:42:14.504 INFO:tasks.workunit.client.0.smithi090.stdout:5/125: fsync d4/f13 0 2024-04-26T04:42:14.510 INFO:tasks.workunit.client.1.smithi104.stdout:2/539: mknod d6/de/d1d/cb5 0 2024-04-26T04:42:14.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:14 smithi104 ceph-mon[32369]: pgmap v249: 65 pgs: 65 active+clean; 606 MiB data, 2.0 GiB used, 534 GiB / 536 GiB avail; 42 MiB/s rd, 63 MiB/s wr, 1.01k op/s 2024-04-26T04:42:14.511 INFO:tasks.workunit.client.0.smithi090.stdout:5/126: write d4/d1a/f14 [3405927,13372] 0 2024-04-26T04:42:14.514 INFO:tasks.workunit.client.0.smithi090.stdout:5/127: mkdir d4/d1a/d1f/d27 0 2024-04-26T04:42:14.515 INFO:tasks.workunit.client.0.smithi090.stdout:5/128: creat d4/d1a/df/f28 x:0 0 0 2024-04-26T04:42:14.519 INFO:tasks.workunit.client.0.smithi090.stdout:4/69: dwrite d3/f6 [0,4194304] 0 2024-04-26T04:42:14.521 INFO:tasks.workunit.client.0.smithi090.stdout:4/70: symlink d3/db/l16 0 2024-04-26T04:42:14.521 INFO:tasks.workunit.client.0.smithi090.stdout:2/55: dwrite f2 [0,4194304] 0 2024-04-26T04:42:14.522 INFO:tasks.workunit.client.1.smithi104.stdout:6/490: dwrite d0/d9/d13/d15/f42 [0,4194304] 0 2024-04-26T04:42:14.522 INFO:tasks.workunit.client.1.smithi104.stdout:9/275: dwrite f10 [0,4194304] 0 2024-04-26T04:42:14.522 INFO:tasks.workunit.client.0.smithi090.stdout:4/71: getdents d3/db/d12 0 2024-04-26T04:42:14.524 INFO:tasks.workunit.client.0.smithi090.stdout:2/56: symlink db/dc/ld 0 2024-04-26T04:42:14.524 INFO:tasks.workunit.client.1.smithi104.stdout:9/276: mkdir d11/d15/d32/d4c/d54 0 2024-04-26T04:42:14.524 INFO:tasks.workunit.client.1.smithi104.stdout:9/277: fsync d11/f16 0 2024-04-26T04:42:14.525 INFO:tasks.workunit.client.1.smithi104.stdout:9/278: dread - d11/d15/f3d zero size 2024-04-26T04:42:14.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/72: rename d3/f6 to d3/db/d12/f17 0 2024-04-26T04:42:14.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/73: chown d3/f9 1281 1 2024-04-26T04:42:14.530 INFO:tasks.workunit.client.0.smithi090.stdout:4/74: chown d3/db/d12/c15 390 1 2024-04-26T04:42:14.532 INFO:tasks.workunit.client.0.smithi090.stdout:4/75: symlink d3/l18 0 2024-04-26T04:42:14.539 INFO:tasks.workunit.client.1.smithi104.stdout:2/540: dwrite d6/de/d1d/f83 [0,4194304] 0 2024-04-26T04:42:14.542 INFO:tasks.workunit.client.1.smithi104.stdout:2/541: link d6/d8/d5a/fb0 d6/d1b/d51/d6a/fb6 0 2024-04-26T04:42:14.543 INFO:tasks.workunit.client.1.smithi104.stdout:2/542: mknod d6/d8/d3e/cb7 0 2024-04-26T04:42:14.549 INFO:tasks.workunit.client.0.smithi090.stdout:4/76: write d3/db/d12/f17 [2829549,83973] 0 2024-04-26T04:42:14.550 INFO:tasks.workunit.client.0.smithi090.stdout:4/77: link d3/f9 d3/db/f19 0 2024-04-26T04:42:14.550 INFO:tasks.workunit.client.0.smithi090.stdout:4/78: write d3/db/d12/ff [922103,20544] 0 2024-04-26T04:42:14.551 INFO:tasks.workunit.client.0.smithi090.stdout:4/79: read - d3/db/fc zero size 2024-04-26T04:42:14.553 INFO:tasks.workunit.client.1.smithi104.stdout:5/361: sync 2024-04-26T04:42:14.554 INFO:tasks.workunit.client.0.smithi090.stdout:2/57: truncate f2 149999 0 2024-04-26T04:42:14.555 INFO:tasks.workunit.client.1.smithi104.stdout:5/362: symlink d0/d1/d4/d2a/d2b/d5c/l75 0 2024-04-26T04:42:14.555 INFO:tasks.workunit.client.1.smithi104.stdout:5/363: readlink d0/d1/d4/la 0 2024-04-26T04:42:14.555 INFO:tasks.workunit.client.0.smithi090.stdout:2/58: mknod db/ce 0 2024-04-26T04:42:14.557 INFO:tasks.workunit.client.0.smithi090.stdout:5/129: dwrite d4/d1a/f14 [0,4194304] 0 2024-04-26T04:42:14.558 INFO:tasks.workunit.client.1.smithi104.stdout:5/364: rename d0/d1/c67 to d0/d41/c76 0 2024-04-26T04:42:14.558 INFO:tasks.workunit.client.1.smithi104.stdout:5/365: write d0/d1/f3d [675755,78779] 0 2024-04-26T04:42:14.558 INFO:tasks.workunit.client.1.smithi104.stdout:5/366: creat d0/d1/d64/f77 x:0 0 0 2024-04-26T04:42:14.558 INFO:tasks.workunit.client.1.smithi104.stdout:5/367: chown d0/d1/d3e/f51 2321524 1 2024-04-26T04:42:14.558 INFO:tasks.workunit.client.1.smithi104.stdout:9/279: dwrite d11/f16 [4194304,4194304] 0 2024-04-26T04:42:14.559 INFO:tasks.workunit.client.1.smithi104.stdout:2/543: write d6/d8/d5a/f54 [843606,107668] 0 2024-04-26T04:42:14.559 INFO:tasks.workunit.client.1.smithi104.stdout:9/280: write d11/d1d/f49 [689070,81508] 0 2024-04-26T04:42:14.559 INFO:tasks.workunit.client.0.smithi090.stdout:5/130: write d4/f13 [695939,89998] 0 2024-04-26T04:42:14.560 INFO:tasks.workunit.client.1.smithi104.stdout:5/368: symlink d0/d1/d4/d2a/d6b/l78 0 2024-04-26T04:42:14.561 INFO:tasks.workunit.client.1.smithi104.stdout:9/281: dread d11/d15/f30 [0,4194304] 0 2024-04-26T04:42:14.561 INFO:tasks.workunit.client.1.smithi104.stdout:6/491: dwrite d0/d9/d13/d41/f7a [0,4194304] 0 2024-04-26T04:42:14.561 INFO:tasks.workunit.client.1.smithi104.stdout:6/492: dread - d0/d9/f93 zero size 2024-04-26T04:42:14.563 INFO:tasks.workunit.client.0.smithi090.stdout:2/59: fsync f2 0 2024-04-26T04:42:14.565 INFO:tasks.workunit.client.0.smithi090.stdout:5/131: creat d4/d1a/d1f/d27/f29 x:0 0 0 2024-04-26T04:42:14.565 INFO:tasks.workunit.client.0.smithi090.stdout:5/132: fdatasync d4/d1a/df/f28 0 2024-04-26T04:42:14.565 INFO:tasks.workunit.client.0.smithi090.stdout:5/133: dread - d4/d1a/df/f28 zero size 2024-04-26T04:42:14.567 INFO:tasks.workunit.client.0.smithi090.stdout:2/60: link ca db/cf 0 2024-04-26T04:42:14.567 INFO:tasks.workunit.client.0.smithi090.stdout:2/61: write f2 [976572,74116] 0 2024-04-26T04:42:14.569 INFO:tasks.workunit.client.0.smithi090.stdout:2/62: creat db/f10 x:0 0 0 2024-04-26T04:42:14.569 INFO:tasks.workunit.client.0.smithi090.stdout:4/80: dwrite d3/f9 [4194304,4194304] 0 2024-04-26T04:42:14.569 INFO:tasks.workunit.client.1.smithi104.stdout:5/369: rename d0/d41/f55 to d0/d1/d3e/f79 0 2024-04-26T04:42:14.569 INFO:tasks.workunit.client.1.smithi104.stdout:9/282: stat d11/d1d/l33 0 2024-04-26T04:42:14.569 INFO:tasks.workunit.client.1.smithi104.stdout:9/283: readlink ld 0 2024-04-26T04:42:14.570 INFO:tasks.workunit.client.0.smithi090.stdout:2/63: mkdir db/d11 0 2024-04-26T04:42:14.570 INFO:tasks.workunit.client.0.smithi090.stdout:2/64: dread - db/f10 zero size 2024-04-26T04:42:14.571 INFO:tasks.workunit.client.1.smithi104.stdout:6/493: truncate d0/d9/d13/f43 989880 0 2024-04-26T04:42:14.571 INFO:tasks.workunit.client.1.smithi104.stdout:6/494: chown d0/d9/d13/d44/d47/f4c 7775 1 2024-04-26T04:42:14.571 INFO:tasks.workunit.client.1.smithi104.stdout:6/495: creat d0/f9d x:0 0 0 2024-04-26T04:42:14.571 INFO:tasks.workunit.client.1.smithi104.stdout:6/496: write d0/d9/f3b [857098,48450] 0 2024-04-26T04:42:14.571 INFO:tasks.workunit.client.0.smithi090.stdout:2/65: read f2 [458471,11973] 0 2024-04-26T04:42:14.573 INFO:tasks.workunit.client.1.smithi104.stdout:9/284: dread d11/d1d/f4f [0,4194304] 0 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:8/36: sync 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:9/29: sync 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:3/15: sync 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:6/148: sync 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:6/149: chown d9 107 1 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:3/16: truncate d1/f3 398841 0 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:0/144: sync 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:7/78: sync 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:0/145: chown d0/d9/la 2 1 2024-04-26T04:42:14.575 INFO:tasks.workunit.client.0.smithi090.stdout:1/58: sync 2024-04-26T04:42:14.576 INFO:tasks.workunit.client.1.smithi104.stdout:5/370: symlink d0/d1/d4/d2a/d2b/d37/d48/l7a 0 2024-04-26T04:42:14.577 INFO:tasks.workunit.client.1.smithi104.stdout:9/285: mknod d11/d53/c55 0 2024-04-26T04:42:14.577 INFO:tasks.workunit.client.1.smithi104.stdout:9/286: fdatasync d11/d15/f3d 0 2024-04-26T04:42:14.578 INFO:tasks.workunit.client.0.smithi090.stdout:8/37: write f7 [939986,113194] 0 2024-04-26T04:42:14.578 INFO:tasks.workunit.client.0.smithi090.stdout:8/38: rename dc to dc/dd 22 2024-04-26T04:42:14.579 INFO:tasks.workunit.client.0.smithi090.stdout:3/17: mknod d1/c5 0 2024-04-26T04:42:14.580 INFO:tasks.workunit.client.1.smithi104.stdout:9/287: creat d11/d15/d32/f56 x:0 0 0 2024-04-26T04:42:14.580 INFO:tasks.workunit.client.1.smithi104.stdout:9/288: write f10 [4751384,76623] 0 2024-04-26T04:42:14.582 INFO:tasks.workunit.client.1.smithi104.stdout:9/289: creat d11/d15/d32/d35/f57 x:0 0 0 2024-04-26T04:42:14.585 INFO:tasks.workunit.client.1.smithi104.stdout:9/290: mknod d11/d15/d32/c58 0 2024-04-26T04:42:14.590 INFO:tasks.workunit.client.0.smithi090.stdout:4/81: dread d3/db/d12/f17 [0,4194304] 0 2024-04-26T04:42:14.590 INFO:tasks.workunit.client.0.smithi090.stdout:0/146: dread d0/de/f14 [0,4194304] 0 2024-04-26T04:42:14.590 INFO:tasks.workunit.client.0.smithi090.stdout:0/147: write d0/df/f25 [259975,6274] 0 2024-04-26T04:42:14.591 INFO:tasks.workunit.client.1.smithi104.stdout:4/392: sync 2024-04-26T04:42:14.591 INFO:tasks.workunit.client.1.smithi104.stdout:1/397: sync 2024-04-26T04:42:14.592 INFO:tasks.workunit.client.1.smithi104.stdout:8/228: sync 2024-04-26T04:42:14.592 INFO:tasks.workunit.client.1.smithi104.stdout:3/384: sync 2024-04-26T04:42:14.592 INFO:tasks.workunit.client.1.smithi104.stdout:7/279: sync 2024-04-26T04:42:14.592 INFO:tasks.workunit.client.1.smithi104.stdout:7/280: fdatasync d2/d4/d32/d3d/f50 0 2024-04-26T04:42:14.592 INFO:tasks.workunit.client.1.smithi104.stdout:4/393: creat d4/d7/d1d/d1e/d24/f81 x:0 0 0 2024-04-26T04:42:14.592 INFO:tasks.workunit.client.0.smithi090.stdout:0/148: dread d0/f2d [0,4194304] 0 2024-04-26T04:42:14.593 INFO:tasks.workunit.client.0.smithi090.stdout:0/149: mkdir d0/d9/d33 0 2024-04-26T04:42:14.595 INFO:tasks.workunit.client.0.smithi090.stdout:0/150: rename d0/df/f31 to d0/d26/f34 0 2024-04-26T04:42:14.595 INFO:tasks.workunit.client.1.smithi104.stdout:8/229: rename d0/fb to d0/d1c/d2d/f48 0 2024-04-26T04:42:14.597 INFO:tasks.workunit.client.1.smithi104.stdout:1/398: creat d0/d6/d19/d1b/f7f x:0 0 0 2024-04-26T04:42:14.597 INFO:tasks.workunit.client.1.smithi104.stdout:1/399: chown d0/d6/d2e/d32/d6a 15 1 2024-04-26T04:42:14.597 INFO:tasks.workunit.client.1.smithi104.stdout:1/400: read d0/d6/d49/f68 [439362,99275] 0 2024-04-26T04:42:14.598 INFO:tasks.workunit.client.0.smithi090.stdout:0/151: unlink d0/df/c29 0 2024-04-26T04:42:14.598 INFO:tasks.workunit.client.1.smithi104.stdout:7/281: rmdir d2/d4/d32/d35/d44 39 2024-04-26T04:42:14.600 INFO:tasks.workunit.client.1.smithi104.stdout:4/394: creat d4/d7/d1d/d1e/d24/d4a/f82 x:0 0 0 2024-04-26T04:42:14.600 INFO:tasks.workunit.client.0.smithi090.stdout:0/152: mknod d0/d9/c35 0 2024-04-26T04:42:14.603 INFO:tasks.workunit.client.1.smithi104.stdout:8/230: link d0/d33/d35/f45 d0/f49 0 2024-04-26T04:42:14.603 INFO:tasks.workunit.client.1.smithi104.stdout:8/231: write d0/d1c/d27/d47/f42 [463338,39928] 0 2024-04-26T04:42:14.603 INFO:tasks.workunit.client.1.smithi104.stdout:8/232: readlink d0/l6 0 2024-04-26T04:42:14.603 INFO:tasks.workunit.client.1.smithi104.stdout:8/233: write d0/f2e [155929,74492] 0 2024-04-26T04:42:14.603 INFO:tasks.workunit.client.1.smithi104.stdout:8/234: write d0/f1e [1268353,104120] 0 2024-04-26T04:42:14.603 INFO:tasks.workunit.client.1.smithi104.stdout:8/235: fsync d0/d1c/d27/d47/f25 0 2024-04-26T04:42:14.604 INFO:tasks.workunit.client.1.smithi104.stdout:7/282: link d2/d4/d32/d35/c43 d2/d4/c54 0 2024-04-26T04:42:14.604 INFO:tasks.workunit.client.1.smithi104.stdout:7/283: fsync d2/d4/d1d/f30 0 2024-04-26T04:42:14.604 INFO:tasks.workunit.client.1.smithi104.stdout:4/395: dread d4/d7/d1d/d1e/d24/d48/f60 [0,4194304] 0 2024-04-26T04:42:14.605 INFO:tasks.workunit.client.0.smithi090.stdout:3/18: dread d1/f3 [0,4194304] 0 2024-04-26T04:42:14.605 INFO:tasks.workunit.client.0.smithi090.stdout:3/19: fsync d1/f3 0 2024-04-26T04:42:14.605 INFO:tasks.workunit.client.0.smithi090.stdout:3/20: truncate d1/f3 1370633 0 2024-04-26T04:42:14.605 INFO:tasks.workunit.client.0.smithi090.stdout:3/21: write d1/f3 [2057770,86398] 0 2024-04-26T04:42:14.605 INFO:tasks.workunit.client.1.smithi104.stdout:3/385: rename d5/d9/dc/f15 to d5/d9/d17/d60/f84 0 2024-04-26T04:42:14.606 INFO:tasks.workunit.client.1.smithi104.stdout:3/386: mkdir d5/d9/d17/d72/d85 0 2024-04-26T04:42:14.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/22: mkdir d1/d6 0 2024-04-26T04:42:14.608 INFO:tasks.workunit.client.1.smithi104.stdout:4/396: rename d4/d20/d55/d56/f58 to d4/d7/d1d/d1e/f83 0 2024-04-26T04:42:14.609 INFO:tasks.workunit.client.1.smithi104.stdout:3/387: mkdir d5/d9/d86 0 2024-04-26T04:42:14.609 INFO:tasks.workunit.client.0.smithi090.stdout:3/23: symlink d1/l7 0 2024-04-26T04:42:14.610 INFO:tasks.workunit.client.0.smithi090.stdout:3/24: read d1/f3 [1251907,90305] 0 2024-04-26T04:42:14.611 INFO:tasks.workunit.client.1.smithi104.stdout:4/397: unlink d4/db/d6b/f6c 0 2024-04-26T04:42:14.611 INFO:tasks.workunit.client.1.smithi104.stdout:4/398: fsync d4/d20/f36 0 2024-04-26T04:42:14.611 INFO:tasks.workunit.client.1.smithi104.stdout:3/388: symlink d5/d9/dc/d69/l87 0 2024-04-26T04:42:14.611 INFO:tasks.workunit.client.1.smithi104.stdout:3/389: write d5/d9/d17/f3b [1359105,61100] 0 2024-04-26T04:42:14.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/401: dread d0/d6/d19/f3a [0,4194304] 0 2024-04-26T04:42:14.614 INFO:tasks.workunit.client.0.smithi090.stdout:3/25: unlink d1/c5 0 2024-04-26T04:42:14.614 INFO:tasks.workunit.client.1.smithi104.stdout:3/390: rename d5/d9/d17/l7c to d5/d9/d17/d46/l88 0 2024-04-26T04:42:14.614 INFO:tasks.workunit.client.1.smithi104.stdout:3/391: dread - d5/d9/dc/f36 zero size 2024-04-26T04:42:14.616 INFO:tasks.workunit.client.1.smithi104.stdout:3/392: creat d5/d9/d17/d72/d85/f89 x:0 0 0 2024-04-26T04:42:14.616 INFO:tasks.workunit.client.1.smithi104.stdout:3/393: chown d5/d9/ff 807 1 2024-04-26T04:42:14.618 INFO:tasks.workunit.client.0.smithi090.stdout:3/26: creat d1/f8 x:0 0 0 2024-04-26T04:42:14.618 INFO:tasks.workunit.client.0.smithi090.stdout:3/27: write d1/f3 [2632107,129404] 0 2024-04-26T04:42:14.618 INFO:tasks.workunit.client.0.smithi090.stdout:3/28: dread - d1/f8 zero size 2024-04-26T04:42:14.619 INFO:tasks.workunit.client.0.smithi090.stdout:3/29: chown d1/f3 159 1 2024-04-26T04:42:14.619 INFO:tasks.workunit.client.1.smithi104.stdout:3/394: read d5/f24 [603041,94890] 0 2024-04-26T04:42:14.620 INFO:tasks.workunit.client.0.smithi090.stdout:5/134: dwrite d4/fe [0,4194304] 0 2024-04-26T04:42:14.621 INFO:tasks.workunit.client.0.smithi090.stdout:3/30: mkdir d1/d9 0 2024-04-26T04:42:14.622 INFO:tasks.workunit.client.1.smithi104.stdout:3/395: mknod d5/d9/dc/d69/c8a 0 2024-04-26T04:42:14.625 INFO:tasks.workunit.client.1.smithi104.stdout:7/284: dread d2/d4/d1d/f30 [0,4194304] 0 2024-04-26T04:42:14.625 INFO:tasks.workunit.client.1.smithi104.stdout:7/285: readlink d2/d4/l2d 0 2024-04-26T04:42:14.625 INFO:tasks.workunit.client.1.smithi104.stdout:7/286: read - d2/d4/d1d/d34/f51 zero size 2024-04-26T04:42:14.626 INFO:tasks.workunit.client.1.smithi104.stdout:7/287: mknod d2/d4/d1d/c55 0 2024-04-26T04:42:14.627 INFO:tasks.workunit.client.1.smithi104.stdout:7/288: creat d2/f56 x:0 0 0 2024-04-26T04:42:14.628 INFO:tasks.workunit.client.1.smithi104.stdout:7/289: truncate d2/f2f 7775645 0 2024-04-26T04:42:14.629 INFO:tasks.workunit.client.1.smithi104.stdout:7/290: symlink d2/d4/d32/d35/l57 0 2024-04-26T04:42:14.630 INFO:tasks.workunit.client.1.smithi104.stdout:7/291: mknod d2/d4/d1d/d34/c58 0 2024-04-26T04:42:14.635 INFO:tasks.workunit.client.0.smithi090.stdout:9/30: dwrite f9 [0,4194304] 0 2024-04-26T04:42:14.637 INFO:tasks.workunit.client.0.smithi090.stdout:9/31: mkdir db 0 2024-04-26T04:42:14.655 INFO:tasks.workunit.client.0.smithi090.stdout:9/32: dread f3 [0,4194304] 0 2024-04-26T04:42:14.655 INFO:tasks.workunit.client.0.smithi090.stdout:9/33: chown f2 20544983 1 2024-04-26T04:42:14.655 INFO:tasks.workunit.client.1.smithi104.stdout:3/396: dread d5/d9/f14 [0,4194304] 0 2024-04-26T04:42:14.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/397: fdatasync d5/d9/dc/d69/f3e 0 2024-04-26T04:42:14.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/398: creat d5/d9/dc/d29/d49/d59/f8b x:0 0 0 2024-04-26T04:42:14.656 INFO:tasks.workunit.client.1.smithi104.stdout:3/399: read d5/d9/dc/d29/d49/d59/f74 [165936,15303] 0 2024-04-26T04:42:14.657 INFO:tasks.workunit.client.1.smithi104.stdout:3/400: mknod d5/d9/d17/c8c 0 2024-04-26T04:42:14.662 INFO:tasks.workunit.client.0.smithi090.stdout:6/150: dwrite d9/f12 [0,4194304] 0 2024-04-26T04:42:14.664 INFO:tasks.workunit.client.0.smithi090.stdout:6/151: symlink d9/l39 0 2024-04-26T04:42:14.665 INFO:tasks.workunit.client.1.smithi104.stdout:3/401: write d5/d9/dc/d69/f55 [782283,107082] 0 2024-04-26T04:42:14.665 INFO:tasks.workunit.client.1.smithi104.stdout:3/402: truncate d5/d9/f4a 1561797 0 2024-04-26T04:42:14.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/59: dwrite d0/d4/d7/fa [0,4194304] 0 2024-04-26T04:42:14.667 INFO:tasks.workunit.client.1.smithi104.stdout:5/371: dwrite d0/d1/d4/f17 [0,4194304] 0 2024-04-26T04:42:14.667 INFO:tasks.workunit.client.1.smithi104.stdout:4/399: dwrite d4/d7/f8 [0,4194304] 0 2024-04-26T04:42:14.667 INFO:tasks.workunit.client.1.smithi104.stdout:8/236: dwrite d0/f1a [4194304,4194304] 0 2024-04-26T04:42:14.668 INFO:tasks.workunit.client.0.smithi090.stdout:4/82: dwrite f1 [0,4194304] 0 2024-04-26T04:42:14.669 INFO:tasks.workunit.client.1.smithi104.stdout:1/402: dwrite d0/d6/d19/d21/d28/f36 [0,4194304] 0 2024-04-26T04:42:14.669 INFO:tasks.workunit.client.1.smithi104.stdout:3/403: mknod d5/d9/dc/d29/c8d 0 2024-04-26T04:42:14.669 INFO:tasks.workunit.client.0.smithi090.stdout:5/135: dwrite d4/d1a/f23 [0,4194304] 0 2024-04-26T04:42:14.670 INFO:tasks.workunit.client.0.smithi090.stdout:8/39: dwrite f8 [0,4194304] 0 2024-04-26T04:42:14.673 INFO:tasks.workunit.client.1.smithi104.stdout:4/400: getdents d4/d20 0 2024-04-26T04:42:14.678 INFO:tasks.workunit.client.1.smithi104.stdout:1/403: creat d0/f80 x:0 0 0 2024-04-26T04:42:14.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/404: mkdir d5/d9/dc/d29/d7d/d8e 0 2024-04-26T04:42:14.683 INFO:tasks.workunit.client.0.smithi090.stdout:4/83: symlink d3/l1a 0 2024-04-26T04:42:14.683 INFO:tasks.workunit.client.0.smithi090.stdout:4/84: chown d3/f8 818110 1 2024-04-26T04:42:14.683 INFO:tasks.workunit.client.1.smithi104.stdout:4/401: creat d4/db/d6b/d7f/f84 x:0 0 0 2024-04-26T04:42:14.684 INFO:tasks.workunit.client.1.smithi104.stdout:1/404: mkdir d0/d6/d19/d21/d66/d30/d81 0 2024-04-26T04:42:14.688 INFO:tasks.workunit.client.1.smithi104.stdout:3/405: unlink d5/d9/dc/d29/d49/d59/l7a 0 2024-04-26T04:42:14.688 INFO:tasks.workunit.client.0.smithi090.stdout:5/136: mkdir d4/d1a/d2a 0 2024-04-26T04:42:14.689 INFO:tasks.workunit.client.0.smithi090.stdout:6/152: dwrite d9/fd [0,4194304] 0 2024-04-26T04:42:14.689 INFO:tasks.workunit.client.0.smithi090.stdout:6/153: chown d9/c20 21994461 1 2024-04-26T04:42:14.690 INFO:tasks.workunit.client.1.smithi104.stdout:4/402: mknod d4/d7/d1d/d2e/d74/c85 0 2024-04-26T04:42:14.690 INFO:tasks.workunit.client.1.smithi104.stdout:4/403: truncate d4/d7/d1d/d2e/f7e 1167562 0 2024-04-26T04:42:14.690 INFO:tasks.workunit.client.1.smithi104.stdout:4/404: dread - d4/d7/d1d/d1e/d24/d4a/f50 zero size 2024-04-26T04:42:14.694 INFO:tasks.workunit.client.1.smithi104.stdout:3/406: mknod d5/d9/dc/d3a/c8f 0 2024-04-26T04:42:14.694 INFO:tasks.workunit.client.1.smithi104.stdout:3/407: chown d5/d9/d17/d60/f84 3 1 2024-04-26T04:42:14.695 INFO:tasks.workunit.client.1.smithi104.stdout:4/405: mkdir d4/db/d6b/d86 0 2024-04-26T04:42:14.695 INFO:tasks.workunit.client.0.smithi090.stdout:4/85: creat d3/d13/f1b x:0 0 0 2024-04-26T04:42:14.695 INFO:tasks.workunit.client.0.smithi090.stdout:4/86: fsync d3/db/fc 0 2024-04-26T04:42:14.696 INFO:tasks.workunit.client.1.smithi104.stdout:4/406: truncate d4/f37 2860601 0 2024-04-26T04:42:14.696 INFO:tasks.workunit.client.1.smithi104.stdout:1/405: read d0/f14 [6235426,124837] 0 2024-04-26T04:42:14.696 INFO:tasks.workunit.client.1.smithi104.stdout:4/407: write f2 [4311428,123840] 0 2024-04-26T04:42:14.697 INFO:tasks.workunit.client.0.smithi090.stdout:5/137: link d4/f20 d4/d1a/d1f/f2b 0 2024-04-26T04:42:14.698 INFO:tasks.workunit.client.1.smithi104.stdout:1/406: unlink d0/d6/d2e/d32/c38 0 2024-04-26T04:42:14.699 INFO:tasks.workunit.client.0.smithi090.stdout:9/34: truncate f9 4006090 0 2024-04-26T04:42:14.700 INFO:tasks.workunit.client.0.smithi090.stdout:8/40: dread f4 [0,4194304] 0 2024-04-26T04:42:14.700 INFO:tasks.workunit.client.0.smithi090.stdout:6/154: creat d9/d1c/d31/f3a x:0 0 0 2024-04-26T04:42:14.701 INFO:tasks.workunit.client.1.smithi104.stdout:4/408: creat d4/d20/d55/d72/f87 x:0 0 0 2024-04-26T04:42:14.701 INFO:tasks.workunit.client.1.smithi104.stdout:4/409: creat d4/d20/d55/d72/f88 x:0 0 0 2024-04-26T04:42:14.701 INFO:tasks.workunit.client.1.smithi104.stdout:4/410: readlink d4/d7/d1d/d1e/d24/d59/l62 0 2024-04-26T04:42:14.702 INFO:tasks.workunit.client.0.smithi090.stdout:5/138: truncate d4/f13 122378 0 2024-04-26T04:42:14.703 INFO:tasks.workunit.client.1.smithi104.stdout:1/407: rename d0/d6/d19/d21/d66/d20/d2a/d56/d67/c69 to d0/d6/d19/d21/d66/d29/c82 0 2024-04-26T04:42:14.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/35: creat db/fc x:0 0 0 2024-04-26T04:42:14.705 INFO:tasks.workunit.client.1.smithi104.stdout:4/411: symlink d4/db/l89 0 2024-04-26T04:42:14.706 INFO:tasks.workunit.client.1.smithi104.stdout:7/292: write d2/d4/f10 [3865847,17184] 0 2024-04-26T04:42:14.708 INFO:tasks.workunit.client.1.smithi104.stdout:1/408: creat d0/d6/d19/d21/d66/f83 x:0 0 0 2024-04-26T04:42:14.708 INFO:tasks.workunit.client.1.smithi104.stdout:1/409: dread - d0/d6/d19/d21/d66/f83 zero size 2024-04-26T04:42:14.708 INFO:tasks.workunit.client.1.smithi104.stdout:1/410: creat d0/d6/d2e/f84 x:0 0 0 2024-04-26T04:42:14.708 INFO:tasks.workunit.client.1.smithi104.stdout:1/411: chown d0/d6/d19/f75 510764617 1 2024-04-26T04:42:14.708 INFO:tasks.workunit.client.0.smithi090.stdout:9/36: mkdir db/dd 0 2024-04-26T04:42:14.708 INFO:tasks.workunit.client.0.smithi090.stdout:9/37: creat db/fe x:0 0 0 2024-04-26T04:42:14.709 INFO:tasks.workunit.client.0.smithi090.stdout:8/41: rename c1 to dc/ce 0 2024-04-26T04:42:14.709 INFO:tasks.workunit.client.0.smithi090.stdout:8/42: chown fa 5137600 1 2024-04-26T04:42:14.709 INFO:tasks.workunit.client.1.smithi104.stdout:4/412: getdents d4/d20/d55 0 2024-04-26T04:42:14.709 INFO:tasks.workunit.client.1.smithi104.stdout:4/413: readlink d4/d7/d1d/l27 0 2024-04-26T04:42:14.710 INFO:tasks.workunit.client.0.smithi090.stdout:6/155: dread d9/fd [0,4194304] 0 2024-04-26T04:42:14.711 INFO:tasks.workunit.client.1.smithi104.stdout:4/414: mknod d4/d20/d55/c8a 0 2024-04-26T04:42:14.712 INFO:tasks.workunit.client.1.smithi104.stdout:4/415: truncate d4/db/fe 227373 0 2024-04-26T04:42:14.712 INFO:tasks.workunit.client.1.smithi104.stdout:1/412: dread d0/d6/d19/d21/d66/f41 [0,4194304] 0 2024-04-26T04:42:14.715 INFO:tasks.workunit.client.1.smithi104.stdout:4/416: mknod d4/db/d6b/c8b 0 2024-04-26T04:42:14.716 INFO:tasks.workunit.client.1.smithi104.stdout:1/413: creat d0/d6/d19/d21/d28/f85 x:0 0 0 2024-04-26T04:42:14.717 INFO:tasks.workunit.client.1.smithi104.stdout:4/417: mknod d4/d7/d1d/d1e/d24/d48/c8c 0 2024-04-26T04:42:14.717 INFO:tasks.workunit.client.1.smithi104.stdout:4/418: dread - d4/d7/d1d/d1e/d24/d4a/f82 zero size 2024-04-26T04:42:14.719 INFO:tasks.workunit.client.1.smithi104.stdout:4/419: rename d4/d7/d1d/d1e/d24/l44 to d4/d7/l8d 0 2024-04-26T04:42:14.720 INFO:tasks.workunit.client.0.smithi090.stdout:6/156: dread f3 [0,4194304] 0 2024-04-26T04:42:14.721 INFO:tasks.workunit.client.1.smithi104.stdout:4/420: mkdir d4/d7/d1d/d8e 0 2024-04-26T04:42:14.721 INFO:tasks.workunit.client.1.smithi104.stdout:4/421: chown d4/f37 6 1 2024-04-26T04:42:14.722 INFO:tasks.workunit.client.0.smithi090.stdout:6/157: mkdir d9/d21/d3b 0 2024-04-26T04:42:14.723 INFO:tasks.workunit.client.1.smithi104.stdout:4/422: mkdir d4/d7/d1d/d1e/d8f 0 2024-04-26T04:42:14.723 INFO:tasks.workunit.client.1.smithi104.stdout:4/423: fdatasync d4/d7/f8 0 2024-04-26T04:42:14.724 INFO:tasks.workunit.client.1.smithi104.stdout:4/424: symlink d4/db/d6b/l90 0 2024-04-26T04:42:14.725 INFO:tasks.workunit.client.1.smithi104.stdout:4/425: symlink d4/d7/l91 0 2024-04-26T04:42:14.733 INFO:tasks.workunit.client.0.smithi090.stdout:8/43: write f5 [2595982,69226] 0 2024-04-26T04:42:14.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/31: dwrite d1/f3 [0,4194304] 0 2024-04-26T04:42:14.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/32: write d1/f8 [93570,129340] 0 2024-04-26T04:42:14.735 INFO:tasks.workunit.client.0.smithi090.stdout:8/44: rename f7 to dc/ff 0 2024-04-26T04:42:14.735 INFO:tasks.workunit.client.0.smithi090.stdout:8/45: fsync f8 0 2024-04-26T04:42:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/33: creat d1/d9/fa x:0 0 0 2024-04-26T04:42:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/34: creat d1/d9/fb x:0 0 0 2024-04-26T04:42:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/35: truncate d1/f8 1176393 0 2024-04-26T04:42:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/36: creat d1/fc x:0 0 0 2024-04-26T04:42:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/37: stat d1/l7 0 2024-04-26T04:42:14.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/38: dread - d1/fc zero size 2024-04-26T04:42:14.741 INFO:tasks.workunit.client.0.smithi090.stdout:1/60: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:14.741 INFO:tasks.workunit.client.1.smithi104.stdout:8/237: dwrite d0/d33/d35/f45 [0,4194304] 0 2024-04-26T04:42:14.742 INFO:tasks.workunit.client.0.smithi090.stdout:1/61: rename d0/cc to d0/d4/c13 0 2024-04-26T04:42:14.743 INFO:tasks.workunit.client.0.smithi090.stdout:1/62: link d0/f12 d0/d4/f14 0 2024-04-26T04:42:14.745 INFO:tasks.workunit.client.1.smithi104.stdout:8/238: symlink d0/d1c/d27/d46/l4a 0 2024-04-26T04:42:14.745 INFO:tasks.workunit.client.0.smithi090.stdout:1/63: write d0/f12 [58267,98055] 0 2024-04-26T04:42:14.748 INFO:tasks.workunit.client.1.smithi104.stdout:3/408: dwrite d5/d9/dc/d29/d49/d59/f8b [0,4194304] 0 2024-04-26T04:42:14.749 INFO:tasks.workunit.client.1.smithi104.stdout:7/293: dwrite d2/d4/f11 [0,4194304] 0 2024-04-26T04:42:14.749 INFO:tasks.workunit.client.1.smithi104.stdout:7/294: getdents d2/d4/d32/d3d/d47 0 2024-04-26T04:42:14.749 INFO:tasks.workunit.client.0.smithi090.stdout:4/87: dwrite d3/db/d12/ff [0,4194304] 0 2024-04-26T04:42:14.749 INFO:tasks.workunit.client.0.smithi090.stdout:5/139: dwrite d4/f20 [0,4194304] 0 2024-04-26T04:42:14.750 INFO:tasks.workunit.client.0.smithi090.stdout:5/140: creat d4/f2c x:0 0 0 2024-04-26T04:42:14.753 INFO:tasks.workunit.client.1.smithi104.stdout:7/295: mknod d2/d4/d32/d35/d44/c59 0 2024-04-26T04:42:14.753 INFO:tasks.workunit.client.1.smithi104.stdout:7/296: readlink d2/le 0 2024-04-26T04:42:14.753 INFO:tasks.workunit.client.1.smithi104.stdout:4/426: dwrite d4/d7/d1d/f45 [0,4194304] 0 2024-04-26T04:42:14.753 INFO:tasks.workunit.client.1.smithi104.stdout:4/427: dread - d4/d7/d1d/d1e/d24/f81 zero size 2024-04-26T04:42:14.753 INFO:tasks.workunit.client.1.smithi104.stdout:1/414: dwrite d0/d6/d19/d21/d66/d20/d2a/f31 [0,4194304] 0 2024-04-26T04:42:14.753 INFO:tasks.workunit.client.1.smithi104.stdout:1/415: write d0/d6/d2e/f84 [362471,109166] 0 2024-04-26T04:42:14.754 INFO:tasks.workunit.client.1.smithi104.stdout:1/416: chown d0/d6/d19/d21/d66/d20/d2a/f2c 23790794 1 2024-04-26T04:42:14.754 INFO:tasks.workunit.client.1.smithi104.stdout:1/417: truncate d0/d6/d19/d21/d66/d29/d3f/f7b 762256 0 2024-04-26T04:42:14.754 INFO:tasks.workunit.client.1.smithi104.stdout:3/409: rmdir d5/d9/d17/d72/d85 39 2024-04-26T04:42:14.755 INFO:tasks.workunit.client.0.smithi090.stdout:1/64: dread d0/d4/f14 [0,4194304] 0 2024-04-26T04:42:14.755 INFO:tasks.workunit.client.0.smithi090.stdout:1/65: chown d0/l10 6718 1 2024-04-26T04:42:14.755 INFO:tasks.workunit.client.0.smithi090.stdout:1/66: write d0/d4/f14 [267489,55306] 0 2024-04-26T04:42:14.755 INFO:tasks.workunit.client.0.smithi090.stdout:4/88: fsync d3/d13/f1b 0 2024-04-26T04:42:14.755 INFO:tasks.workunit.client.0.smithi090.stdout:4/89: dread - d3/db/fc zero size 2024-04-26T04:42:14.756 INFO:tasks.workunit.client.1.smithi104.stdout:7/297: mkdir d2/d4/d5a 0 2024-04-26T04:42:14.757 INFO:tasks.workunit.client.1.smithi104.stdout:4/428: dread d4/d20/f36 [0,4194304] 0 2024-04-26T04:42:14.757 INFO:tasks.workunit.client.1.smithi104.stdout:4/429: read - d4/d7/d1d/d1e/d24/d4a/f50 zero size 2024-04-26T04:42:14.759 INFO:tasks.workunit.client.1.smithi104.stdout:3/410: link d5/d9/dc/d29/d49/d59/f8b d5/f90 0 2024-04-26T04:42:14.760 INFO:tasks.workunit.client.1.smithi104.stdout:3/411: creat d5/d9/d17/f91 x:0 0 0 2024-04-26T04:42:14.760 INFO:tasks.workunit.client.1.smithi104.stdout:3/412: truncate d5/d9/dc/d69/f3e 2018449 0 2024-04-26T04:42:14.760 INFO:tasks.workunit.client.1.smithi104.stdout:3/413: write d5/d9/d17/f6c [970135,129899] 0 2024-04-26T04:42:14.760 INFO:tasks.workunit.client.0.smithi090.stdout:6/158: rmdir d9/d1c 39 2024-04-26T04:42:14.761 INFO:tasks.workunit.client.1.smithi104.stdout:7/298: mkdir d2/d4/d32/d35/d44/d5b 0 2024-04-26T04:42:14.762 INFO:tasks.workunit.client.1.smithi104.stdout:4/430: mknod d4/d7/d1d/d1e/d15/c92 0 2024-04-26T04:42:14.762 INFO:tasks.workunit.client.1.smithi104.stdout:4/431: stat d4/db/d6b/c8b 0 2024-04-26T04:42:14.764 INFO:tasks.workunit.client.0.smithi090.stdout:1/67: link d0/f12 d0/d4/f15 0 2024-04-26T04:42:14.764 INFO:tasks.workunit.client.0.smithi090.stdout:1/68: read d0/d4/f15 [549603,69380] 0 2024-04-26T04:42:14.765 INFO:tasks.workunit.client.0.smithi090.stdout:4/90: mknod d3/c1c 0 2024-04-26T04:42:14.770 INFO:tasks.workunit.client.0.smithi090.stdout:5/141: rmdir d4/d1a 39 2024-04-26T04:42:14.770 INFO:tasks.workunit.client.1.smithi104.stdout:7/299: dread d2/f2b [0,4194304] 0 2024-04-26T04:42:14.770 INFO:tasks.workunit.client.1.smithi104.stdout:7/300: dread - d2/d4/d32/d3d/f50 zero size 2024-04-26T04:42:14.771 INFO:tasks.workunit.client.0.smithi090.stdout:6/159: rmdir d9/d1c 39 2024-04-26T04:42:14.771 INFO:tasks.workunit.client.1.smithi104.stdout:3/414: symlink d5/d9/dc/d69/l92 0 2024-04-26T04:42:14.772 INFO:tasks.workunit.client.1.smithi104.stdout:3/415: dread - d5/d9/dc/d32/f5a zero size 2024-04-26T04:42:14.773 INFO:tasks.workunit.client.0.smithi090.stdout:4/91: mkdir d3/d13/d1d 0 2024-04-26T04:42:14.773 INFO:tasks.workunit.client.0.smithi090.stdout:4/92: chown d3/le 0 1 2024-04-26T04:42:14.774 INFO:tasks.workunit.client.1.smithi104.stdout:4/432: creat d4/d7/d1d/d2e/d74/f93 x:0 0 0 2024-04-26T04:42:14.774 INFO:tasks.workunit.client.1.smithi104.stdout:4/433: write d4/d7/d1d/d1e/f5a [647034,5507] 0 2024-04-26T04:42:14.774 INFO:tasks.workunit.client.1.smithi104.stdout:4/434: chown d4/db/f28 633214664 1 2024-04-26T04:42:14.774 INFO:tasks.workunit.client.0.smithi090.stdout:9/38: dwrite f9 [0,4194304] 0 2024-04-26T04:42:14.775 INFO:tasks.workunit.client.0.smithi090.stdout:5/142: symlink d4/l2d 0 2024-04-26T04:42:14.775 INFO:tasks.workunit.client.0.smithi090.stdout:1/69: dread d0/d4/d7/fa [0,4194304] 0 2024-04-26T04:42:14.777 INFO:tasks.workunit.client.1.smithi104.stdout:3/416: symlink d5/d9/dc/d29/d7d/l93 0 2024-04-26T04:42:14.777 INFO:tasks.workunit.client.1.smithi104.stdout:3/417: chown d5/d9/d17/d60/l81 25 1 2024-04-26T04:42:14.777 INFO:tasks.workunit.client.1.smithi104.stdout:7/301: link d2/d4/l27 d2/d4/d32/d3d/d41/l5c 0 2024-04-26T04:42:14.778 INFO:tasks.workunit.client.0.smithi090.stdout:9/39: creat db/dd/ff x:0 0 0 2024-04-26T04:42:14.778 INFO:tasks.workunit.client.0.smithi090.stdout:3/39: truncate d1/f3 1392720 0 2024-04-26T04:42:14.778 INFO:tasks.workunit.client.0.smithi090.stdout:5/143: symlink d4/d1a/d1f/d27/l2e 0 2024-04-26T04:42:14.780 INFO:tasks.workunit.client.1.smithi104.stdout:1/418: rmdir d0/d6/d19/d21 39 2024-04-26T04:42:14.780 INFO:tasks.workunit.client.1.smithi104.stdout:1/419: creat d0/d6/d2e/f86 x:0 0 0 2024-04-26T04:42:14.780 INFO:tasks.workunit.client.1.smithi104.stdout:1/420: creat d0/d6/d49/f87 x:0 0 0 2024-04-26T04:42:14.786 INFO:tasks.workunit.client.0.smithi090.stdout:6/160: dread f1 [4194304,4194304] 0 2024-04-26T04:42:14.786 INFO:tasks.workunit.client.0.smithi090.stdout:9/40: rename c4 to db/c10 0 2024-04-26T04:42:14.789 INFO:tasks.workunit.client.0.smithi090.stdout:9/41: dread f6 [0,4194304] 0 2024-04-26T04:42:14.789 INFO:tasks.workunit.client.0.smithi090.stdout:9/42: truncate db/fe 232561 0 2024-04-26T04:42:14.789 INFO:tasks.workunit.client.0.smithi090.stdout:9/43: creat db/dd/f11 x:0 0 0 2024-04-26T04:42:14.790 INFO:tasks.workunit.client.1.smithi104.stdout:4/435: getdents d4/d7/d1d/d1e/d24/d48 0 2024-04-26T04:42:14.790 INFO:tasks.workunit.client.0.smithi090.stdout:9/44: read f6 [529659,53073] 0 2024-04-26T04:42:14.791 INFO:tasks.workunit.client.0.smithi090.stdout:9/45: read f6 [719107,107106] 0 2024-04-26T04:42:14.791 INFO:tasks.workunit.client.1.smithi104.stdout:3/418: rename d5/d9/d17/d46/f70 to d5/d9/d80/f94 0 2024-04-26T04:42:14.791 INFO:tasks.workunit.client.1.smithi104.stdout:3/419: chown d5/d9/d17/d72 11025519 1 2024-04-26T04:42:14.792 INFO:tasks.workunit.client.1.smithi104.stdout:3/420: chown d5/d9/dc 1753002575 1 2024-04-26T04:42:14.792 INFO:tasks.workunit.client.1.smithi104.stdout:3/421: dread - d5/d9/dc/d69/f6e zero size 2024-04-26T04:42:14.792 INFO:tasks.workunit.client.1.smithi104.stdout:3/422: chown d5/d9/cb 39375589 1 2024-04-26T04:42:14.792 INFO:tasks.workunit.client.0.smithi090.stdout:5/144: rename d4/f2c to d4/d1a/d1f/f2f 0 2024-04-26T04:42:14.792 INFO:tasks.workunit.client.1.smithi104.stdout:7/302: symlink d2/d4/d1d/l5d 0 2024-04-26T04:42:14.794 INFO:tasks.workunit.client.0.smithi090.stdout:9/46: symlink db/dd/l12 0 2024-04-26T04:42:14.795 INFO:tasks.workunit.client.1.smithi104.stdout:0/320: sync 2024-04-26T04:42:14.796 INFO:tasks.workunit.client.1.smithi104.stdout:1/421: symlink d0/d6/d19/d21/d66/d20/l88 0 2024-04-26T04:42:14.797 INFO:tasks.workunit.client.0.smithi090.stdout:4/93: dwrite f1 [0,4194304] 0 2024-04-26T04:42:14.797 INFO:tasks.workunit.client.0.smithi090.stdout:4/94: chown d3/f4 2 1 2024-04-26T04:42:14.797 INFO:tasks.workunit.client.0.smithi090.stdout:4/95: chown d3/f9 4 1 2024-04-26T04:42:14.797 INFO:tasks.workunit.client.0.smithi090.stdout:4/96: chown d3/db 21039540 1 2024-04-26T04:42:14.798 INFO:tasks.workunit.client.1.smithi104.stdout:4/436: creat d4/d20/d55/d56/f94 x:0 0 0 2024-04-26T04:42:14.802 INFO:tasks.workunit.client.1.smithi104.stdout:0/321: truncate d0/d4/f29 649783 0 2024-04-26T04:42:14.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/422: symlink d0/d6/l89 0 2024-04-26T04:42:14.807 INFO:tasks.workunit.client.0.smithi090.stdout:4/97: dread d3/f4 [0,4194304] 0 2024-04-26T04:42:14.808 INFO:tasks.workunit.client.1.smithi104.stdout:7/303: getdents d2/d4/d32/d35 0 2024-04-26T04:42:14.809 INFO:tasks.workunit.client.0.smithi090.stdout:4/98: mkdir d3/d13/d1e 0 2024-04-26T04:42:14.810 INFO:tasks.workunit.client.1.smithi104.stdout:1/423: unlink d0/d6/d19/d21/l1d 0 2024-04-26T04:42:14.810 INFO:tasks.workunit.client.0.smithi090.stdout:4/99: unlink d3/l11 0 2024-04-26T04:42:14.810 INFO:tasks.workunit.client.0.smithi090.stdout:4/100: creat d3/d13/f1f x:0 0 0 2024-04-26T04:42:14.811 INFO:tasks.workunit.client.1.smithi104.stdout:7/304: mkdir d2/d4/d32/d5e 0 2024-04-26T04:42:14.811 INFO:tasks.workunit.client.0.smithi090.stdout:5/145: read d4/d1a/f14 [724700,115292] 0 2024-04-26T04:42:14.811 INFO:tasks.workunit.client.0.smithi090.stdout:5/146: write d4/f20 [4249524,90122] 0 2024-04-26T04:42:14.812 INFO:tasks.workunit.client.0.smithi090.stdout:5/147: fsync d4/d1a/f23 0 2024-04-26T04:42:14.812 INFO:tasks.workunit.client.0.smithi090.stdout:5/148: chown d4/d1a/d1f/d27/l2e 10 1 2024-04-26T04:42:14.812 INFO:tasks.workunit.client.0.smithi090.stdout:5/149: chown d4/d1a/d1f/d27/f29 16 1 2024-04-26T04:42:14.813 INFO:tasks.workunit.client.0.smithi090.stdout:4/101: unlink d3/fa 0 2024-04-26T04:42:14.813 INFO:tasks.workunit.client.1.smithi104.stdout:1/424: truncate d0/d6/ff 9472862 0 2024-04-26T04:42:14.815 INFO:tasks.workunit.client.0.smithi090.stdout:5/150: truncate d4/f1e 1650993 0 2024-04-26T04:42:14.815 INFO:tasks.workunit.client.0.smithi090.stdout:5/151: read d4/f13 [102241,89859] 0 2024-04-26T04:42:14.815 INFO:tasks.workunit.client.1.smithi104.stdout:7/305: mkdir d2/d4/d32/d3d/d41/d5f 0 2024-04-26T04:42:14.816 INFO:tasks.workunit.client.1.smithi104.stdout:0/322: dread d0/d2/d21/d27/f30 [0,4194304] 0 2024-04-26T04:42:14.817 INFO:tasks.workunit.client.1.smithi104.stdout:7/306: mkdir d2/d4/d32/d60 0 2024-04-26T04:42:14.817 INFO:tasks.workunit.client.0.smithi090.stdout:4/102: rename d3/d13/d1e to d3/db/d12/d20 0 2024-04-26T04:42:14.818 INFO:tasks.workunit.client.0.smithi090.stdout:1/70: dwrite d0/d4/f14 [0,4194304] 0 2024-04-26T04:42:14.818 INFO:tasks.workunit.client.1.smithi104.stdout:0/323: truncate d0/d4/d3c/f46 1713584 0 2024-04-26T04:42:14.819 INFO:tasks.workunit.client.1.smithi104.stdout:0/324: read d0/d2/d21/d38/f4f [442248,110822] 0 2024-04-26T04:42:14.820 INFO:tasks.workunit.client.1.smithi104.stdout:7/307: creat d2/d4/d32/d3d/d47/f61 x:0 0 0 2024-04-26T04:42:14.820 INFO:tasks.workunit.client.1.smithi104.stdout:0/325: write d0/d2/d21/d38/f54 [997222,88362] 0 2024-04-26T04:42:14.820 INFO:tasks.workunit.client.0.smithi090.stdout:1/71: mkdir d0/d16 0 2024-04-26T04:42:14.820 INFO:tasks.workunit.client.1.smithi104.stdout:7/308: truncate d2/f42 295424 0 2024-04-26T04:42:14.822 INFO:tasks.workunit.client.1.smithi104.stdout:0/326: link d0/d4/d15/f56 d0/d2/d10/f64 0 2024-04-26T04:42:14.822 INFO:tasks.workunit.client.0.smithi090.stdout:1/72: rename d0/d16 to d0/d4/d7/d17 0 2024-04-26T04:42:14.822 INFO:tasks.workunit.client.0.smithi090.stdout:1/73: fsync d0/d4/f8 0 2024-04-26T04:42:14.823 INFO:tasks.workunit.client.1.smithi104.stdout:7/309: mknod d2/d4/c62 0 2024-04-26T04:42:14.824 INFO:tasks.workunit.client.1.smithi104.stdout:0/327: creat d0/d2/d21/d27/d1e/f65 x:0 0 0 2024-04-26T04:42:14.826 INFO:tasks.workunit.client.0.smithi090.stdout:1/74: write d0/d4/d7/fa [615859,99273] 0 2024-04-26T04:42:14.826 INFO:tasks.workunit.client.1.smithi104.stdout:0/328: unlink d0/d2/d10/f32 0 2024-04-26T04:42:14.826 INFO:tasks.workunit.client.1.smithi104.stdout:0/329: readlink d0/d2/d21/d27/l3a 0 2024-04-26T04:42:14.827 INFO:tasks.workunit.client.1.smithi104.stdout:0/330: creat d0/d4/d15/f66 x:0 0 0 2024-04-26T04:42:14.827 INFO:tasks.workunit.client.1.smithi104.stdout:0/331: fdatasync d0/f42 0 2024-04-26T04:42:14.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/75: rename d0/d4/f8 to d0/d4/d7/d17/f18 0 2024-04-26T04:42:14.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/76: chown d0/d4/d7/fa 33091164 1 2024-04-26T04:42:14.828 INFO:tasks.workunit.client.0.smithi090.stdout:1/77: chown d0/d4/c13 31506940 1 2024-04-26T04:42:14.829 INFO:tasks.workunit.client.0.smithi090.stdout:9/47: dwrite db/fe [0,4194304] 0 2024-04-26T04:42:14.829 INFO:tasks.workunit.client.0.smithi090.stdout:1/78: dread d0/d4/d7/d17/f18 [0,4194304] 0 2024-04-26T04:42:14.830 INFO:tasks.workunit.client.1.smithi104.stdout:0/332: rename d0/d2/d21/c40 to d0/d4/d15/c67 0 2024-04-26T04:42:14.830 INFO:tasks.workunit.client.1.smithi104.stdout:0/333: write d0/d2/d21/d38/f36 [4775126,59479] 0 2024-04-26T04:42:14.830 INFO:tasks.workunit.client.0.smithi090.stdout:4/103: dread d3/f4 [4194304,4194304] 0 2024-04-26T04:42:14.830 INFO:tasks.workunit.client.0.smithi090.stdout:4/104: write d3/db/fc [302783,112131] 0 2024-04-26T04:42:14.831 INFO:tasks.workunit.client.0.smithi090.stdout:4/105: creat d3/d13/f21 x:0 0 0 2024-04-26T04:42:14.831 INFO:tasks.workunit.client.0.smithi090.stdout:4/106: stat d3/d13 0 2024-04-26T04:42:14.831 INFO:tasks.workunit.client.0.smithi090.stdout:4/107: write d3/d13/f1b [949811,61528] 0 2024-04-26T04:42:14.831 INFO:tasks.workunit.client.1.smithi104.stdout:0/334: truncate d0/fe 6928905 0 2024-04-26T04:42:14.831 INFO:tasks.workunit.client.1.smithi104.stdout:0/335: truncate d0/d2/d21/f52 28417 0 2024-04-26T04:42:14.831 INFO:tasks.workunit.client.1.smithi104.stdout:0/336: stat d0/d4/d3c/f4b 0 2024-04-26T04:42:14.832 INFO:tasks.workunit.client.1.smithi104.stdout:0/337: read d0/d2/d21/d27/f30 [1524496,15771] 0 2024-04-26T04:42:14.832 INFO:tasks.workunit.client.0.smithi090.stdout:3/40: dwrite d1/d9/fa [0,4194304] 0 2024-04-26T04:42:14.832 INFO:tasks.workunit.client.0.smithi090.stdout:3/41: write d1/fc [957298,73808] 0 2024-04-26T04:42:14.832 INFO:tasks.workunit.client.1.smithi104.stdout:7/310: dread d2/d4/f13 [0,4194304] 0 2024-04-26T04:42:14.833 INFO:tasks.workunit.client.0.smithi090.stdout:1/79: dread d0/d4/d7/d17/f18 [0,4194304] 0 2024-04-26T04:42:14.833 INFO:tasks.workunit.client.1.smithi104.stdout:3/423: dwrite d5/d9/dc/d29/f58 [0,4194304] 0 2024-04-26T04:42:14.834 INFO:tasks.workunit.client.1.smithi104.stdout:4/437: dwrite d4/d7/d1d/d1e/d24/f76 [0,4194304] 0 2024-04-26T04:42:14.835 INFO:tasks.workunit.client.1.smithi104.stdout:0/338: creat d0/d2/d21/d27/d1e/d57/f68 x:0 0 0 2024-04-26T04:42:14.837 INFO:tasks.workunit.client.1.smithi104.stdout:7/311: creat d2/d4/d32/d60/f63 x:0 0 0 2024-04-26T04:42:14.837 INFO:tasks.workunit.client.1.smithi104.stdout:7/312: truncate d2/d4/d32/d60/f63 513153 0 2024-04-26T04:42:14.837 INFO:tasks.workunit.client.1.smithi104.stdout:7/313: chown d2/d4/d1d/d34/f40 4495 1 2024-04-26T04:42:14.838 INFO:tasks.workunit.client.0.smithi090.stdout:4/108: symlink d3/db/d12/l22 0 2024-04-26T04:42:14.838 INFO:tasks.workunit.client.0.smithi090.stdout:4/109: readlink d3/db/d12/l22 0 2024-04-26T04:42:14.839 INFO:tasks.workunit.client.1.smithi104.stdout:3/424: mkdir d5/d9/d17/d46/d95 0 2024-04-26T04:42:14.840 INFO:tasks.workunit.client.0.smithi090.stdout:5/152: getdents d4/d1a/d1f 0 2024-04-26T04:42:14.845 INFO:tasks.workunit.client.0.smithi090.stdout:5/153: getdents d4/d1a/df 0 2024-04-26T04:42:14.845 INFO:tasks.workunit.client.0.smithi090.stdout:9/48: rmdir db 39 2024-04-26T04:42:14.845 INFO:tasks.workunit.client.0.smithi090.stdout:9/49: write f2 [360535,52612] 0 2024-04-26T04:42:14.846 INFO:tasks.workunit.client.0.smithi090.stdout:4/110: write d3/f8 [129164,8246] 0 2024-04-26T04:42:14.847 INFO:tasks.workunit.client.0.smithi090.stdout:5/154: read d4/d1a/f17 [429846,68147] 0 2024-04-26T04:42:14.847 INFO:tasks.workunit.client.0.smithi090.stdout:5/155: chown d4/d1a/l12 2619 1 2024-04-26T04:42:14.847 INFO:tasks.workunit.client.0.smithi090.stdout:4/111: write d3/d13/f1f [979796,115597] 0 2024-04-26T04:42:14.847 INFO:tasks.workunit.client.0.smithi090.stdout:4/112: readlink d3/le 0 2024-04-26T04:42:14.848 INFO:tasks.workunit.client.0.smithi090.stdout:9/50: creat db/f13 x:0 0 0 2024-04-26T04:42:14.849 INFO:tasks.workunit.client.0.smithi090.stdout:9/51: chown db/c10 48067436 1 2024-04-26T04:42:14.849 INFO:tasks.workunit.client.0.smithi090.stdout:9/52: creat db/f14 x:0 0 0 2024-04-26T04:42:14.850 INFO:tasks.workunit.client.0.smithi090.stdout:5/156: symlink d4/d1a/d1f/l30 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:9/53: dread f3 [0,4194304] 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:9/54: chown db/dd/l12 501136 1 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:9/55: truncate db/fc 841764 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/113: write d3/db/d12/f10 [575676,121283] 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/114: creat d3/db/f23 x:0 0 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/115: chown d3/db 0 1 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/116: truncate d3/db/fc 637755 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/117: truncate d3/db/fc 770762 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/118: fsync d3/f14 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/119: stat d3/d13/f21 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:4/120: truncate d3/d13/f21 178919 0 2024-04-26T04:42:14.852 INFO:tasks.workunit.client.0.smithi090.stdout:5/157: dread d4/f13 [0,4194304] 0 2024-04-26T04:42:14.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/56: unlink db/fe 0 2024-04-26T04:42:14.857 INFO:tasks.workunit.client.1.smithi104.stdout:0/339: unlink d0/d4/d15/c47 0 2024-04-26T04:42:14.859 INFO:tasks.workunit.client.0.smithi090.stdout:5/158: mkdir d4/d1a/d2a/d31 0 2024-04-26T04:42:14.859 INFO:tasks.workunit.client.0.smithi090.stdout:5/159: fsync d4/f13 0 2024-04-26T04:42:14.861 INFO:tasks.workunit.client.0.smithi090.stdout:9/57: truncate f9 445828 0 2024-04-26T04:42:14.863 INFO:tasks.workunit.client.1.smithi104.stdout:0/340: unlink d0/d2/d10/d3d/f50 0 2024-04-26T04:42:14.863 INFO:tasks.workunit.client.0.smithi090.stdout:5/160: creat d4/d1a/df/d21/f32 x:0 0 0 2024-04-26T04:42:14.866 INFO:tasks.workunit.client.1.smithi104.stdout:4/438: dwrite d4/d7/d1d/d1e/d15/f66 [0,4194304] 0 2024-04-26T04:42:14.867 INFO:tasks.workunit.client.0.smithi090.stdout:5/161: rename d4/d1a/c1c to d4/d1a/d2a/d31/c33 0 2024-04-26T04:42:14.868 INFO:tasks.workunit.client.1.smithi104.stdout:7/314: mknod d2/c64 0 2024-04-26T04:42:14.869 INFO:tasks.workunit.client.0.smithi090.stdout:3/42: dwrite d1/f8 [0,4194304] 0 2024-04-26T04:42:14.872 INFO:tasks.workunit.client.0.smithi090.stdout:5/162: link d4/d1a/df/d21/f32 d4/d1a/d2a/d31/f34 0 2024-04-26T04:42:14.873 INFO:tasks.workunit.client.1.smithi104.stdout:7/315: symlink d2/d4/d32/d35/d37/l65 0 2024-04-26T04:42:14.873 INFO:tasks.workunit.client.1.smithi104.stdout:7/316: fsync d2/d4/f13 0 2024-04-26T04:42:14.876 INFO:tasks.workunit.client.1.smithi104.stdout:0/341: rename d0/d4/d15/l20 to d0/d2/d21/d27/d1e/l69 0 2024-04-26T04:42:14.877 INFO:tasks.workunit.client.1.smithi104.stdout:3/425: dwrite d5/f23 [0,4194304] 0 2024-04-26T04:42:14.877 INFO:tasks.workunit.client.1.smithi104.stdout:7/317: rmdir d2/d4/d32/d3d/d41 39 2024-04-26T04:42:14.879 INFO:tasks.workunit.client.1.smithi104.stdout:0/342: creat d0/d4/d15/f6a x:0 0 0 2024-04-26T04:42:14.880 INFO:tasks.workunit.client.1.smithi104.stdout:0/343: chown d0/f2f 0 1 2024-04-26T04:42:14.882 INFO:tasks.workunit.client.1.smithi104.stdout:4/439: rename d4/d20/c70 to d4/d7/d1d/d1e/c95 0 2024-04-26T04:42:14.885 INFO:tasks.workunit.client.0.smithi090.stdout:4/121: dwrite d3/f9 [0,4194304] 0 2024-04-26T04:42:14.886 INFO:tasks.workunit.client.1.smithi104.stdout:3/426: rmdir d5/d9 39 2024-04-26T04:42:14.886 INFO:tasks.workunit.client.0.smithi090.stdout:5/163: write d4/d1a/f23 [2632865,93982] 0 2024-04-26T04:42:14.886 INFO:tasks.workunit.client.0.smithi090.stdout:1/80: dwrite d0/d4/f15 [4194304,4194304] 0 2024-04-26T04:42:14.888 INFO:tasks.workunit.client.1.smithi104.stdout:7/318: symlink d2/d4/d32/d60/l66 0 2024-04-26T04:42:14.888 INFO:tasks.workunit.client.0.smithi090.stdout:4/122: dread d3/f8 [0,4194304] 0 2024-04-26T04:42:14.889 INFO:tasks.workunit.client.1.smithi104.stdout:7/319: read d2/d4/d32/d60/f63 [414791,12139] 0 2024-04-26T04:42:14.889 INFO:tasks.workunit.client.0.smithi090.stdout:1/81: link d0/d4/cd d0/c19 0 2024-04-26T04:42:14.890 INFO:tasks.workunit.client.0.smithi090.stdout:4/123: mkdir d3/db/d12/d20/d24 0 2024-04-26T04:42:14.890 INFO:tasks.workunit.client.0.smithi090.stdout:4/124: write d3/db/d12/f17 [3177643,106471] 0 2024-04-26T04:42:14.891 INFO:tasks.workunit.client.1.smithi104.stdout:0/344: rename d0/f17 to d0/d2/d21/d38/f6b 0 2024-04-26T04:42:14.891 INFO:tasks.workunit.client.1.smithi104.stdout:0/345: truncate d0/d2/d21/d49/f55 693761 0 2024-04-26T04:42:14.892 INFO:tasks.workunit.client.0.smithi090.stdout:4/125: rename d3/c1c to d3/db/d12/d20/c25 0 2024-04-26T04:42:14.893 INFO:tasks.workunit.client.1.smithi104.stdout:4/440: link d4/d7/ff d4/d20/d55/d56/f96 0 2024-04-26T04:42:14.893 INFO:tasks.workunit.client.1.smithi104.stdout:4/441: readlink d4/d20/d55/l57 0 2024-04-26T04:42:14.894 INFO:tasks.workunit.client.0.smithi090.stdout:3/43: dwrite d1/d9/fa [0,4194304] 0 2024-04-26T04:42:14.895 INFO:tasks.workunit.client.1.smithi104.stdout:3/427: mknod d5/d9/c96 0 2024-04-26T04:42:14.895 INFO:tasks.workunit.client.0.smithi090.stdout:4/126: creat d3/db/d12/d20/f26 x:0 0 0 2024-04-26T04:42:14.895 INFO:tasks.workunit.client.0.smithi090.stdout:4/127: fdatasync d3/d13/f1b 0 2024-04-26T04:42:14.896 INFO:tasks.workunit.client.1.smithi104.stdout:3/428: dread d5/f3f [0,4194304] 0 2024-04-26T04:42:14.897 INFO:tasks.workunit.client.0.smithi090.stdout:1/82: dread d0/d4/d7/fa [0,4194304] 0 2024-04-26T04:42:14.897 INFO:tasks.workunit.client.0.smithi090.stdout:9/58: dwrite db/f14 [0,4194304] 0 2024-04-26T04:42:14.898 INFO:tasks.workunit.client.1.smithi104.stdout:3/429: mknod d5/d9/d17/d60/c97 0 2024-04-26T04:42:14.898 INFO:tasks.workunit.client.1.smithi104.stdout:3/430: chown d5/d9/d17/d60/c97 61 1 2024-04-26T04:42:14.898 INFO:tasks.workunit.client.1.smithi104.stdout:3/431: write d5/d9/dc/d32/f77 [659408,123335] 0 2024-04-26T04:42:14.899 INFO:tasks.workunit.client.0.smithi090.stdout:4/128: dread d3/d13/f1b [0,4194304] 0 2024-04-26T04:42:14.899 INFO:tasks.workunit.client.0.smithi090.stdout:4/129: creat d3/f27 x:0 0 0 2024-04-26T04:42:14.899 INFO:tasks.workunit.client.1.smithi104.stdout:2/544: sync 2024-04-26T04:42:14.902 INFO:tasks.workunit.client.1.smithi104.stdout:2/545: mkdir d6/d1b/d51/db8 0 2024-04-26T04:42:14.902 INFO:tasks.workunit.client.0.smithi090.stdout:3/44: dread d1/f3 [0,4194304] 0 2024-04-26T04:42:14.903 INFO:tasks.workunit.client.1.smithi104.stdout:2/546: getdents d6/d1b/d75 0 2024-04-26T04:42:14.903 INFO:tasks.workunit.client.1.smithi104.stdout:2/547: fdatasync d6/f62 0 2024-04-26T04:42:14.903 INFO:tasks.workunit.client.0.smithi090.stdout:9/59: mknod db/dd/c15 0 2024-04-26T04:42:14.904 INFO:tasks.workunit.client.1.smithi104.stdout:2/548: symlink d6/d1b/d68/d81/lb9 0 2024-04-26T04:42:14.906 INFO:tasks.workunit.client.0.smithi090.stdout:1/83: getdents d0/d4/d7 0 2024-04-26T04:42:14.906 INFO:tasks.workunit.client.0.smithi090.stdout:1/84: readlink d0/l6 0 2024-04-26T04:42:14.908 INFO:tasks.workunit.client.0.smithi090.stdout:3/45: mknod d1/cd 0 2024-04-26T04:42:14.909 INFO:tasks.workunit.client.0.smithi090.stdout:5/164: dwrite d4/fe [0,4194304] 0 2024-04-26T04:42:14.909 INFO:tasks.workunit.client.0.smithi090.stdout:5/165: chown d4/d1a/d1f/d27/l2e 6 1 2024-04-26T04:42:14.909 INFO:tasks.workunit.client.0.smithi090.stdout:5/166: write d4/d1a/d1f/f2f [630736,122154] 0 2024-04-26T04:42:14.910 INFO:tasks.workunit.client.0.smithi090.stdout:3/46: mknod d1/d6/ce 0 2024-04-26T04:42:14.915 INFO:tasks.workunit.client.0.smithi090.stdout:3/47: mknod d1/d6/cf 0 2024-04-26T04:42:14.915 INFO:tasks.workunit.client.0.smithi090.stdout:3/48: write d1/d9/fa [4284398,97571] 0 2024-04-26T04:42:14.915 INFO:tasks.workunit.client.0.smithi090.stdout:3/49: dread - d1/d9/fb zero size 2024-04-26T04:42:14.915 INFO:tasks.workunit.client.0.smithi090.stdout:9/60: dread f2 [0,4194304] 0 2024-04-26T04:42:14.916 INFO:tasks.workunit.client.0.smithi090.stdout:3/50: rmdir d1/d6 39 2024-04-26T04:42:14.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/61: mkdir db/d16 0 2024-04-26T04:42:14.919 INFO:tasks.workunit.client.0.smithi090.stdout:3/51: unlink d1/f3 0 2024-04-26T04:42:14.919 INFO:tasks.workunit.client.0.smithi090.stdout:5/167: dread d4/fe [0,4194304] 0 2024-04-26T04:42:14.920 INFO:tasks.workunit.client.0.smithi090.stdout:3/52: mknod d1/c10 0 2024-04-26T04:42:14.920 INFO:tasks.workunit.client.0.smithi090.stdout:3/53: creat d1/f11 x:0 0 0 2024-04-26T04:42:14.920 INFO:tasks.workunit.client.0.smithi090.stdout:3/54: fdatasync d1/f8 0 2024-04-26T04:42:14.922 INFO:tasks.workunit.client.0.smithi090.stdout:5/168: rename d4/d1a/d2a/d31/c33 to d4/d1a/df/d21/c35 0 2024-04-26T04:42:14.925 INFO:tasks.workunit.client.0.smithi090.stdout:3/55: link d1/f8 d1/d9/f12 0 2024-04-26T04:42:14.929 INFO:tasks.workunit.client.0.smithi090.stdout:2/66: sync 2024-04-26T04:42:14.929 INFO:tasks.workunit.client.0.smithi090.stdout:0/153: sync 2024-04-26T04:42:14.929 INFO:tasks.workunit.client.0.smithi090.stdout:7/79: sync 2024-04-26T04:42:14.929 INFO:tasks.workunit.client.0.smithi090.stdout:7/80: fdatasync d0/d1/d1f/f16 0 2024-04-26T04:42:14.930 INFO:tasks.workunit.client.0.smithi090.stdout:2/67: creat db/dc/f12 x:0 0 0 2024-04-26T04:42:14.931 INFO:tasks.workunit.client.0.smithi090.stdout:2/68: fsync f2 0 2024-04-26T04:42:14.931 INFO:tasks.workunit.client.0.smithi090.stdout:7/81: getdents d0/d1/d4 0 2024-04-26T04:42:14.932 INFO:tasks.workunit.client.1.smithi104.stdout:0/346: rename d0/d4/d15 to d0/d2/d21/d27/d1e/d57/d6c 0 2024-04-26T04:42:14.933 INFO:tasks.workunit.client.0.smithi090.stdout:1/85: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:14.934 INFO:tasks.workunit.client.0.smithi090.stdout:2/69: symlink db/l13 0 2024-04-26T04:42:14.934 INFO:tasks.workunit.client.1.smithi104.stdout:4/442: rename d4/d7/d1d/d1e/d15/f46 to d4/d7/d1d/d8e/f97 0 2024-04-26T04:42:14.935 INFO:tasks.workunit.client.0.smithi090.stdout:2/70: mkdir db/d11/d14 0 2024-04-26T04:42:14.935 INFO:tasks.workunit.client.0.smithi090.stdout:2/71: chown l5 510880 1 2024-04-26T04:42:14.936 INFO:tasks.workunit.client.1.smithi104.stdout:0/347: creat d0/d2/d21/d27/f6d x:0 0 0 2024-04-26T04:42:14.936 INFO:tasks.workunit.client.1.smithi104.stdout:2/549: dread d6/d8/d3e/f2a [4194304,4194304] 0 2024-04-26T04:42:14.936 INFO:tasks.workunit.client.1.smithi104.stdout:2/550: fsync d6/d8/d3e/f7a 0 2024-04-26T04:42:14.937 INFO:tasks.workunit.client.0.smithi090.stdout:8/46: sync 2024-04-26T04:42:14.937 INFO:tasks.workunit.client.0.smithi090.stdout:8/47: write f4 [969686,120756] 0 2024-04-26T04:42:14.938 INFO:tasks.workunit.client.0.smithi090.stdout:8/48: mknod dc/c10 0 2024-04-26T04:42:14.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/49: symlink dc/l11 0 2024-04-26T04:42:14.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/50: mkdir dc/d12 0 2024-04-26T04:42:14.944 INFO:tasks.workunit.client.1.smithi104.stdout:3/432: rename d5/d9/f1d to d5/d9/dc/d29/d7d/d8e/f98 0 2024-04-26T04:42:14.944 INFO:tasks.workunit.client.1.smithi104.stdout:3/433: dread - d5/d9/dc/d29/d49/d59/f7b zero size 2024-04-26T04:42:14.944 INFO:tasks.workunit.client.1.smithi104.stdout:6/497: sync 2024-04-26T04:42:14.944 INFO:tasks.workunit.client.1.smithi104.stdout:6/498: creat d0/d1b/f9e x:0 0 0 2024-04-26T04:42:14.944 INFO:tasks.workunit.client.1.smithi104.stdout:6/499: read - d0/d9/d13/d15/d80/f61 zero size 2024-04-26T04:42:14.945 INFO:tasks.workunit.client.1.smithi104.stdout:2/551: truncate d6/d8/f37 2152681 0 2024-04-26T04:42:14.945 INFO:tasks.workunit.client.1.smithi104.stdout:2/552: readlink d6/d1b/l64 0 2024-04-26T04:42:14.945 INFO:tasks.workunit.client.1.smithi104.stdout:4/443: rename d4/d7/d1d/d1e/d24/d59/f5d to d4/f98 0 2024-04-26T04:42:14.945 INFO:tasks.workunit.client.1.smithi104.stdout:6/500: creat d0/d9/d13/d41/d82/f9f x:0 0 0 2024-04-26T04:42:14.945 INFO:tasks.workunit.client.1.smithi104.stdout:2/553: mknod d6/de/cba 0 2024-04-26T04:42:14.946 INFO:tasks.workunit.client.1.smithi104.stdout:3/434: rename d5/d9/f4a to d5/d9/d17/d72/f99 0 2024-04-26T04:42:14.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/51: unlink f8 0 2024-04-26T04:42:14.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/52: fsync fa 0 2024-04-26T04:42:14.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/53: readlink l3 0 2024-04-26T04:42:14.946 INFO:tasks.workunit.client.0.smithi090.stdout:8/54: creat dc/d12/f13 x:0 0 0 2024-04-26T04:42:14.947 INFO:tasks.workunit.client.0.smithi090.stdout:7/82: dread d0/d1/d1f/f16 [0,4194304] 0 2024-04-26T04:42:14.947 INFO:tasks.workunit.client.1.smithi104.stdout:4/444: symlink d4/db/d6b/d86/l99 0 2024-04-26T04:42:14.947 INFO:tasks.workunit.client.0.smithi090.stdout:3/56: fsync d1/f8 0 2024-04-26T04:42:14.948 INFO:tasks.workunit.client.1.smithi104.stdout:6/501: stat d0/d9/c1e 0 2024-04-26T04:42:14.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/154: dwrite d0/d26/f24 [0,4194304] 0 2024-04-26T04:42:14.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/155: chown d0/de 21055 1 2024-04-26T04:42:14.950 INFO:tasks.workunit.client.1.smithi104.stdout:2/554: mknod d6/d1b/d93/cbb 0 2024-04-26T04:42:14.950 INFO:tasks.workunit.client.1.smithi104.stdout:2/555: write d6/de/d1d/f8f [509011,58248] 0 2024-04-26T04:42:14.950 INFO:tasks.workunit.client.1.smithi104.stdout:2/556: write d6/d8/d5a/f54 [868513,100284] 0 2024-04-26T04:42:14.951 INFO:tasks.workunit.client.1.smithi104.stdout:2/557: dread d6/de/d8e/f4c [0,4194304] 0 2024-04-26T04:42:14.951 INFO:tasks.workunit.client.0.smithi090.stdout:3/57: rename d1/c10 to d1/d6/c13 0 2024-04-26T04:42:14.951 INFO:tasks.workunit.client.0.smithi090.stdout:3/58: write d1/fc [998155,129027] 0 2024-04-26T04:42:14.951 INFO:tasks.workunit.client.1.smithi104.stdout:3/435: getdents d5/d9/d80 0 2024-04-26T04:42:14.952 INFO:tasks.workunit.client.1.smithi104.stdout:3/436: dread - d5/d9/dc/d29/f56 zero size 2024-04-26T04:42:14.953 INFO:tasks.workunit.client.0.smithi090.stdout:0/156: creat d0/de/f36 x:0 0 0 2024-04-26T04:42:14.954 INFO:tasks.workunit.client.1.smithi104.stdout:2/558: truncate d6/d1b/f38 649272 0 2024-04-26T04:42:14.955 INFO:tasks.workunit.client.1.smithi104.stdout:2/559: mknod d6/d1b/d51/d6a/cbc 0 2024-04-26T04:42:14.955 INFO:tasks.workunit.client.1.smithi104.stdout:2/560: readlink d6/d1b/d51/l65 0 2024-04-26T04:42:14.956 INFO:tasks.workunit.client.1.smithi104.stdout:3/437: truncate d5/d9/dc/f41 2022886 0 2024-04-26T04:42:14.956 INFO:tasks.workunit.client.1.smithi104.stdout:2/561: symlink d6/d1b/d93/lbd 0 2024-04-26T04:42:14.957 INFO:tasks.workunit.client.1.smithi104.stdout:2/562: fsync d6/de/d1d/f83 0 2024-04-26T04:42:14.957 INFO:tasks.workunit.client.1.smithi104.stdout:2/563: mknod d6/d1b/d75/cbe 0 2024-04-26T04:42:14.958 INFO:tasks.workunit.client.1.smithi104.stdout:3/438: symlink d5/d9/dc/l9a 0 2024-04-26T04:42:14.958 INFO:tasks.workunit.client.1.smithi104.stdout:2/564: symlink d6/d1b/d68/d81/lbf 0 2024-04-26T04:42:14.960 INFO:tasks.workunit.client.1.smithi104.stdout:3/439: creat d5/d9/dc/d32/d47/f9b x:0 0 0 2024-04-26T04:42:14.961 INFO:tasks.workunit.client.1.smithi104.stdout:3/440: getdents d5/d9/dc/d69 0 2024-04-26T04:42:14.962 INFO:tasks.workunit.client.0.smithi090.stdout:7/83: write d0/d1/d4/f10 [1088753,21346] 0 2024-04-26T04:42:14.962 INFO:tasks.workunit.client.1.smithi104.stdout:3/441: symlink d5/d9/dc/d32/l9c 0 2024-04-26T04:42:14.964 INFO:tasks.workunit.client.1.smithi104.stdout:3/442: rename d5/d9/d17/c2b to d5/d9/dc/d32/d47/c9d 0 2024-04-26T04:42:14.964 INFO:tasks.workunit.client.1.smithi104.stdout:3/443: write d5/f3f [2355745,50214] 0 2024-04-26T04:42:15.019 INFO:tasks.workunit.client.0.smithi090.stdout:7/84: read d0/f8 [357909,88121] 0 2024-04-26T04:42:15.034 INFO:tasks.workunit.client.1.smithi104.stdout:4/445: dread d4/d7/d1d/f45 [0,4194304] 0 2024-04-26T04:42:15.034 INFO:tasks.workunit.client.1.smithi104.stdout:2/565: dread d6/d1b/d51/f87 [0,4194304] 0 2024-04-26T04:42:15.034 INFO:tasks.workunit.client.1.smithi104.stdout:2/566: creat d6/d8/d3e/fc0 x:0 0 0 2024-04-26T04:42:15.036 INFO:tasks.workunit.client.1.smithi104.stdout:2/567: mknod d6/de/d8e/cc1 0 2024-04-26T04:42:15.038 INFO:tasks.workunit.client.1.smithi104.stdout:2/568: unlink d6/d1b/d93/lbd 0 2024-04-26T04:42:15.068 INFO:tasks.workunit.client.0.smithi090.stdout:8/55: dwrite fa [0,4194304] 0 2024-04-26T04:42:15.068 INFO:tasks.workunit.client.0.smithi090.stdout:1/86: dwrite d0/f12 [0,4194304] 0 2024-04-26T04:42:15.068 INFO:tasks.workunit.client.0.smithi090.stdout:5/169: dwrite d4/d1a/d2a/d31/f34 [0,4194304] 0 2024-04-26T04:42:15.070 INFO:tasks.workunit.client.1.smithi104.stdout:3/444: dwrite d5/d9/dc/f41 [0,4194304] 0 2024-04-26T04:42:15.070 INFO:tasks.workunit.client.0.smithi090.stdout:8/56: truncate f5 734010 0 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:3/59: dwrite d1/d9/f12 [0,4194304] 0 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:1/87: mknod d0/d4/d7/c1a 0 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/72: dwrite f2 [0,4194304] 0 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/73: chown l6 339 1 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/74: truncate db/dc/f12 786598 0 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/75: readlink l6 0 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:2/76: fsync db/dc/f12 0 2024-04-26T04:42:15.071 INFO:tasks.workunit.client.0.smithi090.stdout:7/85: dwrite d0/d1/d4/d9/f18 [0,4194304] 0 2024-04-26T04:42:15.072 INFO:tasks.workunit.client.1.smithi104.stdout:6/502: dwrite d0/d9/d13/d15/d80/f61 [0,4194304] 0 2024-04-26T04:42:15.073 INFO:tasks.workunit.client.0.smithi090.stdout:5/170: mkdir d4/d1a/df/d36 0 2024-04-26T04:42:15.074 INFO:tasks.workunit.client.1.smithi104.stdout:2/569: dwrite d6/d8/d5a/f2e [0,4194304] 0 2024-04-26T04:42:15.074 INFO:tasks.workunit.client.1.smithi104.stdout:4/446: dwrite d4/d7/d1d/d1e/d24/d59/f6a [0,4194304] 0 2024-04-26T04:42:15.074 INFO:tasks.workunit.client.1.smithi104.stdout:4/447: write d4/d7/d1d/d1e/d24/f68 [511950,69647] 0 2024-04-26T04:42:15.076 INFO:tasks.workunit.client.1.smithi104.stdout:6/503: rmdir d0/d9/d13 39 2024-04-26T04:42:15.077 INFO:tasks.workunit.client.0.smithi090.stdout:8/57: dread f4 [0,4194304] 0 2024-04-26T04:42:15.077 INFO:tasks.workunit.client.0.smithi090.stdout:1/88: symlink d0/l1b 0 2024-04-26T04:42:15.078 INFO:tasks.workunit.client.1.smithi104.stdout:4/448: rename d4/d7/d1d/d1e/f83 to d4/d20/f9a 0 2024-04-26T04:42:15.078 INFO:tasks.workunit.client.1.smithi104.stdout:4/449: truncate d4/d20/d55/d72/f87 507986 0 2024-04-26T04:42:15.078 INFO:tasks.workunit.client.1.smithi104.stdout:4/450: stat d4/d7/d1d/d1e/d15/f1a 0 2024-04-26T04:42:15.078 INFO:tasks.workunit.client.1.smithi104.stdout:4/451: fsync d4/d20/d55/d56/f96 0 2024-04-26T04:42:15.082 INFO:tasks.workunit.client.1.smithi104.stdout:3/445: dread d5/d9/d17/f34 [0,4194304] 0 2024-04-26T04:42:15.082 INFO:tasks.workunit.client.1.smithi104.stdout:4/452: rename d4/db/d6b/c8b to d4/d7/d1d/d1e/d8f/c9b 0 2024-04-26T04:42:15.082 INFO:tasks.workunit.client.1.smithi104.stdout:4/453: creat d4/d20/d55/d72/f9c x:0 0 0 2024-04-26T04:42:15.083 INFO:tasks.workunit.client.0.smithi090.stdout:1/89: dread d0/d4/d7/fa [0,4194304] 0 2024-04-26T04:42:15.083 INFO:tasks.workunit.client.1.smithi104.stdout:6/504: rmdir d0/d1b 39 2024-04-26T04:42:15.083 INFO:tasks.workunit.client.1.smithi104.stdout:6/505: readlink d0/d9/l99 0 2024-04-26T04:42:15.085 INFO:tasks.workunit.client.0.smithi090.stdout:8/58: creat dc/f14 x:0 0 0 2024-04-26T04:42:15.085 INFO:tasks.workunit.client.0.smithi090.stdout:8/59: readlink l3 0 2024-04-26T04:42:15.085 INFO:tasks.workunit.client.0.smithi090.stdout:8/60: write f4 [2278898,105216] 0 2024-04-26T04:42:15.087 INFO:tasks.workunit.client.0.smithi090.stdout:8/61: mknod dc/d12/c15 0 2024-04-26T04:42:15.090 INFO:tasks.workunit.client.0.smithi090.stdout:8/62: symlink dc/d12/l16 0 2024-04-26T04:42:15.090 INFO:tasks.workunit.client.0.smithi090.stdout:8/63: chown l3 1534243 1 2024-04-26T04:42:15.090 INFO:tasks.workunit.client.0.smithi090.stdout:8/64: creat dc/d12/f17 x:0 0 0 2024-04-26T04:42:15.096 INFO:tasks.workunit.client.1.smithi104.stdout:3/446: read d5/d9/d17/f2a [22962,65692] 0 2024-04-26T04:42:15.099 INFO:tasks.workunit.client.1.smithi104.stdout:3/447: rmdir d5/d9/dc/d3a 39 2024-04-26T04:42:15.099 INFO:tasks.workunit.client.0.smithi090.stdout:3/60: dread d1/f8 [0,4194304] 0 2024-04-26T04:42:15.099 INFO:tasks.workunit.client.0.smithi090.stdout:3/61: chown d1/l4 725807473 1 2024-04-26T04:42:15.106 INFO:tasks.workunit.client.0.smithi090.stdout:3/62: creat d1/d6/f14 x:0 0 0 2024-04-26T04:42:15.108 INFO:tasks.workunit.client.1.smithi104.stdout:9/291: sync 2024-04-26T04:42:15.108 INFO:tasks.workunit.client.1.smithi104.stdout:1/425: sync 2024-04-26T04:42:15.108 INFO:tasks.workunit.client.1.smithi104.stdout:7/320: sync 2024-04-26T04:42:15.110 INFO:tasks.workunit.client.0.smithi090.stdout:3/63: mknod d1/d6/c15 0 2024-04-26T04:42:15.111 INFO:tasks.workunit.client.1.smithi104.stdout:7/321: mknod d2/d4/d32/d60/c67 0 2024-04-26T04:42:15.111 INFO:tasks.workunit.client.1.smithi104.stdout:7/322: chown d2/d4/d32/d35/d37/c4c 2668647 1 2024-04-26T04:42:15.111 INFO:tasks.workunit.client.1.smithi104.stdout:7/323: fdatasync d2/d4/d1d/d34/f51 0 2024-04-26T04:42:15.111 INFO:tasks.workunit.client.1.smithi104.stdout:7/324: dread - d2/d4/d32/d3d/f50 zero size 2024-04-26T04:42:15.112 INFO:tasks.workunit.client.1.smithi104.stdout:9/292: rmdir d11/d15 39 2024-04-26T04:42:15.112 INFO:tasks.workunit.client.1.smithi104.stdout:9/293: read - d11/f44 zero size 2024-04-26T04:42:15.113 INFO:tasks.workunit.client.1.smithi104.stdout:9/294: creat d11/d15/f59 x:0 0 0 2024-04-26T04:42:15.113 INFO:tasks.workunit.client.0.smithi090.stdout:3/64: getdents d1 0 2024-04-26T04:42:15.114 INFO:tasks.workunit.client.1.smithi104.stdout:9/295: symlink d11/d53/l5a 0 2024-04-26T04:42:15.115 INFO:tasks.workunit.client.1.smithi104.stdout:9/296: stat d11/d15 0 2024-04-26T04:42:15.116 INFO:tasks.workunit.client.1.smithi104.stdout:8/239: sync 2024-04-26T04:42:15.116 INFO:tasks.workunit.client.1.smithi104.stdout:5/372: sync 2024-04-26T04:42:15.117 INFO:tasks.workunit.client.1.smithi104.stdout:8/240: symlink d0/d1c/d27/l4b 0 2024-04-26T04:42:15.117 INFO:tasks.workunit.client.0.smithi090.stdout:1/90: dwrite d0/d4/f9 [0,4194304] 0 2024-04-26T04:42:15.118 INFO:tasks.workunit.client.1.smithi104.stdout:5/373: mkdir d0/d7b 0 2024-04-26T04:42:15.118 INFO:tasks.workunit.client.1.smithi104.stdout:5/374: write d0/f58 [554558,19323] 0 2024-04-26T04:42:15.119 INFO:tasks.workunit.client.0.smithi090.stdout:3/65: dread d1/fc [0,4194304] 0 2024-04-26T04:42:15.119 INFO:tasks.workunit.client.1.smithi104.stdout:8/241: creat d0/d1c/f4c x:0 0 0 2024-04-26T04:42:15.120 INFO:tasks.workunit.client.0.smithi090.stdout:3/66: mknod d1/d9/c16 0 2024-04-26T04:42:15.121 INFO:tasks.workunit.client.0.smithi090.stdout:3/67: truncate d1/d9/fa 2363894 0 2024-04-26T04:42:15.121 INFO:tasks.workunit.client.0.smithi090.stdout:3/68: chown d1/cd 1150340127 1 2024-04-26T04:42:15.122 INFO:tasks.workunit.client.1.smithi104.stdout:5/375: rename d0/d1/d4/d2a/d2b/d5c/f5e to d0/d1/d4/d2a/d2b/d37/f7c 0 2024-04-26T04:42:15.122 INFO:tasks.workunit.client.0.smithi090.stdout:3/69: mkdir d1/d17 0 2024-04-26T04:42:15.122 INFO:tasks.workunit.client.0.smithi090.stdout:3/70: write d1/d9/fb [1025698,99048] 0 2024-04-26T04:42:15.123 INFO:tasks.workunit.client.1.smithi104.stdout:5/376: symlink d0/d1/d4/d2a/d2b/d37/d48/l7d 0 2024-04-26T04:42:15.123 INFO:tasks.workunit.client.0.smithi090.stdout:3/71: mknod d1/d17/c18 0 2024-04-26T04:42:15.126 INFO:tasks.workunit.client.0.smithi090.stdout:3/72: write d1/fc [616156,21362] 0 2024-04-26T04:42:15.159 INFO:tasks.workunit.client.0.smithi090.stdout:6/161: sync 2024-04-26T04:42:15.159 INFO:tasks.workunit.client.0.smithi090.stdout:6/162: chown d9/c1b 6 1 2024-04-26T04:42:15.161 INFO:tasks.workunit.client.0.smithi090.stdout:1/91: dread d0/f12 [4194304,4194304] 0 2024-04-26T04:42:15.161 INFO:tasks.workunit.client.0.smithi090.stdout:6/163: mkdir d9/d3c 0 2024-04-26T04:42:15.161 INFO:tasks.workunit.client.0.smithi090.stdout:6/164: dread - d9/d22/f28 zero size 2024-04-26T04:42:15.166 INFO:tasks.workunit.client.0.smithi090.stdout:7/86: dwrite d0/d1/d1f/f16 [0,4194304] 0 2024-04-26T04:42:15.169 INFO:tasks.workunit.client.1.smithi104.stdout:2/570: dwrite d6/d8/d3e/f7a [0,4194304] 0 2024-04-26T04:42:15.169 INFO:tasks.workunit.client.1.smithi104.stdout:4/454: dwrite d4/d20/d55/d72/f88 [0,4194304] 0 2024-04-26T04:42:15.170 INFO:tasks.workunit.client.0.smithi090.stdout:2/77: dwrite f2 [4194304,4194304] 0 2024-04-26T04:42:15.170 INFO:tasks.workunit.client.1.smithi104.stdout:2/571: truncate d6/d1b/d51/d6a/f7d 172181 0 2024-04-26T04:42:15.172 INFO:tasks.workunit.client.1.smithi104.stdout:4/455: mknod d4/d7/d1d/d1e/d24/d59/c9d 0 2024-04-26T04:42:15.173 INFO:tasks.workunit.client.1.smithi104.stdout:2/572: truncate d6/d8/f70 3832184 0 2024-04-26T04:42:15.173 INFO:tasks.workunit.client.1.smithi104.stdout:2/573: read d6/d1b/d51/f87 [413701,76639] 0 2024-04-26T04:42:15.175 INFO:tasks.workunit.client.1.smithi104.stdout:2/574: unlink d6/l6e 0 2024-04-26T04:42:15.176 INFO:tasks.workunit.client.0.smithi090.stdout:2/78: rename c9 to db/d11/c15 0 2024-04-26T04:42:15.177 INFO:tasks.workunit.client.0.smithi090.stdout:2/79: dread db/dc/f12 [0,4194304] 0 2024-04-26T04:42:15.177 INFO:tasks.workunit.client.0.smithi090.stdout:2/80: fsync f2 0 2024-04-26T04:42:15.177 INFO:tasks.workunit.client.0.smithi090.stdout:2/81: creat db/f16 x:0 0 0 2024-04-26T04:42:15.177 INFO:tasks.workunit.client.0.smithi090.stdout:2/82: stat l6 0 2024-04-26T04:42:15.178 INFO:tasks.workunit.client.0.smithi090.stdout:5/171: dwrite d4/d1a/f1d [0,4194304] 0 2024-04-26T04:42:15.178 INFO:tasks.workunit.client.1.smithi104.stdout:7/325: dwrite d2/d4/d32/d35/d37/f3a [0,4194304] 0 2024-04-26T04:42:15.180 INFO:tasks.workunit.client.1.smithi104.stdout:9/297: dwrite f4 [0,4194304] 0 2024-04-26T04:42:15.181 INFO:tasks.workunit.client.0.smithi090.stdout:8/65: dwrite fa [0,4194304] 0 2024-04-26T04:42:15.181 INFO:tasks.workunit.client.0.smithi090.stdout:8/66: stat dc/d12/f17 0 2024-04-26T04:42:15.182 INFO:tasks.workunit.client.1.smithi104.stdout:9/298: link d11/d15/l2b d11/d15/d32/d35/l5b 0 2024-04-26T04:42:15.183 INFO:tasks.workunit.client.1.smithi104.stdout:6/506: dwrite d0/d5/d8e/f9b [0,4194304] 0 2024-04-26T04:42:15.183 INFO:tasks.workunit.client.1.smithi104.stdout:4/456: dread d4/d7/f53 [0,4194304] 0 2024-04-26T04:42:15.183 INFO:tasks.workunit.client.1.smithi104.stdout:3/448: dwrite d5/d9/f12 [4194304,4194304] 0 2024-04-26T04:42:15.183 INFO:tasks.workunit.client.0.smithi090.stdout:2/83: creat db/d11/f17 x:0 0 0 2024-04-26T04:42:15.184 INFO:tasks.workunit.client.1.smithi104.stdout:9/299: symlink d11/d15/d32/l5c 0 2024-04-26T04:42:15.184 INFO:tasks.workunit.client.1.smithi104.stdout:9/300: fsync f10 0 2024-04-26T04:42:15.184 INFO:tasks.workunit.client.1.smithi104.stdout:9/301: readlink d11/d15/d32/d35/l5b 0 2024-04-26T04:42:15.184 INFO:tasks.workunit.client.1.smithi104.stdout:5/377: dwrite d0/d1/d4/d2a/d2b/f42 [0,4194304] 0 2024-04-26T04:42:15.186 INFO:tasks.workunit.client.1.smithi104.stdout:6/507: symlink d0/d5/d8e/la0 0 2024-04-26T04:42:15.187 INFO:tasks.workunit.client.0.smithi090.stdout:7/87: write d0/d1/d1f/f16 [2817671,51803] 0 2024-04-26T04:42:15.187 INFO:tasks.workunit.client.0.smithi090.stdout:5/172: link d4/d1a/d1f/f2f d4/f37 0 2024-04-26T04:42:15.190 INFO:tasks.workunit.client.1.smithi104.stdout:3/449: creat d5/d9/d80/f9e x:0 0 0 2024-04-26T04:42:15.191 INFO:tasks.workunit.client.1.smithi104.stdout:3/450: read d5/d9/dc/d32/f53 [7376,61349] 0 2024-04-26T04:42:15.191 INFO:tasks.workunit.client.1.smithi104.stdout:4/457: mkdir d4/d7/d1d/d1e/d24/d4a/d9e 0 2024-04-26T04:42:15.191 INFO:tasks.workunit.client.1.smithi104.stdout:3/451: truncate d5/d9/dc/d29/d49/d59/f7b 1036346 0 2024-04-26T04:42:15.192 INFO:tasks.workunit.client.1.smithi104.stdout:9/302: unlink fb 0 2024-04-26T04:42:15.193 INFO:tasks.workunit.client.0.smithi090.stdout:7/88: unlink d0/db/df/l1e 0 2024-04-26T04:42:15.193 INFO:tasks.workunit.client.0.smithi090.stdout:7/89: fsync d0/d1/d4/f10 0 2024-04-26T04:42:15.193 INFO:tasks.workunit.client.0.smithi090.stdout:7/90: read d0/f8 [1248689,117451] 0 2024-04-26T04:42:15.194 INFO:tasks.workunit.client.1.smithi104.stdout:6/508: dread d0/d9/d13/d44/d71/f74 [0,4194304] 0 2024-04-26T04:42:15.195 INFO:tasks.workunit.client.0.smithi090.stdout:7/91: symlink d0/db/df/d1c/l24 0 2024-04-26T04:42:15.195 INFO:tasks.workunit.client.0.smithi090.stdout:7/92: creat d0/d1/d1f/f25 x:0 0 0 2024-04-26T04:42:15.197 INFO:tasks.workunit.client.0.smithi090.stdout:1/92: dwrite d0/f12 [4194304,4194304] 0 2024-04-26T04:42:15.197 INFO:tasks.workunit.client.0.smithi090.stdout:1/93: write d0/d4/d7/d17/f18 [665315,123899] 0 2024-04-26T04:42:15.198 INFO:tasks.workunit.client.1.smithi104.stdout:5/378: rename d0/d41/c62 to d0/d1/d4/c7e 0 2024-04-26T04:42:15.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/93: getdents d0/db 0 2024-04-26T04:42:15.199 INFO:tasks.workunit.client.0.smithi090.stdout:6/165: dwrite d9/d1c/f36 [0,4194304] 0 2024-04-26T04:42:15.199 INFO:tasks.workunit.client.1.smithi104.stdout:4/458: symlink d4/d7/d1d/d1e/d15/l9f 0 2024-04-26T04:42:15.200 INFO:tasks.workunit.client.0.smithi090.stdout:3/73: truncate d1/fc 711203 0 2024-04-26T04:42:15.200 INFO:tasks.workunit.client.0.smithi090.stdout:3/74: stat d1/f11 0 2024-04-26T04:42:15.201 INFO:tasks.workunit.client.1.smithi104.stdout:3/452: link d5/d9/f12 d5/d9/d17/d72/d85/f9f 0 2024-04-26T04:42:15.201 INFO:tasks.workunit.client.1.smithi104.stdout:3/453: fsync d5/d9/f26 0 2024-04-26T04:42:15.203 INFO:tasks.workunit.client.1.smithi104.stdout:9/303: getdents d11/d15/d1b 0 2024-04-26T04:42:15.203 INFO:tasks.workunit.client.1.smithi104.stdout:4/459: dread d4/d20/d55/d56/f31 [0,4194304] 0 2024-04-26T04:42:15.204 INFO:tasks.workunit.client.0.smithi090.stdout:1/94: link d0/f2 d0/f1c 0 2024-04-26T04:42:15.204 INFO:tasks.workunit.client.0.smithi090.stdout:5/173: dread d4/d1a/f1d [0,4194304] 0 2024-04-26T04:42:15.204 INFO:tasks.workunit.client.0.smithi090.stdout:5/174: fdatasync d4/d1a/d1f/f2b 0 2024-04-26T04:42:15.204 INFO:tasks.workunit.client.0.smithi090.stdout:5/175: write d4/d1a/d2a/d31/f34 [4609115,45648] 0 2024-04-26T04:42:15.205 INFO:tasks.workunit.client.1.smithi104.stdout:3/454: dread d5/d9/d17/f1b [0,4194304] 0 2024-04-26T04:42:15.205 INFO:tasks.workunit.client.0.smithi090.stdout:1/95: write d0/d4/d7/d17/f18 [542966,7850] 0 2024-04-26T04:42:15.205 INFO:tasks.workunit.client.0.smithi090.stdout:1/96: fdatasync d0/f2 0 2024-04-26T04:42:15.206 INFO:tasks.workunit.client.1.smithi104.stdout:6/509: mkdir d0/d9/d13/d15/d80/da1 0 2024-04-26T04:42:15.206 INFO:tasks.workunit.client.1.smithi104.stdout:5/379: dread d0/d1/d4/d2a/d2b/d37/f57 [0,4194304] 0 2024-04-26T04:42:15.208 INFO:tasks.workunit.client.1.smithi104.stdout:3/455: creat d5/d9/dc/d29/d7d/fa0 x:0 0 0 2024-04-26T04:42:15.209 INFO:tasks.workunit.client.1.smithi104.stdout:3/456: write d5/d9/f12 [9254194,118082] 0 2024-04-26T04:42:15.211 INFO:tasks.workunit.client.0.smithi090.stdout:6/166: getdents d9/d37 0 2024-04-26T04:42:15.211 INFO:tasks.workunit.client.1.smithi104.stdout:5/380: symlink d0/d41/l7f 0 2024-04-26T04:42:15.213 INFO:tasks.workunit.client.0.smithi090.stdout:1/97: mknod d0/d4/c1d 0 2024-04-26T04:42:15.213 INFO:tasks.workunit.client.0.smithi090.stdout:1/98: chown d0/d4/d7/d17 3909 1 2024-04-26T04:42:15.214 INFO:tasks.workunit.client.0.smithi090.stdout:1/99: chown d0/l5 77206335 1 2024-04-26T04:42:15.214 INFO:tasks.workunit.client.1.smithi104.stdout:3/457: symlink d5/d9/dc/d29/d7d/d8e/la1 0 2024-04-26T04:42:15.214 INFO:tasks.workunit.client.1.smithi104.stdout:3/458: write d5/d9/dc/d32/f5a [524572,96275] 0 2024-04-26T04:42:15.215 INFO:tasks.workunit.client.1.smithi104.stdout:5/381: dread d0/f58 [0,4194304] 0 2024-04-26T04:42:15.215 INFO:tasks.workunit.client.1.smithi104.stdout:5/382: chown d0/d1/d4/d2a/d2b/d37/d48/l7a 102479 1 2024-04-26T04:42:15.216 INFO:tasks.workunit.client.0.smithi090.stdout:8/67: dwrite f5 [0,4194304] 0 2024-04-26T04:42:15.216 INFO:tasks.workunit.client.0.smithi090.stdout:6/167: mkdir d9/d1c/d31/d3d 0 2024-04-26T04:42:15.219 INFO:tasks.workunit.client.0.smithi090.stdout:1/100: mknod d0/d4/c1e 0 2024-04-26T04:42:15.220 INFO:tasks.workunit.client.1.smithi104.stdout:9/304: dread d11/d15/d32/d35/f3a [0,4194304] 0 2024-04-26T04:42:15.220 INFO:tasks.workunit.client.1.smithi104.stdout:9/305: creat d11/d1d/d34/f5d x:0 0 0 2024-04-26T04:42:15.220 INFO:tasks.workunit.client.1.smithi104.stdout:9/306: stat d11/d15/d32 0 2024-04-26T04:42:15.221 INFO:tasks.workunit.client.0.smithi090.stdout:1/101: mknod d0/c1f 0 2024-04-26T04:42:15.222 INFO:tasks.workunit.client.1.smithi104.stdout:3/459: mkdir d5/d9/d80/da2 0 2024-04-26T04:42:15.223 INFO:tasks.workunit.client.1.smithi104.stdout:3/460: write d5/d9/dc/f21 [775488,47638] 0 2024-04-26T04:42:15.226 INFO:tasks.workunit.client.0.smithi090.stdout:2/84: dwrite f2 [0,4194304] 0 2024-04-26T04:42:15.226 INFO:tasks.workunit.client.0.smithi090.stdout:2/85: read - db/f16 zero size 2024-04-26T04:42:15.227 INFO:tasks.workunit.client.1.smithi104.stdout:3/461: mkdir d5/d9/d17/da3 0 2024-04-26T04:42:15.227 INFO:tasks.workunit.client.0.smithi090.stdout:2/86: dread db/dc/f12 [0,4194304] 0 2024-04-26T04:42:15.228 INFO:tasks.workunit.client.0.smithi090.stdout:2/87: chown l6 62574233 1 2024-04-26T04:42:15.228 INFO:tasks.workunit.client.1.smithi104.stdout:5/383: dread d0/d1/d1e/f2e [0,4194304] 0 2024-04-26T04:42:15.228 INFO:tasks.workunit.client.1.smithi104.stdout:3/462: link d5/d9/dc/d29/d7d/d8e/f98 d5/d9/d17/da3/fa4 0 2024-04-26T04:42:15.229 INFO:tasks.workunit.client.1.smithi104.stdout:3/463: write d5/f24 [1276355,107925] 0 2024-04-26T04:42:15.229 INFO:tasks.workunit.client.0.smithi090.stdout:2/88: symlink db/d11/d14/l18 0 2024-04-26T04:42:15.229 INFO:tasks.workunit.client.0.smithi090.stdout:2/89: creat db/dc/f19 x:0 0 0 2024-04-26T04:42:15.230 INFO:tasks.workunit.client.1.smithi104.stdout:5/384: link d0/d1/f38 d0/d1/d4/d2a/d2b/d37/d48/f80 0 2024-04-26T04:42:15.231 INFO:tasks.workunit.client.0.smithi090.stdout:7/94: rmdir d0/db/df 39 2024-04-26T04:42:15.231 INFO:tasks.workunit.client.1.smithi104.stdout:3/464: creat d5/d9/d17/d46/d95/fa5 x:0 0 0 2024-04-26T04:42:15.232 INFO:tasks.workunit.client.1.smithi104.stdout:3/465: chown d5/d9/f5d 681155301 1 2024-04-26T04:42:15.233 INFO:tasks.workunit.client.1.smithi104.stdout:5/385: link d0/d1/d64/d26/l46 d0/d1/d4/l81 0 2024-04-26T04:42:15.233 INFO:tasks.workunit.client.0.smithi090.stdout:7/95: link d0/d1/d1f/f16 d0/db/df/d1c/d23/f26 0 2024-04-26T04:42:15.233 INFO:tasks.workunit.client.0.smithi090.stdout:9/62: sync 2024-04-26T04:42:15.233 INFO:tasks.workunit.client.0.smithi090.stdout:0/157: sync 2024-04-26T04:42:15.234 INFO:tasks.workunit.client.0.smithi090.stdout:4/130: sync 2024-04-26T04:42:15.235 INFO:tasks.workunit.client.1.smithi104.stdout:5/386: unlink d0/d1/d4/d2a/d2b/d5c/l75 0 2024-04-26T04:42:15.235 INFO:tasks.workunit.client.0.smithi090.stdout:7/96: symlink d0/d1/l27 0 2024-04-26T04:42:15.237 INFO:tasks.workunit.client.0.smithi090.stdout:9/63: mknod db/dd/c17 0 2024-04-26T04:42:15.237 INFO:tasks.workunit.client.0.smithi090.stdout:7/97: mknod d0/d1/c28 0 2024-04-26T04:42:15.240 INFO:tasks.workunit.client.0.smithi090.stdout:0/158: dread d0/df/f25 [0,4194304] 0 2024-04-26T04:42:15.241 INFO:tasks.workunit.client.1.smithi104.stdout:5/387: symlink d0/d1/l82 0 2024-04-26T04:42:15.243 INFO:tasks.workunit.client.0.smithi090.stdout:0/159: mknod d0/df/c37 0 2024-04-26T04:42:15.245 INFO:tasks.workunit.client.1.smithi104.stdout:0/348: sync 2024-04-26T04:42:15.249 INFO:tasks.workunit.client.1.smithi104.stdout:0/349: link d0/d2/d21/d27/d1e/f2c d0/d2/d21/d49/d62/f6e 0 2024-04-26T04:42:15.249 INFO:tasks.workunit.client.0.smithi090.stdout:2/90: dread f2 [4194304,4194304] 0 2024-04-26T04:42:15.250 INFO:tasks.workunit.client.0.smithi090.stdout:0/160: write d0/f2d [477087,42788] 0 2024-04-26T04:42:15.251 INFO:tasks.workunit.client.1.smithi104.stdout:0/350: mkdir d0/d4/d3c/d53/d5d/d6f 0 2024-04-26T04:42:15.251 INFO:tasks.workunit.client.1.smithi104.stdout:0/351: truncate d0/d2/d21/d27/f6d 166473 0 2024-04-26T04:42:15.252 INFO:tasks.workunit.client.1.smithi104.stdout:4/460: dwrite d4/d7/d1d/f6f [0,4194304] 0 2024-04-26T04:42:15.254 INFO:tasks.workunit.client.1.smithi104.stdout:4/461: symlink d4/d20/d55/d56/la0 0 2024-04-26T04:42:15.255 INFO:tasks.workunit.client.1.smithi104.stdout:4/462: rmdir d4/d7/d1d/d1e/d24/d48 39 2024-04-26T04:42:15.256 INFO:tasks.workunit.client.0.smithi090.stdout:3/75: dwrite d1/d6/f14 [0,4194304] 0 2024-04-26T04:42:15.256 INFO:tasks.workunit.client.0.smithi090.stdout:5/176: dwrite d4/d1a/df/f28 [0,4194304] 0 2024-04-26T04:42:15.259 INFO:tasks.workunit.client.0.smithi090.stdout:2/91: dread f2 [4194304,4194304] 0 2024-04-26T04:42:15.264 INFO:tasks.workunit.client.0.smithi090.stdout:0/161: dread d0/de/f2c [0,4194304] 0 2024-04-26T04:42:15.264 INFO:tasks.workunit.client.0.smithi090.stdout:0/162: write d0/d9/f13 [450270,14277] 0 2024-04-26T04:42:15.265 INFO:tasks.workunit.client.0.smithi090.stdout:0/163: write d0/d26/f27 [579164,115051] 0 2024-04-26T04:42:15.269 INFO:tasks.workunit.client.0.smithi090.stdout:0/164: creat d0/f38 x:0 0 0 2024-04-26T04:42:15.269 INFO:tasks.workunit.client.0.smithi090.stdout:0/165: fsync d0/df/f25 0 2024-04-26T04:42:15.270 INFO:tasks.workunit.client.1.smithi104.stdout:2/575: dwrite d6/d8/f70 [0,4194304] 0 2024-04-26T04:42:15.270 INFO:tasks.workunit.client.0.smithi090.stdout:0/166: creat d0/de/f39 x:0 0 0 2024-04-26T04:42:15.273 INFO:tasks.workunit.client.1.smithi104.stdout:2/576: rename d6/d1b/d68/l9b to d6/d1b/d51/d6a/lc2 0 2024-04-26T04:42:15.273 INFO:tasks.workunit.client.0.smithi090.stdout:0/167: link d0/l4 d0/l3a 0 2024-04-26T04:42:15.273 INFO:tasks.workunit.client.0.smithi090.stdout:0/168: chown d0/de/f2f 71 1 2024-04-26T04:42:15.274 INFO:tasks.workunit.client.0.smithi090.stdout:0/169: write d0/df/f25 [1114718,40613] 0 2024-04-26T04:42:15.275 INFO:tasks.workunit.client.1.smithi104.stdout:2/577: dread d6/d1b/f71 [0,4194304] 0 2024-04-26T04:42:15.276 INFO:tasks.workunit.client.0.smithi090.stdout:4/131: dwrite d3/f9 [4194304,4194304] 0 2024-04-26T04:42:15.276 INFO:tasks.workunit.client.0.smithi090.stdout:4/132: dread - d3/f27 zero size 2024-04-26T04:42:15.278 INFO:tasks.workunit.client.1.smithi104.stdout:6/510: rmdir d0/d9 39 2024-04-26T04:42:15.280 INFO:tasks.workunit.client.0.smithi090.stdout:0/170: dread d0/d9/f17 [0,4194304] 0 2024-04-26T04:42:15.285 INFO:tasks.workunit.client.1.smithi104.stdout:6/511: symlink d0/d9/d13/d15/d80/la2 0 2024-04-26T04:42:15.288 INFO:tasks.workunit.client.1.smithi104.stdout:6/512: creat d0/d9/d13/d44/d71/fa3 x:0 0 0 2024-04-26T04:42:15.288 INFO:tasks.workunit.client.1.smithi104.stdout:6/513: chown d0/l14 118108425 1 2024-04-26T04:42:15.289 INFO:tasks.workunit.client.1.smithi104.stdout:6/514: mkdir d0/d9/d13/d44/d47/da4 0 2024-04-26T04:42:15.291 INFO:tasks.workunit.client.0.smithi090.stdout:9/64: dwrite db/dd/f11 [0,4194304] 0 2024-04-26T04:42:15.291 INFO:tasks.workunit.client.0.smithi090.stdout:9/65: readlink db/dd/l12 0 2024-04-26T04:42:15.291 INFO:tasks.workunit.client.0.smithi090.stdout:7/98: dwrite d0/d1/d4/d9/f14 [0,4194304] 0 2024-04-26T04:42:15.291 INFO:tasks.workunit.client.1.smithi104.stdout:6/515: mkdir d0/d9/d13/d44/d71/da5 0 2024-04-26T04:42:15.291 INFO:tasks.workunit.client.1.smithi104.stdout:3/466: dwrite d5/d9/f26 [0,4194304] 0 2024-04-26T04:42:15.292 INFO:tasks.workunit.client.1.smithi104.stdout:6/516: mkdir d0/d9/d13/d44/d47/da4/da6 0 2024-04-26T04:42:15.292 INFO:tasks.workunit.client.1.smithi104.stdout:6/517: dread - d0/d1b/f7c zero size 2024-04-26T04:42:15.293 INFO:tasks.workunit.client.0.smithi090.stdout:2/92: fsync db/dc/f19 0 2024-04-26T04:42:15.295 INFO:tasks.workunit.client.1.smithi104.stdout:3/467: symlink d5/d9/d17/d72/d85/la6 0 2024-04-26T04:42:15.295 INFO:tasks.workunit.client.1.smithi104.stdout:3/468: write d5/d9/dc/d32/f77 [784336,125760] 0 2024-04-26T04:42:15.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/518: creat d0/d9/d13/d15/d80/fa7 x:0 0 0 2024-04-26T04:42:15.298 INFO:tasks.workunit.client.0.smithi090.stdout:2/93: symlink db/dc/l1a 0 2024-04-26T04:42:15.299 INFO:tasks.workunit.client.0.smithi090.stdout:2/94: chown db/d11/c15 1616657 1 2024-04-26T04:42:15.299 INFO:tasks.workunit.client.0.smithi090.stdout:2/95: write db/dc/f12 [946255,66298] 0 2024-04-26T04:42:15.299 INFO:tasks.workunit.client.1.smithi104.stdout:3/469: chown d5/d9/dc/c4b 6785849 1 2024-04-26T04:42:15.301 INFO:tasks.workunit.client.1.smithi104.stdout:6/519: mkdir d0/d9/da8 0 2024-04-26T04:42:15.304 INFO:tasks.workunit.client.1.smithi104.stdout:6/520: unlink d0/d9/d13/d15/d80/c6a 0 2024-04-26T04:42:15.305 INFO:tasks.workunit.client.1.smithi104.stdout:6/521: dread d0/d5/f22 [0,4194304] 0 2024-04-26T04:42:15.305 INFO:tasks.workunit.client.1.smithi104.stdout:6/522: write d0/d9/d13/d44/d71/fa3 [284972,18168] 0 2024-04-26T04:42:15.307 INFO:tasks.workunit.client.1.smithi104.stdout:0/352: dwrite d0/d2/d21/d49/f55 [0,4194304] 0 2024-04-26T04:42:15.307 INFO:tasks.workunit.client.1.smithi104.stdout:6/523: mknod d0/d9/d13/d44/d71/da5/ca9 0 2024-04-26T04:42:15.307 INFO:tasks.workunit.client.0.smithi090.stdout:3/76: dwrite d1/d9/f12 [0,4194304] 0 2024-04-26T04:42:15.309 INFO:tasks.workunit.client.0.smithi090.stdout:3/77: mknod d1/d9/c19 0 2024-04-26T04:42:15.310 INFO:tasks.workunit.client.1.smithi104.stdout:0/353: link d0/d2/d21/d27/c2a d0/d2/d21/d27/c70 0 2024-04-26T04:42:15.310 INFO:tasks.workunit.client.1.smithi104.stdout:0/354: creat d0/d2/d10/f71 x:0 0 0 2024-04-26T04:42:15.312 INFO:tasks.workunit.client.1.smithi104.stdout:6/524: mkdir d0/d9/d13/d15/d80/da1/daa 0 2024-04-26T04:42:15.312 INFO:tasks.workunit.client.1.smithi104.stdout:6/525: write d0/d5/f4e [595749,80113] 0 2024-04-26T04:42:15.312 INFO:tasks.workunit.client.1.smithi104.stdout:6/526: write d0/f9c [790304,37887] 0 2024-04-26T04:42:15.315 INFO:tasks.workunit.client.1.smithi104.stdout:6/527: truncate d0/d9/d25/f39 1786100 0 2024-04-26T04:42:15.315 INFO:tasks.workunit.client.1.smithi104.stdout:0/355: write d0/d2/d21/d49/d62/f6e [122145,66831] 0 2024-04-26T04:42:15.315 INFO:tasks.workunit.client.1.smithi104.stdout:0/356: truncate d0/d2/d21/d27/d1e/f65 623075 0 2024-04-26T04:42:15.317 INFO:tasks.workunit.client.1.smithi104.stdout:6/528: link d0/c38 d0/d1b/cab 0 2024-04-26T04:42:15.318 INFO:tasks.workunit.client.1.smithi104.stdout:6/529: rmdir d0/d5/d8e 39 2024-04-26T04:42:15.320 INFO:tasks.workunit.client.0.smithi090.stdout:7/99: write d0/d1/d4/d9/f18 [4096080,116449] 0 2024-04-26T04:42:15.321 INFO:tasks.workunit.client.0.smithi090.stdout:5/177: dwrite d4/d1a/f23 [0,4194304] 0 2024-04-26T04:42:15.322 INFO:tasks.workunit.client.1.smithi104.stdout:6/530: rename d0/d5 to d0/d9/d13/d41/d96/dac 0 2024-04-26T04:42:15.324 INFO:tasks.workunit.client.0.smithi090.stdout:5/178: mknod d4/d1a/df/d21/c38 0 2024-04-26T04:42:15.327 INFO:tasks.workunit.client.0.smithi090.stdout:5/179: dread d4/d1a/f14 [0,4194304] 0 2024-04-26T04:42:15.329 INFO:tasks.workunit.client.0.smithi090.stdout:7/100: rename d0/d1/d4/d9 to d0/db/d15/d29 0 2024-04-26T04:42:15.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/101: mknod d0/db/d15/c2a 0 2024-04-26T04:42:15.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/102: write d0/f8 [428890,80508] 0 2024-04-26T04:42:15.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/103: stat d0/d1/d1f 0 2024-04-26T04:42:15.334 INFO:tasks.workunit.client.0.smithi090.stdout:7/104: creat d0/db/d15/d29/f2b x:0 0 0 2024-04-26T04:42:15.334 INFO:tasks.workunit.client.0.smithi090.stdout:7/105: getdents d0/db/df/d1c/d23 0 2024-04-26T04:42:15.335 INFO:tasks.workunit.client.0.smithi090.stdout:7/106: write d0/d1/d1f/f16 [83017,13400] 0 2024-04-26T04:42:15.338 INFO:tasks.workunit.client.0.smithi090.stdout:5/180: dread d4/d1a/df/f22 [0,4194304] 0 2024-04-26T04:42:15.339 INFO:tasks.workunit.client.0.smithi090.stdout:4/133: dwrite d3/db/f19 [8388608,4194304] 0 2024-04-26T04:42:15.339 INFO:tasks.workunit.client.0.smithi090.stdout:0/171: dwrite d0/d26/f34 [0,4194304] 0 2024-04-26T04:42:15.340 INFO:tasks.workunit.client.0.smithi090.stdout:4/134: creat d3/db/d12/f28 x:0 0 0 2024-04-26T04:42:15.346 INFO:tasks.workunit.client.1.smithi104.stdout:2/578: dwrite d6/d1b/d93/f9d [0,4194304] 0 2024-04-26T04:42:15.347 INFO:tasks.workunit.client.0.smithi090.stdout:7/107: write d0/db/d15/d29/fc [4249273,129215] 0 2024-04-26T04:42:15.347 INFO:tasks.workunit.client.0.smithi090.stdout:7/108: fsync d0/d1/d1f/f17 0 2024-04-26T04:42:15.347 INFO:tasks.workunit.client.1.smithi104.stdout:4/463: dwrite d4/d20/d55/d56/f94 [0,4194304] 0 2024-04-26T04:42:15.348 INFO:tasks.workunit.client.1.smithi104.stdout:4/464: chown d4/d7/d1d/d1e/d8f/c9b 13429 1 2024-04-26T04:42:15.348 INFO:tasks.workunit.client.1.smithi104.stdout:2/579: symlink d6/de/lc3 0 2024-04-26T04:42:15.349 INFO:tasks.workunit.client.0.smithi090.stdout:5/181: write d4/d1a/df/f28 [2321609,43699] 0 2024-04-26T04:42:15.354 INFO:tasks.workunit.client.0.smithi090.stdout:5/182: mknod d4/d1a/c39 0 2024-04-26T04:42:15.354 INFO:tasks.workunit.client.0.smithi090.stdout:9/66: dwrite f2 [0,4194304] 0 2024-04-26T04:42:15.354 INFO:tasks.workunit.client.0.smithi090.stdout:9/67: chown f6 31 1 2024-04-26T04:42:15.354 INFO:tasks.workunit.client.0.smithi090.stdout:4/135: dread d3/f9 [8388608,4194304] 0 2024-04-26T04:42:15.355 INFO:tasks.workunit.client.0.smithi090.stdout:5/183: unlink d4/d1a/f14 0 2024-04-26T04:42:15.356 INFO:tasks.workunit.client.0.smithi090.stdout:5/184: creat d4/d1a/d1f/f3a x:0 0 0 2024-04-26T04:42:15.356 INFO:tasks.workunit.client.0.smithi090.stdout:2/96: dwrite db/f16 [0,4194304] 0 2024-04-26T04:42:15.356 INFO:tasks.workunit.client.0.smithi090.stdout:2/97: fsync f2 0 2024-04-26T04:42:15.356 INFO:tasks.workunit.client.0.smithi090.stdout:2/98: write db/dc/f19 [669820,13775] 0 2024-04-26T04:42:15.356 INFO:tasks.workunit.client.1.smithi104.stdout:4/465: rmdir d4/db 39 2024-04-26T04:42:15.357 INFO:tasks.workunit.client.1.smithi104.stdout:2/580: creat d6/d1b/d93/fc4 x:0 0 0 2024-04-26T04:42:15.357 INFO:tasks.workunit.client.1.smithi104.stdout:3/470: dwrite d5/f3f [0,4194304] 0 2024-04-26T04:42:15.357 INFO:tasks.workunit.client.1.smithi104.stdout:4/466: creat d4/d20/d55/d72/d7d/fa1 x:0 0 0 2024-04-26T04:42:15.357 INFO:tasks.workunit.client.1.smithi104.stdout:4/467: write d4/d7/d1d/d1e/d15/f22 [1034761,96103] 0 2024-04-26T04:42:15.357 INFO:tasks.workunit.client.1.smithi104.stdout:2/581: creat d6/d1b/d51/d6a/fc5 x:0 0 0 2024-04-26T04:42:15.357 INFO:tasks.workunit.client.1.smithi104.stdout:0/357: dwrite d0/d2/d21/d27/d1e/d57/d6c/f59 [0,4194304] 0 2024-04-26T04:42:15.357 INFO:tasks.workunit.client.1.smithi104.stdout:6/531: dwrite d0/d9/d13/d15/f2b [0,4194304] 0 2024-04-26T04:42:15.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/136: symlink d3/db/d12/l29 0 2024-04-26T04:42:15.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/137: chown d3/db/f23 172216211 1 2024-04-26T04:42:15.358 INFO:tasks.workunit.client.0.smithi090.stdout:3/78: dwrite d1/d9/f12 [0,4194304] 0 2024-04-26T04:42:15.358 INFO:tasks.workunit.client.0.smithi090.stdout:3/79: read - d1/f11 zero size 2024-04-26T04:42:15.362 INFO:tasks.workunit.client.0.smithi090.stdout:2/99: mkdir db/d11/d14/d1b 0 2024-04-26T04:42:15.368 INFO:tasks.workunit.client.1.smithi104.stdout:4/468: creat d4/db/d6b/fa2 x:0 0 0 2024-04-26T04:42:15.371 INFO:tasks.workunit.client.1.smithi104.stdout:2/582: creat d6/d1b/d68/d81/fc6 x:0 0 0 2024-04-26T04:42:15.371 INFO:tasks.workunit.client.1.smithi104.stdout:2/583: creat d6/d8/d5a/fc7 x:0 0 0 2024-04-26T04:42:15.372 INFO:tasks.workunit.client.0.smithi090.stdout:0/172: dread d0/df/f25 [0,4194304] 0 2024-04-26T04:42:15.372 INFO:tasks.workunit.client.0.smithi090.stdout:0/173: creat d0/f3b x:0 0 0 2024-04-26T04:42:15.379 INFO:tasks.workunit.client.1.smithi104.stdout:3/471: truncate d5/d9/dc/f28 1805583 0 2024-04-26T04:42:15.379 INFO:tasks.workunit.client.1.smithi104.stdout:3/472: creat d5/d9/d17/d60/fa7 x:0 0 0 2024-04-26T04:42:15.379 INFO:tasks.workunit.client.1.smithi104.stdout:3/473: dread - d5/d9/dc/d32/d48/f78 zero size 2024-04-26T04:42:15.380 INFO:tasks.workunit.client.1.smithi104.stdout:4/469: rmdir d4/d20 39 2024-04-26T04:42:15.381 INFO:tasks.workunit.client.1.smithi104.stdout:2/584: rename d6/d8/d5a/c22 to d6/d8/d3e/cc8 0 2024-04-26T04:42:15.384 INFO:tasks.workunit.client.0.smithi090.stdout:3/80: creat d1/d17/f1a x:0 0 0 2024-04-26T04:42:15.387 INFO:tasks.workunit.client.0.smithi090.stdout:0/174: creat d0/df/f3c x:0 0 0 2024-04-26T04:42:15.387 INFO:tasks.workunit.client.0.smithi090.stdout:0/175: readlink d0/de/l18 0 2024-04-26T04:42:15.387 INFO:tasks.workunit.client.0.smithi090.stdout:0/176: stat d0/d9/l10 0 2024-04-26T04:42:15.390 INFO:tasks.workunit.client.0.smithi090.stdout:3/81: mkdir d1/d6/d1b 0 2024-04-26T04:42:15.392 INFO:tasks.workunit.client.0.smithi090.stdout:0/177: rename d0/de/l18 to d0/d26/l3d 0 2024-04-26T04:42:15.393 INFO:tasks.workunit.client.1.smithi104.stdout:4/470: dread d4/f37 [0,4194304] 0 2024-04-26T04:42:15.398 INFO:tasks.workunit.client.0.smithi090.stdout:3/82: getdents d1/d9 0 2024-04-26T04:42:15.400 INFO:tasks.workunit.client.0.smithi090.stdout:3/83: symlink d1/l1c 0 2024-04-26T04:42:15.404 INFO:tasks.workunit.client.1.smithi104.stdout:6/532: dwrite d0/d9/f4a [0,4194304] 0 2024-04-26T04:42:15.405 INFO:tasks.workunit.client.0.smithi090.stdout:3/84: dread d1/d9/fb [0,4194304] 0 2024-04-26T04:42:15.406 INFO:tasks.workunit.client.1.smithi104.stdout:6/533: mknod d0/cad 0 2024-04-26T04:42:15.407 INFO:tasks.workunit.client.1.smithi104.stdout:6/534: mknod d0/d1b/cae 0 2024-04-26T04:42:15.407 INFO:tasks.workunit.client.0.smithi090.stdout:3/85: dread d1/d9/fb [0,4194304] 0 2024-04-26T04:42:15.409 INFO:tasks.workunit.client.1.smithi104.stdout:6/535: rename d0/d9/d13/d44/d71/f19 to d0/d9/d13/d15/d80/faf 0 2024-04-26T04:42:15.409 INFO:tasks.workunit.client.0.smithi090.stdout:3/86: dread d1/d9/fb [0,4194304] 0 2024-04-26T04:42:15.409 INFO:tasks.workunit.client.0.smithi090.stdout:3/87: chown d1/d9 12 1 2024-04-26T04:42:15.410 INFO:tasks.workunit.client.1.smithi104.stdout:6/536: creat d0/d9/d13/d41/d96/fb0 x:0 0 0 2024-04-26T04:42:15.410 INFO:tasks.workunit.client.1.smithi104.stdout:6/537: truncate d0/d9/d25/f68 1473842 0 2024-04-26T04:42:15.411 INFO:tasks.workunit.client.1.smithi104.stdout:6/538: rename d0/d9/d13/d41/d82/f9f to d0/d9/d13/d44/d47/fb1 0 2024-04-26T04:42:15.411 INFO:tasks.workunit.client.1.smithi104.stdout:6/539: chown d0/d9/d13/d41/d96/dac/d8e/c92 10570 1 2024-04-26T04:42:15.412 INFO:tasks.workunit.client.0.smithi090.stdout:9/68: dwrite db/dd/ff [0,4194304] 0 2024-04-26T04:42:15.412 INFO:tasks.workunit.client.0.smithi090.stdout:9/69: write f8 [4083606,103206] 0 2024-04-26T04:42:15.412 INFO:tasks.workunit.client.0.smithi090.stdout:4/138: dwrite f1 [0,4194304] 0 2024-04-26T04:42:15.412 INFO:tasks.workunit.client.1.smithi104.stdout:6/540: symlink d0/d9/d13/d15/d80/da1/daa/lb2 0 2024-04-26T04:42:15.413 INFO:tasks.workunit.client.1.smithi104.stdout:6/541: mkdir d0/d9/d13/d41/d96/dac/db3 0 2024-04-26T04:42:15.413 INFO:tasks.workunit.client.0.smithi090.stdout:9/70: mkdir db/dd/d18 0 2024-04-26T04:42:15.414 INFO:tasks.workunit.client.1.smithi104.stdout:6/542: creat d0/d9/d13/d41/d96/dac/d8e/fb4 x:0 0 0 2024-04-26T04:42:15.414 INFO:tasks.workunit.client.0.smithi090.stdout:4/139: rename d3/f14 to d3/db/d12/d20/f2a 0 2024-04-26T04:42:15.415 INFO:tasks.workunit.client.1.smithi104.stdout:6/543: creat d0/d1b/fb5 x:0 0 0 2024-04-26T04:42:15.416 INFO:tasks.workunit.client.1.smithi104.stdout:6/544: creat d0/d9/d13/d15/fb6 x:0 0 0 2024-04-26T04:42:15.417 INFO:tasks.workunit.client.1.smithi104.stdout:6/545: fdatasync d0/d9/d13/d15/f5e 0 2024-04-26T04:42:15.417 INFO:tasks.workunit.client.0.smithi090.stdout:9/71: unlink db/dd/f11 0 2024-04-26T04:42:15.417 INFO:tasks.workunit.client.0.smithi090.stdout:9/72: write f6 [1775895,29583] 0 2024-04-26T04:42:15.418 INFO:tasks.workunit.client.1.smithi104.stdout:6/546: creat d0/d9/d13/d44/d71/da5/fb7 x:0 0 0 2024-04-26T04:42:15.418 INFO:tasks.workunit.client.1.smithi104.stdout:6/547: mkdir d0/d9/d13/d15/d80/da1/daa/db8 0 2024-04-26T04:42:15.426 INFO:tasks.workunit.client.0.smithi090.stdout:3/88: write d1/f8 [1270228,11630] 0 2024-04-26T04:42:15.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/185: dwrite d4/f1e [0,4194304] 0 2024-04-26T04:42:15.428 INFO:tasks.workunit.client.1.smithi104.stdout:0/358: dwrite d0/d2/d10/f71 [0,4194304] 0 2024-04-26T04:42:15.428 INFO:tasks.workunit.client.1.smithi104.stdout:0/359: creat d0/d2/d10/d3d/f72 x:0 0 0 2024-04-26T04:42:15.428 INFO:tasks.workunit.client.1.smithi104.stdout:6/548: dread d0/d9/d13/d15/d80/f61 [0,4194304] 0 2024-04-26T04:42:15.428 INFO:tasks.workunit.client.1.smithi104.stdout:6/549: chown d0/d9/d13/d44/d47/f4c 2913263 1 2024-04-26T04:42:15.428 INFO:tasks.workunit.client.1.smithi104.stdout:6/550: creat d0/d9/d13/d44/d71/fb9 x:0 0 0 2024-04-26T04:42:15.430 INFO:tasks.workunit.client.1.smithi104.stdout:3/474: dwrite d5/d9/dc/d29/f56 [0,4194304] 0 2024-04-26T04:42:15.430 INFO:tasks.workunit.client.1.smithi104.stdout:4/471: dwrite d4/d7/d1d/d1e/f5a [0,4194304] 0 2024-04-26T04:42:15.430 INFO:tasks.workunit.client.1.smithi104.stdout:2/585: dwrite d6/f10 [4194304,4194304] 0 2024-04-26T04:42:15.430 INFO:tasks.workunit.client.1.smithi104.stdout:2/586: chown d6/de/cba 4 1 2024-04-26T04:42:15.430 INFO:tasks.workunit.client.0.smithi090.stdout:3/89: mknod d1/d17/c1d 0 2024-04-26T04:42:15.430 INFO:tasks.workunit.client.0.smithi090.stdout:3/90: fdatasync d1/d9/fa 0 2024-04-26T04:42:15.431 INFO:tasks.workunit.client.0.smithi090.stdout:4/140: dread d3/db/d12/f17 [0,4194304] 0 2024-04-26T04:42:15.431 INFO:tasks.workunit.client.0.smithi090.stdout:0/178: dwrite d0/de/f2c [4194304,4194304] 0 2024-04-26T04:42:15.431 INFO:tasks.workunit.client.0.smithi090.stdout:4/141: write d3/f27 [343244,18898] 0 2024-04-26T04:42:15.431 INFO:tasks.workunit.client.0.smithi090.stdout:4/142: creat d3/f2b x:0 0 0 2024-04-26T04:42:15.431 INFO:tasks.workunit.client.0.smithi090.stdout:4/143: creat d3/f2c x:0 0 0 2024-04-26T04:42:15.431 INFO:tasks.workunit.client.0.smithi090.stdout:9/73: fsync f6 0 2024-04-26T04:42:15.431 INFO:tasks.workunit.client.0.smithi090.stdout:3/91: mkdir d1/d6/d1b/d1e 0 2024-04-26T04:42:15.432 INFO:tasks.workunit.client.0.smithi090.stdout:3/92: creat d1/d17/f1f x:0 0 0 2024-04-26T04:42:15.432 INFO:tasks.workunit.client.1.smithi104.stdout:0/360: link d0/d2/d21/d27/d1e/d57/d6c/c67 d0/d2/c73 0 2024-04-26T04:42:15.434 INFO:tasks.workunit.client.0.smithi090.stdout:0/179: unlink d0/d26/f34 0 2024-04-26T04:42:15.437 INFO:tasks.workunit.client.1.smithi104.stdout:3/475: mknod d5/d9/d17/d60/ca8 0 2024-04-26T04:42:15.437 INFO:tasks.workunit.client.0.smithi090.stdout:5/186: dread d4/d1a/f1d [0,4194304] 0 2024-04-26T04:42:15.437 INFO:tasks.workunit.client.0.smithi090.stdout:4/144: stat d3/db/d12/d20/c25 0 2024-04-26T04:42:15.438 INFO:tasks.workunit.client.0.smithi090.stdout:4/145: readlink d3/l1a 0 2024-04-26T04:42:15.439 INFO:tasks.workunit.client.1.smithi104.stdout:1/426: sync 2024-04-26T04:42:15.439 INFO:tasks.workunit.client.1.smithi104.stdout:7/326: sync 2024-04-26T04:42:15.439 INFO:tasks.workunit.client.1.smithi104.stdout:4/472: mkdir d4/d20/d55/d56/da3 0 2024-04-26T04:42:15.439 INFO:tasks.workunit.client.1.smithi104.stdout:8/242: sync 2024-04-26T04:42:15.440 INFO:tasks.workunit.client.1.smithi104.stdout:8/243: fsync d0/d33/d35/f45 0 2024-04-26T04:42:15.440 INFO:tasks.workunit.client.0.smithi090.stdout:9/74: creat db/dd/f19 x:0 0 0 2024-04-26T04:42:15.440 INFO:tasks.workunit.client.0.smithi090.stdout:9/75: chown f3 1050 1 2024-04-26T04:42:15.441 INFO:tasks.workunit.client.0.smithi090.stdout:1/102: sync 2024-04-26T04:42:15.441 INFO:tasks.workunit.client.0.smithi090.stdout:8/68: sync 2024-04-26T04:42:15.442 INFO:tasks.workunit.client.0.smithi090.stdout:6/168: sync 2024-04-26T04:42:15.442 INFO:tasks.workunit.client.0.smithi090.stdout:1/103: chown d0/d4/cd 58 1 2024-04-26T04:42:15.442 INFO:tasks.workunit.client.1.smithi104.stdout:2/587: unlink d6/d8/d5a/fc7 0 2024-04-26T04:42:15.442 INFO:tasks.workunit.client.1.smithi104.stdout:2/588: write d6/d8/fb [393284,117654] 0 2024-04-26T04:42:15.443 INFO:tasks.workunit.client.1.smithi104.stdout:2/589: fdatasync d6/d1b/d51/f58 0 2024-04-26T04:42:15.443 INFO:tasks.workunit.client.0.smithi090.stdout:3/93: creat d1/d9/f20 x:0 0 0 2024-04-26T04:42:15.443 INFO:tasks.workunit.client.0.smithi090.stdout:3/94: write d1/fc [1596993,33664] 0 2024-04-26T04:42:15.443 INFO:tasks.workunit.client.1.smithi104.stdout:0/361: mknod d0/d4/d3c/c74 0 2024-04-26T04:42:15.443 INFO:tasks.workunit.client.1.smithi104.stdout:0/362: fsync d0/d2/d10/f4e 0 2024-04-26T04:42:15.445 INFO:tasks.workunit.client.0.smithi090.stdout:0/180: unlink d0/l4 0 2024-04-26T04:42:15.445 INFO:tasks.workunit.client.0.smithi090.stdout:0/181: write d0/d26/f27 [1456594,103572] 0 2024-04-26T04:42:15.447 INFO:tasks.workunit.client.1.smithi104.stdout:8/244: creat d0/f4d x:0 0 0 2024-04-26T04:42:15.447 INFO:tasks.workunit.client.1.smithi104.stdout:8/245: fsync d0/d33/d35/f3e 0 2024-04-26T04:42:15.447 INFO:tasks.workunit.client.0.smithi090.stdout:4/146: mkdir d3/db/d12/d20/d2d 0 2024-04-26T04:42:15.448 INFO:tasks.workunit.client.1.smithi104.stdout:1/427: creat d0/d6/d19/d21/d66/d30/d81/f8a x:0 0 0 2024-04-26T04:42:15.448 INFO:tasks.workunit.client.1.smithi104.stdout:1/428: write d0/d6/d19/d21/d66/d30/d81/f8a [913173,39016] 0 2024-04-26T04:42:15.451 INFO:tasks.workunit.client.0.smithi090.stdout:8/69: rename c9 to dc/c18 0 2024-04-26T04:42:15.452 INFO:tasks.workunit.client.1.smithi104.stdout:1/429: dread d0/d6/d2e/f7c [0,4194304] 0 2024-04-26T04:42:15.453 INFO:tasks.workunit.client.1.smithi104.stdout:1/430: readlink d0/d6/d2e/d32/d4c/l4e 0 2024-04-26T04:42:15.453 INFO:tasks.workunit.client.1.smithi104.stdout:1/431: chown d0/d6/d19/d1b/d35/c70 242 1 2024-04-26T04:42:15.453 INFO:tasks.workunit.client.1.smithi104.stdout:1/432: creat d0/d6/d19/d21/d66/d30/d81/f8b x:0 0 0 2024-04-26T04:42:15.457 INFO:tasks.workunit.client.1.smithi104.stdout:1/433: mkdir d0/d6/d19/d21/d66/d20/d2a/d56/d8c 0 2024-04-26T04:42:15.457 INFO:tasks.workunit.client.1.smithi104.stdout:0/363: dread d0/f2f [0,4194304] 0 2024-04-26T04:42:15.458 INFO:tasks.workunit.client.0.smithi090.stdout:1/104: creat d0/d4/f20 x:0 0 0 2024-04-26T04:42:15.459 INFO:tasks.workunit.client.1.smithi104.stdout:1/434: mknod d0/d6/d49/c8d 0 2024-04-26T04:42:15.459 INFO:tasks.workunit.client.1.smithi104.stdout:1/435: readlink d0/d6/d2e/d32/l4f 0 2024-04-26T04:42:15.461 INFO:tasks.workunit.client.0.smithi090.stdout:6/169: creat d9/d1c/d31/d3d/f3e x:0 0 0 2024-04-26T04:42:15.462 INFO:tasks.workunit.client.1.smithi104.stdout:1/436: symlink d0/d6/d19/d1b/l8e 0 2024-04-26T04:42:15.463 INFO:tasks.workunit.client.0.smithi090.stdout:0/182: truncate d0/de/f14 7505739 0 2024-04-26T04:42:15.466 INFO:tasks.workunit.client.1.smithi104.stdout:1/437: mknod d0/d6/d19/d21/d66/d20/d2a/d56/d8c/c8f 0 2024-04-26T04:42:15.466 INFO:tasks.workunit.client.1.smithi104.stdout:1/438: fdatasync d0/d6/d19/d21/d66/d20/d2a/f78 0 2024-04-26T04:42:15.466 INFO:tasks.workunit.client.1.smithi104.stdout:1/439: write d0/d6/d19/d21/d59/f6e [360276,34193] 0 2024-04-26T04:42:15.469 INFO:tasks.workunit.client.1.smithi104.stdout:6/551: rename d0/d9/d13/d15/d80/da1 to d0/d9/d13/d15/dba 0 2024-04-26T04:42:15.469 INFO:tasks.workunit.client.1.smithi104.stdout:6/552: fdatasync d0/d9/d25/f39 0 2024-04-26T04:42:15.469 INFO:tasks.workunit.client.1.smithi104.stdout:6/553: fsync d0/d1b/f9e 0 2024-04-26T04:42:15.473 INFO:tasks.workunit.client.0.smithi090.stdout:1/105: creat d0/d4/d7/d17/f21 x:0 0 0 2024-04-26T04:42:15.474 INFO:tasks.workunit.client.1.smithi104.stdout:7/327: rename d2/d4/d1d/f30 to d2/d4/d5a/f68 0 2024-04-26T04:42:15.474 INFO:tasks.workunit.client.1.smithi104.stdout:9/307: sync 2024-04-26T04:42:15.474 INFO:tasks.workunit.client.1.smithi104.stdout:5/388: sync 2024-04-26T04:42:15.474 INFO:tasks.workunit.client.1.smithi104.stdout:5/389: truncate d0/d1/d3e/f51 996520 0 2024-04-26T04:42:15.476 INFO:tasks.workunit.client.1.smithi104.stdout:6/554: link d0/d9/d13/d41/c45 d0/d9/da8/cbb 0 2024-04-26T04:42:15.477 INFO:tasks.workunit.client.0.smithi090.stdout:5/187: link d4/f37 d4/f3b 0 2024-04-26T04:42:15.479 INFO:tasks.workunit.client.1.smithi104.stdout:4/473: rename d4/d7/l8d to d4/d7/d1d/d1e/d15/la4 0 2024-04-26T04:42:15.480 INFO:tasks.workunit.client.0.smithi090.stdout:6/170: rename d9/c1a to d9/d22/c3f 0 2024-04-26T04:42:15.480 INFO:tasks.workunit.client.0.smithi090.stdout:6/171: write d9/d1c/d31/d3d/f3e [100522,44277] 0 2024-04-26T04:42:15.480 INFO:tasks.workunit.client.0.smithi090.stdout:9/76: fsync db/dd/f19 0 2024-04-26T04:42:15.480 INFO:tasks.workunit.client.0.smithi090.stdout:9/77: chown f2 25210885 1 2024-04-26T04:42:15.481 INFO:tasks.workunit.client.0.smithi090.stdout:9/78: truncate f8 1940102 0 2024-04-26T04:42:15.482 INFO:tasks.workunit.client.1.smithi104.stdout:7/328: unlink d2/d4/d32/d35/l3b 0 2024-04-26T04:42:15.482 INFO:tasks.workunit.client.1.smithi104.stdout:7/329: write d2/d4/d32/d35/f4d [309565,31551] 0 2024-04-26T04:42:15.483 INFO:tasks.workunit.client.1.smithi104.stdout:9/308: creat d11/d15/d32/d35/f5e x:0 0 0 2024-04-26T04:42:15.484 INFO:tasks.workunit.client.1.smithi104.stdout:5/390: mknod d0/d1/d4/d2a/d6b/c83 0 2024-04-26T04:42:15.487 INFO:tasks.workunit.client.1.smithi104.stdout:8/246: rename d0/d33/d35/f44 to d0/f4e 0 2024-04-26T04:42:15.489 INFO:tasks.workunit.client.1.smithi104.stdout:4/474: link d4/d7/d1d/d2e/c4d d4/d7/d1d/d1e/d15/ca5 0 2024-04-26T04:42:15.489 INFO:tasks.workunit.client.1.smithi104.stdout:4/475: write d4/d20/f51 [2438208,29970] 0 2024-04-26T04:42:15.489 INFO:tasks.workunit.client.1.smithi104.stdout:4/476: fdatasync d4/f69 0 2024-04-26T04:42:15.491 INFO:tasks.workunit.client.1.smithi104.stdout:7/330: creat d2/d4/d32/f69 x:0 0 0 2024-04-26T04:42:15.491 INFO:tasks.workunit.client.1.smithi104.stdout:9/309: mknod d11/d1d/d34/c5f 0 2024-04-26T04:42:15.493 INFO:tasks.workunit.client.1.smithi104.stdout:8/247: symlink d0/l4f 0 2024-04-26T04:42:15.496 INFO:tasks.workunit.client.1.smithi104.stdout:4/477: mknod d4/d20/ca6 0 2024-04-26T04:42:15.497 INFO:tasks.workunit.client.1.smithi104.stdout:7/331: truncate d2/f2f 1487180 0 2024-04-26T04:42:15.498 INFO:tasks.workunit.client.1.smithi104.stdout:9/310: rename d11/d15/d1b/f46 to d11/d15/d1b/f60 0 2024-04-26T04:42:15.499 INFO:tasks.workunit.client.1.smithi104.stdout:9/311: read - d11/d15/d1b/f37 zero size 2024-04-26T04:42:15.499 INFO:tasks.workunit.client.1.smithi104.stdout:9/312: truncate d11/f52 926606 0 2024-04-26T04:42:15.500 INFO:tasks.workunit.client.1.smithi104.stdout:4/478: rmdir d4/d7/d1d/d2e 39 2024-04-26T04:42:15.500 INFO:tasks.workunit.client.0.smithi090.stdout:8/70: dwrite f4 [0,4194304] 0 2024-04-26T04:42:15.503 INFO:tasks.workunit.client.1.smithi104.stdout:7/332: rename d2/d4/d1d/d34/c58 to d2/d4/d32/d35/d44/d5b/c6a 0 2024-04-26T04:42:15.503 INFO:tasks.workunit.client.1.smithi104.stdout:7/333: truncate d2/f42 1007797 0 2024-04-26T04:42:15.503 INFO:tasks.workunit.client.1.smithi104.stdout:7/334: dread - d2/d4/d32/d3d/f50 zero size 2024-04-26T04:42:15.503 INFO:tasks.workunit.client.0.smithi090.stdout:8/71: rename dc/c10 to dc/c19 0 2024-04-26T04:42:15.503 INFO:tasks.workunit.client.0.smithi090.stdout:6/172: dread d9/d22/f25 [0,4194304] 0 2024-04-26T04:42:15.504 INFO:tasks.workunit.client.1.smithi104.stdout:9/313: creat d11/d15/d32/d4c/d54/f61 x:0 0 0 2024-04-26T04:42:15.504 INFO:tasks.workunit.client.0.smithi090.stdout:5/188: write d4/d1a/df/f10 [3070906,1312] 0 2024-04-26T04:42:15.505 INFO:tasks.workunit.client.1.smithi104.stdout:4/479: mknod d4/d20/d55/d72/ca7 0 2024-04-26T04:42:15.506 INFO:tasks.workunit.client.0.smithi090.stdout:6/173: rename d9/d37 to d9/d21/d40 0 2024-04-26T04:42:15.506 INFO:tasks.workunit.client.0.smithi090.stdout:6/174: readlink d9/d22/l27 0 2024-04-26T04:42:15.506 INFO:tasks.workunit.client.0.smithi090.stdout:6/175: stat d9/f15 0 2024-04-26T04:42:15.506 INFO:tasks.workunit.client.0.smithi090.stdout:8/72: rmdir dc/d12 39 2024-04-26T04:42:15.506 INFO:tasks.workunit.client.0.smithi090.stdout:8/73: creat dc/f1a x:0 0 0 2024-04-26T04:42:15.508 INFO:tasks.workunit.client.1.smithi104.stdout:7/335: mknod d2/d4/d32/d3d/d41/c6b 0 2024-04-26T04:42:15.508 INFO:tasks.workunit.client.0.smithi090.stdout:5/189: chown d4/c1b 73531 1 2024-04-26T04:42:15.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/190: chown d4/d1a/d1f/f2f 2366950 1 2024-04-26T04:42:15.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/191: truncate d4/d1a/f17 1582651 0 2024-04-26T04:42:15.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/192: creat d4/d1a/df/d21/f3c x:0 0 0 2024-04-26T04:42:15.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/193: fdatasync d4/d1a/d1f/f3a 0 2024-04-26T04:42:15.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/194: creat d4/d1a/d1f/f3d x:0 0 0 2024-04-26T04:42:15.509 INFO:tasks.workunit.client.0.smithi090.stdout:6/176: creat d9/d1c/d31/f41 x:0 0 0 2024-04-26T04:42:15.511 INFO:tasks.workunit.client.0.smithi090.stdout:8/74: link dc/c18 dc/d12/c1b 0 2024-04-26T04:42:15.511 INFO:tasks.workunit.client.0.smithi090.stdout:8/75: write dc/d12/f17 [687445,910] 0 2024-04-26T04:42:15.511 INFO:tasks.workunit.client.1.smithi104.stdout:7/336: mkdir d2/d4/d32/d3d/d41/d5f/d6c 0 2024-04-26T04:42:15.512 INFO:tasks.workunit.client.0.smithi090.stdout:5/195: write d4/f1e [5175002,130523] 0 2024-04-26T04:42:15.513 INFO:tasks.workunit.client.0.smithi090.stdout:6/177: rmdir d9 39 2024-04-26T04:42:15.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/76: symlink dc/d12/l1c 0 2024-04-26T04:42:15.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/77: fdatasync dc/f1a 0 2024-04-26T04:42:15.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/78: creat dc/f1d x:0 0 0 2024-04-26T04:42:15.514 INFO:tasks.workunit.client.0.smithi090.stdout:8/79: write dc/d12/f17 [618733,58330] 0 2024-04-26T04:42:15.515 INFO:tasks.workunit.client.1.smithi104.stdout:7/337: mkdir d2/d4/d32/d3d/d41/d5f/d6d 0 2024-04-26T04:42:15.515 INFO:tasks.workunit.client.0.smithi090.stdout:6/178: unlink d9/d22/c2f 0 2024-04-26T04:42:15.516 INFO:tasks.workunit.client.0.smithi090.stdout:6/179: mknod d9/d21/c42 0 2024-04-26T04:42:15.518 INFO:tasks.workunit.client.1.smithi104.stdout:7/338: creat d2/d4/f6e x:0 0 0 2024-04-26T04:42:15.523 INFO:tasks.workunit.client.1.smithi104.stdout:8/248: dwrite d0/d1c/d2d/f32 [0,4194304] 0 2024-04-26T04:42:15.524 INFO:tasks.workunit.client.1.smithi104.stdout:8/249: chown d0/d33/f28 1763 1 2024-04-26T04:42:15.529 INFO:tasks.workunit.client.0.smithi090.stdout:0/183: dwrite d0/f3b [0,4194304] 0 2024-04-26T04:42:15.530 INFO:tasks.workunit.client.0.smithi090.stdout:4/147: dwrite d3/db/d12/ff [4194304,4194304] 0 2024-04-26T04:42:15.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/80: dread fa [0,4194304] 0 2024-04-26T04:42:15.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/81: creat dc/f1e x:0 0 0 2024-04-26T04:42:15.531 INFO:tasks.workunit.client.1.smithi104.stdout:3/476: dwrite d5/d9/f26 [0,4194304] 0 2024-04-26T04:42:15.537 INFO:tasks.workunit.client.1.smithi104.stdout:8/250: write d0/d1c/d2d/f32 [2146487,62916] 0 2024-04-26T04:42:15.538 INFO:tasks.workunit.client.1.smithi104.stdout:4/480: rename d4/d20/d55/d72 to d4/d7/d1d/d1e/d24/d59/da8 0 2024-04-26T04:42:15.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/148: mkdir d3/db/d12/d20/d2d/d2e 0 2024-04-26T04:42:15.539 INFO:tasks.workunit.client.1.smithi104.stdout:0/364: dwrite d0/d2/d21/d27/d1e/d57/d6c/f6a [0,4194304] 0 2024-04-26T04:42:15.539 INFO:tasks.workunit.client.1.smithi104.stdout:1/440: dwrite d0/d6/d19/d21/d28/f36 [0,4194304] 0 2024-04-26T04:42:15.540 INFO:tasks.workunit.client.1.smithi104.stdout:1/441: chown d0 42257017 1 2024-04-26T04:42:15.540 INFO:tasks.workunit.client.1.smithi104.stdout:6/555: dwrite d0/d9/d13/d15/f42 [0,4194304] 0 2024-04-26T04:42:15.541 INFO:tasks.workunit.client.1.smithi104.stdout:4/481: mknod d4/d20/d55/d56/ca9 0 2024-04-26T04:42:15.541 INFO:tasks.workunit.client.0.smithi090.stdout:0/184: rmdir d0/d9/d33 0 2024-04-26T04:42:15.543 INFO:tasks.workunit.client.1.smithi104.stdout:5/391: dwrite d0/d1/d64/f6c [0,4194304] 0 2024-04-26T04:42:15.544 INFO:tasks.workunit.client.1.smithi104.stdout:0/365: rmdir d0/d4/d3b 39 2024-04-26T04:42:15.545 INFO:tasks.workunit.client.1.smithi104.stdout:6/556: unlink d0/d9/d13/d15/l3c 0 2024-04-26T04:42:15.545 INFO:tasks.workunit.client.1.smithi104.stdout:4/482: rmdir d4/db/d6b/d86 39 2024-04-26T04:42:15.548 INFO:tasks.workunit.client.1.smithi104.stdout:6/557: dread d0/d9/d13/d41/f7f [0,4194304] 0 2024-04-26T04:42:15.548 INFO:tasks.workunit.client.1.smithi104.stdout:6/558: write d0/d9/d13/d44/f97 [880292,92364] 0 2024-04-26T04:42:15.548 INFO:tasks.workunit.client.0.smithi090.stdout:0/185: rename d0/de/f14 to d0/d9/f3e 0 2024-04-26T04:42:15.549 INFO:tasks.workunit.client.0.smithi090.stdout:1/106: fsync d0/d4/d7/d17/f21 0 2024-04-26T04:42:15.549 INFO:tasks.workunit.client.0.smithi090.stdout:1/107: chown d0/c1f 1744 1 2024-04-26T04:42:15.558 INFO:tasks.workunit.client.1.smithi104.stdout:0/366: creat d0/d4/d3c/d53/d5d/f75 x:0 0 0 2024-04-26T04:42:15.558 INFO:tasks.workunit.client.1.smithi104.stdout:0/367: write d0/d4/d3c/f46 [109815,40438] 0 2024-04-26T04:42:15.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/149: dread f1 [0,4194304] 0 2024-04-26T04:42:15.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/150: chown d3/f2b 26344 1 2024-04-26T04:42:15.563 INFO:tasks.workunit.client.0.smithi090.stdout:4/151: rename d3/f8 to d3/db/f2f 0 2024-04-26T04:42:15.563 INFO:tasks.workunit.client.0.smithi090.stdout:4/152: mkdir d3/db/d12/d20/d24/d30 0 2024-04-26T04:42:15.564 INFO:tasks.workunit.client.1.smithi104.stdout:6/559: creat d0/d9/fbc x:0 0 0 2024-04-26T04:42:15.564 INFO:tasks.workunit.client.1.smithi104.stdout:4/483: creat d4/d7/d1d/d2e/faa x:0 0 0 2024-04-26T04:42:15.565 INFO:tasks.workunit.client.0.smithi090.stdout:4/153: truncate d3/f4 4278356 0 2024-04-26T04:42:15.565 INFO:tasks.workunit.client.0.smithi090.stdout:4/154: stat d3/f9 0 2024-04-26T04:42:15.565 INFO:tasks.workunit.client.0.smithi090.stdout:4/155: stat d3/db/d12/l22 0 2024-04-26T04:42:15.566 INFO:tasks.workunit.client.1.smithi104.stdout:6/560: mkdir d0/d9/d13/d41/d96/dac/d8e/dbd 0 2024-04-26T04:42:15.567 INFO:tasks.workunit.client.0.smithi090.stdout:4/156: unlink d3/d13/f1b 0 2024-04-26T04:42:15.567 INFO:tasks.workunit.client.1.smithi104.stdout:4/484: creat d4/d20/fab x:0 0 0 2024-04-26T04:42:15.567 INFO:tasks.workunit.client.0.smithi090.stdout:9/79: dwrite f2 [0,4194304] 0 2024-04-26T04:42:15.567 INFO:tasks.workunit.client.0.smithi090.stdout:9/80: stat db/d16 0 2024-04-26T04:42:15.567 INFO:tasks.workunit.client.0.smithi090.stdout:9/81: chown la 867221084 1 2024-04-26T04:42:15.568 INFO:tasks.workunit.client.1.smithi104.stdout:0/368: mknod d0/d2/d21/d27/d1e/c76 0 2024-04-26T04:42:15.571 INFO:tasks.workunit.client.0.smithi090.stdout:4/157: dread d3/f27 [0,4194304] 0 2024-04-26T04:42:15.571 INFO:tasks.workunit.client.1.smithi104.stdout:4/485: truncate d4/d7/d1d/d2e/f7e 995081 0 2024-04-26T04:42:15.573 INFO:tasks.workunit.client.0.smithi090.stdout:4/158: symlink d3/db/d12/d20/l31 0 2024-04-26T04:42:15.573 INFO:tasks.workunit.client.0.smithi090.stdout:4/159: readlink d3/db/d12/d20/l31 0 2024-04-26T04:42:15.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/160: rmdir d3/db/d12/d20/d24 39 2024-04-26T04:42:15.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/161: fdatasync d3/d13/f21 0 2024-04-26T04:42:15.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/162: getdents d3/d13/d1d 0 2024-04-26T04:42:15.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/163: chown d3/db/d12/d20/d2d/d2e 55107613 1 2024-04-26T04:42:15.577 INFO:tasks.workunit.client.1.smithi104.stdout:0/369: creat d0/d4/d3b/f77 x:0 0 0 2024-04-26T04:42:15.577 INFO:tasks.workunit.client.1.smithi104.stdout:6/561: rmdir d0/d1b 39 2024-04-26T04:42:15.577 INFO:tasks.workunit.client.0.smithi090.stdout:1/108: write d0/d4/f15 [338443,21027] 0 2024-04-26T04:42:15.578 INFO:tasks.workunit.client.1.smithi104.stdout:0/370: creat d0/d2/d21/d49/d5e/f78 x:0 0 0 2024-04-26T04:42:15.580 INFO:tasks.workunit.client.0.smithi090.stdout:1/109: mkdir d0/d4/d7/d17/d22 0 2024-04-26T04:42:15.582 INFO:tasks.workunit.client.1.smithi104.stdout:4/486: write f2 [3300007,673] 0 2024-04-26T04:42:15.582 INFO:tasks.workunit.client.1.smithi104.stdout:6/562: creat d0/d9/d13/d44/fbe x:0 0 0 2024-04-26T04:42:15.582 INFO:tasks.workunit.client.0.smithi090.stdout:1/110: unlink d0/d4/d7/fa 0 2024-04-26T04:42:15.584 INFO:tasks.workunit.client.0.smithi090.stdout:1/111: creat d0/d4/d7/d17/d22/f23 x:0 0 0 2024-04-26T04:42:15.584 INFO:tasks.workunit.client.1.smithi104.stdout:9/314: dwrite d11/d15/d1b/f60 [0,4194304] 0 2024-04-26T04:42:15.585 INFO:tasks.workunit.client.1.smithi104.stdout:4/487: link d4/d7/f42 d4/d7/d1d/d1e/d24/d4a/fac 0 2024-04-26T04:42:15.585 INFO:tasks.workunit.client.0.smithi090.stdout:5/196: dwrite d4/d1a/df/d21/f32 [0,4194304] 0 2024-04-26T04:42:15.586 INFO:tasks.workunit.client.0.smithi090.stdout:1/112: mknod d0/c24 0 2024-04-26T04:42:15.587 INFO:tasks.workunit.client.1.smithi104.stdout:6/563: rename d0/f6e to d0/d9/d13/d41/fbf 0 2024-04-26T04:42:15.587 INFO:tasks.workunit.client.1.smithi104.stdout:6/564: truncate d0/d1b/f9e 154592 0 2024-04-26T04:42:15.590 INFO:tasks.workunit.client.1.smithi104.stdout:4/488: unlink d4/d7/d1d/d1e/d15/l9f 0 2024-04-26T04:42:15.591 INFO:tasks.workunit.client.1.smithi104.stdout:4/489: creat d4/d7/d1d/d1e/fad x:0 0 0 2024-04-26T04:42:15.594 INFO:tasks.workunit.client.1.smithi104.stdout:4/490: creat d4/d7/d1d/d1e/d24/d59/fae x:0 0 0 2024-04-26T04:42:15.594 INFO:tasks.workunit.client.1.smithi104.stdout:4/491: chown d4/d7/d1d/d1e/d24/d59/c6e 206372 1 2024-04-26T04:42:15.596 INFO:tasks.workunit.client.1.smithi104.stdout:4/492: creat d4/d20/d55/d56/da3/faf x:0 0 0 2024-04-26T04:42:15.604 INFO:tasks.workunit.client.1.smithi104.stdout:4/493: dread d4/d20/f36 [0,4194304] 0 2024-04-26T04:42:15.605 INFO:tasks.workunit.client.1.smithi104.stdout:4/494: mknod d4/d7/d1d/d1e/d8f/cb0 0 2024-04-26T04:42:15.607 INFO:tasks.workunit.client.1.smithi104.stdout:4/495: unlink d4/d7/d1d/d1e/d15/f66 0 2024-04-26T04:42:15.607 INFO:tasks.workunit.client.1.smithi104.stdout:4/496: fdatasync d4/d7/d1d/d1e/d24/d59/da8/f9c 0 2024-04-26T04:42:15.607 INFO:tasks.workunit.client.1.smithi104.stdout:4/497: fsync d4/d7/f53 0 2024-04-26T04:42:15.608 INFO:tasks.workunit.client.1.smithi104.stdout:4/498: mknod d4/db/d6b/cb1 0 2024-04-26T04:42:15.609 INFO:tasks.workunit.client.1.smithi104.stdout:4/499: write d4/d7/d1d/d1e/d24/f68 [1148966,109861] 0 2024-04-26T04:42:15.609 INFO:tasks.workunit.client.1.smithi104.stdout:4/500: chown d4/d7/d1d/d1e/d24/l29 554885450 1 2024-04-26T04:42:15.609 INFO:tasks.workunit.client.1.smithi104.stdout:4/501: read d4/f98 [410487,59051] 0 2024-04-26T04:42:15.610 INFO:tasks.workunit.client.1.smithi104.stdout:3/477: dwrite d5/f54 [0,4194304] 0 2024-04-26T04:42:15.612 INFO:tasks.workunit.client.1.smithi104.stdout:4/502: unlink d4/d7/d1d/d1e/f5a 0 2024-04-26T04:42:15.613 INFO:tasks.workunit.client.1.smithi104.stdout:9/315: dread d11/f17 [0,4194304] 0 2024-04-26T04:42:15.614 INFO:tasks.workunit.client.1.smithi104.stdout:3/478: unlink d5/d9/f14 0 2024-04-26T04:42:15.615 INFO:tasks.workunit.client.1.smithi104.stdout:4/503: write d4/f37 [1112997,66402] 0 2024-04-26T04:42:15.615 INFO:tasks.workunit.client.1.smithi104.stdout:9/316: dread d11/f52 [0,4194304] 0 2024-04-26T04:42:15.615 INFO:tasks.workunit.client.1.smithi104.stdout:9/317: stat f10 0 2024-04-26T04:42:15.616 INFO:tasks.workunit.client.1.smithi104.stdout:3/479: mkdir d5/d9/dc/d29/d7d/da9 0 2024-04-26T04:42:15.617 INFO:tasks.workunit.client.1.smithi104.stdout:4/504: symlink d4/d7/d1d/d1e/d24/lb2 0 2024-04-26T04:42:15.617 INFO:tasks.workunit.client.1.smithi104.stdout:7/339: dwrite d2/d4/f26 [0,4194304] 0 2024-04-26T04:42:15.617 INFO:tasks.workunit.client.1.smithi104.stdout:7/340: creat d2/d4/d32/d35/d37/f6f x:0 0 0 2024-04-26T04:42:15.618 INFO:tasks.workunit.client.0.smithi090.stdout:6/180: dwrite d9/d22/f28 [0,4194304] 0 2024-04-26T04:42:15.618 INFO:tasks.workunit.client.0.smithi090.stdout:8/82: dwrite f4 [0,4194304] 0 2024-04-26T04:42:15.618 INFO:tasks.workunit.client.0.smithi090.stdout:6/181: readlink d9/d1c/l34 0 2024-04-26T04:42:15.618 INFO:tasks.workunit.client.0.smithi090.stdout:6/182: fdatasync d9/d1c/d31/f3a 0 2024-04-26T04:42:15.618 INFO:tasks.workunit.client.0.smithi090.stdout:6/183: write d9/f24 [358144,122997] 0 2024-04-26T04:42:15.619 INFO:tasks.workunit.client.1.smithi104.stdout:9/318: rmdir d11/d15/d32/d4e 0 2024-04-26T04:42:15.620 INFO:tasks.workunit.client.1.smithi104.stdout:9/319: read d11/d1d/f4f [63552,116968] 0 2024-04-26T04:42:15.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/83: mkdir dc/d1f 0 2024-04-26T04:42:15.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/84: readlink dc/d12/l16 0 2024-04-26T04:42:15.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/85: fsync dc/f1d 0 2024-04-26T04:42:15.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/86: chown f5 543450793 1 2024-04-26T04:42:15.621 INFO:tasks.workunit.client.1.smithi104.stdout:4/505: mkdir d4/d7/d1d/d1e/d24/d59/db3 0 2024-04-26T04:42:15.623 INFO:tasks.workunit.client.1.smithi104.stdout:8/251: dwrite d0/d1c/f26 [0,4194304] 0 2024-04-26T04:42:15.623 INFO:tasks.workunit.client.1.smithi104.stdout:5/392: dwrite d0/d1/d4/d2a/f50 [0,4194304] 0 2024-04-26T04:42:15.623 INFO:tasks.workunit.client.1.smithi104.stdout:5/393: fdatasync d0/f58 0 2024-04-26T04:42:15.624 INFO:tasks.workunit.client.1.smithi104.stdout:1/442: dwrite d0/d6/d19/d21/d66/d20/d2a/f78 [0,4194304] 0 2024-04-26T04:42:15.624 INFO:tasks.workunit.client.1.smithi104.stdout:1/443: fdatasync d0/d6/d19/d21/d66/d20/d2a/f61 0 2024-04-26T04:42:15.624 INFO:tasks.workunit.client.0.smithi090.stdout:9/82: dwrite db/f13 [0,4194304] 0 2024-04-26T04:42:15.624 INFO:tasks.workunit.client.0.smithi090.stdout:5/197: dwrite d4/d1a/d2a/d31/f34 [4194304,4194304] 0 2024-04-26T04:42:15.624 INFO:tasks.workunit.client.0.smithi090.stdout:6/184: truncate d9/f12 4930807 0 2024-04-26T04:42:15.624 INFO:tasks.workunit.client.0.smithi090.stdout:6/185: getdents d9/d21/d3b 0 2024-04-26T04:42:15.625 INFO:tasks.workunit.client.0.smithi090.stdout:4/164: dwrite d3/f2c [0,4194304] 0 2024-04-26T04:42:15.625 INFO:tasks.workunit.client.0.smithi090.stdout:0/186: dwrite d0/d9/f13 [0,4194304] 0 2024-04-26T04:42:15.625 INFO:tasks.workunit.client.0.smithi090.stdout:4/165: chown d3/c7 7430040 1 2024-04-26T04:42:15.625 INFO:tasks.workunit.client.1.smithi104.stdout:6/565: dwrite d0/d9/d13/d44/fbe [0,4194304] 0 2024-04-26T04:42:15.625 INFO:tasks.workunit.client.0.smithi090.stdout:5/198: dread d4/f37 [0,4194304] 0 2024-04-26T04:42:15.627 INFO:tasks.workunit.client.1.smithi104.stdout:7/341: rename d2/c3f to d2/d4/d32/d3d/c70 0 2024-04-26T04:42:15.628 INFO:tasks.workunit.client.1.smithi104.stdout:3/480: getdents d5/d9/dc/d32 0 2024-04-26T04:42:15.629 INFO:tasks.workunit.client.0.smithi090.stdout:5/199: fdatasync d4/d1a/df/f10 0 2024-04-26T04:42:15.629 INFO:tasks.workunit.client.0.smithi090.stdout:5/200: chown d4/d1a/d1f/f3a 29567 1 2024-04-26T04:42:15.629 INFO:tasks.workunit.client.0.smithi090.stdout:9/83: rename l7 to db/dd/d18/l1a 0 2024-04-26T04:42:15.629 INFO:tasks.workunit.client.0.smithi090.stdout:9/84: chown db/dd 1 1 2024-04-26T04:42:15.629 INFO:tasks.workunit.client.0.smithi090.stdout:9/85: creat db/f1b x:0 0 0 2024-04-26T04:42:15.631 INFO:tasks.workunit.client.1.smithi104.stdout:4/506: mknod d4/d7/d1d/d1e/d24/d4a/d9e/cb4 0 2024-04-26T04:42:15.631 INFO:tasks.workunit.client.1.smithi104.stdout:1/444: dread d0/d6/d19/d21/d66/f41 [0,4194304] 0 2024-04-26T04:42:15.631 INFO:tasks.workunit.client.1.smithi104.stdout:1/445: truncate d0/d6/d49/f87 64343 0 2024-04-26T04:42:15.634 INFO:tasks.workunit.client.1.smithi104.stdout:8/252: mkdir d0/d50 0 2024-04-26T04:42:15.638 INFO:tasks.workunit.client.1.smithi104.stdout:3/481: dread d5/d9/d17/f3b [0,4194304] 0 2024-04-26T04:42:15.639 INFO:tasks.workunit.client.1.smithi104.stdout:5/394: mknod d0/d1/d4/d2a/d2b/d5c/c84 0 2024-04-26T04:42:15.643 INFO:tasks.workunit.client.0.smithi090.stdout:0/187: chown d0/f38 3432 1 2024-04-26T04:42:15.645 INFO:tasks.workunit.client.0.smithi090.stdout:4/166: creat d3/db/d12/d20/d24/f32 x:0 0 0 2024-04-26T04:42:15.646 INFO:tasks.workunit.client.1.smithi104.stdout:6/566: mknod d0/d9/d13/d44/cc0 0 2024-04-26T04:42:15.648 INFO:tasks.workunit.client.0.smithi090.stdout:6/186: rename f3 to d9/d22/f43 0 2024-04-26T04:42:15.649 INFO:tasks.workunit.client.1.smithi104.stdout:7/342: link d2/d4/f11 d2/d4/d32/d3d/f71 0 2024-04-26T04:42:15.650 INFO:tasks.workunit.client.0.smithi090.stdout:4/167: truncate d3/d13/f21 93042 0 2024-04-26T04:42:15.650 INFO:tasks.workunit.client.1.smithi104.stdout:4/507: truncate d4/d7/f8 1693667 0 2024-04-26T04:42:15.652 INFO:tasks.workunit.client.1.smithi104.stdout:1/446: rename d0/d6/d19/c1e to d0/d6/d19/d21/c90 0 2024-04-26T04:42:15.652 INFO:tasks.workunit.client.1.smithi104.stdout:1/447: fdatasync d0/d6/d2e/f7c 0 2024-04-26T04:42:15.652 INFO:tasks.workunit.client.0.smithi090.stdout:9/86: rename l5 to db/d16/l1c 0 2024-04-26T04:42:15.653 INFO:tasks.workunit.client.1.smithi104.stdout:5/395: dread d0/d1/d4/d2a/d2b/f35 [0,4194304] 0 2024-04-26T04:42:15.653 INFO:tasks.workunit.client.1.smithi104.stdout:5/396: creat d0/d1/d64/f85 x:0 0 0 2024-04-26T04:42:15.654 INFO:tasks.workunit.client.0.smithi090.stdout:6/187: mknod d9/d22/c44 0 2024-04-26T04:42:15.654 INFO:tasks.workunit.client.0.smithi090.stdout:8/87: dwrite dc/f1a [0,4194304] 0 2024-04-26T04:42:15.654 INFO:tasks.workunit.client.1.smithi104.stdout:8/253: mkdir d0/d33/d51 0 2024-04-26T04:42:15.655 INFO:tasks.workunit.client.1.smithi104.stdout:8/254: chown d0/f4d 200539 1 2024-04-26T04:42:15.655 INFO:tasks.workunit.client.1.smithi104.stdout:6/567: unlink d0/d9/fc 0 2024-04-26T04:42:15.656 INFO:tasks.workunit.client.1.smithi104.stdout:6/568: fsync d0/d9/d13/d15/d80/faf 0 2024-04-26T04:42:15.656 INFO:tasks.workunit.client.1.smithi104.stdout:6/569: creat d0/d9/d13/d44/d71/fc1 x:0 0 0 2024-04-26T04:42:15.656 INFO:tasks.workunit.client.0.smithi090.stdout:0/188: link d0/d26/l1d d0/d26/l3f 0 2024-04-26T04:42:15.656 INFO:tasks.workunit.client.0.smithi090.stdout:0/189: fsync d0/d26/f27 0 2024-04-26T04:42:15.656 INFO:tasks.workunit.client.0.smithi090.stdout:0/190: truncate d0/d26/f27 2092435 0 2024-04-26T04:42:15.657 INFO:tasks.workunit.client.0.smithi090.stdout:0/191: readlink d0/d26/l3f 0 2024-04-26T04:42:15.657 INFO:tasks.workunit.client.0.smithi090.stdout:1/113: dwrite d0/f12 [0,4194304] 0 2024-04-26T04:42:15.657 INFO:tasks.workunit.client.0.smithi090.stdout:4/168: truncate d3/db/d12/f17 3463623 0 2024-04-26T04:42:15.658 INFO:tasks.workunit.client.0.smithi090.stdout:4/169: readlink d3/db/d12/l29 0 2024-04-26T04:42:15.658 INFO:tasks.workunit.client.1.smithi104.stdout:4/508: creat d4/d7/d1d/d1e/d24/d59/db3/fb5 x:0 0 0 2024-04-26T04:42:15.658 INFO:tasks.workunit.client.1.smithi104.stdout:1/448: symlink d0/d6/d19/d1b/d35/l91 0 2024-04-26T04:42:15.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/201: dwrite d4/d1a/df/d21/f3c [0,4194304] 0 2024-04-26T04:42:15.661 INFO:tasks.workunit.client.1.smithi104.stdout:8/255: unlink d0/d1c/d27/c2a 0 2024-04-26T04:42:15.661 INFO:tasks.workunit.client.1.smithi104.stdout:8/256: chown d0/d33/l23 10 1 2024-04-26T04:42:15.666 INFO:tasks.workunit.client.1.smithi104.stdout:7/343: getdents d2 0 2024-04-26T04:42:15.666 INFO:tasks.workunit.client.1.smithi104.stdout:7/344: write d2/d4/d32/d3d/f50 [48416,42885] 0 2024-04-26T04:42:15.666 INFO:tasks.workunit.client.1.smithi104.stdout:7/345: chown d2/d4/d32/d3d/d41/l4b 1441 1 2024-04-26T04:42:15.667 INFO:tasks.workunit.client.0.smithi090.stdout:6/188: dread d9/d22/f26 [0,4194304] 0 2024-04-26T04:42:15.668 INFO:tasks.workunit.client.0.smithi090.stdout:4/170: symlink d3/l33 0 2024-04-26T04:42:15.668 INFO:tasks.workunit.client.0.smithi090.stdout:4/171: creat d3/db/f34 x:0 0 0 2024-04-26T04:42:15.668 INFO:tasks.workunit.client.0.smithi090.stdout:4/172: readlink d3/db/d12/l29 0 2024-04-26T04:42:15.669 INFO:tasks.workunit.client.1.smithi104.stdout:4/509: rename d4/d7/d1d/d1e/d24/d59/da8 to d4/d7/d1d/d1e/d8f/db6 0 2024-04-26T04:42:15.670 INFO:tasks.workunit.client.1.smithi104.stdout:1/449: truncate d0/d6/d49/f68 1128108 0 2024-04-26T04:42:15.672 INFO:tasks.workunit.client.1.smithi104.stdout:8/257: symlink d0/d50/l52 0 2024-04-26T04:42:15.673 INFO:tasks.workunit.client.0.smithi090.stdout:2/100: sync 2024-04-26T04:42:15.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/95: sync 2024-04-26T04:42:15.674 INFO:tasks.workunit.client.0.smithi090.stdout:3/96: creat d1/d6/f21 x:0 0 0 2024-04-26T04:42:15.674 INFO:tasks.workunit.client.0.smithi090.stdout:7/109: sync 2024-04-26T04:42:15.675 INFO:tasks.workunit.client.0.smithi090.stdout:5/202: mkdir d4/d1a/df/d36/d3e 0 2024-04-26T04:42:15.676 INFO:tasks.workunit.client.1.smithi104.stdout:4/510: rename d4/d7/f42 to d4/d7/d1d/d1e/d24/d59/fb7 0 2024-04-26T04:42:15.677 INFO:tasks.workunit.client.1.smithi104.stdout:7/346: dread d2/f42 [0,4194304] 0 2024-04-26T04:42:15.678 INFO:tasks.workunit.client.1.smithi104.stdout:1/450: link d0/d6/d49/f4a d0/d6/d2e/d32/f92 0 2024-04-26T04:42:15.678 INFO:tasks.workunit.client.0.smithi090.stdout:0/192: rmdir d0/d9 39 2024-04-26T04:42:15.680 INFO:tasks.workunit.client.1.smithi104.stdout:8/258: truncate d0/d1c/d27/d47/f25 1227633 0 2024-04-26T04:42:15.680 INFO:tasks.workunit.client.1.smithi104.stdout:8/259: truncate d0/d1c/d27/f40 604649 0 2024-04-26T04:42:15.680 INFO:tasks.workunit.client.1.smithi104.stdout:8/260: read - d0/d1c/d27/d47/f1d zero size 2024-04-26T04:42:15.682 INFO:tasks.workunit.client.1.smithi104.stdout:7/347: rename d2/c64 to d2/d4/d1d/d34/c72 0 2024-04-26T04:42:15.682 INFO:tasks.workunit.client.1.smithi104.stdout:7/348: fsync d2/d4/f13 0 2024-04-26T04:42:15.683 INFO:tasks.workunit.client.1.smithi104.stdout:1/451: mknod d0/d6/d19/d21/d58/c93 0 2024-04-26T04:42:15.684 INFO:tasks.workunit.client.0.smithi090.stdout:9/87: rmdir db 39 2024-04-26T04:42:15.684 INFO:tasks.workunit.client.0.smithi090.stdout:4/173: creat d3/db/d12/d20/d2d/d2e/f35 x:0 0 0 2024-04-26T04:42:15.684 INFO:tasks.workunit.client.0.smithi090.stdout:4/174: write f0 [346834,10114] 0 2024-04-26T04:42:15.686 INFO:tasks.workunit.client.1.smithi104.stdout:7/349: rename d2/d4/d32/d3d/c4e to d2/d4/d32/d3d/d41/d5f/d6c/c73 0 2024-04-26T04:42:15.688 INFO:tasks.workunit.client.1.smithi104.stdout:8/261: getdents d0/d33/d35 0 2024-04-26T04:42:15.688 INFO:tasks.workunit.client.1.smithi104.stdout:8/262: write d0/d1c/d27/f40 [981936,67266] 0 2024-04-26T04:42:15.688 INFO:tasks.workunit.client.0.smithi090.stdout:3/97: mknod d1/d6/d1b/c22 0 2024-04-26T04:42:15.690 INFO:tasks.workunit.client.1.smithi104.stdout:8/263: mknod d0/c53 0 2024-04-26T04:42:15.691 INFO:tasks.workunit.client.1.smithi104.stdout:8/264: stat d0/d50 0 2024-04-26T04:42:15.691 INFO:tasks.workunit.client.0.smithi090.stdout:2/101: dread db/f16 [0,4194304] 0 2024-04-26T04:42:15.692 INFO:tasks.workunit.client.0.smithi090.stdout:5/203: rename d4/d1a/d1f/d27 to d4/d1a/d3f 0 2024-04-26T04:42:15.693 INFO:tasks.workunit.client.0.smithi090.stdout:0/193: fsync d0/d9/f13 0 2024-04-26T04:42:15.696 INFO:tasks.workunit.client.1.smithi104.stdout:6/570: dwrite d0/d9/d13/d15/fb6 [0,4194304] 0 2024-04-26T04:42:15.696 INFO:tasks.workunit.client.1.smithi104.stdout:6/571: chown d0/d9/d13/d44/d71/d3f/f5d 430 1 2024-04-26T04:42:15.698 INFO:tasks.workunit.client.0.smithi090.stdout:7/110: dread d0/db/d15/d29/f14 [0,4194304] 0 2024-04-26T04:42:15.699 INFO:tasks.workunit.client.0.smithi090.stdout:2/102: link db/d11/c15 db/d11/c1c 0 2024-04-26T04:42:15.699 INFO:tasks.workunit.client.1.smithi104.stdout:6/572: rmdir d0/d9/d13/d15/dba/daa/db8 0 2024-04-26T04:42:15.699 INFO:tasks.workunit.client.1.smithi104.stdout:6/573: creat d0/d9/d13/d15/d80/fc2 x:0 0 0 2024-04-26T04:42:15.701 INFO:tasks.workunit.client.0.smithi090.stdout:4/175: rename d3/db/f34 to d3/f36 0 2024-04-26T04:42:15.702 INFO:tasks.workunit.client.0.smithi090.stdout:4/176: fdatasync d3/db/fc 0 2024-04-26T04:42:15.703 INFO:tasks.workunit.client.0.smithi090.stdout:1/114: dwrite d0/f12 [0,4194304] 0 2024-04-26T04:42:15.703 INFO:tasks.workunit.client.0.smithi090.stdout:5/204: symlink d4/d1a/d3f/l40 0 2024-04-26T04:42:15.703 INFO:tasks.workunit.client.0.smithi090.stdout:5/205: chown d4/d1a/df/f22 1214 1 2024-04-26T04:42:15.704 INFO:tasks.workunit.client.0.smithi090.stdout:5/206: fdatasync d4/d1a/d2a/d31/f34 0 2024-04-26T04:42:15.704 INFO:tasks.workunit.client.0.smithi090.stdout:5/207: fsync d4/d1a/d1f/f3d 0 2024-04-26T04:42:15.705 INFO:tasks.workunit.client.0.smithi090.stdout:9/88: write f2 [3508033,6533] 0 2024-04-26T04:42:15.706 INFO:tasks.workunit.client.1.smithi104.stdout:8/265: write d0/d33/f1f [1637779,29908] 0 2024-04-26T04:42:15.706 INFO:tasks.workunit.client.0.smithi090.stdout:7/111: truncate d0/db/df/d1c/d23/f26 2007460 0 2024-04-26T04:42:15.707 INFO:tasks.workunit.client.0.smithi090.stdout:3/98: write d1/d6/f14 [2875374,114251] 0 2024-04-26T04:42:15.707 INFO:tasks.workunit.client.1.smithi104.stdout:8/266: symlink d0/d1c/d2d/l54 0 2024-04-26T04:42:15.708 INFO:tasks.workunit.client.0.smithi090.stdout:4/177: creat d3/db/d12/d20/d24/d30/f37 x:0 0 0 2024-04-26T04:42:15.708 INFO:tasks.workunit.client.0.smithi090.stdout:4/178: fsync f1 0 2024-04-26T04:42:15.709 INFO:tasks.workunit.client.1.smithi104.stdout:8/267: creat d0/d50/f55 x:0 0 0 2024-04-26T04:42:15.709 INFO:tasks.workunit.client.0.smithi090.stdout:1/115: mknod d0/d4/d7/c25 0 2024-04-26T04:42:15.709 INFO:tasks.workunit.client.0.smithi090.stdout:1/116: chown d0/l6 5671070 1 2024-04-26T04:42:15.710 INFO:tasks.workunit.client.0.smithi090.stdout:1/117: write d0/d4/d7/d17/d22/f23 [669465,105130] 0 2024-04-26T04:42:15.710 INFO:tasks.workunit.client.0.smithi090.stdout:5/208: creat d4/d1a/d2a/f41 x:0 0 0 2024-04-26T04:42:15.710 INFO:tasks.workunit.client.0.smithi090.stdout:5/209: readlink d4/d1a/d3f/l2e 0 2024-04-26T04:42:15.710 INFO:tasks.workunit.client.0.smithi090.stdout:0/194: truncate d0/df/f25 984918 0 2024-04-26T04:42:15.711 INFO:tasks.workunit.client.0.smithi090.stdout:7/112: symlink d0/db/df/l2c 0 2024-04-26T04:42:15.712 INFO:tasks.workunit.client.0.smithi090.stdout:3/99: rename d1/d17/c1d to d1/d17/c23 0 2024-04-26T04:42:15.713 INFO:tasks.workunit.client.0.smithi090.stdout:4/179: link d3/d13/f21 d3/d13/d1d/f38 0 2024-04-26T04:42:15.713 INFO:tasks.workunit.client.0.smithi090.stdout:4/180: creat d3/d13/f39 x:0 0 0 2024-04-26T04:42:15.714 INFO:tasks.workunit.client.0.smithi090.stdout:4/181: chown d3/db/d12/d20 29452971 1 2024-04-26T04:42:15.714 INFO:tasks.workunit.client.0.smithi090.stdout:4/182: creat d3/db/d12/d20/d2d/d2e/f3a x:0 0 0 2024-04-26T04:42:15.714 INFO:tasks.workunit.client.0.smithi090.stdout:4/183: fsync d3/f9 0 2024-04-26T04:42:15.715 INFO:tasks.workunit.client.0.smithi090.stdout:5/210: unlink d4/d1a/f23 0 2024-04-26T04:42:15.715 INFO:tasks.workunit.client.0.smithi090.stdout:5/211: write d4/d1a/d3f/f29 [293692,106701] 0 2024-04-26T04:42:15.716 INFO:tasks.workunit.client.0.smithi090.stdout:3/100: creat d1/d6/d1b/d1e/f24 x:0 0 0 2024-04-26T04:42:15.720 INFO:tasks.workunit.client.0.smithi090.stdout:9/89: write db/f14 [2256596,57033] 0 2024-04-26T04:42:15.721 INFO:tasks.workunit.client.0.smithi090.stdout:5/212: mkdir d4/d1a/df/d36/d42 0 2024-04-26T04:42:15.721 INFO:tasks.workunit.client.0.smithi090.stdout:9/90: dread db/fc [0,4194304] 0 2024-04-26T04:42:15.722 INFO:tasks.workunit.client.0.smithi090.stdout:4/184: mkdir d3/db/d12/d3b 0 2024-04-26T04:42:15.722 INFO:tasks.workunit.client.0.smithi090.stdout:4/185: write d3/db/fc [1770893,99089] 0 2024-04-26T04:42:15.723 INFO:tasks.workunit.client.1.smithi104.stdout:6/574: dread d0/d9/d13/d15/fb6 [0,4194304] 0 2024-04-26T04:42:15.724 INFO:tasks.workunit.client.1.smithi104.stdout:6/575: mknod d0/d9/d13/d41/d96/dac/db3/cc3 0 2024-04-26T04:42:15.724 INFO:tasks.workunit.client.1.smithi104.stdout:6/576: stat d0/d9/d13/d44 0 2024-04-26T04:42:15.724 INFO:tasks.workunit.client.1.smithi104.stdout:6/577: read - d0/d9/d13/d41/f73 zero size 2024-04-26T04:42:15.724 INFO:tasks.workunit.client.1.smithi104.stdout:6/578: truncate d0/d9/d25/f39 636332 0 2024-04-26T04:42:15.725 INFO:tasks.workunit.client.1.smithi104.stdout:8/268: write d0/d33/f2b [389424,78969] 0 2024-04-26T04:42:15.726 INFO:tasks.workunit.client.0.smithi090.stdout:5/213: rename d4/d1a/df/f10 to d4/d1a/f43 0 2024-04-26T04:42:15.726 INFO:tasks.workunit.client.0.smithi090.stdout:8/88: dwrite dc/f1a [0,4194304] 0 2024-04-26T04:42:15.728 INFO:tasks.workunit.client.0.smithi090.stdout:4/186: unlink d3/f2b 0 2024-04-26T04:42:15.728 INFO:tasks.workunit.client.1.smithi104.stdout:8/269: unlink d0/f3f 0 2024-04-26T04:42:15.728 INFO:tasks.workunit.client.1.smithi104.stdout:8/270: truncate d0/d50/f55 335986 0 2024-04-26T04:42:15.729 INFO:tasks.workunit.client.1.smithi104.stdout:8/271: truncate d0/d1c/d2d/f48 3504696 0 2024-04-26T04:42:15.731 INFO:tasks.workunit.client.0.smithi090.stdout:5/214: unlink d4/f3b 0 2024-04-26T04:42:15.731 INFO:tasks.workunit.client.1.smithi104.stdout:8/272: mknod d0/d33/d51/c56 0 2024-04-26T04:42:15.732 INFO:tasks.workunit.client.1.smithi104.stdout:8/273: mknod d0/d33/d35/c57 0 2024-04-26T04:42:15.733 INFO:tasks.workunit.client.0.smithi090.stdout:6/189: dwrite d9/d1c/f2b [0,4194304] 0 2024-04-26T04:42:15.733 INFO:tasks.workunit.client.0.smithi090.stdout:6/190: readlink d9/l33 0 2024-04-26T04:42:15.734 INFO:tasks.workunit.client.0.smithi090.stdout:9/91: dread db/f13 [0,4194304] 0 2024-04-26T04:42:15.735 INFO:tasks.workunit.client.0.smithi090.stdout:9/92: link f6 db/dd/f1d 0 2024-04-26T04:42:15.736 INFO:tasks.workunit.client.0.smithi090.stdout:9/93: mkdir db/d1e 0 2024-04-26T04:42:15.737 INFO:tasks.workunit.client.0.smithi090.stdout:9/94: symlink db/d16/l1f 0 2024-04-26T04:42:15.737 INFO:tasks.workunit.client.0.smithi090.stdout:9/95: write db/dd/f19 [485336,107310] 0 2024-04-26T04:42:15.737 INFO:tasks.workunit.client.1.smithi104.stdout:4/511: dwrite d4/f98 [0,4194304] 0 2024-04-26T04:42:15.738 INFO:tasks.workunit.client.0.smithi090.stdout:8/89: write f4 [3566486,45916] 0 2024-04-26T04:42:15.740 INFO:tasks.workunit.client.0.smithi090.stdout:8/90: creat dc/d12/f20 x:0 0 0 2024-04-26T04:42:15.741 INFO:tasks.workunit.client.1.smithi104.stdout:6/579: dread d0/d1b/f2c [0,4194304] 0 2024-04-26T04:42:15.743 INFO:tasks.workunit.client.1.smithi104.stdout:4/512: mkdir d4/d7/d1d/d1e/d24/d59/db8 0 2024-04-26T04:42:15.745 INFO:tasks.workunit.client.1.smithi104.stdout:4/513: symlink d4/d20/d55/d56/lb9 0 2024-04-26T04:42:15.748 INFO:tasks.workunit.client.1.smithi104.stdout:4/514: link d4/d7/d1d/d1e/fad d4/d7/d1d/d2e/fba 0 2024-04-26T04:42:15.750 INFO:tasks.workunit.client.1.smithi104.stdout:4/515: mkdir d4/d7/d1d/d1e/d24/d4a/dbb 0 2024-04-26T04:42:15.750 INFO:tasks.workunit.client.1.smithi104.stdout:4/516: fsync d4/d7/d1d/f45 0 2024-04-26T04:42:15.751 INFO:tasks.workunit.client.1.smithi104.stdout:4/517: creat d4/d7/d1d/d1e/d24/d59/fbc x:0 0 0 2024-04-26T04:42:15.752 INFO:tasks.workunit.client.1.smithi104.stdout:1/452: dwrite d0/d6/d49/f4a [0,4194304] 0 2024-04-26T04:42:15.752 INFO:tasks.workunit.client.1.smithi104.stdout:5/397: dwrite d0/d1/d4/d2a/f50 [4194304,4194304] 0 2024-04-26T04:42:15.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/191: write d9/f14 [12112710,37398] 0 2024-04-26T04:42:15.753 INFO:tasks.workunit.client.0.smithi090.stdout:1/118: dwrite d0/d4/d7/d17/f21 [0,4194304] 0 2024-04-26T04:42:15.753 INFO:tasks.workunit.client.0.smithi090.stdout:1/119: truncate d0/d4/f20 179005 0 2024-04-26T04:42:15.753 INFO:tasks.workunit.client.0.smithi090.stdout:1/120: creat d0/d4/f26 x:0 0 0 2024-04-26T04:42:15.754 INFO:tasks.workunit.client.1.smithi104.stdout:7/350: dwrite d2/d4/d32/d35/d37/f6f [0,4194304] 0 2024-04-26T04:42:15.754 INFO:tasks.workunit.client.1.smithi104.stdout:7/351: creat d2/d4/f74 x:0 0 0 2024-04-26T04:42:15.754 INFO:tasks.workunit.client.1.smithi104.stdout:7/352: chown d2/d4/d32/d35/l57 11066 1 2024-04-26T04:42:15.754 INFO:tasks.workunit.client.1.smithi104.stdout:7/353: chown d2/d4 7236307 1 2024-04-26T04:42:15.754 INFO:tasks.workunit.client.1.smithi104.stdout:4/518: mkdir d4/d7/d1d/d2e/d74/dbd 0 2024-04-26T04:42:15.754 INFO:tasks.workunit.client.1.smithi104.stdout:4/519: readlink d4/db/d6b/d86/l99 0 2024-04-26T04:42:15.754 INFO:tasks.workunit.client.1.smithi104.stdout:8/274: write d0/f5 [1355656,26826] 0 2024-04-26T04:42:15.756 INFO:tasks.workunit.client.0.smithi090.stdout:6/192: creat d9/d35/f45 x:0 0 0 2024-04-26T04:42:15.756 INFO:tasks.workunit.client.1.smithi104.stdout:5/398: mknod d0/d1/d1e/c86 0 2024-04-26T04:42:15.757 INFO:tasks.workunit.client.0.smithi090.stdout:8/91: dwrite dc/d12/f13 [0,4194304] 0 2024-04-26T04:42:15.757 INFO:tasks.workunit.client.0.smithi090.stdout:3/101: dwrite d1/f8 [0,4194304] 0 2024-04-26T04:42:15.757 INFO:tasks.workunit.client.0.smithi090.stdout:0/195: dwrite d0/d9/f17 [0,4194304] 0 2024-04-26T04:42:15.757 INFO:tasks.workunit.client.0.smithi090.stdout:0/196: fdatasync d0/d26/f27 0 2024-04-26T04:42:15.757 INFO:tasks.workunit.client.0.smithi090.stdout:3/102: creat d1/d17/f25 x:0 0 0 2024-04-26T04:42:15.758 INFO:tasks.workunit.client.1.smithi104.stdout:1/453: creat d0/d6/d19/d21/d59/f94 x:0 0 0 2024-04-26T04:42:15.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/121: link d0/f2 d0/d4/f27 0 2024-04-26T04:42:15.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/122: readlink d0/l6 0 2024-04-26T04:42:15.762 INFO:tasks.workunit.client.1.smithi104.stdout:1/454: dread d0/d6/d19/d21/f6d [0,4194304] 0 2024-04-26T04:42:15.767 INFO:tasks.workunit.client.1.smithi104.stdout:7/354: unlink d2/d4/d1d/d34/f51 0 2024-04-26T04:42:15.767 INFO:tasks.workunit.client.1.smithi104.stdout:1/455: read d0/d6/d2e/f84 [89256,127093] 0 2024-04-26T04:42:15.767 INFO:tasks.workunit.client.1.smithi104.stdout:6/580: dwrite d0/d1b/f24 [0,4194304] 0 2024-04-26T04:42:15.767 INFO:tasks.workunit.client.1.smithi104.stdout:4/520: symlink d4/d7/d1d/lbe 0 2024-04-26T04:42:15.767 INFO:tasks.workunit.client.1.smithi104.stdout:6/581: write d0/d9/d13/d44/d71/fc1 [1041218,111881] 0 2024-04-26T04:42:15.767 INFO:tasks.workunit.client.1.smithi104.stdout:6/582: write d0/d9/d13/d15/d80/faf [1749265,60956] 0 2024-04-26T04:42:15.768 INFO:tasks.workunit.client.0.smithi090.stdout:8/92: creat dc/d1f/f21 x:0 0 0 2024-04-26T04:42:15.768 INFO:tasks.workunit.client.0.smithi090.stdout:8/93: creat dc/d1f/f22 x:0 0 0 2024-04-26T04:42:15.768 INFO:tasks.workunit.client.0.smithi090.stdout:8/94: fdatasync dc/f1a 0 2024-04-26T04:42:15.769 INFO:tasks.workunit.client.0.smithi090.stdout:8/95: fsync dc/f14 0 2024-04-26T04:42:15.769 INFO:tasks.workunit.client.0.smithi090.stdout:2/103: rmdir db/d11 39 2024-04-26T04:42:15.769 INFO:tasks.workunit.client.1.smithi104.stdout:8/275: mknod d0/d1c/d2d/c58 0 2024-04-26T04:42:15.772 INFO:tasks.workunit.client.0.smithi090.stdout:2/104: dread db/dc/f19 [0,4194304] 0 2024-04-26T04:42:15.772 INFO:tasks.workunit.client.0.smithi090.stdout:2/105: readlink l6 0 2024-04-26T04:42:15.772 INFO:tasks.workunit.client.1.smithi104.stdout:1/456: dread d0/d6/d49/f68 [0,4194304] 0 2024-04-26T04:42:15.773 INFO:tasks.workunit.client.1.smithi104.stdout:7/355: truncate d2/d4/f13 678448 0 2024-04-26T04:42:15.775 INFO:tasks.workunit.client.1.smithi104.stdout:8/276: dread d0/d1c/d2d/f32 [0,4194304] 0 2024-04-26T04:42:15.776 INFO:tasks.workunit.client.0.smithi090.stdout:3/103: mknod d1/d6/d1b/d1e/c26 0 2024-04-26T04:42:15.776 INFO:tasks.workunit.client.0.smithi090.stdout:3/104: chown d1/d6/d1b 1478098650 1 2024-04-26T04:42:15.776 INFO:tasks.workunit.client.0.smithi090.stdout:3/105: read - d1/d9/f20 zero size 2024-04-26T04:42:15.776 INFO:tasks.workunit.client.1.smithi104.stdout:4/521: mkdir d4/d20/d55/d56/da3/dbf 0 2024-04-26T04:42:15.776 INFO:tasks.workunit.client.1.smithi104.stdout:4/522: chown d4/d7/d1d/d1e/d8f/db6/f9c 381454 1 2024-04-26T04:42:15.776 INFO:tasks.workunit.client.1.smithi104.stdout:4/523: write d4/d7/d1d/d1e/d8f/db6/d7d/fa1 [752950,16584] 0 2024-04-26T04:42:15.779 INFO:tasks.workunit.client.1.smithi104.stdout:8/277: write d0/d1c/d27/f40 [698079,114466] 0 2024-04-26T04:42:15.779 INFO:tasks.workunit.client.1.smithi104.stdout:6/583: unlink d0/d9/d25/c2e 0 2024-04-26T04:42:15.781 INFO:tasks.workunit.client.1.smithi104.stdout:5/399: mknod d0/d1/c87 0 2024-04-26T04:42:15.781 INFO:tasks.workunit.client.1.smithi104.stdout:5/400: creat d0/f88 x:0 0 0 2024-04-26T04:42:15.782 INFO:tasks.workunit.client.1.smithi104.stdout:8/278: write d0/f1e [653030,120065] 0 2024-04-26T04:42:15.782 INFO:tasks.workunit.client.0.smithi090.stdout:3/106: getdents d1 0 2024-04-26T04:42:15.786 INFO:tasks.workunit.client.0.smithi090.stdout:8/96: rename c2 to dc/d12/c23 0 2024-04-26T04:42:15.786 INFO:tasks.workunit.client.0.smithi090.stdout:8/97: dread - dc/f1d zero size 2024-04-26T04:42:15.786 INFO:tasks.workunit.client.0.smithi090.stdout:8/98: dread - dc/f1d zero size 2024-04-26T04:42:15.788 INFO:tasks.workunit.client.0.smithi090.stdout:1/123: dwrite d0/d4/f27 [0,4194304] 0 2024-04-26T04:42:15.789 INFO:tasks.workunit.client.1.smithi104.stdout:4/524: truncate d4/d7/d1d/f43 209092 0 2024-04-26T04:42:15.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/187: getdents d3/d13/d1d 0 2024-04-26T04:42:15.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/188: write d3/d13/f39 [872392,47209] 0 2024-04-26T04:42:15.790 INFO:tasks.workunit.client.1.smithi104.stdout:6/584: mkdir d0/d9/d13/d41/d96/dac/dc4 0 2024-04-26T04:42:15.790 INFO:tasks.workunit.client.1.smithi104.stdout:4/525: dread d4/d7/d1d/d1e/d24/d59/fb7 [0,4194304] 0 2024-04-26T04:42:15.791 INFO:tasks.workunit.client.1.smithi104.stdout:4/526: readlink d4/d7/d1d/d2e/l79 0 2024-04-26T04:42:15.793 INFO:tasks.workunit.client.1.smithi104.stdout:5/401: mkdir d0/d89 0 2024-04-26T04:42:15.793 INFO:tasks.workunit.client.1.smithi104.stdout:5/402: creat d0/d1/d4/d2a/d2b/d5c/f8a x:0 0 0 2024-04-26T04:42:15.793 INFO:tasks.workunit.client.1.smithi104.stdout:5/403: fsync d0/d1/d4/d2a/f50 0 2024-04-26T04:42:15.793 INFO:tasks.workunit.client.1.smithi104.stdout:5/404: getdents d0/d1/d64/d26/d36/d72 0 2024-04-26T04:42:15.794 INFO:tasks.workunit.client.1.smithi104.stdout:1/457: getdents d0/d6/d19/d21 0 2024-04-26T04:42:15.794 INFO:tasks.workunit.client.1.smithi104.stdout:1/458: truncate d0/d6/d19/d21/d66/d30/d81/f8b 796539 0 2024-04-26T04:42:15.794 INFO:tasks.workunit.client.1.smithi104.stdout:1/459: fdatasync d0/d6/d49/f87 0 2024-04-26T04:42:15.794 INFO:tasks.workunit.client.1.smithi104.stdout:1/460: chown d0/d6/d19/d21/d66/d20/d2a/f78 2 1 2024-04-26T04:42:15.794 INFO:tasks.workunit.client.1.smithi104.stdout:1/461: creat d0/d6/d2e/f95 x:0 0 0 2024-04-26T04:42:15.794 INFO:tasks.workunit.client.1.smithi104.stdout:7/356: dread d2/d4/d1d/f29 [0,4194304] 0 2024-04-26T04:42:15.795 INFO:tasks.workunit.client.1.smithi104.stdout:8/279: mkdir d0/d1c/d59 0 2024-04-26T04:42:15.795 INFO:tasks.workunit.client.1.smithi104.stdout:8/280: stat d0/d1c/d27/f40 0 2024-04-26T04:42:15.797 INFO:tasks.workunit.client.0.smithi090.stdout:9/96: dwrite f2 [0,4194304] 0 2024-04-26T04:42:15.800 INFO:tasks.workunit.client.1.smithi104.stdout:6/585: mknod d0/d9/d13/cc5 0 2024-04-26T04:42:15.800 INFO:tasks.workunit.client.1.smithi104.stdout:6/586: readlink d0/d9/d13/l3a 0 2024-04-26T04:42:15.800 INFO:tasks.workunit.client.1.smithi104.stdout:6/587: chown d0/d1b/c65 0 1 2024-04-26T04:42:15.802 INFO:tasks.workunit.client.0.smithi090.stdout:5/215: rmdir d4/d1a 39 2024-04-26T04:42:15.803 INFO:tasks.workunit.client.1.smithi104.stdout:4/527: unlink d4/d7/d1d/d1e/d24/f81 0 2024-04-26T04:42:15.803 INFO:tasks.workunit.client.1.smithi104.stdout:4/528: readlink d4/d7/l91 0 2024-04-26T04:42:15.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/462: symlink d0/d6/d2e/d32/d6a/l96 0 2024-04-26T04:42:15.807 INFO:tasks.workunit.client.0.smithi090.stdout:4/189: write d3/db/d12/ff [1949304,37060] 0 2024-04-26T04:42:15.808 INFO:tasks.workunit.client.1.smithi104.stdout:7/357: link d2/d4/f11 d2/f75 0 2024-04-26T04:42:15.808 INFO:tasks.workunit.client.1.smithi104.stdout:7/358: dread - d2/f56 zero size 2024-04-26T04:42:15.808 INFO:tasks.workunit.client.1.smithi104.stdout:7/359: fdatasync d2/d4/d32/d35/d37/f6f 0 2024-04-26T04:42:15.808 INFO:tasks.workunit.client.1.smithi104.stdout:7/360: chown d2/d4/l25 10 1 2024-04-26T04:42:15.808 INFO:tasks.workunit.client.0.smithi090.stdout:9/97: link db/dd/f19 db/dd/d18/f20 0 2024-04-26T04:42:15.809 INFO:tasks.workunit.client.1.smithi104.stdout:8/281: symlink d0/d1c/d27/l5a 0 2024-04-26T04:42:15.809 INFO:tasks.workunit.client.1.smithi104.stdout:8/282: chown d0/d1c/d27/d47/f1d 32073594 1 2024-04-26T04:42:15.809 INFO:tasks.workunit.client.1.smithi104.stdout:8/283: chown d0/d1c/d2d/f48 34 1 2024-04-26T04:42:15.809 INFO:tasks.workunit.client.1.smithi104.stdout:8/284: dread - d0/d33/d35/f36 zero size 2024-04-26T04:42:15.809 INFO:tasks.workunit.client.1.smithi104.stdout:8/285: truncate d0/f4e 5862 0 2024-04-26T04:42:15.809 INFO:tasks.workunit.client.1.smithi104.stdout:4/529: dread d4/d7/d1d/d1e/d24/f68 [0,4194304] 0 2024-04-26T04:42:15.810 INFO:tasks.workunit.client.1.smithi104.stdout:6/588: rename d0/d1b to d0/d9/dc6 0 2024-04-26T04:42:15.810 INFO:tasks.workunit.client.1.smithi104.stdout:1/463: readlink d0/l23 0 2024-04-26T04:42:15.811 INFO:tasks.workunit.client.0.smithi090.stdout:4/190: mknod d3/d13/d1d/c3c 0 2024-04-26T04:42:15.811 INFO:tasks.workunit.client.1.smithi104.stdout:1/464: dread d0/d6/d49/f87 [0,4194304] 0 2024-04-26T04:42:15.812 INFO:tasks.workunit.client.0.smithi090.stdout:5/216: creat d4/d1a/d3f/f44 x:0 0 0 2024-04-26T04:42:15.814 INFO:tasks.workunit.client.1.smithi104.stdout:8/286: unlink d0/f3b 0 2024-04-26T04:42:15.814 INFO:tasks.workunit.client.1.smithi104.stdout:8/287: chown d0/c37 2856495 1 2024-04-26T04:42:15.815 INFO:tasks.workunit.client.0.smithi090.stdout:4/191: symlink d3/db/d12/l3d 0 2024-04-26T04:42:15.815 INFO:tasks.workunit.client.0.smithi090.stdout:6/193: rmdir d9/d35 39 2024-04-26T04:42:15.816 INFO:tasks.workunit.client.1.smithi104.stdout:4/530: mknod d4/d7/d1d/d1e/cc0 0 2024-04-26T04:42:15.816 INFO:tasks.workunit.client.1.smithi104.stdout:4/531: truncate d4/db/d6b/d7f/f84 76671 0 2024-04-26T04:42:15.818 INFO:tasks.workunit.client.0.smithi090.stdout:5/217: dread d4/d1a/f43 [0,4194304] 0 2024-04-26T04:42:15.818 INFO:tasks.workunit.client.0.smithi090.stdout:0/197: dwrite d0/f2d [0,4194304] 0 2024-04-26T04:42:15.819 INFO:tasks.workunit.client.0.smithi090.stdout:7/113: dwrite d0/db/df/d1c/d23/f26 [0,4194304] 0 2024-04-26T04:42:15.819 INFO:tasks.workunit.client.0.smithi090.stdout:7/114: read d0/d1/d1f/f17 [42752,52507] 0 2024-04-26T04:42:15.820 INFO:tasks.workunit.client.1.smithi104.stdout:5/405: dwrite d0/d1/d64/f31 [0,4194304] 0 2024-04-26T04:42:15.820 INFO:tasks.workunit.client.1.smithi104.stdout:5/406: stat d0/d1/d4/f17 0 2024-04-26T04:42:15.820 INFO:tasks.workunit.client.1.smithi104.stdout:5/407: stat d0/d1/d4/d2a/d2b/d37/d48/c60 0 2024-04-26T04:42:15.820 INFO:tasks.workunit.client.1.smithi104.stdout:5/408: readlink d0/d1/d64/l1f 0 2024-04-26T04:42:15.820 INFO:tasks.workunit.client.1.smithi104.stdout:5/409: dread - d0/d1/d3e/f59 zero size 2024-04-26T04:42:15.820 INFO:tasks.workunit.client.1.smithi104.stdout:7/361: rename d2/d4/d32/d3d/f71 to d2/d4/d32/f76 0 2024-04-26T04:42:15.820 INFO:tasks.workunit.client.0.smithi090.stdout:3/107: dwrite d1/f11 [0,4194304] 0 2024-04-26T04:42:15.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/218: unlink d4/d1a/df/d21/c38 0 2024-04-26T04:42:15.823 INFO:tasks.workunit.client.1.smithi104.stdout:8/288: stat d0/d33/l23 0 2024-04-26T04:42:15.825 INFO:tasks.workunit.client.1.smithi104.stdout:5/410: symlink d0/d1/d4/d2a/d6b/l8b 0 2024-04-26T04:42:15.825 INFO:tasks.workunit.client.0.smithi090.stdout:0/198: symlink d0/d26/l40 0 2024-04-26T04:42:15.828 INFO:tasks.workunit.client.0.smithi090.stdout:7/115: unlink d0/d1/d1f/f25 0 2024-04-26T04:42:15.828 INFO:tasks.workunit.client.0.smithi090.stdout:7/116: chown d0/d1/d1f 1036031 1 2024-04-26T04:42:15.829 INFO:tasks.workunit.client.0.smithi090.stdout:7/117: creat d0/d1/d1f/f2d x:0 0 0 2024-04-26T04:42:15.829 INFO:tasks.workunit.client.1.smithi104.stdout:1/465: rename d0/d6/d19/d21/d66/d20/l88 to d0/d6/d2e/d32/l97 0 2024-04-26T04:42:15.829 INFO:tasks.workunit.client.1.smithi104.stdout:6/589: rmdir d0/d9/d13/d41/d96/dac/d8e 39 2024-04-26T04:42:15.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/99: getdents dc/d12 0 2024-04-26T04:42:15.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/100: creat dc/d1f/f24 x:0 0 0 2024-04-26T04:42:15.831 INFO:tasks.workunit.client.1.smithi104.stdout:5/411: truncate d0/d1/d4/d2a/d2b/d37/d48/f70 2833254 0 2024-04-26T04:42:15.834 INFO:tasks.workunit.client.1.smithi104.stdout:7/362: rename d2/d4/l27 to d2/d4/d1d/d34/l77 0 2024-04-26T04:42:15.834 INFO:tasks.workunit.client.1.smithi104.stdout:7/363: chown d2/d4/d32/d3d/d47 83771 1 2024-04-26T04:42:15.834 INFO:tasks.workunit.client.1.smithi104.stdout:7/364: dread - d2/d4/d32/f69 zero size 2024-04-26T04:42:15.834 INFO:tasks.workunit.client.1.smithi104.stdout:7/365: stat d2/d4/d32/d35/l57 0 2024-04-26T04:42:15.834 INFO:tasks.workunit.client.1.smithi104.stdout:7/366: readlink d2/d4/l17 0 2024-04-26T04:42:15.835 INFO:tasks.workunit.client.1.smithi104.stdout:8/289: rename d0/d1c/d27/d47/f42 to d0/d1c/d27/f5b 0 2024-04-26T04:42:15.837 INFO:tasks.workunit.client.1.smithi104.stdout:7/367: link d2/d4/d32/d3d/d41/l4b d2/d4/d32/d35/l78 0 2024-04-26T04:42:15.837 INFO:tasks.workunit.client.0.smithi090.stdout:1/124: truncate d0/d4/d7/d17/d22/f23 136222 0 2024-04-26T04:42:15.838 INFO:tasks.workunit.client.1.smithi104.stdout:8/290: creat d0/d1c/d27/f5c x:0 0 0 2024-04-26T04:42:15.839 INFO:tasks.workunit.client.1.smithi104.stdout:2/590: sync 2024-04-26T04:42:15.839 INFO:tasks.workunit.client.0.smithi090.stdout:7/118: link d0/d1/d1f/fa d0/d1/d4/f2e 0 2024-04-26T04:42:15.840 INFO:tasks.workunit.client.0.smithi090.stdout:7/119: chown d0/d1/d1f/f22 2 1 2024-04-26T04:42:15.840 INFO:tasks.workunit.client.1.smithi104.stdout:7/368: unlink d2/f2f 0 2024-04-26T04:42:15.842 INFO:tasks.workunit.client.0.smithi090.stdout:8/101: mknod dc/d12/c25 0 2024-04-26T04:42:15.842 INFO:tasks.workunit.client.1.smithi104.stdout:2/591: creat d6/d1b/d75/fc9 x:0 0 0 2024-04-26T04:42:15.844 INFO:tasks.workunit.client.1.smithi104.stdout:1/466: dread d0/d6/d19/d21/d66/d20/d2a/f31 [0,4194304] 0 2024-04-26T04:42:15.844 INFO:tasks.workunit.client.1.smithi104.stdout:1/467: fsync d0/d6/d49/f4a 0 2024-04-26T04:42:15.845 INFO:tasks.workunit.client.0.smithi090.stdout:0/199: unlink d0/de/c1e 0 2024-04-26T04:42:15.845 INFO:tasks.workunit.client.1.smithi104.stdout:7/369: rename d2/d4/d1d/c4f to d2/d4/d5a/c79 0 2024-04-26T04:42:15.846 INFO:tasks.workunit.client.1.smithi104.stdout:1/468: getdents d0/d6/d2e/d32/d4c 0 2024-04-26T04:42:15.846 INFO:tasks.workunit.client.1.smithi104.stdout:1/469: fdatasync d0/d6/d19/f3a 0 2024-04-26T04:42:15.846 INFO:tasks.workunit.client.1.smithi104.stdout:1/470: readlink d0/d6/d19/d21/d66/l10 0 2024-04-26T04:42:15.847 INFO:tasks.workunit.client.0.smithi090.stdout:7/120: symlink d0/d1/l2f 0 2024-04-26T04:42:15.848 INFO:tasks.workunit.client.1.smithi104.stdout:7/370: creat d2/d4/d32/d35/f7a x:0 0 0 2024-04-26T04:42:15.848 INFO:tasks.workunit.client.1.smithi104.stdout:7/371: fsync d2/d4/d32/f69 0 2024-04-26T04:42:15.849 INFO:tasks.workunit.client.0.smithi090.stdout:0/200: creat d0/d9/f41 x:0 0 0 2024-04-26T04:42:15.849 INFO:tasks.workunit.client.0.smithi090.stdout:0/201: dread - d0/de/f36 zero size 2024-04-26T04:42:15.849 INFO:tasks.workunit.client.1.smithi104.stdout:1/471: mkdir d0/d6/d19/d21/d66/d20/d2a/d98 0 2024-04-26T04:42:15.850 INFO:tasks.workunit.client.0.smithi090.stdout:9/98: dwrite db/f13 [0,4194304] 0 2024-04-26T04:42:15.850 INFO:tasks.workunit.client.0.smithi090.stdout:7/121: mkdir d0/db/d15/d30 0 2024-04-26T04:42:15.851 INFO:tasks.workunit.client.1.smithi104.stdout:7/372: creat d2/d4/d5a/f7b x:0 0 0 2024-04-26T04:42:15.851 INFO:tasks.workunit.client.1.smithi104.stdout:7/373: chown d2/d4/d1d/l2c 376 1 2024-04-26T04:42:15.852 INFO:tasks.workunit.client.0.smithi090.stdout:9/99: mknod db/dd/d18/c21 0 2024-04-26T04:42:15.852 INFO:tasks.workunit.client.0.smithi090.stdout:9/100: stat db/dd 0 2024-04-26T04:42:15.852 INFO:tasks.workunit.client.1.smithi104.stdout:1/472: creat d0/d6/d19/d21/d66/d20/d2a/d56/d8c/f99 x:0 0 0 2024-04-26T04:42:15.853 INFO:tasks.workunit.client.0.smithi090.stdout:9/101: unlink f8 0 2024-04-26T04:42:15.854 INFO:tasks.workunit.client.1.smithi104.stdout:7/374: creat d2/d4/d32/d35/d44/d5b/f7c x:0 0 0 2024-04-26T04:42:15.854 INFO:tasks.workunit.client.1.smithi104.stdout:7/375: readlink d2/d4/d32/d3d/d41/l4b 0 2024-04-26T04:42:15.854 INFO:tasks.workunit.client.1.smithi104.stdout:7/376: rename d2/d4/d32/d3d/d41 to d2/d4/d32/d3d/d41/d5f/d6d/d7d 22 2024-04-26T04:42:15.855 INFO:tasks.workunit.client.1.smithi104.stdout:1/473: mkdir d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a 0 2024-04-26T04:42:15.858 INFO:tasks.workunit.client.1.smithi104.stdout:8/291: write d0/f2f [4134921,81011] 0 2024-04-26T04:42:15.859 INFO:tasks.workunit.client.1.smithi104.stdout:8/292: fsync d0/fe 0 2024-04-26T04:42:15.859 INFO:tasks.workunit.client.0.smithi090.stdout:7/122: dread d0/f8 [0,4194304] 0 2024-04-26T04:42:15.859 INFO:tasks.workunit.client.0.smithi090.stdout:7/123: chown d0/d1/d1f 1578564 1 2024-04-26T04:42:15.859 INFO:tasks.workunit.client.0.smithi090.stdout:7/124: chown d0/db/df/d1c/d23/f26 2 1 2024-04-26T04:42:15.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/125: creat d0/db/df/d1c/d23/f31 x:0 0 0 2024-04-26T04:42:15.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/126: truncate d0/d1/d1f/f22 606047 0 2024-04-26T04:42:15.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/127: stat d0/db/df/d1c/d23/f31 0 2024-04-26T04:42:15.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/128: stat d0/db/d15/d30 0 2024-04-26T04:42:15.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/129: stat d0/db 0 2024-04-26T04:42:15.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/130: readlink d0/db/df/l12 0 2024-04-26T04:42:15.862 INFO:tasks.workunit.client.0.smithi090.stdout:7/131: truncate d0/d1/d1f/fa 6363310 0 2024-04-26T04:42:15.862 INFO:tasks.workunit.client.1.smithi104.stdout:8/293: symlink d0/d1c/l5d 0 2024-04-26T04:42:15.862 INFO:tasks.workunit.client.1.smithi104.stdout:0/371: sync 2024-04-26T04:42:15.862 INFO:tasks.workunit.client.1.smithi104.stdout:0/372: truncate d0/d2/d21/f52 377131 0 2024-04-26T04:42:15.862 INFO:tasks.workunit.client.1.smithi104.stdout:3/482: sync 2024-04-26T04:42:15.862 INFO:tasks.workunit.client.1.smithi104.stdout:9/320: sync 2024-04-26T04:42:15.862 INFO:tasks.workunit.client.1.smithi104.stdout:9/321: creat d11/d15/d32/d4c/d54/f62 x:0 0 0 2024-04-26T04:42:15.864 INFO:tasks.workunit.client.1.smithi104.stdout:0/373: mknod d0/d2/d21/d27/c79 0 2024-04-26T04:42:15.867 INFO:tasks.workunit.client.1.smithi104.stdout:9/322: mknod d11/c63 0 2024-04-26T04:42:15.867 INFO:tasks.workunit.client.1.smithi104.stdout:9/323: write d11/d15/d1b/f37 [107844,46050] 0 2024-04-26T04:42:15.867 INFO:tasks.workunit.client.1.smithi104.stdout:9/324: write d11/f20 [635190,6124] 0 2024-04-26T04:42:15.868 INFO:tasks.workunit.client.1.smithi104.stdout:3/483: getdents d5/d9/d17/d46/d95 0 2024-04-26T04:42:15.868 INFO:tasks.workunit.client.1.smithi104.stdout:3/484: chown d5/d9/cd 1 1 2024-04-26T04:42:15.870 INFO:tasks.workunit.client.1.smithi104.stdout:3/485: truncate d5/d9/d80/f94 753181 0 2024-04-26T04:42:15.870 INFO:tasks.workunit.client.1.smithi104.stdout:3/486: fdatasync d5/d9/dc/f36 0 2024-04-26T04:42:15.873 INFO:tasks.workunit.client.1.smithi104.stdout:9/325: dread d11/d15/d32/d35/f3a [0,4194304] 0 2024-04-26T04:42:15.874 INFO:tasks.workunit.client.0.smithi090.stdout:6/194: dwrite d9/d1c/f2b [4194304,4194304] 0 2024-04-26T04:42:15.875 INFO:tasks.workunit.client.0.smithi090.stdout:3/108: dwrite d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:15.875 INFO:tasks.workunit.client.0.smithi090.stdout:1/125: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:15.875 INFO:tasks.workunit.client.1.smithi104.stdout:3/487: link d5/d9/d17/d72/d85/f9f d5/d9/dc/d29/faa 0 2024-04-26T04:42:15.875 INFO:tasks.workunit.client.1.smithi104.stdout:7/377: write d2/d4/f26 [676705,99864] 0 2024-04-26T04:42:15.875 INFO:tasks.workunit.client.1.smithi104.stdout:7/378: read d2/f2b [2285773,21036] 0 2024-04-26T04:42:15.875 INFO:tasks.workunit.client.1.smithi104.stdout:7/379: truncate d2/d4/d1d/f3e 121715 0 2024-04-26T04:42:15.876 INFO:tasks.workunit.client.1.smithi104.stdout:9/326: unlink fc 0 2024-04-26T04:42:15.877 INFO:tasks.workunit.client.1.smithi104.stdout:9/327: dread - d11/f44 zero size 2024-04-26T04:42:15.877 INFO:tasks.workunit.client.1.smithi104.stdout:4/532: dread d4/d7/d1d/f43 [0,4194304] 0 2024-04-26T04:42:15.878 INFO:tasks.workunit.client.0.smithi090.stdout:9/102: read f2 [2703702,32252] 0 2024-04-26T04:42:15.878 INFO:tasks.workunit.client.0.smithi090.stdout:9/103: chown db/f13 0 1 2024-04-26T04:42:15.879 INFO:tasks.workunit.client.0.smithi090.stdout:8/102: write dc/f1a [1223619,5103] 0 2024-04-26T04:42:15.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/192: dwrite d3/f36 [0,4194304] 0 2024-04-26T04:42:15.881 INFO:tasks.workunit.client.0.smithi090.stdout:6/195: creat d9/d21/d3b/f46 x:0 0 0 2024-04-26T04:42:15.881 INFO:tasks.workunit.client.1.smithi104.stdout:4/533: rmdir d4/d7/d1d/d1e/d24/d4a/d9e 39 2024-04-26T04:42:15.881 INFO:tasks.workunit.client.1.smithi104.stdout:2/592: dwrite d6/d8/f17 [4194304,4194304] 0 2024-04-26T04:42:15.881 INFO:tasks.workunit.client.1.smithi104.stdout:2/593: chown d6/d8/d5a/f54 3 1 2024-04-26T04:42:15.882 INFO:tasks.workunit.client.1.smithi104.stdout:6/590: dread d0/d9/d13/d44/d71/fc1 [0,4194304] 0 2024-04-26T04:42:15.887 INFO:tasks.workunit.client.1.smithi104.stdout:2/594: symlink d6/d1b/d51/lca 0 2024-04-26T04:42:15.889 INFO:tasks.workunit.client.1.smithi104.stdout:3/488: write d5/d9/dc/f42 [45404,117754] 0 2024-04-26T04:42:15.889 INFO:tasks.workunit.client.1.smithi104.stdout:3/489: stat d5/d9/d17/d60/c76 0 2024-04-26T04:42:15.890 INFO:tasks.workunit.client.1.smithi104.stdout:2/595: mknod d6/de/ccb 0 2024-04-26T04:42:15.890 INFO:tasks.workunit.client.1.smithi104.stdout:4/534: dread d4/db/f28 [0,4194304] 0 2024-04-26T04:42:15.892 INFO:tasks.workunit.client.0.smithi090.stdout:1/126: mknod d0/c28 0 2024-04-26T04:42:15.892 INFO:tasks.workunit.client.0.smithi090.stdout:7/132: getdents d0/d1/d1f 0 2024-04-26T04:42:15.893 INFO:tasks.workunit.client.1.smithi104.stdout:0/374: read d0/d4/d3c/f4b [335211,67829] 0 2024-04-26T04:42:15.894 INFO:tasks.workunit.client.1.smithi104.stdout:4/535: mkdir d4/d7/d1d/d1e/d24/d59/dc1 0 2024-04-26T04:42:15.895 INFO:tasks.workunit.client.1.smithi104.stdout:8/294: dwrite d0/f4e [0,4194304] 0 2024-04-26T04:42:15.895 INFO:tasks.workunit.client.1.smithi104.stdout:8/295: truncate d0/d33/f2c 475142 0 2024-04-26T04:42:15.895 INFO:tasks.workunit.client.1.smithi104.stdout:1/474: dwrite d0/d6/d2e/f7c [0,4194304] 0 2024-04-26T04:42:15.895 INFO:tasks.workunit.client.1.smithi104.stdout:8/296: dread - d0/d1c/f4c zero size 2024-04-26T04:42:15.895 INFO:tasks.workunit.client.1.smithi104.stdout:1/475: dread - d0/d6/d19/d21/d59/f94 zero size 2024-04-26T04:42:15.897 INFO:tasks.workunit.client.1.smithi104.stdout:3/490: rmdir d5/d9/d80 39 2024-04-26T04:42:15.897 INFO:tasks.workunit.client.1.smithi104.stdout:4/536: mknod d4/d7/d1d/d1e/d24/d59/db8/cc2 0 2024-04-26T04:42:15.897 INFO:tasks.workunit.client.1.smithi104.stdout:4/537: chown d4/d7/d1d/d2e/l79 312947699 1 2024-04-26T04:42:15.897 INFO:tasks.workunit.client.1.smithi104.stdout:4/538: write d4/d7/d1d/d1e/d24/f68 [1985346,103851] 0 2024-04-26T04:42:15.898 INFO:tasks.workunit.client.1.smithi104.stdout:6/591: write d0/d9/d13/d41/f7a [779644,79229] 0 2024-04-26T04:42:15.902 INFO:tasks.workunit.client.0.smithi090.stdout:5/219: rename d4/d1a/df/d21 to d4/d1a/d45 0 2024-04-26T04:42:15.904 INFO:tasks.workunit.client.1.smithi104.stdout:1/476: symlink d0/d6/d19/d21/d58/l9b 0 2024-04-26T04:42:15.907 INFO:tasks.workunit.client.1.smithi104.stdout:4/539: creat d4/d7/d1d/d8e/fc3 x:0 0 0 2024-04-26T04:42:15.907 INFO:tasks.workunit.client.0.smithi090.stdout:8/103: write dc/d12/f17 [1142054,117162] 0 2024-04-26T04:42:15.908 INFO:tasks.workunit.client.1.smithi104.stdout:4/540: write d4/d7/d1d/d1e/d24/d48/f60 [1471227,66455] 0 2024-04-26T04:42:15.908 INFO:tasks.workunit.client.1.smithi104.stdout:5/412: dread d0/d1/d4/d2a/d2b/d37/d48/f70 [0,4194304] 0 2024-04-26T04:42:15.908 INFO:tasks.workunit.client.1.smithi104.stdout:5/413: creat d0/d1/d4/f8c x:0 0 0 2024-04-26T04:42:15.908 INFO:tasks.workunit.client.1.smithi104.stdout:5/414: write d0/d1/d3e/f79 [593518,43370] 0 2024-04-26T04:42:15.909 INFO:tasks.workunit.client.1.smithi104.stdout:6/592: mknod d0/d9/d13/cc7 0 2024-04-26T04:42:15.911 INFO:tasks.workunit.client.0.smithi090.stdout:3/109: rmdir d1/d17 39 2024-04-26T04:42:15.915 INFO:tasks.workunit.client.1.smithi104.stdout:3/491: dread f4 [4194304,4194304] 0 2024-04-26T04:42:15.917 INFO:tasks.workunit.client.0.smithi090.stdout:1/127: creat d0/d4/d7/d17/f29 x:0 0 0 2024-04-26T04:42:15.918 INFO:tasks.workunit.client.0.smithi090.stdout:7/133: creat d0/db/d15/d30/f32 x:0 0 0 2024-04-26T04:42:15.920 INFO:tasks.workunit.client.0.smithi090.stdout:5/220: unlink d4/d1a/d1f/l30 0 2024-04-26T04:42:15.921 INFO:tasks.workunit.client.1.smithi104.stdout:0/375: dread d0/fe [0,4194304] 0 2024-04-26T04:42:15.924 INFO:tasks.workunit.client.0.smithi090.stdout:3/110: creat d1/f27 x:0 0 0 2024-04-26T04:42:15.924 INFO:tasks.workunit.client.0.smithi090.stdout:3/111: dread - d1/d17/f1a zero size 2024-04-26T04:42:15.924 INFO:tasks.workunit.client.1.smithi104.stdout:1/477: getdents d0/d6/d19/d21/d66/d30/d81 0 2024-04-26T04:42:15.924 INFO:tasks.workunit.client.1.smithi104.stdout:5/415: getdents d0/d1/d4/d2a/d2b 0 2024-04-26T04:42:15.924 INFO:tasks.workunit.client.1.smithi104.stdout:4/541: read d4/d7/d1d/d1e/d24/f76 [78654,4503] 0 2024-04-26T04:42:15.925 INFO:tasks.workunit.client.1.smithi104.stdout:0/376: dread d0/d2/d21/d27/d1e/f2c [0,4194304] 0 2024-04-26T04:42:15.925 INFO:tasks.workunit.client.1.smithi104.stdout:0/377: readlink d0/d2/l9 0 2024-04-26T04:42:15.925 INFO:tasks.workunit.client.0.smithi090.stdout:7/134: mknod d0/db/d15/d29/c33 0 2024-04-26T04:42:15.928 INFO:tasks.workunit.client.1.smithi104.stdout:0/378: getdents d0/d2/d21/d49/d62 0 2024-04-26T04:42:15.929 INFO:tasks.workunit.client.0.smithi090.stdout:8/104: rename dc/d12/l1c to dc/l26 0 2024-04-26T04:42:15.929 INFO:tasks.workunit.client.0.smithi090.stdout:8/105: creat dc/d1f/f27 x:0 0 0 2024-04-26T04:42:15.929 INFO:tasks.workunit.client.0.smithi090.stdout:8/106: chown dc/d1f/f27 63 1 2024-04-26T04:42:15.929 INFO:tasks.workunit.client.0.smithi090.stdout:1/128: dread d0/f1c [0,4194304] 0 2024-04-26T04:42:15.930 INFO:tasks.workunit.client.1.smithi104.stdout:6/593: rename d0/d9/d13/d15 to d0/d9/d13/d41/d82/dc8 0 2024-04-26T04:42:15.930 INFO:tasks.workunit.client.1.smithi104.stdout:6/594: readlink d0/d9/d13/d41/d96/dac/d8e/la0 0 2024-04-26T04:42:15.931 INFO:tasks.workunit.client.0.smithi090.stdout:3/112: symlink d1/d6/d1b/d1e/l28 0 2024-04-26T04:42:15.931 INFO:tasks.workunit.client.0.smithi090.stdout:3/113: readlink d1/l1c 0 2024-04-26T04:42:15.931 INFO:tasks.workunit.client.0.smithi090.stdout:3/114: chown d1/d6/d1b/d1e 6112094 1 2024-04-26T04:42:15.931 INFO:tasks.workunit.client.1.smithi104.stdout:6/595: symlink d0/d9/d13/d41/d96/dac/d8e/dbd/lc9 0 2024-04-26T04:42:15.932 INFO:tasks.workunit.client.1.smithi104.stdout:6/596: fdatasync d0/d9/d13/f3d 0 2024-04-26T04:42:15.932 INFO:tasks.workunit.client.1.smithi104.stdout:6/597: chown d0/d9/d13/d44/d71/c26 0 1 2024-04-26T04:42:15.932 INFO:tasks.workunit.client.0.smithi090.stdout:7/135: mknod d0/db/df/c34 0 2024-04-26T04:42:15.932 INFO:tasks.workunit.client.0.smithi090.stdout:7/136: read d0/db/d15/d29/f18 [4010159,36970] 0 2024-04-26T04:42:15.933 INFO:tasks.workunit.client.0.smithi090.stdout:3/115: rmdir d1/d17 39 2024-04-26T04:42:15.933 INFO:tasks.workunit.client.1.smithi104.stdout:3/492: rename d5/d9/dc/d29/d49/d59/f7b to d5/d9/dc/d69/fab 0 2024-04-26T04:42:15.934 INFO:tasks.workunit.client.0.smithi090.stdout:3/116: truncate d1/fc 863797 0 2024-04-26T04:42:15.934 INFO:tasks.workunit.client.0.smithi090.stdout:3/117: write d1/d6/f14 [3277993,109302] 0 2024-04-26T04:42:15.935 INFO:tasks.workunit.client.1.smithi104.stdout:6/598: mkdir d0/d9/d13/d41/d82/dc8/d80/dca 0 2024-04-26T04:42:15.935 INFO:tasks.workunit.client.1.smithi104.stdout:5/416: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:15.936 INFO:tasks.workunit.client.1.smithi104.stdout:5/417: chown d0/d1/d3e/f79 1377556 1 2024-04-26T04:42:15.937 INFO:tasks.workunit.client.1.smithi104.stdout:6/599: symlink d0/d9/d13/d44/lcb 0 2024-04-26T04:42:15.937 INFO:tasks.workunit.client.1.smithi104.stdout:6/600: chown d0/d9/d13/d41/d96/dac/db3 1880387 1 2024-04-26T04:42:15.937 INFO:tasks.workunit.client.0.smithi090.stdout:7/137: dread d0/d1/d1f/f17 [0,4194304] 0 2024-04-26T04:42:15.938 INFO:tasks.workunit.client.1.smithi104.stdout:4/542: write d4/d7/d1d/d1e/d24/d48/f4e [1409528,15867] 0 2024-04-26T04:42:15.939 INFO:tasks.workunit.client.1.smithi104.stdout:4/543: creat d4/d20/d55/d56/da3/fc4 x:0 0 0 2024-04-26T04:42:15.939 INFO:tasks.workunit.client.1.smithi104.stdout:4/544: readlink d4/d7/d1d/d1e/d24/l29 0 2024-04-26T04:42:15.941 INFO:tasks.workunit.client.1.smithi104.stdout:5/418: rename d0/d1/d64/f77 to d0/d7b/f8d 0 2024-04-26T04:42:15.941 INFO:tasks.workunit.client.1.smithi104.stdout:6/601: rename d0/d9/d13/d44/d71/d3f/l78 to d0/d9/d13/d44/d71/d3f/lcc 0 2024-04-26T04:42:15.946 INFO:tasks.workunit.client.1.smithi104.stdout:4/545: creat d4/d20/d55/d56/da3/dbf/fc5 x:0 0 0 2024-04-26T04:42:15.947 INFO:tasks.workunit.client.0.smithi090.stdout:8/107: write fa [3713586,81622] 0 2024-04-26T04:42:15.948 INFO:tasks.workunit.client.1.smithi104.stdout:5/419: link d0/d1/d4/d2a/d2b/d5c/f8a d0/d1/d4/d2a/d2b/d37/f8e 0 2024-04-26T04:42:15.950 INFO:tasks.workunit.client.1.smithi104.stdout:5/420: truncate d0/d1/d64/f19 391123 0 2024-04-26T04:42:15.951 INFO:tasks.workunit.client.0.smithi090.stdout:8/108: creat dc/d12/f28 x:0 0 0 2024-04-26T04:42:15.951 INFO:tasks.workunit.client.0.smithi090.stdout:8/109: truncate dc/f14 456091 0 2024-04-26T04:42:15.956 INFO:tasks.workunit.client.0.smithi090.stdout:5/221: write d4/fe [46381,116264] 0 2024-04-26T04:42:15.958 INFO:tasks.workunit.client.0.smithi090.stdout:4/193: dwrite d3/f36 [0,4194304] 0 2024-04-26T04:42:15.958 INFO:tasks.workunit.client.0.smithi090.stdout:4/194: read - d3/db/d12/d20/d24/d30/f37 zero size 2024-04-26T04:42:15.962 INFO:tasks.workunit.client.1.smithi104.stdout:2/596: dwrite d6/d1b/d51/d6a/fb6 [0,4194304] 0 2024-04-26T04:42:15.964 INFO:tasks.workunit.client.0.smithi090.stdout:1/129: read d0/f12 [4940886,18564] 0 2024-04-26T04:42:15.965 INFO:tasks.workunit.client.1.smithi104.stdout:2/597: creat d6/de/d85/fcc x:0 0 0 2024-04-26T04:42:15.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/130: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:15.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/131: chown d0/d4/d7/c25 6337 1 2024-04-26T04:42:15.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/104: dwrite db/f14 [0,4194304] 0 2024-04-26T04:42:15.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/105: fsync db/dd/f19 0 2024-04-26T04:42:15.971 INFO:tasks.workunit.client.0.smithi090.stdout:9/106: symlink db/dd/l22 0 2024-04-26T04:42:15.973 INFO:tasks.workunit.client.0.smithi090.stdout:9/107: link db/d16/l1c db/d16/l23 0 2024-04-26T04:42:15.974 INFO:tasks.workunit.client.0.smithi090.stdout:9/108: symlink db/dd/l24 0 2024-04-26T04:42:15.983 INFO:tasks.workunit.client.1.smithi104.stdout:5/421: dread d0/d1/f2 [4194304,4194304] 0 2024-04-26T04:42:15.983 INFO:tasks.workunit.client.1.smithi104.stdout:5/422: write d0/d1/d3e/f68 [1038605,44816] 0 2024-04-26T04:42:15.983 INFO:tasks.workunit.client.1.smithi104.stdout:5/423: write d0/d1/d1e/f56 [114730,37840] 0 2024-04-26T04:42:15.985 INFO:tasks.workunit.client.1.smithi104.stdout:5/424: getdents d0/d1/d3e 0 2024-04-26T04:42:15.985 INFO:tasks.workunit.client.1.smithi104.stdout:5/425: fdatasync d0/d1/d4/d2a/d2b/d37/f8e 0 2024-04-26T04:42:15.985 INFO:tasks.workunit.client.1.smithi104.stdout:8/297: dwrite d0/f49 [0,4194304] 0 2024-04-26T04:42:15.985 INFO:tasks.workunit.client.1.smithi104.stdout:5/426: truncate d0/f58 313565 0 2024-04-26T04:42:15.986 INFO:tasks.workunit.client.1.smithi104.stdout:5/427: fsync d0/d1/d4/f24 0 2024-04-26T04:42:15.987 INFO:tasks.workunit.client.1.smithi104.stdout:5/428: write d0/d1/d4/f24 [673118,20969] 0 2024-04-26T04:42:15.988 INFO:tasks.workunit.client.1.smithi104.stdout:5/429: link d0/d1/d64/f5b d0/d1/d1e/f8f 0 2024-04-26T04:42:15.988 INFO:tasks.workunit.client.1.smithi104.stdout:5/430: getdents d0/d1/d3e 0 2024-04-26T04:42:15.989 INFO:tasks.workunit.client.1.smithi104.stdout:5/431: mkdir d0/d1/d4/d90 0 2024-04-26T04:42:15.989 INFO:tasks.workunit.client.1.smithi104.stdout:8/298: dread d0/f2f [0,4194304] 0 2024-04-26T04:42:15.991 INFO:tasks.workunit.client.1.smithi104.stdout:5/432: mkdir d0/d1/d4/d2a/d6b/d91 0 2024-04-26T04:42:15.991 INFO:tasks.workunit.client.1.smithi104.stdout:5/433: creat d0/d1/d64/f92 x:0 0 0 2024-04-26T04:42:15.991 INFO:tasks.workunit.client.1.smithi104.stdout:5/434: chown d0/d1/d4/d2a/d2b/d37/f57 16020459 1 2024-04-26T04:42:15.991 INFO:tasks.workunit.client.1.smithi104.stdout:5/435: stat d0/d1/d3e/f51 0 2024-04-26T04:42:16.003 INFO:tasks.workunit.client.1.smithi104.stdout:8/299: dread d0/d33/f28 [0,4194304] 0 2024-04-26T04:42:16.003 INFO:tasks.workunit.client.1.smithi104.stdout:8/300: chown d0/d1c/d27/f40 738 1 2024-04-26T04:42:16.003 INFO:tasks.workunit.client.1.smithi104.stdout:1/478: dwrite d0/d6/d49/f68 [0,4194304] 0 2024-04-26T04:42:16.003 INFO:tasks.workunit.client.1.smithi104.stdout:1/479: fdatasync d0/d6/d2e/f7c 0 2024-04-26T04:42:16.004 INFO:tasks.workunit.client.1.smithi104.stdout:8/301: mknod d0/c5e 0 2024-04-26T04:42:16.005 INFO:tasks.workunit.client.0.smithi090.stdout:3/118: dwrite d1/d6/d1b/d1e/f24 [0,4194304] 0 2024-04-26T04:42:16.005 INFO:tasks.workunit.client.0.smithi090.stdout:3/119: chown d1/cd 2649 1 2024-04-26T04:42:16.005 INFO:tasks.workunit.client.0.smithi090.stdout:3/120: creat d1/f29 x:0 0 0 2024-04-26T04:42:16.005 INFO:tasks.workunit.client.0.smithi090.stdout:3/121: chown d1/d9/c19 5 1 2024-04-26T04:42:16.006 INFO:tasks.workunit.client.1.smithi104.stdout:1/480: mknod d0/d6/d19/d21/d66/d30/d7e/c9c 0 2024-04-26T04:42:16.006 INFO:tasks.workunit.client.1.smithi104.stdout:5/436: dread d0/d1/d64/d26/d36/f43 [0,4194304] 0 2024-04-26T04:42:16.006 INFO:tasks.workunit.client.1.smithi104.stdout:5/437: chown d0/d1/d4/c7e 9295671 1 2024-04-26T04:42:16.007 INFO:tasks.workunit.client.1.smithi104.stdout:5/438: creat d0/f93 x:0 0 0 2024-04-26T04:42:16.007 INFO:tasks.workunit.client.1.smithi104.stdout:8/302: getdents d0/d33/d51 0 2024-04-26T04:42:16.007 INFO:tasks.workunit.client.1.smithi104.stdout:8/303: write d0/d1c/d2d/f48 [422645,51392] 0 2024-04-26T04:42:16.007 INFO:tasks.workunit.client.1.smithi104.stdout:8/304: write d0/f5 [4800761,43145] 0 2024-04-26T04:42:16.007 INFO:tasks.workunit.client.1.smithi104.stdout:1/481: mkdir d0/d6/d19/d21/d66/d30/d81/d9d 0 2024-04-26T04:42:16.008 INFO:tasks.workunit.client.1.smithi104.stdout:5/439: rmdir d0/d1/d64/d26 39 2024-04-26T04:42:16.008 INFO:tasks.workunit.client.1.smithi104.stdout:8/305: rmdir d0/d33/d35 39 2024-04-26T04:42:16.009 INFO:tasks.workunit.client.1.smithi104.stdout:1/482: mkdir d0/d6/d9e 0 2024-04-26T04:42:16.010 INFO:tasks.workunit.client.1.smithi104.stdout:1/483: readlink d0/d6/d19/d21/d66/d30/l6c 0 2024-04-26T04:42:16.010 INFO:tasks.workunit.client.1.smithi104.stdout:1/484: stat d0/d6/d19/d21/d66/d20 0 2024-04-26T04:42:16.010 INFO:tasks.workunit.client.1.smithi104.stdout:5/440: rename d0/d1/d64/l1f to d0/d1/d1e/l94 0 2024-04-26T04:42:16.011 INFO:tasks.workunit.client.1.smithi104.stdout:8/306: symlink d0/d1c/d27/l5f 0 2024-04-26T04:42:16.012 INFO:tasks.workunit.client.1.smithi104.stdout:5/441: mkdir d0/d1/d4/d2a/d6b/d73/d95 0 2024-04-26T04:42:16.012 INFO:tasks.workunit.client.1.smithi104.stdout:8/307: symlink d0/d33/l60 0 2024-04-26T04:42:16.014 INFO:tasks.workunit.client.1.smithi104.stdout:0/379: dwrite d0/d2/d21/f52 [0,4194304] 0 2024-04-26T04:42:16.014 INFO:tasks.workunit.client.0.smithi090.stdout:5/222: dwrite d4/d1a/df/f28 [0,4194304] 0 2024-04-26T04:42:16.015 INFO:tasks.workunit.client.1.smithi104.stdout:5/442: truncate d0/f1d 3325036 0 2024-04-26T04:42:16.015 INFO:tasks.workunit.client.1.smithi104.stdout:9/328: dwrite d11/d15/d1b/f37 [0,4194304] 0 2024-04-26T04:42:16.015 INFO:tasks.workunit.client.1.smithi104.stdout:8/308: symlink d0/d33/d35/l61 0 2024-04-26T04:42:16.016 INFO:tasks.workunit.client.0.smithi090.stdout:4/195: truncate d3/d13/f39 808081 0 2024-04-26T04:42:16.016 INFO:tasks.workunit.client.0.smithi090.stdout:3/122: write d1/f8 [3329101,49862] 0 2024-04-26T04:42:16.016 INFO:tasks.workunit.client.0.smithi090.stdout:6/196: dwrite d9/d21/d3b/f46 [0,4194304] 0 2024-04-26T04:42:16.016 INFO:tasks.workunit.client.0.smithi090.stdout:3/123: fsync d1/d6/f14 0 2024-04-26T04:42:16.016 INFO:tasks.workunit.client.0.smithi090.stdout:3/124: write d1/d17/f1f [986462,114979] 0 2024-04-26T04:42:16.017 INFO:tasks.workunit.client.1.smithi104.stdout:0/380: link d0/d2/d21/d27/d1e/d57/d6c/l25 d0/d2/d21/d27/d1e/d57/l7a 0 2024-04-26T04:42:16.017 INFO:tasks.workunit.client.0.smithi090.stdout:5/223: dread d4/d1a/d3f/f29 [0,4194304] 0 2024-04-26T04:42:16.017 INFO:tasks.workunit.client.0.smithi090.stdout:1/132: truncate d0/d4/f27 3578782 0 2024-04-26T04:42:16.018 INFO:tasks.workunit.client.1.smithi104.stdout:4/546: dwrite d4/d20/d55/d56/f3e [0,4194304] 0 2024-04-26T04:42:16.019 INFO:tasks.workunit.client.1.smithi104.stdout:5/443: symlink d0/d1/d64/d26/l96 0 2024-04-26T04:42:16.020 INFO:tasks.workunit.client.0.smithi090.stdout:4/196: dread d3/db/fc [0,4194304] 0 2024-04-26T04:42:16.020 INFO:tasks.workunit.client.0.smithi090.stdout:4/197: truncate d3/f27 766836 0 2024-04-26T04:42:16.020 INFO:tasks.workunit.client.0.smithi090.stdout:4/198: truncate d3/db/d12/d20/f2a 437399 0 2024-04-26T04:42:16.020 INFO:tasks.workunit.client.0.smithi090.stdout:7/138: dwrite d0/db/d15/d29/f2b [0,4194304] 0 2024-04-26T04:42:16.021 INFO:tasks.workunit.client.1.smithi104.stdout:6/602: dwrite d0/d9/d13/f43 [0,4194304] 0 2024-04-26T04:42:16.021 INFO:tasks.workunit.client.1.smithi104.stdout:6/603: fsync d0/d9/d13/d41/d96/dac/d8e/fb4 0 2024-04-26T04:42:16.021 INFO:tasks.workunit.client.0.smithi090.stdout:8/110: dwrite dc/d12/f20 [0,4194304] 0 2024-04-26T04:42:16.022 INFO:tasks.workunit.client.0.smithi090.stdout:9/109: dwrite db/dd/f19 [0,4194304] 0 2024-04-26T04:42:16.022 INFO:tasks.workunit.client.0.smithi090.stdout:8/111: write dc/f1e [158149,32400] 0 2024-04-26T04:42:16.022 INFO:tasks.workunit.client.0.smithi090.stdout:8/112: dread - dc/d1f/f22 zero size 2024-04-26T04:42:16.023 INFO:tasks.workunit.client.1.smithi104.stdout:1/485: dread d0/d6/d2e/f7c [0,4194304] 0 2024-04-26T04:42:16.023 INFO:tasks.workunit.client.1.smithi104.stdout:2/598: dwrite d6/d8/d5a/f54 [0,4194304] 0 2024-04-26T04:42:16.023 INFO:tasks.workunit.client.1.smithi104.stdout:1/486: dread - d0/d6/d19/d21/d66/d20/d2a/d56/d8c/f99 zero size 2024-04-26T04:42:16.024 INFO:tasks.workunit.client.1.smithi104.stdout:8/309: rmdir d0/d1c/d2d 39 2024-04-26T04:42:16.025 INFO:tasks.workunit.client.0.smithi090.stdout:6/197: unlink d9/d1c/l1f 0 2024-04-26T04:42:16.030 INFO:tasks.workunit.client.1.smithi104.stdout:2/599: dread d6/d8/f37 [0,4194304] 0 2024-04-26T04:42:16.035 INFO:tasks.workunit.client.0.smithi090.stdout:3/125: truncate d1/d9/f20 3097018 0 2024-04-26T04:42:16.035 INFO:tasks.workunit.client.0.smithi090.stdout:3/126: write d1/d9/fa [2851033,2346] 0 2024-04-26T04:42:16.037 INFO:tasks.workunit.client.0.smithi090.stdout:5/224: rmdir d4/d1a/df/d36/d3e 0 2024-04-26T04:42:16.038 INFO:tasks.workunit.client.1.smithi104.stdout:4/547: mkdir d4/d7/d1d/d1e/d15/dc6 0 2024-04-26T04:42:16.039 INFO:tasks.workunit.client.0.smithi090.stdout:1/133: creat d0/d4/d7/d17/f2a x:0 0 0 2024-04-26T04:42:16.039 INFO:tasks.workunit.client.1.smithi104.stdout:5/444: mkdir d0/d1/d4/d90/d97 0 2024-04-26T04:42:16.040 INFO:tasks.workunit.client.0.smithi090.stdout:4/199: mkdir d3/d3e 0 2024-04-26T04:42:16.043 INFO:tasks.workunit.client.0.smithi090.stdout:8/113: dwrite dc/d12/f20 [0,4194304] 0 2024-04-26T04:42:16.044 INFO:tasks.workunit.client.1.smithi104.stdout:2/600: write d6/d8/d5a/fb0 [2335653,128901] 0 2024-04-26T04:42:16.044 INFO:tasks.workunit.client.1.smithi104.stdout:2/601: creat d6/d8/fcd x:0 0 0 2024-04-26T04:42:16.044 INFO:tasks.workunit.client.1.smithi104.stdout:6/604: mkdir d0/d9/d13/dcd 0 2024-04-26T04:42:16.044 INFO:tasks.workunit.client.0.smithi090.stdout:7/139: symlink d0/db/df/l35 0 2024-04-26T04:42:16.044 INFO:tasks.workunit.client.0.smithi090.stdout:7/140: write d0/db/d15/d29/f14 [4651245,62724] 0 2024-04-26T04:42:16.046 INFO:tasks.workunit.client.1.smithi104.stdout:9/329: symlink d11/d15/d32/l64 0 2024-04-26T04:42:16.051 INFO:tasks.workunit.client.0.smithi090.stdout:9/110: rename db/dd/f19 to db/dd/d18/f25 0 2024-04-26T04:42:16.052 INFO:tasks.workunit.client.0.smithi090.stdout:6/198: creat d9/d1c/f47 x:0 0 0 2024-04-26T04:42:16.053 INFO:tasks.workunit.client.0.smithi090.stdout:6/199: dread - d9/d1c/d31/f3a zero size 2024-04-26T04:42:16.057 INFO:tasks.workunit.client.0.smithi090.stdout:5/225: creat d4/f46 x:0 0 0 2024-04-26T04:42:16.063 INFO:tasks.workunit.client.0.smithi090.stdout:7/141: fdatasync d0/db/f19 0 2024-04-26T04:42:16.063 INFO:tasks.workunit.client.0.smithi090.stdout:7/142: truncate d0/db/d15/d30/f32 951043 0 2024-04-26T04:42:16.063 INFO:tasks.workunit.client.0.smithi090.stdout:7/143: chown d0/db/df/l35 102134845 1 2024-04-26T04:42:16.065 INFO:tasks.workunit.client.0.smithi090.stdout:9/111: creat db/f26 x:0 0 0 2024-04-26T04:42:16.067 INFO:tasks.workunit.client.0.smithi090.stdout:6/200: creat d9/d21/f48 x:0 0 0 2024-04-26T04:42:16.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/226: symlink d4/d1a/d2a/d31/l47 0 2024-04-26T04:42:16.069 INFO:tasks.workunit.client.0.smithi090.stdout:1/134: getdents d0/d4/d7/d17/d22 0 2024-04-26T04:42:16.070 INFO:tasks.workunit.client.1.smithi104.stdout:1/487: creat d0/d6/d19/d21/d66/d30/d81/d9d/f9f x:0 0 0 2024-04-26T04:42:16.070 INFO:tasks.workunit.client.1.smithi104.stdout:1/488: dread d0/d6/d19/d21/d66/d30/d81/f8b [0,4194304] 0 2024-04-26T04:42:16.071 INFO:tasks.workunit.client.0.smithi090.stdout:8/114: dwrite dc/d1f/f27 [0,4194304] 0 2024-04-26T04:42:16.072 INFO:tasks.workunit.client.0.smithi090.stdout:3/127: dwrite d1/f8 [4194304,4194304] 0 2024-04-26T04:42:16.073 INFO:tasks.workunit.client.1.smithi104.stdout:4/548: rename d4/d7/d1d/d1e/d24/c40 to d4/d7/d1d/d1e/d8f/db6/d7d/cc7 0 2024-04-26T04:42:16.073 INFO:tasks.workunit.client.0.smithi090.stdout:8/115: dread dc/ff [0,4194304] 0 2024-04-26T04:42:16.073 INFO:tasks.workunit.client.0.smithi090.stdout:8/116: chown cb 598076 1 2024-04-26T04:42:16.074 INFO:tasks.workunit.client.0.smithi090.stdout:9/112: dread db/f14 [0,4194304] 0 2024-04-26T04:42:16.074 INFO:tasks.workunit.client.0.smithi090.stdout:4/200: dwrite d3/db/d12/f10 [0,4194304] 0 2024-04-26T04:42:16.074 INFO:tasks.workunit.client.0.smithi090.stdout:4/201: dread - d3/db/d12/d20/d24/f32 zero size 2024-04-26T04:42:16.075 INFO:tasks.workunit.client.1.smithi104.stdout:5/445: unlink d0/d1/d4/d2a/d2b/d37/d48/c60 0 2024-04-26T04:42:16.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/227: creat d4/d1a/f48 x:0 0 0 2024-04-26T04:42:16.077 INFO:tasks.workunit.client.0.smithi090.stdout:5/228: write d4/d1a/d1f/f3a [483753,95885] 0 2024-04-26T04:42:16.079 INFO:tasks.workunit.client.0.smithi090.stdout:1/135: mknod d0/d4/d7/d17/d22/c2b 0 2024-04-26T04:42:16.079 INFO:tasks.workunit.client.0.smithi090.stdout:1/136: truncate d0/d4/f15 8737075 0 2024-04-26T04:42:16.080 INFO:tasks.workunit.client.0.smithi090.stdout:8/117: dread dc/d12/f17 [0,4194304] 0 2024-04-26T04:42:16.082 INFO:tasks.workunit.client.1.smithi104.stdout:2/602: unlink d6/d1b/d68/c6b 0 2024-04-26T04:42:16.086 INFO:tasks.workunit.client.1.smithi104.stdout:6/605: creat d0/d9/dc6/fce x:0 0 0 2024-04-26T04:42:16.088 INFO:tasks.workunit.client.1.smithi104.stdout:9/330: unlink d11/d1d/l33 0 2024-04-26T04:42:16.088 INFO:tasks.workunit.client.0.smithi090.stdout:7/144: dread d0/db/d15/d29/f14 [0,4194304] 0 2024-04-26T04:42:16.089 INFO:tasks.workunit.client.0.smithi090.stdout:7/145: fdatasync d0/db/f19 0 2024-04-26T04:42:16.093 INFO:tasks.workunit.client.1.smithi104.stdout:0/381: write d0/d2/d21/f52 [4421075,105805] 0 2024-04-26T04:42:16.097 INFO:tasks.workunit.client.0.smithi090.stdout:5/229: dread d4/d1a/d1f/f2b [0,4194304] 0 2024-04-26T04:42:16.102 INFO:tasks.workunit.client.0.smithi090.stdout:7/146: dread d0/db/f19 [0,4194304] 0 2024-04-26T04:42:16.102 INFO:tasks.workunit.client.0.smithi090.stdout:7/147: write d0/f8 [271857,49659] 0 2024-04-26T04:42:16.108 INFO:tasks.workunit.client.0.smithi090.stdout:9/113: getdents db/dd/d18 0 2024-04-26T04:42:16.109 INFO:tasks.workunit.client.1.smithi104.stdout:4/549: dwrite d4/f98 [0,4194304] 0 2024-04-26T04:42:16.112 INFO:tasks.workunit.client.1.smithi104.stdout:5/446: dwrite d0/f58 [0,4194304] 0 2024-04-26T04:42:16.113 INFO:tasks.workunit.client.1.smithi104.stdout:5/447: chown d0/d1/c1b 0 1 2024-04-26T04:42:16.113 INFO:tasks.workunit.client.1.smithi104.stdout:5/448: readlink d0/d1/d64/d26/l46 0 2024-04-26T04:42:16.113 INFO:tasks.workunit.client.0.smithi090.stdout:1/137: link d0/f1c d0/d4/d7/f2c 0 2024-04-26T04:42:16.113 INFO:tasks.workunit.client.0.smithi090.stdout:1/138: chown d0/d4/d7/d17/f29 39 1 2024-04-26T04:42:16.113 INFO:tasks.workunit.client.0.smithi090.stdout:1/139: truncate d0/d4/d7/d17/f2a 974110 0 2024-04-26T04:42:16.114 INFO:tasks.workunit.client.0.smithi090.stdout:6/201: rename d9/d22 to d9/d3c/d49 0 2024-04-26T04:42:16.114 INFO:tasks.workunit.client.0.smithi090.stdout:6/202: fdatasync d9/d1c/f36 0 2024-04-26T04:42:16.115 INFO:tasks.workunit.client.1.smithi104.stdout:5/449: dread d0/d1/d4/f24 [0,4194304] 0 2024-04-26T04:42:16.118 INFO:tasks.workunit.client.1.smithi104.stdout:9/331: dwrite d11/d1d/d34/f5d [0,4194304] 0 2024-04-26T04:42:16.118 INFO:tasks.workunit.client.0.smithi090.stdout:5/230: mkdir d4/d1a/df/d36/d42/d49 0 2024-04-26T04:42:16.118 INFO:tasks.workunit.client.0.smithi090.stdout:5/231: write d4/d1a/f48 [310120,101983] 0 2024-04-26T04:42:16.119 INFO:tasks.workunit.client.1.smithi104.stdout:6/606: dwrite d0/d9/f3b [0,4194304] 0 2024-04-26T04:42:16.120 INFO:tasks.workunit.client.0.smithi090.stdout:9/114: dread db/dd/ff [0,4194304] 0 2024-04-26T04:42:16.123 INFO:tasks.workunit.client.0.smithi090.stdout:4/202: rmdir d3/db/d12/d20/d24 39 2024-04-26T04:42:16.125 INFO:tasks.workunit.client.0.smithi090.stdout:1/140: mknod d0/c2d 0 2024-04-26T04:42:16.129 INFO:tasks.workunit.client.0.smithi090.stdout:7/148: rename d0/d1/l2f to d0/db/d15/l36 0 2024-04-26T04:42:16.129 INFO:tasks.workunit.client.0.smithi090.stdout:7/149: fsync d0/f8 0 2024-04-26T04:42:16.130 INFO:tasks.workunit.client.1.smithi104.stdout:6/607: dread d0/d9/dc6/f24 [0,4194304] 0 2024-04-26T04:42:16.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/203: creat d9/d21/d40/f4a x:0 0 0 2024-04-26T04:42:16.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/204: dread - d9/d1c/f47 zero size 2024-04-26T04:42:16.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/205: fsync f1 0 2024-04-26T04:42:16.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/206: readlink d9/d21/d40/l38 0 2024-04-26T04:42:16.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/207: stat d9/d3c/d49/c44 0 2024-04-26T04:42:16.132 INFO:tasks.workunit.client.0.smithi090.stdout:6/208: creat d9/d21/d40/f4b x:0 0 0 2024-04-26T04:42:16.132 INFO:tasks.workunit.client.0.smithi090.stdout:6/209: fsync d9/f15 0 2024-04-26T04:42:16.132 INFO:tasks.workunit.client.0.smithi090.stdout:6/210: dread - d9/d1c/d31/f41 zero size 2024-04-26T04:42:16.132 INFO:tasks.workunit.client.0.smithi090.stdout:5/232: mkdir d4/d1a/df/d4a 0 2024-04-26T04:42:16.135 INFO:tasks.workunit.client.0.smithi090.stdout:4/203: dread d3/f2c [0,4194304] 0 2024-04-26T04:42:16.137 INFO:tasks.workunit.client.1.smithi104.stdout:1/489: creat d0/d6/d19/d21/d66/d29/d3f/fa0 x:0 0 0 2024-04-26T04:42:16.138 INFO:tasks.workunit.client.0.smithi090.stdout:1/141: rmdir d0/d4/d7/d17 39 2024-04-26T04:42:16.138 INFO:tasks.workunit.client.0.smithi090.stdout:1/142: creat d0/f2e x:0 0 0 2024-04-26T04:42:16.139 INFO:tasks.workunit.client.0.smithi090.stdout:9/115: dwrite db/f26 [0,4194304] 0 2024-04-26T04:42:16.139 INFO:tasks.workunit.client.0.smithi090.stdout:7/150: stat d0/db/l21 0 2024-04-26T04:42:16.140 INFO:tasks.workunit.client.1.smithi104.stdout:6/608: write d0/d9/d13/d41/d82/dc8/d80/f61 [949761,26395] 0 2024-04-26T04:42:16.140 INFO:tasks.workunit.client.1.smithi104.stdout:4/550: mknod d4/d7/d1d/d2e/cc8 0 2024-04-26T04:42:16.141 INFO:tasks.workunit.client.0.smithi090.stdout:1/143: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:16.141 INFO:tasks.workunit.client.0.smithi090.stdout:1/144: chown d0/d4/d7/d17/f18 1872512 1 2024-04-26T04:42:16.141 INFO:tasks.workunit.client.0.smithi090.stdout:1/145: chown d0/l6 91 1 2024-04-26T04:42:16.141 INFO:tasks.workunit.client.0.smithi090.stdout:1/146: write d0/d4/f26 [480430,70401] 0 2024-04-26T04:42:16.142 INFO:tasks.workunit.client.0.smithi090.stdout:6/211: mkdir d9/d21/d40/d4c 0 2024-04-26T04:42:16.143 INFO:tasks.workunit.client.1.smithi104.stdout:5/450: mkdir d0/d1/d1e/d98 0 2024-04-26T04:42:16.145 INFO:tasks.workunit.client.0.smithi090.stdout:4/204: truncate d3/f4 605715 0 2024-04-26T04:42:16.147 INFO:tasks.workunit.client.0.smithi090.stdout:6/212: write d9/d3c/d49/f43 [4444192,126924] 0 2024-04-26T04:42:16.147 INFO:tasks.workunit.client.0.smithi090.stdout:6/213: chown d9/d21/c2d 7366 1 2024-04-26T04:42:16.148 INFO:tasks.workunit.client.1.smithi104.stdout:1/490: mkdir d0/d6/d2e/d32/d4d/da1 0 2024-04-26T04:42:16.150 INFO:tasks.workunit.client.1.smithi104.stdout:5/451: mkdir d0/d1/d4/d2a/d2b/d37/d99 0 2024-04-26T04:42:16.152 INFO:tasks.workunit.client.0.smithi090.stdout:9/116: link f3 db/d1e/f27 0 2024-04-26T04:42:16.155 INFO:tasks.workunit.client.0.smithi090.stdout:7/151: link d0/db/d15/d29/f2b d0/d1/d1f/f37 0 2024-04-26T04:42:16.155 INFO:tasks.workunit.client.1.smithi104.stdout:6/609: dread d0/d9/d13/f91 [0,4194304] 0 2024-04-26T04:42:16.155 INFO:tasks.workunit.client.1.smithi104.stdout:6/610: getdents d0/d9/d13/d41/d96/dac/dc4 0 2024-04-26T04:42:16.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/147: creat d0/d4/d7/d17/d22/f2f x:0 0 0 2024-04-26T04:42:16.157 INFO:tasks.workunit.client.1.smithi104.stdout:9/332: write d11/d15/f18 [3278547,111004] 0 2024-04-26T04:42:16.158 INFO:tasks.workunit.client.1.smithi104.stdout:9/333: creat d11/d15/d32/d35/f65 x:0 0 0 2024-04-26T04:42:16.158 INFO:tasks.workunit.client.1.smithi104.stdout:9/334: stat d11/d15/d32/l64 0 2024-04-26T04:42:16.160 INFO:tasks.workunit.client.0.smithi090.stdout:4/205: symlink d3/db/d12/d20/d24/l3f 0 2024-04-26T04:42:16.162 INFO:tasks.workunit.client.1.smithi104.stdout:6/611: write d0/d9/d13/d44/d71/f74 [692904,8178] 0 2024-04-26T04:42:16.163 INFO:tasks.workunit.client.0.smithi090.stdout:6/214: mkdir d9/d1c/d4d 0 2024-04-26T04:42:16.164 INFO:tasks.workunit.client.0.smithi090.stdout:6/215: truncate d9/d21/d40/f4a 987645 0 2024-04-26T04:42:16.164 INFO:tasks.workunit.client.0.smithi090.stdout:6/216: readlink d9/d1c/l30 0 2024-04-26T04:42:16.165 INFO:tasks.workunit.client.1.smithi104.stdout:6/612: creat d0/d9/d13/d41/d96/dac/dc4/fcf x:0 0 0 2024-04-26T04:42:16.166 INFO:tasks.workunit.client.0.smithi090.stdout:9/117: mknod db/d1e/c28 0 2024-04-26T04:42:16.168 INFO:tasks.workunit.client.1.smithi104.stdout:9/335: dread d11/d15/f30 [0,4194304] 0 2024-04-26T04:42:16.169 INFO:tasks.workunit.client.0.smithi090.stdout:7/152: link d0/f8 d0/d1/f38 0 2024-04-26T04:42:16.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/128: truncate d1/d9/f20 1618112 0 2024-04-26T04:42:16.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/129: chown d1/d6/c15 300113551 1 2024-04-26T04:42:16.174 INFO:tasks.workunit.client.1.smithi104.stdout:6/613: symlink d0/d9/d13/d41/d82/dc8/d80/dca/ld0 0 2024-04-26T04:42:16.175 INFO:tasks.workunit.client.1.smithi104.stdout:6/614: write d0/d9/dc6/f9e [635972,13058] 0 2024-04-26T04:42:16.175 INFO:tasks.workunit.client.1.smithi104.stdout:6/615: link d0/d9/da8/cbb d0/d9/d13/d41/d82/dc8/cd1 0 2024-04-26T04:42:16.175 INFO:tasks.workunit.client.1.smithi104.stdout:6/616: write d0/f9c [1488809,72076] 0 2024-04-26T04:42:16.175 INFO:tasks.workunit.client.0.smithi090.stdout:6/217: rename d9/d35 to d9/d21/d40/d4e 0 2024-04-26T04:42:16.177 INFO:tasks.workunit.client.0.smithi090.stdout:7/153: symlink d0/db/d15/d30/l39 0 2024-04-26T04:42:16.178 INFO:tasks.workunit.client.0.smithi090.stdout:3/130: mkdir d1/d6/d1b/d1e/d2a 0 2024-04-26T04:42:16.178 INFO:tasks.workunit.client.0.smithi090.stdout:3/131: truncate d1/d9/fb 1205749 0 2024-04-26T04:42:16.179 INFO:tasks.workunit.client.0.smithi090.stdout:3/132: dread - d1/d17/f25 zero size 2024-04-26T04:42:16.179 INFO:tasks.workunit.client.0.smithi090.stdout:3/133: rename d1/d6 to d1/d6/d1b/d1e/d2b 22 2024-04-26T04:42:16.181 INFO:tasks.workunit.client.0.smithi090.stdout:3/134: dread d1/fc [0,4194304] 0 2024-04-26T04:42:16.181 INFO:tasks.workunit.client.0.smithi090.stdout:7/154: rmdir d0/db/df 39 2024-04-26T04:42:16.183 INFO:tasks.workunit.client.0.smithi090.stdout:7/155: unlink d0/d1/l27 0 2024-04-26T04:42:16.183 INFO:tasks.workunit.client.0.smithi090.stdout:7/156: creat d0/db/d15/d30/f3a x:0 0 0 2024-04-26T04:42:16.191 INFO:tasks.workunit.client.0.smithi090.stdout:5/233: dwrite d4/d1a/d1f/f2f [0,4194304] 0 2024-04-26T04:42:16.192 INFO:tasks.workunit.client.1.smithi104.stdout:1/491: dwrite d0/f14 [4194304,4194304] 0 2024-04-26T04:42:16.192 INFO:tasks.workunit.client.1.smithi104.stdout:4/551: dwrite d4/d7/d1d/d1e/d24/d48/f4e [0,4194304] 0 2024-04-26T04:42:16.192 INFO:tasks.workunit.client.0.smithi090.stdout:3/135: dread d1/d6/f14 [0,4194304] 0 2024-04-26T04:42:16.193 INFO:tasks.workunit.client.0.smithi090.stdout:9/118: write db/f26 [4913156,28856] 0 2024-04-26T04:42:16.197 INFO:tasks.workunit.client.0.smithi090.stdout:5/234: symlink d4/d1a/d3f/l4b 0 2024-04-26T04:42:16.198 INFO:tasks.workunit.client.1.smithi104.stdout:9/336: dread d11/d15/f18 [0,4194304] 0 2024-04-26T04:42:16.198 INFO:tasks.workunit.client.0.smithi090.stdout:6/218: dread d9/f12 [0,4194304] 0 2024-04-26T04:42:16.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/206: dwrite d3/f2c [0,4194304] 0 2024-04-26T04:42:16.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/207: fsync d3/db/d12/ff 0 2024-04-26T04:42:16.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/208: chown d3/d13/d1d/f38 22714172 1 2024-04-26T04:42:16.200 INFO:tasks.workunit.client.0.smithi090.stdout:5/235: link d4/d1a/d1f/c26 d4/d1a/d1f/c4c 0 2024-04-26T04:42:16.200 INFO:tasks.workunit.client.0.smithi090.stdout:5/236: fsync d4/d1a/d45/f3c 0 2024-04-26T04:42:16.201 INFO:tasks.workunit.client.1.smithi104.stdout:9/337: creat d11/d15/d32/f66 x:0 0 0 2024-04-26T04:42:16.202 INFO:tasks.workunit.client.0.smithi090.stdout:1/148: dwrite d0/d4/f27 [0,4194304] 0 2024-04-26T04:42:16.202 INFO:tasks.workunit.client.0.smithi090.stdout:1/149: write d0/d4/d7/d17/f29 [178568,39817] 0 2024-04-26T04:42:16.203 INFO:tasks.workunit.client.1.smithi104.stdout:9/338: dread - d11/d15/f59 zero size 2024-04-26T04:42:16.204 INFO:tasks.workunit.client.0.smithi090.stdout:9/119: dread db/dd/d18/f20 [0,4194304] 0 2024-04-26T04:42:16.204 INFO:tasks.workunit.client.0.smithi090.stdout:9/120: readlink db/dd/d18/l1a 0 2024-04-26T04:42:16.204 INFO:tasks.workunit.client.0.smithi090.stdout:6/219: rename d9/d3c/d49/c44 to d9/d3c/d49/c4f 0 2024-04-26T04:42:16.207 INFO:tasks.workunit.client.0.smithi090.stdout:5/237: rename d4/d1a/f1d to d4/d1a/df/d36/f4d 0 2024-04-26T04:42:16.207 INFO:tasks.workunit.client.0.smithi090.stdout:4/209: dread d3/db/f2f [0,4194304] 0 2024-04-26T04:42:16.208 INFO:tasks.workunit.client.0.smithi090.stdout:4/210: write d3/f4 [1641385,69825] 0 2024-04-26T04:42:16.209 INFO:tasks.workunit.client.0.smithi090.stdout:1/150: creat d0/d4/d7/f30 x:0 0 0 2024-04-26T04:42:16.211 INFO:tasks.workunit.client.0.smithi090.stdout:6/220: symlink d9/d1c/l50 0 2024-04-26T04:42:16.213 INFO:tasks.workunit.client.0.smithi090.stdout:5/238: mknod d4/d1a/df/c4e 0 2024-04-26T04:42:16.214 INFO:tasks.workunit.client.0.smithi090.stdout:5/239: chown d4/d1a/c39 944 1 2024-04-26T04:42:16.214 INFO:tasks.workunit.client.0.smithi090.stdout:5/240: write d4/f46 [273267,89065] 0 2024-04-26T04:42:16.214 INFO:tasks.workunit.client.0.smithi090.stdout:4/211: creat d3/db/d12/f40 x:0 0 0 2024-04-26T04:42:16.214 INFO:tasks.workunit.client.0.smithi090.stdout:4/212: read f0 [3864612,75070] 0 2024-04-26T04:42:16.216 INFO:tasks.workunit.client.0.smithi090.stdout:6/221: rmdir d9 39 2024-04-26T04:42:16.219 INFO:tasks.workunit.client.1.smithi104.stdout:0/382: rename d0/d2/d21/d49/d62/f6e to d0/d2/d21/d27/f7b 0 2024-04-26T04:42:16.220 INFO:tasks.workunit.client.0.smithi090.stdout:5/241: creat d4/d1a/d45/f4f x:0 0 0 2024-04-26T04:42:16.221 INFO:tasks.workunit.client.1.smithi104.stdout:4/552: rename d4/d20/d55/d56/f96 to d4/db/d6b/fc9 0 2024-04-26T04:42:16.221 INFO:tasks.workunit.client.1.smithi104.stdout:4/553: stat d4/d7/d1d/d2e/f7e 0 2024-04-26T04:42:16.221 INFO:tasks.workunit.client.1.smithi104.stdout:4/554: write d4/db/f12 [1661606,128215] 0 2024-04-26T04:42:16.222 INFO:tasks.workunit.client.0.smithi090.stdout:4/213: rename d3/db/fc to d3/db/d12/d20/f41 0 2024-04-26T04:42:16.222 INFO:tasks.workunit.client.1.smithi104.stdout:4/555: symlink d4/d7/d1d/d1e/d24/d59/dc1/lca 0 2024-04-26T04:42:16.223 INFO:tasks.workunit.client.1.smithi104.stdout:4/556: read - d4/d20/f9a zero size 2024-04-26T04:42:16.223 INFO:tasks.workunit.client.1.smithi104.stdout:4/557: fsync d4/f37 0 2024-04-26T04:42:16.225 INFO:tasks.workunit.client.0.smithi090.stdout:2/106: sync 2024-04-26T04:42:16.225 INFO:tasks.workunit.client.0.smithi090.stdout:0/202: sync 2024-04-26T04:42:16.225 INFO:tasks.workunit.client.0.smithi090.stdout:0/203: dread - d0/de/f36 zero size 2024-04-26T04:42:16.227 INFO:tasks.workunit.client.1.smithi104.stdout:1/492: dwrite d0/d6/d19/d21/d28/f85 [0,4194304] 0 2024-04-26T04:42:16.229 INFO:tasks.workunit.client.0.smithi090.stdout:9/121: dwrite f9 [0,4194304] 0 2024-04-26T04:42:16.229 INFO:tasks.workunit.client.0.smithi090.stdout:6/222: dread d9/d3c/d49/f26 [0,4194304] 0 2024-04-26T04:42:16.229 INFO:tasks.workunit.client.0.smithi090.stdout:9/122: write f2 [1907882,84122] 0 2024-04-26T04:42:16.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/223: creat d9/d21/d3b/f51 x:0 0 0 2024-04-26T04:42:16.230 INFO:tasks.workunit.client.0.smithi090.stdout:1/151: dread d0/d4/d7/d17/f29 [0,4194304] 0 2024-04-26T04:42:16.231 INFO:tasks.workunit.client.0.smithi090.stdout:7/157: dwrite d0/db/df/d1c/d23/f26 [0,4194304] 0 2024-04-26T04:42:16.231 INFO:tasks.workunit.client.0.smithi090.stdout:4/214: rename d3/d13/d1d/c3c to d3/db/d12/d20/d24/d30/c42 0 2024-04-26T04:42:16.231 INFO:tasks.workunit.client.0.smithi090.stdout:4/215: stat d3/f4 0 2024-04-26T04:42:16.231 INFO:tasks.workunit.client.0.smithi090.stdout:5/242: read d4/d1a/d1f/f2b [30445,90190] 0 2024-04-26T04:42:16.233 INFO:tasks.workunit.client.0.smithi090.stdout:2/107: symlink db/l1d 0 2024-04-26T04:42:16.234 INFO:tasks.workunit.client.1.smithi104.stdout:4/558: write d4/d7/d1d/d1e/d8f/db6/f88 [2204031,39195] 0 2024-04-26T04:42:16.235 INFO:tasks.workunit.client.1.smithi104.stdout:1/493: rmdir d0/d6/d19/d21/d66/d30/d81/d9d 39 2024-04-26T04:42:16.236 INFO:tasks.workunit.client.1.smithi104.stdout:1/494: stat d0/d6/d19/d21/d66/d20/d2a/d56/c5c 0 2024-04-26T04:42:16.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/204: creat d0/d26/f42 x:0 0 0 2024-04-26T04:42:16.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/205: chown d0/df 978 1 2024-04-26T04:42:16.237 INFO:tasks.workunit.client.0.smithi090.stdout:0/206: creat d0/d9/f43 x:0 0 0 2024-04-26T04:42:16.238 INFO:tasks.workunit.client.1.smithi104.stdout:1/495: symlink d0/d6/d19/d21/d66/d20/d2a/d98/la2 0 2024-04-26T04:42:16.240 INFO:tasks.workunit.client.0.smithi090.stdout:9/123: symlink db/d1e/l29 0 2024-04-26T04:42:16.242 INFO:tasks.workunit.client.0.smithi090.stdout:6/224: unlink d9/d1c/l50 0 2024-04-26T04:42:16.244 INFO:tasks.workunit.client.0.smithi090.stdout:3/136: dwrite d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:16.244 INFO:tasks.workunit.client.0.smithi090.stdout:3/137: readlink d1/l4 0 2024-04-26T04:42:16.245 INFO:tasks.workunit.client.0.smithi090.stdout:7/158: rename d0/db/d15/d29/f2b to d0/d1/d4/f3b 0 2024-04-26T04:42:16.246 INFO:tasks.workunit.client.1.smithi104.stdout:0/383: dwrite d0/d2/d21/d27/d1e/d57/d6c/f59 [0,4194304] 0 2024-04-26T04:42:16.246 INFO:tasks.workunit.client.1.smithi104.stdout:1/496: rmdir d0/d6 39 2024-04-26T04:42:16.246 INFO:tasks.workunit.client.1.smithi104.stdout:1/497: chown d0/d6/d19/d21/d66/d30/l6c 2063856 1 2024-04-26T04:42:16.247 INFO:tasks.workunit.client.0.smithi090.stdout:4/216: unlink d3/f4 0 2024-04-26T04:42:16.247 INFO:tasks.workunit.client.0.smithi090.stdout:4/217: stat d3/db/d12/ff 0 2024-04-26T04:42:16.247 INFO:tasks.workunit.client.0.smithi090.stdout:4/218: stat d3/db/d12/ff 0 2024-04-26T04:42:16.247 INFO:tasks.workunit.client.0.smithi090.stdout:5/243: mknod d4/d1a/df/d36/d42/c50 0 2024-04-26T04:42:16.249 INFO:tasks.workunit.client.0.smithi090.stdout:2/108: link f2 db/dc/f1e 0 2024-04-26T04:42:16.249 INFO:tasks.workunit.client.0.smithi090.stdout:2/109: creat db/dc/f1f x:0 0 0 2024-04-26T04:42:16.250 INFO:tasks.workunit.client.1.smithi104.stdout:0/384: mkdir d0/d2/d21/d38/d7c 0 2024-04-26T04:42:16.251 INFO:tasks.workunit.client.1.smithi104.stdout:0/385: write d0/d2/d21/d27/f6d [1053404,47983] 0 2024-04-26T04:42:16.251 INFO:tasks.workunit.client.1.smithi104.stdout:0/386: write d0/d4/f29 [605940,109671] 0 2024-04-26T04:42:16.252 INFO:tasks.workunit.client.1.smithi104.stdout:1/498: rmdir d0/d6/d19/d1b/d35 39 2024-04-26T04:42:16.253 INFO:tasks.workunit.client.1.smithi104.stdout:0/387: dread d0/d2/d21/d27/d1e/f2c [0,4194304] 0 2024-04-26T04:42:16.254 INFO:tasks.workunit.client.1.smithi104.stdout:1/499: mkdir d0/d6/d19/d21/d66/d20/da3 0 2024-04-26T04:42:16.255 INFO:tasks.workunit.client.1.smithi104.stdout:1/500: truncate d0/d6/d19/d21/d66/d20/d2a/d56/d8c/f99 432273 0 2024-04-26T04:42:16.255 INFO:tasks.workunit.client.1.smithi104.stdout:4/559: dwrite d4/db/d6b/fc9 [4194304,4194304] 0 2024-04-26T04:42:16.255 INFO:tasks.workunit.client.1.smithi104.stdout:4/560: readlink d4/db/d6b/l90 0 2024-04-26T04:42:16.256 INFO:tasks.workunit.client.0.smithi090.stdout:0/207: getdents d0 0 2024-04-26T04:42:16.259 INFO:tasks.workunit.client.1.smithi104.stdout:1/501: rename d0/d6/l27 to d0/d6/d19/d21/d66/d29/la4 0 2024-04-26T04:42:16.261 INFO:tasks.workunit.client.1.smithi104.stdout:4/561: mknod d4/d7/d1d/d1e/d8f/db6/ccb 0 2024-04-26T04:42:16.262 INFO:tasks.workunit.client.0.smithi090.stdout:6/225: symlink d9/d21/d40/l52 0 2024-04-26T04:42:16.262 INFO:tasks.workunit.client.0.smithi090.stdout:6/226: chown d9/d21/c42 26 1 2024-04-26T04:42:16.262 INFO:tasks.workunit.client.0.smithi090.stdout:6/227: write d9/d21/d40/d4e/f45 [415825,67700] 0 2024-04-26T04:42:16.263 INFO:tasks.workunit.client.1.smithi104.stdout:4/562: truncate d4/d20/f36 748793 0 2024-04-26T04:42:16.267 INFO:tasks.workunit.client.1.smithi104.stdout:1/502: dread d0/d6/d19/d21/d66/d29/d3f/f7b [0,4194304] 0 2024-04-26T04:42:16.267 INFO:tasks.workunit.client.1.smithi104.stdout:1/503: dread - d0/d6/d19/d21/d66/f83 zero size 2024-04-26T04:42:16.269 INFO:tasks.workunit.client.1.smithi104.stdout:5/452: rmdir d0/d1/d64 39 2024-04-26T04:42:16.270 INFO:tasks.workunit.client.1.smithi104.stdout:5/453: chown d0/d1/d4/f17 73258238 1 2024-04-26T04:42:16.271 INFO:tasks.workunit.client.1.smithi104.stdout:1/504: mkdir d0/d6/d19/d1b/da5 0 2024-04-26T04:42:16.272 INFO:tasks.workunit.client.0.smithi090.stdout:4/219: mkdir d3/db/d12/d20/d24/d43 0 2024-04-26T04:42:16.273 INFO:tasks.workunit.client.1.smithi104.stdout:5/454: unlink d0/d1/d4/d2a/d2b/f42 0 2024-04-26T04:42:16.273 INFO:tasks.workunit.client.1.smithi104.stdout:5/455: read - d0/f93 zero size 2024-04-26T04:42:16.273 INFO:tasks.workunit.client.0.smithi090.stdout:7/159: dwrite d0/d1/d4/f2e [4194304,4194304] 0 2024-04-26T04:42:16.274 INFO:tasks.workunit.client.0.smithi090.stdout:7/160: creat d0/f3c x:0 0 0 2024-04-26T04:42:16.274 INFO:tasks.workunit.client.0.smithi090.stdout:7/161: fdatasync d0/d1/d4/f3b 0 2024-04-26T04:42:16.274 INFO:tasks.workunit.client.1.smithi104.stdout:1/505: stat d0/d6/d19/d21/d66/l10 0 2024-04-26T04:42:16.276 INFO:tasks.workunit.client.1.smithi104.stdout:5/456: getdents d0/d1 0 2024-04-26T04:42:16.276 INFO:tasks.workunit.client.0.smithi090.stdout:2/110: mkdir db/d11/d14/d1b/d20 0 2024-04-26T04:42:16.277 INFO:tasks.workunit.client.1.smithi104.stdout:1/506: creat d0/d6/d19/d1b/da5/fa6 x:0 0 0 2024-04-26T04:42:16.277 INFO:tasks.workunit.client.1.smithi104.stdout:1/507: fsync d0/d6/d2e/d32/f92 0 2024-04-26T04:42:16.277 INFO:tasks.workunit.client.1.smithi104.stdout:1/508: stat d0/d6/d2e/f84 0 2024-04-26T04:42:16.277 INFO:tasks.workunit.client.0.smithi090.stdout:9/124: getdents db 0 2024-04-26T04:42:16.277 INFO:tasks.workunit.client.0.smithi090.stdout:9/125: write db/f26 [5461005,106544] 0 2024-04-26T04:42:16.280 INFO:tasks.workunit.client.1.smithi104.stdout:9/339: creat d11/d15/f67 x:0 0 0 2024-04-26T04:42:16.281 INFO:tasks.workunit.client.0.smithi090.stdout:6/228: link d9/d21/d40/f4b d9/d3c/f53 0 2024-04-26T04:42:16.284 INFO:tasks.workunit.client.1.smithi104.stdout:9/340: creat d11/f68 x:0 0 0 2024-04-26T04:42:16.284 INFO:tasks.workunit.client.1.smithi104.stdout:9/341: stat d11/d15/d32/f56 0 2024-04-26T04:42:16.286 INFO:tasks.workunit.client.0.smithi090.stdout:5/244: rename d4/f20 to d4/d1a/df/d4a/f51 0 2024-04-26T04:42:16.286 INFO:tasks.workunit.client.0.smithi090.stdout:5/245: creat d4/d1a/d3f/f52 x:0 0 0 2024-04-26T04:42:16.286 INFO:tasks.workunit.client.0.smithi090.stdout:5/246: write d4/d1a/d3f/f44 [33041,54302] 0 2024-04-26T04:42:16.287 INFO:tasks.workunit.client.1.smithi104.stdout:0/388: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:16.287 INFO:tasks.workunit.client.1.smithi104.stdout:0/389: creat d0/d2/d21/d49/d5e/f7d x:0 0 0 2024-04-26T04:42:16.287 INFO:tasks.workunit.client.1.smithi104.stdout:0/390: write d0/f2f [338859,79103] 0 2024-04-26T04:42:16.288 INFO:tasks.workunit.client.0.smithi090.stdout:4/220: symlink d3/db/d12/d20/l44 0 2024-04-26T04:42:16.290 INFO:tasks.workunit.client.1.smithi104.stdout:0/391: rename d0/d2/d10/f4e to d0/d2/d21/d49/f7e 0 2024-04-26T04:42:16.296 INFO:tasks.workunit.client.0.smithi090.stdout:7/162: symlink d0/db/df/d1c/d23/l3d 0 2024-04-26T04:42:16.297 INFO:tasks.workunit.client.0.smithi090.stdout:2/111: mknod db/c21 0 2024-04-26T04:42:16.297 INFO:tasks.workunit.client.0.smithi090.stdout:2/112: getdents db/d11/d14/d1b/d20 0 2024-04-26T04:42:16.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/126: creat db/dd/d18/f2a x:0 0 0 2024-04-26T04:42:16.299 INFO:tasks.workunit.client.0.smithi090.stdout:0/208: dwrite d0/f38 [0,4194304] 0 2024-04-26T04:42:16.299 INFO:tasks.workunit.client.0.smithi090.stdout:0/209: dread - d0/d26/f42 zero size 2024-04-26T04:42:16.299 INFO:tasks.workunit.client.0.smithi090.stdout:0/210: creat d0/d9/f44 x:0 0 0 2024-04-26T04:42:16.300 INFO:tasks.workunit.client.0.smithi090.stdout:6/229: rename d9/d3c/f53 to d9/d1c/d31/f54 0 2024-04-26T04:42:16.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/230: fdatasync d9/fd 0 2024-04-26T04:42:16.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/231: read - d9/d1c/d31/f41 zero size 2024-04-26T04:42:16.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/232: write d9/f17 [2913474,83468] 0 2024-04-26T04:42:16.302 INFO:tasks.workunit.client.0.smithi090.stdout:5/247: mkdir d4/d1a/d1f/d53 0 2024-04-26T04:42:16.303 INFO:tasks.workunit.client.1.smithi104.stdout:4/563: dwrite d4/db/d6b/fc9 [0,4194304] 0 2024-04-26T04:42:16.303 INFO:tasks.workunit.client.1.smithi104.stdout:4/564: write d4/d7/d1d/d1e/d24/d48/f7c [1032482,128909] 0 2024-04-26T04:42:16.303 INFO:tasks.workunit.client.0.smithi090.stdout:4/221: mkdir d3/db/d12/d20/d45 0 2024-04-26T04:42:16.304 INFO:tasks.workunit.client.1.smithi104.stdout:0/392: read d0/d2/d21/f52 [3032266,130474] 0 2024-04-26T04:42:16.305 INFO:tasks.workunit.client.0.smithi090.stdout:7/163: mkdir d0/d1/d4/d3e 0 2024-04-26T04:42:16.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/127: symlink db/dd/l2b 0 2024-04-26T04:42:16.306 INFO:tasks.workunit.client.1.smithi104.stdout:5/457: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:16.307 INFO:tasks.workunit.client.1.smithi104.stdout:0/393: creat d0/f7f x:0 0 0 2024-04-26T04:42:16.307 INFO:tasks.workunit.client.1.smithi104.stdout:0/394: stat d0/d2/d21/d49 0 2024-04-26T04:42:16.308 INFO:tasks.workunit.client.0.smithi090.stdout:0/211: mkdir d0/d9/d45 0 2024-04-26T04:42:16.309 INFO:tasks.workunit.client.0.smithi090.stdout:0/212: readlink d0/df/l1a 0 2024-04-26T04:42:16.309 INFO:tasks.workunit.client.1.smithi104.stdout:5/458: unlink d0/d1/d3e/f68 0 2024-04-26T04:42:16.310 INFO:tasks.workunit.client.0.smithi090.stdout:2/113: dread db/f16 [0,4194304] 0 2024-04-26T04:42:16.310 INFO:tasks.workunit.client.1.smithi104.stdout:5/459: symlink d0/d41/d4f/l9a 0 2024-04-26T04:42:16.310 INFO:tasks.workunit.client.1.smithi104.stdout:5/460: stat d0/d1/d4/f63 0 2024-04-26T04:42:16.311 INFO:tasks.workunit.client.0.smithi090.stdout:6/233: rename d9/d21/d40/f4a to d9/d3c/f55 0 2024-04-26T04:42:16.311 INFO:tasks.workunit.client.1.smithi104.stdout:5/461: symlink d0/d1/l9b 0 2024-04-26T04:42:16.312 INFO:tasks.workunit.client.1.smithi104.stdout:5/462: chown d0/d1/d1e/l94 11779 1 2024-04-26T04:42:16.312 INFO:tasks.workunit.client.0.smithi090.stdout:5/248: creat d4/d1a/df/d36/f54 x:0 0 0 2024-04-26T04:42:16.313 INFO:tasks.workunit.client.1.smithi104.stdout:1/509: dwrite d0/d6/d19/d21/d66/d20/d2a/f6f [0,4194304] 0 2024-04-26T04:42:16.313 INFO:tasks.workunit.client.1.smithi104.stdout:1/510: stat d0/d6/d19/d21/d59/c5e 0 2024-04-26T04:42:16.313 INFO:tasks.workunit.client.1.smithi104.stdout:5/463: link d0/d1/f3d d0/d1/d4/d2a/d2b/d37/f9c 0 2024-04-26T04:42:16.314 INFO:tasks.workunit.client.0.smithi090.stdout:4/222: symlink d3/db/d12/d20/d24/d43/l46 0 2024-04-26T04:42:16.314 INFO:tasks.workunit.client.0.smithi090.stdout:9/128: fdatasync db/f26 0 2024-04-26T04:42:16.314 INFO:tasks.workunit.client.0.smithi090.stdout:9/129: chown la 428915 1 2024-04-26T04:42:16.315 INFO:tasks.workunit.client.1.smithi104.stdout:9/342: dwrite d11/d15/d32/d4c/d54/f62 [0,4194304] 0 2024-04-26T04:42:16.315 INFO:tasks.workunit.client.1.smithi104.stdout:1/511: mkdir d0/d6/d19/d1b/da7 0 2024-04-26T04:42:16.315 INFO:tasks.workunit.client.1.smithi104.stdout:1/512: readlink d0/d6/d19/d1b/d35/l91 0 2024-04-26T04:42:16.316 INFO:tasks.workunit.client.0.smithi090.stdout:0/213: mkdir d0/d9/d46 0 2024-04-26T04:42:16.317 INFO:tasks.workunit.client.1.smithi104.stdout:5/464: mknod d0/d1/d4/d2a/d6b/d73/c9d 0 2024-04-26T04:42:16.317 INFO:tasks.workunit.client.0.smithi090.stdout:7/164: mknod d0/db/d15/d29/c3f 0 2024-04-26T04:42:16.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/165: readlink d0/db/df/l1a 0 2024-04-26T04:42:16.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/166: write d0/f8 [3292452,22887] 0 2024-04-26T04:42:16.318 INFO:tasks.workunit.client.0.smithi090.stdout:7/167: creat d0/db/d15/d30/f40 x:0 0 0 2024-04-26T04:42:16.319 INFO:tasks.workunit.client.0.smithi090.stdout:2/114: mknod db/d11/d14/d1b/d20/c22 0 2024-04-26T04:42:16.320 INFO:tasks.workunit.client.0.smithi090.stdout:6/234: mknod d9/d21/d40/d4c/c56 0 2024-04-26T04:42:16.321 INFO:tasks.workunit.client.0.smithi090.stdout:4/223: mknod d3/db/d12/d20/d2d/d2e/c47 0 2024-04-26T04:42:16.322 INFO:tasks.workunit.client.0.smithi090.stdout:4/224: fsync d3/db/d12/f10 0 2024-04-26T04:42:16.329 INFO:tasks.workunit.client.1.smithi104.stdout:9/343: mkdir d11/d1d/d34/d69 0 2024-04-26T04:42:16.331 INFO:tasks.workunit.client.1.smithi104.stdout:1/513: symlink d0/d6/d19/d21/d66/d20/da3/la8 0 2024-04-26T04:42:16.332 INFO:tasks.workunit.client.0.smithi090.stdout:4/225: dread d3/d13/d1d/f38 [0,4194304] 0 2024-04-26T04:42:16.333 INFO:tasks.workunit.client.0.smithi090.stdout:4/226: creat d3/f48 x:0 0 0 2024-04-26T04:42:16.333 INFO:tasks.workunit.client.0.smithi090.stdout:4/227: write d3/db/d12/d20/d24/d30/f37 [871673,100457] 0 2024-04-26T04:42:16.337 INFO:tasks.workunit.client.0.smithi090.stdout:0/214: dread d0/f3b [0,4194304] 0 2024-04-26T04:42:16.343 INFO:tasks.workunit.client.0.smithi090.stdout:4/228: dread d3/d13/f39 [0,4194304] 0 2024-04-26T04:42:16.344 INFO:tasks.workunit.client.0.smithi090.stdout:4/229: write d3/db/d12/d20/d2d/d2e/f3a [527598,34578] 0 2024-04-26T04:42:16.347 INFO:tasks.workunit.client.1.smithi104.stdout:1/514: write d0/d6/d19/d1b/d35/f40 [2058145,37106] 0 2024-04-26T04:42:16.347 INFO:tasks.workunit.client.1.smithi104.stdout:1/515: chown d0/d6/d19/d21/d66/d29/f39 2 1 2024-04-26T04:42:16.347 INFO:tasks.workunit.client.1.smithi104.stdout:1/516: truncate d0/d6/d19/d1b/da5/fa6 474528 0 2024-04-26T04:42:16.347 INFO:tasks.workunit.client.1.smithi104.stdout:1/517: stat d0/d6/d19/d21/d28/f85 0 2024-04-26T04:42:16.349 INFO:tasks.workunit.client.1.smithi104.stdout:1/518: rmdir d0/d6/d19/d21/d59 39 2024-04-26T04:42:16.349 INFO:tasks.workunit.client.1.smithi104.stdout:1/519: chown d0/d6/d19/d21/d66/l24 51347 1 2024-04-26T04:42:16.349 INFO:tasks.workunit.client.1.smithi104.stdout:1/520: chown d0/d6/d19/d21/d66/f83 54436 1 2024-04-26T04:42:16.351 INFO:tasks.workunit.client.1.smithi104.stdout:1/521: dread d0/d6/d19/d21/d66/d30/d81/f8a [0,4194304] 0 2024-04-26T04:42:16.352 INFO:tasks.workunit.client.1.smithi104.stdout:1/522: symlink d0/d6/la9 0 2024-04-26T04:42:16.353 INFO:tasks.workunit.client.1.smithi104.stdout:1/523: mknod d0/d6/d19/d21/caa 0 2024-04-26T04:42:16.353 INFO:tasks.workunit.client.1.smithi104.stdout:1/524: fdatasync d0/d6/d49/f87 0 2024-04-26T04:42:16.359 INFO:tasks.workunit.client.0.smithi090.stdout:5/249: dwrite d4/f37 [0,4194304] 0 2024-04-26T04:42:16.359 INFO:tasks.workunit.client.1.smithi104.stdout:1/525: dread d0/d6/d19/d21/d66/d29/d3f/f5b [0,4194304] 0 2024-04-26T04:42:16.360 INFO:tasks.workunit.client.1.smithi104.stdout:5/465: dwrite d0/d1/d4/f8c [0,4194304] 0 2024-04-26T04:42:16.360 INFO:tasks.workunit.client.1.smithi104.stdout:5/466: fsync d0/d1/d4/d2a/d2b/d5c/f8a 0 2024-04-26T04:42:16.361 INFO:tasks.workunit.client.0.smithi090.stdout:2/115: dwrite db/f16 [0,4194304] 0 2024-04-26T04:42:16.362 INFO:tasks.workunit.client.0.smithi090.stdout:9/130: dwrite db/f26 [4194304,4194304] 0 2024-04-26T04:42:16.363 INFO:tasks.workunit.client.0.smithi090.stdout:7/168: dwrite d0/db/d15/d29/fc [8388608,4194304] 0 2024-04-26T04:42:16.364 INFO:tasks.workunit.client.0.smithi090.stdout:4/230: dread d3/db/d12/d20/d24/d30/f37 [0,4194304] 0 2024-04-26T04:42:16.365 INFO:tasks.workunit.client.0.smithi090.stdout:4/231: stat d3/db/d12/d20 0 2024-04-26T04:42:16.365 INFO:tasks.workunit.client.1.smithi104.stdout:9/344: dwrite d11/f16 [4194304,4194304] 0 2024-04-26T04:42:16.366 INFO:tasks.workunit.client.0.smithi090.stdout:4/232: write d3/d13/f39 [440523,6849] 0 2024-04-26T04:42:16.366 INFO:tasks.workunit.client.0.smithi090.stdout:2/116: mknod db/d11/d14/d1b/d20/c23 0 2024-04-26T04:42:16.368 INFO:tasks.workunit.client.1.smithi104.stdout:9/345: truncate d11/d15/d1b/f37 2070239 0 2024-04-26T04:42:16.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/235: rename d9/d3c/f55 to d9/d1c/f57 0 2024-04-26T04:42:16.372 INFO:tasks.workunit.client.0.smithi090.stdout:9/131: creat db/d1e/f2c x:0 0 0 2024-04-26T04:42:16.373 INFO:tasks.workunit.client.1.smithi104.stdout:5/467: write d0/f58 [2203412,64110] 0 2024-04-26T04:42:16.374 INFO:tasks.workunit.client.0.smithi090.stdout:5/250: read d4/d1a/d45/f3c [854194,123208] 0 2024-04-26T04:42:16.374 INFO:tasks.workunit.client.0.smithi090.stdout:7/169: symlink d0/d1/d1f/l41 0 2024-04-26T04:42:16.378 INFO:tasks.workunit.client.1.smithi104.stdout:5/468: rmdir d0/d1/d4/d90/d97 0 2024-04-26T04:42:16.380 INFO:tasks.workunit.client.1.smithi104.stdout:8/310: sync 2024-04-26T04:42:16.380 INFO:tasks.workunit.client.1.smithi104.stdout:3/493: sync 2024-04-26T04:42:16.380 INFO:tasks.workunit.client.1.smithi104.stdout:7/380: sync 2024-04-26T04:42:16.380 INFO:tasks.workunit.client.1.smithi104.stdout:7/381: chown d2 234 1 2024-04-26T04:42:16.380 INFO:tasks.workunit.client.1.smithi104.stdout:8/311: write d0/d1c/f4c [1035677,70085] 0 2024-04-26T04:42:16.380 INFO:tasks.workunit.client.0.smithi090.stdout:4/233: creat d3/db/d12/d20/d24/d30/f49 x:0 0 0 2024-04-26T04:42:16.381 INFO:tasks.workunit.client.0.smithi090.stdout:4/234: stat d3/c7 0 2024-04-26T04:42:16.381 INFO:tasks.workunit.client.0.smithi090.stdout:4/235: write d3/db/f23 [72633,103408] 0 2024-04-26T04:42:16.382 INFO:tasks.workunit.client.0.smithi090.stdout:6/236: dread d9/d21/d3b/f46 [0,4194304] 0 2024-04-26T04:42:16.382 INFO:tasks.workunit.client.0.smithi090.stdout:6/237: write d9/d21/d40/f4b [1024031,100999] 0 2024-04-26T04:42:16.383 INFO:tasks.workunit.client.1.smithi104.stdout:3/494: rename d5/d9/d17/d72/d85 to d5/d9/dc/d32/d48/dac 0 2024-04-26T04:42:16.384 INFO:tasks.workunit.client.1.smithi104.stdout:3/495: dread - d5/d9/d17/d60/f7e zero size 2024-04-26T04:42:16.384 INFO:tasks.workunit.client.0.smithi090.stdout:2/117: creat db/d11/d14/d1b/d20/f24 x:0 0 0 2024-04-26T04:42:16.384 INFO:tasks.workunit.client.0.smithi090.stdout:2/118: dread - db/d11/f17 zero size 2024-04-26T04:42:16.384 INFO:tasks.workunit.client.0.smithi090.stdout:2/119: chown db/d11/d14/d1b 1 1 2024-04-26T04:42:16.384 INFO:tasks.workunit.client.0.smithi090.stdout:9/132: symlink db/d16/l2d 0 2024-04-26T04:42:16.386 INFO:tasks.workunit.client.1.smithi104.stdout:5/469: dread d0/d1/d4/f63 [0,4194304] 0 2024-04-26T04:42:16.386 INFO:tasks.workunit.client.1.smithi104.stdout:3/496: creat d5/d9/d80/da2/fad x:0 0 0 2024-04-26T04:42:16.386 INFO:tasks.workunit.client.1.smithi104.stdout:3/497: stat d5/d9/dc/d29/f56 0 2024-04-26T04:42:16.386 INFO:tasks.workunit.client.1.smithi104.stdout:3/498: fdatasync d5/d9/d17/d60/f84 0 2024-04-26T04:42:16.387 INFO:tasks.workunit.client.1.smithi104.stdout:7/382: rmdir d2/d4/d5a 39 2024-04-26T04:42:16.388 INFO:tasks.workunit.client.1.smithi104.stdout:5/470: mknod d0/d7b/c9e 0 2024-04-26T04:42:16.389 INFO:tasks.workunit.client.1.smithi104.stdout:7/383: mkdir d2/d4/d32/d3d/d41/d7e 0 2024-04-26T04:42:16.389 INFO:tasks.workunit.client.1.smithi104.stdout:7/384: fdatasync d2/f56 0 2024-04-26T04:42:16.389 INFO:tasks.workunit.client.1.smithi104.stdout:7/385: chown d2/la 17254139 1 2024-04-26T04:42:16.391 INFO:tasks.workunit.client.0.smithi090.stdout:5/251: creat d4/d1a/df/d4a/f55 x:0 0 0 2024-04-26T04:42:16.391 INFO:tasks.workunit.client.0.smithi090.stdout:5/252: dread - d4/d1a/d1f/f3d zero size 2024-04-26T04:42:16.391 INFO:tasks.workunit.client.1.smithi104.stdout:5/471: creat d0/d1/d64/d26/d36/f9f x:0 0 0 2024-04-26T04:42:16.392 INFO:tasks.workunit.client.1.smithi104.stdout:7/386: mkdir d2/d4/d1d/d34/d7f 0 2024-04-26T04:42:16.393 INFO:tasks.workunit.client.0.smithi090.stdout:7/170: unlink d0/db/d15/d29/le 0 2024-04-26T04:42:16.393 INFO:tasks.workunit.client.0.smithi090.stdout:7/171: chown d0/db/d15/l36 9 1 2024-04-26T04:42:16.393 INFO:tasks.workunit.client.0.smithi090.stdout:7/172: chown d0/db/df/l35 22465 1 2024-04-26T04:42:16.396 INFO:tasks.workunit.client.1.smithi104.stdout:5/472: unlink d0/d1/d64/f5b 0 2024-04-26T04:42:16.397 INFO:tasks.workunit.client.0.smithi090.stdout:6/238: unlink d9/d1c/f36 0 2024-04-26T04:42:16.397 INFO:tasks.workunit.client.0.smithi090.stdout:6/239: write d9/f24 [493371,40234] 0 2024-04-26T04:42:16.398 INFO:tasks.workunit.client.1.smithi104.stdout:7/387: mkdir d2/d4/d32/d3d/d41/d5f/d80 0 2024-04-26T04:42:16.398 INFO:tasks.workunit.client.1.smithi104.stdout:7/388: stat d2/d4/f11 0 2024-04-26T04:42:16.398 INFO:tasks.workunit.client.0.smithi090.stdout:2/120: mkdir db/d11/d14/d25 0 2024-04-26T04:42:16.399 INFO:tasks.workunit.client.1.smithi104.stdout:5/473: creat d0/d41/d4f/fa0 x:0 0 0 2024-04-26T04:42:16.400 INFO:tasks.workunit.client.0.smithi090.stdout:9/133: mknod db/dd/d18/c2e 0 2024-04-26T04:42:16.401 INFO:tasks.workunit.client.1.smithi104.stdout:3/499: dread d5/f54 [0,4194304] 0 2024-04-26T04:42:16.401 INFO:tasks.workunit.client.1.smithi104.stdout:7/389: symlink d2/d4/d32/d3d/d41/d5f/l81 0 2024-04-26T04:42:16.401 INFO:tasks.workunit.client.1.smithi104.stdout:8/312: dwrite d0/d1c/d27/d47/f25 [0,4194304] 0 2024-04-26T04:42:16.401 INFO:tasks.workunit.client.1.smithi104.stdout:7/390: creat d2/d4/d32/d60/f82 x:0 0 0 2024-04-26T04:42:16.402 INFO:tasks.workunit.client.0.smithi090.stdout:5/253: mkdir d4/d1a/df/d36/d42/d56 0 2024-04-26T04:42:16.405 INFO:tasks.workunit.client.0.smithi090.stdout:7/173: rename d0/d1/d1f/f22 to d0/db/df/f42 0 2024-04-26T04:42:16.405 INFO:tasks.workunit.client.1.smithi104.stdout:5/474: creat d0/d1/d4/d2a/d2b/fa1 x:0 0 0 2024-04-26T04:42:16.406 INFO:tasks.workunit.client.1.smithi104.stdout:3/500: readlink d5/d9/dc/l11 0 2024-04-26T04:42:16.407 INFO:tasks.workunit.client.0.smithi090.stdout:6/240: mkdir d9/d1c/d4d/d58 0 2024-04-26T04:42:16.409 INFO:tasks.workunit.client.0.smithi090.stdout:2/121: creat db/d11/f26 x:0 0 0 2024-04-26T04:42:16.409 INFO:tasks.workunit.client.0.smithi090.stdout:2/122: dread - db/dc/f1f zero size 2024-04-26T04:42:16.410 INFO:tasks.workunit.client.1.smithi104.stdout:5/475: rename d0/d1/d4/d2a/d2b/d37/f4b to d0/d89/fa2 0 2024-04-26T04:42:16.410 INFO:tasks.workunit.client.1.smithi104.stdout:8/313: dread d0/fe [0,4194304] 0 2024-04-26T04:42:16.410 INFO:tasks.workunit.client.1.smithi104.stdout:8/314: chown d0/d33/c22 0 1 2024-04-26T04:42:16.411 INFO:tasks.workunit.client.0.smithi090.stdout:9/134: mknod db/d1e/c2f 0 2024-04-26T04:42:16.411 INFO:tasks.workunit.client.1.smithi104.stdout:3/501: rename d5/d9/c35 to d5/d9/dc/d69/cae 0 2024-04-26T04:42:16.413 INFO:tasks.workunit.client.0.smithi090.stdout:4/236: rmdir d3/db/d12/d20 39 2024-04-26T04:42:16.413 INFO:tasks.workunit.client.1.smithi104.stdout:8/315: rename d0/d33/d35/c57 to d0/d1c/d27/d47/c62 0 2024-04-26T04:42:16.414 INFO:tasks.workunit.client.0.smithi090.stdout:5/254: link d4/d1a/d3f/l2e d4/d1a/d1f/l57 0 2024-04-26T04:42:16.414 INFO:tasks.workunit.client.0.smithi090.stdout:5/255: truncate d4/f46 888131 0 2024-04-26T04:42:16.414 INFO:tasks.workunit.client.0.smithi090.stdout:5/256: write d4/d1a/d3f/f44 [229949,57524] 0 2024-04-26T04:42:16.414 INFO:tasks.workunit.client.0.smithi090.stdout:5/257: readlink d4/d1a/d3f/l40 0 2024-04-26T04:42:16.415 INFO:tasks.workunit.client.1.smithi104.stdout:3/502: rename d5/d9/dc/d32/d47/f9b to d5/d9/d17/d60/faf 0 2024-04-26T04:42:16.416 INFO:tasks.workunit.client.0.smithi090.stdout:7/174: mkdir d0/d1/d4/d43 0 2024-04-26T04:42:16.416 INFO:tasks.workunit.client.0.smithi090.stdout:7/175: chown d0/d1 1875 1 2024-04-26T04:42:16.416 INFO:tasks.workunit.client.1.smithi104.stdout:8/316: mknod d0/d1c/c63 0 2024-04-26T04:42:16.418 INFO:tasks.workunit.client.1.smithi104.stdout:8/317: symlink d0/d1c/l64 0 2024-04-26T04:42:16.421 INFO:tasks.workunit.client.1.smithi104.stdout:7/391: getdents d2/d4/d32/d60 0 2024-04-26T04:42:16.422 INFO:tasks.workunit.client.0.smithi090.stdout:2/123: mkdir db/d11/d14/d27 0 2024-04-26T04:42:16.422 INFO:tasks.workunit.client.0.smithi090.stdout:2/124: readlink db/dc/l1a 0 2024-04-26T04:42:16.422 INFO:tasks.workunit.client.1.smithi104.stdout:7/392: symlink d2/l83 0 2024-04-26T04:42:16.424 INFO:tasks.workunit.client.1.smithi104.stdout:7/393: rmdir d2/d4/d32/d3d/d41/d5f/d80 0 2024-04-26T04:42:16.424 INFO:tasks.workunit.client.1.smithi104.stdout:7/394: dread - d2/d4/d32/d35/f7a zero size 2024-04-26T04:42:16.425 INFO:tasks.workunit.client.1.smithi104.stdout:7/395: mknod d2/d4/d32/d3d/d41/d5f/c84 0 2024-04-26T04:42:16.427 INFO:tasks.workunit.client.0.smithi090.stdout:9/135: symlink db/d1e/l30 0 2024-04-26T04:42:16.427 INFO:tasks.workunit.client.0.smithi090.stdout:4/237: mknod d3/db/d12/d20/d24/c4a 0 2024-04-26T04:42:16.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/258: mknod d4/d1a/d2a/c58 0 2024-04-26T04:42:16.427 INFO:tasks.workunit.client.0.smithi090.stdout:4/238: fdatasync d3/db/d12/d20/d24/d30/f37 0 2024-04-26T04:42:16.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/259: write d4/d1a/d1f/f3d [770310,113806] 0 2024-04-26T04:42:16.428 INFO:tasks.workunit.client.0.smithi090.stdout:2/125: mkdir db/d11/d14/d27/d28 0 2024-04-26T04:42:16.428 INFO:tasks.workunit.client.0.smithi090.stdout:9/136: read f3 [1030362,5372] 0 2024-04-26T04:42:16.431 INFO:tasks.workunit.client.1.smithi104.stdout:8/318: dread d0/d33/f2b [0,4194304] 0 2024-04-26T04:42:16.431 INFO:tasks.workunit.client.0.smithi090.stdout:2/126: dread db/dc/f12 [0,4194304] 0 2024-04-26T04:42:16.431 INFO:tasks.workunit.client.0.smithi090.stdout:2/127: write db/d11/d14/d1b/d20/f24 [691711,57980] 0 2024-04-26T04:42:16.432 INFO:tasks.workunit.client.1.smithi104.stdout:8/319: rmdir d0/d33/d51 39 2024-04-26T04:42:16.432 INFO:tasks.workunit.client.0.smithi090.stdout:2/128: link db/dc/ld db/dc/l29 0 2024-04-26T04:42:16.434 INFO:tasks.workunit.client.0.smithi090.stdout:2/129: rename f2 to db/d11/d14/d25/f2a 0 2024-04-26T04:42:16.435 INFO:tasks.workunit.client.1.smithi104.stdout:8/320: rename d0/d33/d51/c56 to d0/d50/c65 0 2024-04-26T04:42:16.435 INFO:tasks.workunit.client.1.smithi104.stdout:8/321: write d0/d1c/d27/f40 [2052555,118551] 0 2024-04-26T04:42:16.435 INFO:tasks.workunit.client.1.smithi104.stdout:8/322: write d0/f4d [593080,75868] 0 2024-04-26T04:42:16.435 INFO:tasks.workunit.client.0.smithi090.stdout:2/130: write db/d11/d14/d1b/d20/f24 [605599,125510] 0 2024-04-26T04:42:16.436 INFO:tasks.workunit.client.0.smithi090.stdout:2/131: chown db/d11/d14/d27 568568565 1 2024-04-26T04:42:16.436 INFO:tasks.workunit.client.0.smithi090.stdout:2/132: fdatasync db/dc/f12 0 2024-04-26T04:42:16.436 INFO:tasks.workunit.client.0.smithi090.stdout:4/239: dread d3/db/f19 [0,4194304] 0 2024-04-26T04:42:16.436 INFO:tasks.workunit.client.0.smithi090.stdout:2/133: symlink db/d11/d14/d27/d28/l2b 0 2024-04-26T04:42:16.436 INFO:tasks.workunit.client.0.smithi090.stdout:4/240: truncate d3/db/d12/d20/d24/d30/f37 700806 0 2024-04-26T04:42:16.436 INFO:tasks.workunit.client.0.smithi090.stdout:4/241: truncate d3/db/d12/f28 802353 0 2024-04-26T04:42:16.437 INFO:tasks.workunit.client.0.smithi090.stdout:4/242: chown d3/db/d12/d20/d24/d30 415985 1 2024-04-26T04:42:16.437 INFO:tasks.workunit.client.0.smithi090.stdout:4/243: chown d3/db/d12/d20/d24/d43 0 1 2024-04-26T04:42:16.437 INFO:tasks.workunit.client.0.smithi090.stdout:4/244: truncate d3/d13/f1f 509149 0 2024-04-26T04:42:16.438 INFO:tasks.workunit.client.1.smithi104.stdout:8/323: dread d0/d1c/d27/f5b [0,4194304] 0 2024-04-26T04:42:16.438 INFO:tasks.workunit.client.0.smithi090.stdout:4/245: symlink d3/db/d12/d3b/l4b 0 2024-04-26T04:42:16.439 INFO:tasks.workunit.client.0.smithi090.stdout:4/246: creat d3/db/d12/f4c x:0 0 0 2024-04-26T04:42:16.442 INFO:tasks.workunit.client.0.smithi090.stdout:4/247: link d3/f9 d3/db/d12/f4d 0 2024-04-26T04:42:16.443 INFO:tasks.workunit.client.0.smithi090.stdout:4/248: creat d3/db/d12/f4e x:0 0 0 2024-04-26T04:42:16.458 INFO:tasks.workunit.client.0.smithi090.stdout:4/249: dread d3/f36 [0,4194304] 0 2024-04-26T04:42:16.459 INFO:tasks.workunit.client.0.smithi090.stdout:4/250: write d3/f9 [2614342,24268] 0 2024-04-26T04:42:16.460 INFO:tasks.workunit.client.1.smithi104.stdout:3/503: dwrite d5/d9/d17/d46/d95/fa5 [0,4194304] 0 2024-04-26T04:42:16.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/251: dread d3/db/d12/d20/f41 [0,4194304] 0 2024-04-26T04:42:16.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/252: write d3/db/d12/d20/d24/d30/f49 [710701,126442] 0 2024-04-26T04:42:16.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/253: stat d3/d13/f1f 0 2024-04-26T04:42:16.462 INFO:tasks.workunit.client.0.smithi090.stdout:6/241: dwrite d9/d3c/d49/f25 [0,4194304] 0 2024-04-26T04:42:16.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/134: fsync db/d11/d14/d1b/d20/f24 0 2024-04-26T04:42:16.462 INFO:tasks.workunit.client.0.smithi090.stdout:7/176: dwrite d0/d1/d1f/f16 [0,4194304] 0 2024-04-26T04:42:16.462 INFO:tasks.workunit.client.0.smithi090.stdout:7/177: chown d0/d1/d4/f2e 238 1 2024-04-26T04:42:16.462 INFO:tasks.workunit.client.0.smithi090.stdout:9/137: dwrite db/f13 [0,4194304] 0 2024-04-26T04:42:16.462 INFO:tasks.workunit.client.0.smithi090.stdout:9/138: read - db/d1e/f2c zero size 2024-04-26T04:42:16.462 INFO:tasks.workunit.client.1.smithi104.stdout:8/324: dwrite d0/d33/f2b [4194304,4194304] 0 2024-04-26T04:42:16.463 INFO:tasks.workunit.client.0.smithi090.stdout:6/242: creat d9/d21/d40/d4c/f59 x:0 0 0 2024-04-26T04:42:16.464 INFO:tasks.workunit.client.1.smithi104.stdout:3/504: creat d5/d9/dc/d29/d7d/da9/fb0 x:0 0 0 2024-04-26T04:42:16.464 INFO:tasks.workunit.client.1.smithi104.stdout:3/505: write d5/d9/dc/f42 [834041,98609] 0 2024-04-26T04:42:16.464 INFO:tasks.workunit.client.1.smithi104.stdout:3/506: fsync d5/d9/d17/f34 0 2024-04-26T04:42:16.464 INFO:tasks.workunit.client.1.smithi104.stdout:3/507: fsync d5/d9/dc/d32/d48/dac/f89 0 2024-04-26T04:42:16.465 INFO:tasks.workunit.client.0.smithi090.stdout:2/135: symlink db/dc/l2c 0 2024-04-26T04:42:16.465 INFO:tasks.workunit.client.0.smithi090.stdout:6/243: dread d9/d3c/d49/f26 [0,4194304] 0 2024-04-26T04:42:16.466 INFO:tasks.workunit.client.0.smithi090.stdout:9/139: mkdir db/dd/d18/d31 0 2024-04-26T04:42:16.466 INFO:tasks.workunit.client.0.smithi090.stdout:6/244: read d9/d21/d40/f4b [609309,113486] 0 2024-04-26T04:42:16.473 INFO:tasks.workunit.client.0.smithi090.stdout:7/178: write d0/d1/d1f/f37 [2871219,128984] 0 2024-04-26T04:42:16.475 INFO:tasks.workunit.client.0.smithi090.stdout:2/136: mkdir db/d11/d14/d25/d2d 0 2024-04-26T04:42:16.476 INFO:tasks.workunit.client.0.smithi090.stdout:9/140: symlink db/d1e/l32 0 2024-04-26T04:42:16.477 INFO:tasks.workunit.client.0.smithi090.stdout:6/245: truncate d9/f11 11444 0 2024-04-26T04:42:16.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/179: stat d0/db/d15/l36 0 2024-04-26T04:42:16.480 INFO:tasks.workunit.client.1.smithi104.stdout:8/325: truncate d0/f34 202001 0 2024-04-26T04:42:16.483 INFO:tasks.workunit.client.1.smithi104.stdout:8/326: dread d0/d33/f2c [0,4194304] 0 2024-04-26T04:42:16.483 INFO:tasks.workunit.client.1.smithi104.stdout:3/508: truncate d5/d9/dc/d29/f56 3402067 0 2024-04-26T04:42:16.483 INFO:tasks.workunit.client.1.smithi104.stdout:8/327: creat d0/d1c/d27/f66 x:0 0 0 2024-04-26T04:42:16.489 INFO:tasks.workunit.client.1.smithi104.stdout:7/396: rmdir d2/d4 39 2024-04-26T04:42:16.494 INFO:tasks.workunit.client.1.smithi104.stdout:3/509: creat d5/d9/dc/d69/fb1 x:0 0 0 2024-04-26T04:42:16.495 INFO:tasks.workunit.client.1.smithi104.stdout:8/328: getdents d0/d33 0 2024-04-26T04:42:16.495 INFO:tasks.workunit.client.0.smithi090.stdout:9/141: unlink db/dd/d18/f20 0 2024-04-26T04:42:16.499 INFO:tasks.workunit.client.1.smithi104.stdout:8/329: mkdir d0/d1c/d27/d47/d67 0 2024-04-26T04:42:16.499 INFO:tasks.workunit.client.1.smithi104.stdout:8/330: write d0/d1c/d27/d47/f38 [529614,68051] 0 2024-04-26T04:42:16.501 INFO:tasks.workunit.client.0.smithi090.stdout:6/246: rename d9/d21/c2d to d9/d1c/d31/d3d/c5a 0 2024-04-26T04:42:16.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/180: creat d0/db/d15/d29/f44 x:0 0 0 2024-04-26T04:42:16.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/181: chown d0/db/d15/d30/f32 153 1 2024-04-26T04:42:16.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/182: write d0/d1/f38 [1434327,57713] 0 2024-04-26T04:42:16.505 INFO:tasks.workunit.client.0.smithi090.stdout:6/247: mknod d9/d21/d40/d4e/c5b 0 2024-04-26T04:42:16.505 INFO:tasks.workunit.client.0.smithi090.stdout:6/248: chown d9/d21/d40/d4e 2912 1 2024-04-26T04:42:16.511 INFO:tasks.workunit.client.0.smithi090.stdout:8/118: sync 2024-04-26T04:42:16.511 INFO:tasks.workunit.client.0.smithi090.stdout:1/152: sync 2024-04-26T04:42:16.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/119: truncate fa 2784502 0 2024-04-26T04:42:16.515 INFO:tasks.workunit.client.0.smithi090.stdout:1/153: mkdir d0/d4/d7/d17/d22/d31 0 2024-04-26T04:42:16.520 INFO:tasks.workunit.client.1.smithi104.stdout:8/331: write d0/d33/d35/f45 [786201,41704] 0 2024-04-26T04:42:16.522 INFO:tasks.workunit.client.1.smithi104.stdout:7/397: dwrite d2/d4/d32/d3d/d47/f61 [0,4194304] 0 2024-04-26T04:42:16.522 INFO:tasks.workunit.client.1.smithi104.stdout:7/398: fdatasync d2/d4/d5a/f68 0 2024-04-26T04:42:16.522 INFO:tasks.workunit.client.1.smithi104.stdout:7/399: chown d2/d4/l2d 4747 1 2024-04-26T04:42:16.522 INFO:tasks.workunit.client.1.smithi104.stdout:7/400: fsync d2/d4/f11 0 2024-04-26T04:42:16.523 INFO:tasks.workunit.client.0.smithi090.stdout:8/120: dread dc/f1a [0,4194304] 0 2024-04-26T04:42:16.524 INFO:tasks.workunit.client.0.smithi090.stdout:8/121: truncate dc/d1f/f27 69896 0 2024-04-26T04:42:16.535 INFO:tasks.workunit.client.1.smithi104.stdout:8/332: write d0/d33/f2b [6342345,124727] 0 2024-04-26T04:42:16.535 INFO:tasks.workunit.client.1.smithi104.stdout:8/333: creat d0/d50/f68 x:0 0 0 2024-04-26T04:42:16.535 INFO:tasks.workunit.client.1.smithi104.stdout:8/334: readlink d0/d33/l60 0 2024-04-26T04:42:16.535 INFO:tasks.workunit.client.1.smithi104.stdout:8/335: write d0/d1c/d2d/f48 [4039721,121686] 0 2024-04-26T04:42:16.537 INFO:tasks.workunit.client.1.smithi104.stdout:7/401: dread d2/d4/f33 [0,4194304] 0 2024-04-26T04:42:16.537 INFO:tasks.workunit.client.1.smithi104.stdout:7/402: fsync d2/d4/d1d/f29 0 2024-04-26T04:42:16.537 INFO:tasks.workunit.client.1.smithi104.stdout:8/336: creat d0/d1c/d59/f69 x:0 0 0 2024-04-26T04:42:16.537 INFO:tasks.workunit.client.1.smithi104.stdout:8/337: write d0/d33/f2b [6940960,53612] 0 2024-04-26T04:42:16.539 INFO:tasks.workunit.client.0.smithi090.stdout:6/249: dwrite d9/d21/d3b/f51 [0,4194304] 0 2024-04-26T04:42:16.539 INFO:tasks.workunit.client.0.smithi090.stdout:6/250: chown d9/c20 2 1 2024-04-26T04:42:16.543 INFO:tasks.workunit.client.0.smithi090.stdout:6/251: rename d9/d1c/d31/d3d/f3e to d9/f5c 0 2024-04-26T04:42:16.543 INFO:tasks.workunit.client.0.smithi090.stdout:6/252: write d9/d3c/d49/f43 [5797452,711] 0 2024-04-26T04:42:16.552 INFO:tasks.workunit.client.0.smithi090.stdout:2/137: dwrite db/d11/d14/d1b/d20/f24 [0,4194304] 0 2024-04-26T04:42:16.552 INFO:tasks.workunit.client.0.smithi090.stdout:2/138: readlink db/l13 0 2024-04-26T04:42:16.552 INFO:tasks.workunit.client.0.smithi090.stdout:2/139: fsync db/dc/f19 0 2024-04-26T04:42:16.552 INFO:tasks.workunit.client.0.smithi090.stdout:7/183: fdatasync d0/d1/d1f/f37 0 2024-04-26T04:42:16.552 INFO:tasks.workunit.client.0.smithi090.stdout:7/184: chown d0/d1/d4/f10 509943201 1 2024-04-26T04:42:16.552 INFO:tasks.workunit.client.0.smithi090.stdout:7/185: readlink d0/db/d15/l36 0 2024-04-26T04:42:16.553 INFO:tasks.workunit.client.0.smithi090.stdout:7/186: creat d0/f45 x:0 0 0 2024-04-26T04:42:16.554 INFO:tasks.workunit.client.0.smithi090.stdout:2/140: symlink db/d11/l2e 0 2024-04-26T04:42:16.556 INFO:tasks.workunit.client.0.smithi090.stdout:9/142: dwrite db/f14 [4194304,4194304] 0 2024-04-26T04:42:16.556 INFO:tasks.workunit.client.0.smithi090.stdout:9/143: fdatasync db/dd/d18/f2a 0 2024-04-26T04:42:16.560 INFO:tasks.workunit.client.1.smithi104.stdout:2/603: sync 2024-04-26T04:42:16.560 INFO:tasks.workunit.client.0.smithi090.stdout:7/187: symlink d0/db/l46 0 2024-04-26T04:42:16.563 INFO:tasks.workunit.client.0.smithi090.stdout:7/188: link d0/db/d15/d29/fc d0/d1/d4/f47 0 2024-04-26T04:42:16.564 INFO:tasks.workunit.client.0.smithi090.stdout:1/154: dread d0/d4/f15 [0,4194304] 0 2024-04-26T04:42:16.566 INFO:tasks.workunit.client.0.smithi090.stdout:7/189: symlink d0/db/d15/d29/l48 0 2024-04-26T04:42:16.566 INFO:tasks.workunit.client.0.smithi090.stdout:7/190: chown d0/db/df/d1c/d23/f31 1249700 1 2024-04-26T04:42:16.568 INFO:tasks.workunit.client.0.smithi090.stdout:7/191: mknod d0/d1/d4/d43/c49 0 2024-04-26T04:42:16.569 INFO:tasks.workunit.client.1.smithi104.stdout:2/604: read d6/fc [880613,9269] 0 2024-04-26T04:42:16.572 INFO:tasks.workunit.client.1.smithi104.stdout:2/605: mkdir d6/d1b/d68/d81/dce 0 2024-04-26T04:42:16.578 INFO:tasks.workunit.client.1.smithi104.stdout:2/606: dread d6/d8/d3e/f49 [0,4194304] 0 2024-04-26T04:42:16.587 INFO:tasks.workunit.client.0.smithi090.stdout:8/122: dwrite dc/f1e [0,4194304] 0 2024-04-26T04:42:16.589 INFO:tasks.workunit.client.1.smithi104.stdout:2/607: dread d6/de/d1d/f43 [0,4194304] 0 2024-04-26T04:42:16.591 INFO:tasks.workunit.client.0.smithi090.stdout:8/123: mknod dc/c29 0 2024-04-26T04:42:16.591 INFO:tasks.workunit.client.0.smithi090.stdout:8/124: dread - dc/d1f/f21 zero size 2024-04-26T04:42:16.591 INFO:tasks.workunit.client.1.smithi104.stdout:2/608: mkdir d6/de/d1d/dcf 0 2024-04-26T04:42:16.593 INFO:tasks.workunit.client.0.smithi090.stdout:8/125: truncate dc/d12/f13 3546143 0 2024-04-26T04:42:16.595 INFO:tasks.workunit.client.0.smithi090.stdout:8/126: unlink f5 0 2024-04-26T04:42:16.595 INFO:tasks.workunit.client.0.smithi090.stdout:8/127: stat dc/f1a 0 2024-04-26T04:42:16.596 INFO:tasks.workunit.client.0.smithi090.stdout:8/128: symlink dc/d1f/l2a 0 2024-04-26T04:42:16.597 INFO:tasks.workunit.client.0.smithi090.stdout:6/253: dwrite d9/d21/d40/f4b [0,4194304] 0 2024-04-26T04:42:16.597 INFO:tasks.workunit.client.0.smithi090.stdout:9/144: dwrite db/dd/ff [0,4194304] 0 2024-04-26T04:42:16.599 INFO:tasks.workunit.client.1.smithi104.stdout:3/510: dwrite d5/d9/dc/f1e [0,4194304] 0 2024-04-26T04:42:16.599 INFO:tasks.workunit.client.1.smithi104.stdout:3/511: chown d5/d9/dc/d3a 1 1 2024-04-26T04:42:16.601 INFO:tasks.workunit.client.0.smithi090.stdout:6/254: rename d9/d21/d40/l52 to d9/d21/d40/d4c/l5d 0 2024-04-26T04:42:16.601 INFO:tasks.workunit.client.0.smithi090.stdout:6/255: write d9/d21/d40/d4c/f59 [748267,119628] 0 2024-04-26T04:42:16.605 INFO:tasks.workunit.client.0.smithi090.stdout:1/155: dwrite d0/d4/d7/f2c [0,4194304] 0 2024-04-26T04:42:16.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/192: dwrite d0/db/d15/d29/f14 [4194304,4194304] 0 2024-04-26T04:42:16.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/193: write d0/d1/d4/f3b [1392107,116185] 0 2024-04-26T04:42:16.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/194: truncate d0/db/d15/d30/f40 374449 0 2024-04-26T04:42:16.606 INFO:tasks.workunit.client.0.smithi090.stdout:9/145: rename db/dd/d18/f25 to db/d16/f33 0 2024-04-26T04:42:16.606 INFO:tasks.workunit.client.0.smithi090.stdout:9/146: creat db/d1e/f34 x:0 0 0 2024-04-26T04:42:16.607 INFO:tasks.workunit.client.1.smithi104.stdout:3/512: rename d5/d9/d17 to d5/db2 0 2024-04-26T04:42:16.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/156: symlink d0/d4/l32 0 2024-04-26T04:42:16.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/157: chown d0/d4/l32 2426 1 2024-04-26T04:42:16.608 INFO:tasks.workunit.client.1.smithi104.stdout:3/513: read d5/db2/f3c [1309532,109824] 0 2024-04-26T04:42:16.608 INFO:tasks.workunit.client.1.smithi104.stdout:3/514: write d5/db2/f91 [632117,93161] 0 2024-04-26T04:42:16.609 INFO:tasks.workunit.client.0.smithi090.stdout:2/141: dwrite db/dc/f19 [0,4194304] 0 2024-04-26T04:42:16.609 INFO:tasks.workunit.client.1.smithi104.stdout:3/515: creat d5/fb3 x:0 0 0 2024-04-26T04:42:16.610 INFO:tasks.workunit.client.1.smithi104.stdout:3/516: truncate d5/d9/dc/d32/d48/dac/f89 502321 0 2024-04-26T04:42:16.610 INFO:tasks.workunit.client.1.smithi104.stdout:3/517: readlink d5/d9/dc/d69/l45 0 2024-04-26T04:42:16.610 INFO:tasks.workunit.client.1.smithi104.stdout:3/518: dread - d5/d9/dc/d29/d7d/fa0 zero size 2024-04-26T04:42:16.617 INFO:tasks.workunit.client.0.smithi090.stdout:7/195: mkdir d0/d1/d4/d43/d4a 0 2024-04-26T04:42:16.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/147: creat db/f35 x:0 0 0 2024-04-26T04:42:16.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/148: creat db/dd/d18/f36 x:0 0 0 2024-04-26T04:42:16.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/149: dread - db/dd/d18/f2a zero size 2024-04-26T04:42:16.618 INFO:tasks.workunit.client.0.smithi090.stdout:1/158: dread d0/d4/d7/f2c [0,4194304] 0 2024-04-26T04:42:16.619 INFO:tasks.workunit.client.0.smithi090.stdout:6/256: rename d9/l13 to d9/l5e 0 2024-04-26T04:42:16.622 INFO:tasks.workunit.client.0.smithi090.stdout:2/142: mkdir db/d11/d14/d27/d2f 0 2024-04-26T04:42:16.622 INFO:tasks.workunit.client.0.smithi090.stdout:2/143: creat db/d11/f30 x:0 0 0 2024-04-26T04:42:16.623 INFO:tasks.workunit.client.0.smithi090.stdout:7/196: unlink d0/db/d15/d29/c33 0 2024-04-26T04:42:16.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/159: truncate d0/d4/d7/d17/f18 377992 0 2024-04-26T04:42:16.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/160: creat d0/d4/d7/f33 x:0 0 0 2024-04-26T04:42:16.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/161: dread - d0/d4/d7/d17/d22/f2f zero size 2024-04-26T04:42:16.626 INFO:tasks.workunit.client.0.smithi090.stdout:1/162: chown d0/c11 12 1 2024-04-26T04:42:16.628 INFO:tasks.workunit.client.0.smithi090.stdout:2/144: symlink db/d11/d14/d25/l31 0 2024-04-26T04:42:16.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/163: creat d0/d4/d7/d17/d22/d31/f34 x:0 0 0 2024-04-26T04:42:16.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/164: stat d0/f2e 0 2024-04-26T04:42:16.632 INFO:tasks.workunit.client.0.smithi090.stdout:2/145: creat db/d11/d14/d27/d2f/f32 x:0 0 0 2024-04-26T04:42:16.632 INFO:tasks.workunit.client.0.smithi090.stdout:2/146: creat db/f33 x:0 0 0 2024-04-26T04:42:16.633 INFO:tasks.workunit.client.0.smithi090.stdout:7/197: rmdir d0/db/d15/d30 39 2024-04-26T04:42:16.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/257: write d9/f12 [4169143,126184] 0 2024-04-26T04:42:16.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/258: creat d9/d21/f5f x:0 0 0 2024-04-26T04:42:16.635 INFO:tasks.workunit.client.1.smithi104.stdout:3/519: dwrite d5/db2/d46/d95/fa5 [0,4194304] 0 2024-04-26T04:42:16.635 INFO:tasks.workunit.client.1.smithi104.stdout:3/520: chown d5/d9/dc/d29/d7d/fa0 186016 1 2024-04-26T04:42:16.635 INFO:tasks.workunit.client.1.smithi104.stdout:3/521: fdatasync d5/d9/fe 0 2024-04-26T04:42:16.635 INFO:tasks.workunit.client.1.smithi104.stdout:3/522: write d5/db2/f3c [2479921,16304] 0 2024-04-26T04:42:16.637 INFO:tasks.workunit.client.1.smithi104.stdout:3/523: mknod d5/d9/d80/cb4 0 2024-04-26T04:42:16.637 INFO:tasks.workunit.client.1.smithi104.stdout:3/524: chown d5/d9/dc/d32/l9c 1059504 1 2024-04-26T04:42:16.639 INFO:tasks.workunit.client.0.smithi090.stdout:8/129: dwrite dc/d12/f28 [0,4194304] 0 2024-04-26T04:42:16.642 INFO:tasks.workunit.client.1.smithi104.stdout:2/609: dwrite d6/d1b/d51/d6a/fc5 [0,4194304] 0 2024-04-26T04:42:16.642 INFO:tasks.workunit.client.1.smithi104.stdout:2/610: write d6/d1b/d51/f58 [802877,71304] 0 2024-04-26T04:42:16.642 INFO:tasks.workunit.client.1.smithi104.stdout:8/338: dwrite d0/d33/f2b [0,4194304] 0 2024-04-26T04:42:16.642 INFO:tasks.workunit.client.1.smithi104.stdout:3/525: link d5/d9/dc/f16 d5/d9/d86/fb5 0 2024-04-26T04:42:16.642 INFO:tasks.workunit.client.1.smithi104.stdout:3/526: dread - d5/d9/dc/d32/d48/f78 zero size 2024-04-26T04:42:16.645 INFO:tasks.workunit.client.0.smithi090.stdout:7/198: dread d0/f8 [0,4194304] 0 2024-04-26T04:42:16.646 INFO:tasks.workunit.client.1.smithi104.stdout:2/611: mknod d6/d1b/d68/d81/d97/d99/cd0 0 2024-04-26T04:42:16.648 INFO:tasks.workunit.client.1.smithi104.stdout:8/339: creat d0/d1c/d2d/f6a x:0 0 0 2024-04-26T04:42:16.649 INFO:tasks.workunit.client.1.smithi104.stdout:8/340: dread - d0/d33/d35/f3e zero size 2024-04-26T04:42:16.650 INFO:tasks.workunit.client.1.smithi104.stdout:3/527: unlink d5/d9/dc/d3a/c83 0 2024-04-26T04:42:16.651 INFO:tasks.workunit.client.1.smithi104.stdout:2/612: getdents d6/d1b 0 2024-04-26T04:42:16.651 INFO:tasks.workunit.client.1.smithi104.stdout:2/613: truncate d6/d8/d3e/fc0 136517 0 2024-04-26T04:42:16.651 INFO:tasks.workunit.client.0.smithi090.stdout:6/259: dread d9/d3c/d49/f29 [0,4194304] 0 2024-04-26T04:42:16.653 INFO:tasks.workunit.client.1.smithi104.stdout:8/341: mkdir d0/d1c/d27/d47/d6b 0 2024-04-26T04:42:16.655 INFO:tasks.workunit.client.1.smithi104.stdout:2/614: mkdir d6/d8/d5a/dd1 0 2024-04-26T04:42:16.655 INFO:tasks.workunit.client.1.smithi104.stdout:2/615: stat d6/d1b/d68/c8d 0 2024-04-26T04:42:16.655 INFO:tasks.workunit.client.1.smithi104.stdout:2/616: truncate d6/d1b/d68/fad 1811 0 2024-04-26T04:42:16.655 INFO:tasks.workunit.client.1.smithi104.stdout:2/617: chown d6/de/d1d/f25 1230 1 2024-04-26T04:42:16.656 INFO:tasks.workunit.client.1.smithi104.stdout:8/342: unlink d0/f34 0 2024-04-26T04:42:16.656 INFO:tasks.workunit.client.0.smithi090.stdout:7/199: read d0/d1/d4/f47 [11645008,505] 0 2024-04-26T04:42:16.657 INFO:tasks.workunit.client.0.smithi090.stdout:7/200: truncate d0/db/d15/d30/f40 1003082 0 2024-04-26T04:42:16.657 INFO:tasks.workunit.client.0.smithi090.stdout:7/201: chown d0/f3c 36 1 2024-04-26T04:42:16.657 INFO:tasks.workunit.client.1.smithi104.stdout:2/618: symlink d6/d1b/ld2 0 2024-04-26T04:42:16.663 INFO:tasks.workunit.client.0.smithi090.stdout:6/260: read d9/f14 [1136611,98768] 0 2024-04-26T04:42:16.665 INFO:tasks.workunit.client.1.smithi104.stdout:3/528: write d5/d9/dc/d3a/f57 [1953386,119103] 0 2024-04-26T04:42:16.665 INFO:tasks.workunit.client.1.smithi104.stdout:3/529: creat d5/d9/dc/d69/fb6 x:0 0 0 2024-04-26T04:42:16.665 INFO:tasks.workunit.client.1.smithi104.stdout:3/530: write d5/db2/f91 [1353034,127060] 0 2024-04-26T04:42:16.673 INFO:tasks.workunit.client.1.smithi104.stdout:8/343: dread d0/d1c/d27/d47/f25 [0,4194304] 0 2024-04-26T04:42:16.673 INFO:tasks.workunit.client.1.smithi104.stdout:8/344: read d0/f2f [2646567,33497] 0 2024-04-26T04:42:16.673 INFO:tasks.workunit.client.1.smithi104.stdout:8/345: readlink d0/d1c/d2d/l54 0 2024-04-26T04:42:16.674 INFO:tasks.workunit.client.0.smithi090.stdout:1/165: dwrite d0/d4/d7/f30 [0,4194304] 0 2024-04-26T04:42:16.674 INFO:tasks.workunit.client.0.smithi090.stdout:1/166: write d0/d4/f15 [6884958,89045] 0 2024-04-26T04:42:16.674 INFO:tasks.workunit.client.0.smithi090.stdout:2/147: dwrite db/d11/d14/d27/d2f/f32 [0,4194304] 0 2024-04-26T04:42:16.675 INFO:tasks.workunit.client.1.smithi104.stdout:2/619: dwrite d6/d8/d3e/fc0 [0,4194304] 0 2024-04-26T04:42:16.675 INFO:tasks.workunit.client.1.smithi104.stdout:2/620: chown d6/d8/d3e/c73 83460 1 2024-04-26T04:42:16.682 INFO:tasks.workunit.client.1.smithi104.stdout:2/621: mknod d6/d1b/d51/d6a/cd3 0 2024-04-26T04:42:16.683 INFO:tasks.workunit.client.1.smithi104.stdout:2/622: fsync d6/d8/f37 0 2024-04-26T04:42:16.683 INFO:tasks.workunit.client.0.smithi090.stdout:2/148: write db/dc/f19 [777497,24275] 0 2024-04-26T04:42:16.683 INFO:tasks.workunit.client.1.smithi104.stdout:2/623: mknod d6/de/d1d/dcf/cd4 0 2024-04-26T04:42:16.687 INFO:tasks.workunit.client.0.smithi090.stdout:6/261: dwrite d9/f12 [0,4194304] 0 2024-04-26T04:42:16.689 INFO:tasks.workunit.client.1.smithi104.stdout:2/624: dread d6/d1b/f38 [0,4194304] 0 2024-04-26T04:42:16.689 INFO:tasks.workunit.client.1.smithi104.stdout:2/625: stat d6/de/d85/fcc 0 2024-04-26T04:42:16.690 INFO:tasks.workunit.client.1.smithi104.stdout:6/617: sync 2024-04-26T04:42:16.691 INFO:tasks.workunit.client.1.smithi104.stdout:2/626: rename d6/d8/f17 to d6/d1b/d68/fd5 0 2024-04-26T04:42:16.693 INFO:tasks.workunit.client.1.smithi104.stdout:6/618: rename d0/d9/d13/l3a to d0/d9/d13/d41/d96/dac/ld2 0 2024-04-26T04:42:16.693 INFO:tasks.workunit.client.1.smithi104.stdout:6/619: readlink d0/d9/d13/d44/d71/l23 0 2024-04-26T04:42:16.693 INFO:tasks.workunit.client.1.smithi104.stdout:6/620: fdatasync d0/d9/f2a 0 2024-04-26T04:42:16.693 INFO:tasks.workunit.client.1.smithi104.stdout:6/621: stat d0/d9/d13/d44/d71/f74 0 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/167: dread d0/f12 [4194304,4194304] 0 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/168: readlink d0/l6 0 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/169: write d0/f2e [692315,107380] 0 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.1.smithi104.stdout:2/627: creat d6/d8/d3e/fd6 x:0 0 0 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.1.smithi104.stdout:2/628: chown d6/d1b/d68/d81/d97/d99 2768 1 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.1.smithi104.stdout:6/622: getdents d0/d9/d13/d44/d47 0 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.1.smithi104.stdout:6/623: truncate d0/d9/d13/d41/fbf 1876966 0 2024-04-26T04:42:16.695 INFO:tasks.workunit.client.1.smithi104.stdout:6/624: write d0/d9/dc6/f24 [4446187,40709] 0 2024-04-26T04:42:16.696 INFO:tasks.workunit.client.1.smithi104.stdout:6/625: mknod d0/d9/d13/d44/d47/cd3 0 2024-04-26T04:42:16.697 INFO:tasks.workunit.client.1.smithi104.stdout:6/626: mknod d0/d9/d13/d44/cd4 0 2024-04-26T04:42:16.697 INFO:tasks.workunit.client.1.smithi104.stdout:2/629: dread d6/fc [0,4194304] 0 2024-04-26T04:42:16.697 INFO:tasks.workunit.client.1.smithi104.stdout:2/630: write d6/de/d1d/f43 [5165036,43630] 0 2024-04-26T04:42:16.699 INFO:tasks.workunit.client.0.smithi090.stdout:8/130: truncate dc/d12/f20 3198827 0 2024-04-26T04:42:16.700 INFO:tasks.workunit.client.1.smithi104.stdout:8/346: dwrite d0/d1c/f26 [4194304,4194304] 0 2024-04-26T04:42:16.700 INFO:tasks.workunit.client.0.smithi090.stdout:7/202: creat d0/d1/d4/d3e/f4b x:0 0 0 2024-04-26T04:42:16.701 INFO:tasks.workunit.client.0.smithi090.stdout:7/203: chown d0/db/df/d1c/d23/f31 0 1 2024-04-26T04:42:16.701 INFO:tasks.workunit.client.1.smithi104.stdout:8/347: mknod d0/d1c/d59/c6c 0 2024-04-26T04:42:16.702 INFO:tasks.workunit.client.1.smithi104.stdout:8/348: mkdir d0/d33/d35/d6d 0 2024-04-26T04:42:16.716 INFO:tasks.workunit.client.0.smithi090.stdout:0/215: sync 2024-04-26T04:42:16.716 INFO:tasks.workunit.client.0.smithi090.stdout:3/138: sync 2024-04-26T04:42:16.717 INFO:tasks.workunit.client.0.smithi090.stdout:3/139: fdatasync d1/d9/f12 0 2024-04-26T04:42:16.717 INFO:tasks.workunit.client.0.smithi090.stdout:6/262: rename c8 to d9/d1c/c60 0 2024-04-26T04:42:16.717 INFO:tasks.workunit.client.0.smithi090.stdout:6/263: chown d9/d1c/d31 346168 1 2024-04-26T04:42:16.717 INFO:tasks.workunit.client.0.smithi090.stdout:6/264: fsync d9/d1c/f47 0 2024-04-26T04:42:16.717 INFO:tasks.workunit.client.0.smithi090.stdout:1/170: mknod d0/c35 0 2024-04-26T04:42:16.718 INFO:tasks.workunit.client.0.smithi090.stdout:1/171: read d0/d4/f20 [135559,79478] 0 2024-04-26T04:42:16.720 INFO:tasks.workunit.client.0.smithi090.stdout:0/216: dread d0/de/f2c [0,4194304] 0 2024-04-26T04:42:16.720 INFO:tasks.workunit.client.0.smithi090.stdout:6/265: creat d9/d3c/f61 x:0 0 0 2024-04-26T04:42:16.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:16 smithi090 ceph-mon[26648]: pgmap v250: 65 pgs: 65 active+clean; 897 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 82 MiB/s rd, 102 MiB/s wr, 772 op/s 2024-04-26T04:42:16.721 INFO:tasks.workunit.client.0.smithi090.stdout:8/131: symlink dc/d12/l2b 0 2024-04-26T04:42:16.721 INFO:tasks.workunit.client.0.smithi090.stdout:8/132: rename dc to dc/d2c 22 2024-04-26T04:42:16.723 INFO:tasks.workunit.client.0.smithi090.stdout:0/217: rmdir d0/d9/d46 0 2024-04-26T04:42:16.723 INFO:tasks.workunit.client.0.smithi090.stdout:9/150: rmdir db/dd/d18 39 2024-04-26T04:42:16.723 INFO:tasks.workunit.client.0.smithi090.stdout:9/151: chown f9 835 1 2024-04-26T04:42:16.725 INFO:tasks.workunit.client.0.smithi090.stdout:8/133: symlink dc/d1f/l2d 0 2024-04-26T04:42:16.725 INFO:tasks.workunit.client.0.smithi090.stdout:2/149: fsync db/d11/f30 0 2024-04-26T04:42:16.726 INFO:tasks.workunit.client.0.smithi090.stdout:0/218: creat d0/de/f47 x:0 0 0 2024-04-26T04:42:16.726 INFO:tasks.workunit.client.0.smithi090.stdout:0/219: dread - d0/de/f2f zero size 2024-04-26T04:42:16.726 INFO:tasks.workunit.client.0.smithi090.stdout:0/220: write d0/d9/f41 [1044458,110485] 0 2024-04-26T04:42:16.727 INFO:tasks.workunit.client.0.smithi090.stdout:0/221: chown d0/f3b 120 1 2024-04-26T04:42:16.729 INFO:tasks.workunit.client.0.smithi090.stdout:2/150: mkdir db/d11/d14/d34 0 2024-04-26T04:42:16.730 INFO:tasks.workunit.client.0.smithi090.stdout:3/140: read d1/d9/f20 [3317127,24311] 0 2024-04-26T04:42:16.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/134: rename dc/d12/f13 to dc/d12/f2e 0 2024-04-26T04:42:16.732 INFO:tasks.workunit.client.0.smithi090.stdout:2/151: symlink db/d11/d14/d1b/l35 0 2024-04-26T04:42:16.735 INFO:tasks.workunit.client.0.smithi090.stdout:2/152: creat db/d11/d14/d25/d2d/f36 x:0 0 0 2024-04-26T04:42:16.735 INFO:tasks.workunit.client.0.smithi090.stdout:2/153: chown db/d11/d14/d1b/d20 499000 1 2024-04-26T04:42:16.740 INFO:tasks.workunit.client.1.smithi104.stdout:2/631: dwrite d6/de/d1d/f43 [4194304,4194304] 0 2024-04-26T04:42:16.741 INFO:tasks.workunit.client.0.smithi090.stdout:4/254: sync 2024-04-26T04:42:16.741 INFO:tasks.workunit.client.0.smithi090.stdout:5/260: sync 2024-04-26T04:42:16.741 INFO:tasks.workunit.client.0.smithi090.stdout:5/261: truncate d4/d1a/d1f/f3a 1198869 0 2024-04-26T04:42:16.742 INFO:tasks.workunit.client.0.smithi090.stdout:5/262: truncate d4/d1a/d45/f3c 831485 0 2024-04-26T04:42:16.744 INFO:tasks.workunit.client.0.smithi090.stdout:5/263: mkdir d4/d1a/d1f/d53/d59 0 2024-04-26T04:42:16.746 INFO:tasks.workunit.client.0.smithi090.stdout:0/222: read d0/d9/f17 [1999814,129230] 0 2024-04-26T04:42:16.747 INFO:tasks.workunit.client.0.smithi090.stdout:5/264: rename d4/d1a/d1f/l57 to d4/d1a/d1f/d53/l5a 0 2024-04-26T04:42:16.747 INFO:tasks.workunit.client.1.smithi104.stdout:8/349: dwrite d0/f1e [0,4194304] 0 2024-04-26T04:42:16.748 INFO:tasks.workunit.client.1.smithi104.stdout:8/350: creat d0/d1c/f6e x:0 0 0 2024-04-26T04:42:16.748 INFO:tasks.workunit.client.1.smithi104.stdout:8/351: stat d0/d1c/d59/c6c 0 2024-04-26T04:42:16.748 INFO:tasks.workunit.client.1.smithi104.stdout:8/352: chown d0/d1c/f26 80 1 2024-04-26T04:42:16.748 INFO:tasks.workunit.client.1.smithi104.stdout:8/353: write d0/d1c/d27/d47/f1d [783369,98215] 0 2024-04-26T04:42:16.748 INFO:tasks.workunit.client.1.smithi104.stdout:6/627: dwrite d0/d9/fbc [0,4194304] 0 2024-04-26T04:42:16.748 INFO:tasks.workunit.client.0.smithi090.stdout:7/204: dwrite d0/db/df/d1c/d23/f26 [0,4194304] 0 2024-04-26T04:42:16.751 INFO:tasks.workunit.client.0.smithi090.stdout:7/205: creat d0/d1/d4/d43/f4c x:0 0 0 2024-04-26T04:42:16.751 INFO:tasks.workunit.client.0.smithi090.stdout:7/206: chown d0/db/d15/d29/f18 4 1 2024-04-26T04:42:16.751 INFO:tasks.workunit.client.0.smithi090.stdout:1/172: dread d0/f2e [0,4194304] 0 2024-04-26T04:42:16.752 INFO:tasks.workunit.client.0.smithi090.stdout:7/207: fdatasync d0/d1/d1f/f37 0 2024-04-26T04:42:16.752 INFO:tasks.workunit.client.0.smithi090.stdout:2/154: dread db/d11/d14/d25/f2a [4194304,4194304] 0 2024-04-26T04:42:16.753 INFO:tasks.workunit.client.0.smithi090.stdout:9/152: dwrite db/f13 [0,4194304] 0 2024-04-26T04:42:16.753 INFO:tasks.workunit.client.0.smithi090.stdout:6/266: dwrite d9/d21/f48 [0,4194304] 0 2024-04-26T04:42:16.757 INFO:tasks.workunit.client.0.smithi090.stdout:7/208: unlink d0/d1/d1f/f37 0 2024-04-26T04:42:16.760 INFO:tasks.workunit.client.0.smithi090.stdout:2/155: creat db/d11/d14/d25/f37 x:0 0 0 2024-04-26T04:42:16.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:16 smithi104 ceph-mon[32369]: pgmap v250: 65 pgs: 65 active+clean; 897 MiB data, 2.6 GiB used, 534 GiB / 536 GiB avail; 82 MiB/s rd, 102 MiB/s wr, 772 op/s 2024-04-26T04:42:16.762 INFO:tasks.workunit.client.0.smithi090.stdout:0/223: write d0/d26/f24 [532603,24121] 0 2024-04-26T04:42:16.762 INFO:tasks.workunit.client.0.smithi090.stdout:0/224: fdatasync d0/d9/f41 0 2024-04-26T04:42:16.763 INFO:tasks.workunit.client.1.smithi104.stdout:6/628: write d0/d9/fbc [268524,57469] 0 2024-04-26T04:42:16.763 INFO:tasks.workunit.client.1.smithi104.stdout:6/629: write d0/d9/d25/f85 [479979,107474] 0 2024-04-26T04:42:16.763 INFO:tasks.workunit.client.1.smithi104.stdout:6/630: dread - d0/d9/f49 zero size 2024-04-26T04:42:16.763 INFO:tasks.workunit.client.1.smithi104.stdout:6/631: chown d0/d9/d13/d41/d96/dac/f4e 114778786 1 2024-04-26T04:42:16.764 INFO:tasks.workunit.client.0.smithi090.stdout:9/153: unlink db/fc 0 2024-04-26T04:42:16.765 INFO:tasks.workunit.client.1.smithi104.stdout:6/632: symlink d0/d9/da8/ld5 0 2024-04-26T04:42:16.766 INFO:tasks.workunit.client.1.smithi104.stdout:6/633: write d0/d9/d13/d41/d96/dac/d8e/fb4 [110565,42024] 0 2024-04-26T04:42:16.766 INFO:tasks.workunit.client.1.smithi104.stdout:6/634: truncate d0/d9/d13/d44/d71/fb9 25273 0 2024-04-26T04:42:16.769 INFO:tasks.workunit.client.0.smithi090.stdout:6/267: getdents d9/d3c/d49 0 2024-04-26T04:42:16.771 INFO:tasks.workunit.client.0.smithi090.stdout:8/135: rmdir dc 39 2024-04-26T04:42:16.772 INFO:tasks.workunit.client.0.smithi090.stdout:1/173: getdents d0/d4 0 2024-04-26T04:42:16.772 INFO:tasks.workunit.client.0.smithi090.stdout:1/174: write d0/d4/d7/d17/f29 [660988,98261] 0 2024-04-26T04:42:16.776 INFO:tasks.workunit.client.0.smithi090.stdout:4/255: dwrite d3/db/f2f [0,4194304] 0 2024-04-26T04:42:16.777 INFO:tasks.workunit.client.1.smithi104.stdout:2/632: dwrite d6/de/d1d/f83 [0,4194304] 0 2024-04-26T04:42:16.780 INFO:tasks.workunit.client.1.smithi104.stdout:2/633: mkdir d6/de/d1d/dd7 0 2024-04-26T04:42:16.787 INFO:tasks.workunit.client.1.smithi104.stdout:8/354: dwrite d0/fe [0,4194304] 0 2024-04-26T04:42:16.787 INFO:tasks.workunit.client.0.smithi090.stdout:5/265: dwrite d4/d1a/df/d4a/f51 [4194304,4194304] 0 2024-04-26T04:42:16.787 INFO:tasks.workunit.client.0.smithi090.stdout:5/266: readlink d4/d1a/d1f/l24 0 2024-04-26T04:42:16.790 INFO:tasks.workunit.client.1.smithi104.stdout:8/355: creat d0/d1c/d27/d46/f6f x:0 0 0 2024-04-26T04:42:16.791 INFO:tasks.workunit.client.1.smithi104.stdout:8/356: unlink d0/d1c/d2d/c30 0 2024-04-26T04:42:16.792 INFO:tasks.workunit.client.1.smithi104.stdout:8/357: write d0/d1c/d59/f69 [199008,46115] 0 2024-04-26T04:42:16.792 INFO:tasks.workunit.client.1.smithi104.stdout:8/358: write d0/d33/f1f [4024646,60195] 0 2024-04-26T04:42:16.793 INFO:tasks.workunit.client.1.smithi104.stdout:2/634: read d6/d1b/d93/faa [2143974,78684] 0 2024-04-26T04:42:16.794 INFO:tasks.workunit.client.1.smithi104.stdout:2/635: mkdir d6/d1b/d51/d6a/dd8 0 2024-04-26T04:42:16.795 INFO:tasks.workunit.client.1.smithi104.stdout:8/359: rmdir d0/d1c/d59 39 2024-04-26T04:42:16.795 INFO:tasks.workunit.client.1.smithi104.stdout:8/360: chown d0/d1c/d27/l4b 28557573 1 2024-04-26T04:42:16.796 INFO:tasks.workunit.client.1.smithi104.stdout:2/636: creat d6/d8/fd9 x:0 0 0 2024-04-26T04:42:16.796 INFO:tasks.workunit.client.0.smithi090.stdout:5/267: dread d4/d1a/df/d36/f4d [0,4194304] 0 2024-04-26T04:42:16.797 INFO:tasks.workunit.client.1.smithi104.stdout:8/361: link d0/f1a d0/d1c/d27/d46/f70 0 2024-04-26T04:42:16.798 INFO:tasks.workunit.client.1.smithi104.stdout:2/637: mknod d6/d1b/d68/d81/d97/cda 0 2024-04-26T04:42:16.798 INFO:tasks.workunit.client.1.smithi104.stdout:2/638: read - d6/d8/d5a/f74 zero size 2024-04-26T04:42:16.799 INFO:tasks.workunit.client.1.smithi104.stdout:8/362: creat d0/d33/d51/f71 x:0 0 0 2024-04-26T04:42:16.799 INFO:tasks.workunit.client.1.smithi104.stdout:8/363: dread - d0/d33/d35/f36 zero size 2024-04-26T04:42:16.799 INFO:tasks.workunit.client.1.smithi104.stdout:8/364: chown d0/l19 3 1 2024-04-26T04:42:16.799 INFO:tasks.workunit.client.1.smithi104.stdout:8/365: fsync d0/d1c/d27/f5b 0 2024-04-26T04:42:16.800 INFO:tasks.workunit.client.1.smithi104.stdout:8/366: truncate d0/f9 83270 0 2024-04-26T04:42:16.800 INFO:tasks.workunit.client.1.smithi104.stdout:8/367: dread - d0/d1c/d2d/f6a zero size 2024-04-26T04:42:16.800 INFO:tasks.workunit.client.1.smithi104.stdout:4/565: sync 2024-04-26T04:42:16.801 INFO:tasks.workunit.client.1.smithi104.stdout:4/566: creat d4/db/d6b/d7f/fcc x:0 0 0 2024-04-26T04:42:16.802 INFO:tasks.workunit.client.1.smithi104.stdout:4/567: creat d4/d7/d1d/d1e/d24/d4a/d9e/fcd x:0 0 0 2024-04-26T04:42:16.804 INFO:tasks.workunit.client.1.smithi104.stdout:4/568: creat d4/d7/d1d/d1e/d24/d48/fce x:0 0 0 2024-04-26T04:42:16.806 INFO:tasks.workunit.client.0.smithi090.stdout:5/268: dread d4/d1a/d45/f32 [0,4194304] 0 2024-04-26T04:42:16.806 INFO:tasks.workunit.client.1.smithi104.stdout:4/569: getdents d4/d7/d1d/d1e/d24/d59/db8 0 2024-04-26T04:42:16.806 INFO:tasks.workunit.client.1.smithi104.stdout:4/570: chown d4/d7/d1d/d1e/d24/f68 2 1 2024-04-26T04:42:16.806 INFO:tasks.workunit.client.1.smithi104.stdout:4/571: chown d4/d7/l3a 1174083 1 2024-04-26T04:42:16.807 INFO:tasks.workunit.client.1.smithi104.stdout:4/572: unlink d4/d7/d1d/d1e/d24/d59/db8/cc2 0 2024-04-26T04:42:16.808 INFO:tasks.workunit.client.1.smithi104.stdout:4/573: truncate d4/d7/d1d/d1e/d24/d48/f5c 2989324 0 2024-04-26T04:42:16.810 INFO:tasks.workunit.client.1.smithi104.stdout:8/368: dread d0/f1a [0,4194304] 0 2024-04-26T04:42:16.810 INFO:tasks.workunit.client.1.smithi104.stdout:8/369: chown d0/f5 622163 1 2024-04-26T04:42:16.814 INFO:tasks.workunit.client.1.smithi104.stdout:2/639: dread d6/de/d1d/f25 [0,4194304] 0 2024-04-26T04:42:16.817 INFO:tasks.workunit.client.1.smithi104.stdout:2/640: dread d6/de/d1d/f6d [0,4194304] 0 2024-04-26T04:42:16.818 INFO:tasks.workunit.client.1.smithi104.stdout:4/574: write d4/d20/d55/d56/f94 [1422788,114095] 0 2024-04-26T04:42:16.818 INFO:tasks.workunit.client.1.smithi104.stdout:4/575: write d4/d7/d1d/d1e/d24/d4a/f82 [751249,112993] 0 2024-04-26T04:42:16.818 INFO:tasks.workunit.client.1.smithi104.stdout:2/641: rename d6/d1b/d51/d6a/lc2 to d6/d1b/d51/d6a/ldb 0 2024-04-26T04:42:16.820 INFO:tasks.workunit.client.1.smithi104.stdout:8/370: read d0/fe [1881348,18841] 0 2024-04-26T04:42:16.821 INFO:tasks.workunit.client.1.smithi104.stdout:2/642: creat d6/d1b/d51/d6a/dd8/fdc x:0 0 0 2024-04-26T04:42:16.821 INFO:tasks.workunit.client.1.smithi104.stdout:2/643: dread - d6/d8/fcd zero size 2024-04-26T04:42:16.823 INFO:tasks.workunit.client.1.smithi104.stdout:8/371: dread d0/f4d [0,4194304] 0 2024-04-26T04:42:16.823 INFO:tasks.workunit.client.1.smithi104.stdout:8/372: chown d0/d1c/d2d/l54 24040 1 2024-04-26T04:42:16.824 INFO:tasks.workunit.client.1.smithi104.stdout:8/373: write d0/d33/d51/f71 [1022445,16648] 0 2024-04-26T04:42:16.824 INFO:tasks.workunit.client.1.smithi104.stdout:2/644: mknod d6/d1b/cdd 0 2024-04-26T04:42:16.827 INFO:tasks.workunit.client.1.smithi104.stdout:8/374: unlink d0/l3 0 2024-04-26T04:42:16.829 INFO:tasks.workunit.client.1.smithi104.stdout:2/645: truncate d6/d1b/d51/f58 482467 0 2024-04-26T04:42:16.834 INFO:tasks.workunit.client.1.smithi104.stdout:4/576: dwrite d4/d7/d1d/d8e/fc3 [0,4194304] 0 2024-04-26T04:42:16.834 INFO:tasks.workunit.client.1.smithi104.stdout:4/577: dread - d4/d7/d1d/d1e/d24/d4a/f50 zero size 2024-04-26T04:42:16.836 INFO:tasks.workunit.client.1.smithi104.stdout:4/578: mknod d4/db/d6b/ccf 0 2024-04-26T04:42:16.840 INFO:tasks.workunit.client.0.smithi090.stdout:7/209: mkdir d0/db/df/d4d 0 2024-04-26T04:42:16.840 INFO:tasks.workunit.client.0.smithi090.stdout:7/210: creat d0/f4e x:0 0 0 2024-04-26T04:42:16.843 INFO:tasks.workunit.client.0.smithi090.stdout:0/225: mkdir d0/d26/d48 0 2024-04-26T04:42:16.844 INFO:tasks.workunit.client.0.smithi090.stdout:9/154: symlink db/l37 0 2024-04-26T04:42:16.845 INFO:tasks.workunit.client.0.smithi090.stdout:9/155: read db/d1e/f27 [1440769,46573] 0 2024-04-26T04:42:16.846 INFO:tasks.workunit.client.0.smithi090.stdout:6/268: rename d9/d3c/d49 to d9/d21/d3b/d62 0 2024-04-26T04:42:16.846 INFO:tasks.workunit.client.0.smithi090.stdout:6/269: write d9/d1c/d31/f3a [370507,125058] 0 2024-04-26T04:42:16.847 INFO:tasks.workunit.client.0.smithi090.stdout:8/136: unlink cb 0 2024-04-26T04:42:16.848 INFO:tasks.workunit.client.1.smithi104.stdout:8/375: dwrite d0/d50/f68 [0,4194304] 0 2024-04-26T04:42:16.851 INFO:tasks.workunit.client.0.smithi090.stdout:4/256: mknod d3/db/d12/d20/d45/c4f 0 2024-04-26T04:42:16.851 INFO:tasks.workunit.client.0.smithi090.stdout:7/211: dread d0/db/d15/d29/fc [4194304,4194304] 0 2024-04-26T04:42:16.851 INFO:tasks.workunit.client.0.smithi090.stdout:7/212: chown d0/db/d15/c1b 3144039 1 2024-04-26T04:42:16.855 INFO:tasks.workunit.client.1.smithi104.stdout:4/579: dwrite d4/db/f28 [0,4194304] 0 2024-04-26T04:42:16.857 INFO:tasks.workunit.client.0.smithi090.stdout:5/269: mkdir d4/d1a/d45/d5b 0 2024-04-26T04:42:16.862 INFO:tasks.workunit.client.0.smithi090.stdout:0/226: mknod d0/d26/c49 0 2024-04-26T04:42:16.865 INFO:tasks.workunit.client.0.smithi090.stdout:8/137: rename l3 to dc/d1f/l2f 0 2024-04-26T04:42:16.867 INFO:tasks.workunit.client.0.smithi090.stdout:1/175: getdents d0/d4/d7 0 2024-04-26T04:42:16.867 INFO:tasks.workunit.client.0.smithi090.stdout:1/176: write d0/d4/f20 [409926,121062] 0 2024-04-26T04:42:16.867 INFO:tasks.workunit.client.0.smithi090.stdout:1/177: readlink d0/l1b 0 2024-04-26T04:42:16.868 INFO:tasks.workunit.client.0.smithi090.stdout:4/257: mkdir d3/d13/d1d/d50 0 2024-04-26T04:42:16.870 INFO:tasks.workunit.client.0.smithi090.stdout:6/270: symlink d9/l63 0 2024-04-26T04:42:16.870 INFO:tasks.workunit.client.0.smithi090.stdout:6/271: creat d9/d3c/f64 x:0 0 0 2024-04-26T04:42:16.872 INFO:tasks.workunit.client.0.smithi090.stdout:8/138: getdents dc 0 2024-04-26T04:42:16.872 INFO:tasks.workunit.client.0.smithi090.stdout:8/139: fsync dc/ff 0 2024-04-26T04:42:16.874 INFO:tasks.workunit.client.0.smithi090.stdout:1/178: unlink d0/c28 0 2024-04-26T04:42:16.875 INFO:tasks.workunit.client.0.smithi090.stdout:4/258: mkdir d3/d13/d1d/d50/d51 0 2024-04-26T04:42:16.877 INFO:tasks.workunit.client.0.smithi090.stdout:0/227: rename d0/l19 to d0/l4a 0 2024-04-26T04:42:16.877 INFO:tasks.workunit.client.0.smithi090.stdout:0/228: readlink d0/d26/l1f 0 2024-04-26T04:42:16.880 INFO:tasks.workunit.client.0.smithi090.stdout:6/272: rename d9/d21/d40/d4c/l5d to d9/d1c/d4d/d58/l65 0 2024-04-26T04:42:16.880 INFO:tasks.workunit.client.0.smithi090.stdout:8/140: getdents dc/d12 0 2024-04-26T04:42:16.882 INFO:tasks.workunit.client.0.smithi090.stdout:1/179: unlink d0/c24 0 2024-04-26T04:42:16.882 INFO:tasks.workunit.client.0.smithi090.stdout:1/180: readlink d0/l3 0 2024-04-26T04:42:16.884 INFO:tasks.workunit.client.0.smithi090.stdout:4/259: unlink d3/d13/f39 0 2024-04-26T04:42:16.884 INFO:tasks.workunit.client.0.smithi090.stdout:4/260: write d3/db/d12/d20/d24/f32 [576537,71975] 0 2024-04-26T04:42:16.885 INFO:tasks.workunit.client.0.smithi090.stdout:7/213: dwrite d0/db/d15/d30/f40 [0,4194304] 0 2024-04-26T04:42:16.886 INFO:tasks.workunit.client.0.smithi090.stdout:8/141: truncate dc/f1e 649198 0 2024-04-26T04:42:16.886 INFO:tasks.workunit.client.1.smithi104.stdout:8/376: dwrite d0/d1c/f6e [0,4194304] 0 2024-04-26T04:42:16.887 INFO:tasks.workunit.client.0.smithi090.stdout:9/156: dwrite db/d1e/f34 [0,4194304] 0 2024-04-26T04:42:16.887 INFO:tasks.workunit.client.0.smithi090.stdout:9/157: write db/dd/d18/f2a [194863,46812] 0 2024-04-26T04:42:16.887 INFO:tasks.workunit.client.0.smithi090.stdout:1/181: creat d0/d4/d7/d17/f36 x:0 0 0 2024-04-26T04:42:16.889 INFO:tasks.workunit.client.0.smithi090.stdout:0/229: mkdir d0/d4b 0 2024-04-26T04:42:16.889 INFO:tasks.workunit.client.0.smithi090.stdout:5/270: dwrite d4/d1a/d1f/f3d [0,4194304] 0 2024-04-26T04:42:16.889 INFO:tasks.workunit.client.0.smithi090.stdout:5/271: fdatasync d4/d1a/d1f/f3d 0 2024-04-26T04:42:16.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/272: chown d4/f37 224 1 2024-04-26T04:42:16.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/273: dread - d4/d1a/d2a/f41 zero size 2024-04-26T04:42:16.890 INFO:tasks.workunit.client.0.smithi090.stdout:5/274: write d4/d1a/d2a/f41 [88653,68118] 0 2024-04-26T04:42:16.890 INFO:tasks.workunit.client.1.smithi104.stdout:4/580: dwrite d4/d7/d1d/d2e/faa [0,4194304] 0 2024-04-26T04:42:16.890 INFO:tasks.workunit.client.1.smithi104.stdout:8/377: link d0/d1c/f26 d0/d1c/d27/f72 0 2024-04-26T04:42:16.890 INFO:tasks.workunit.client.1.smithi104.stdout:8/378: dread - d0/d1c/d27/f5c zero size 2024-04-26T04:42:16.894 INFO:tasks.workunit.client.1.smithi104.stdout:4/581: mknod d4/d7/d1d/d2e/d74/cd0 0 2024-04-26T04:42:16.898 INFO:tasks.workunit.client.0.smithi090.stdout:7/214: mkdir d0/db/d15/d30/d4f 0 2024-04-26T04:42:16.898 INFO:tasks.workunit.client.0.smithi090.stdout:7/215: dread - d0/f4e zero size 2024-04-26T04:42:16.898 INFO:tasks.workunit.client.0.smithi090.stdout:9/158: write db/f26 [6363248,57349] 0 2024-04-26T04:42:16.898 INFO:tasks.workunit.client.1.smithi104.stdout:4/582: link d4/d7/d1d/d1e/c19 d4/d7/d1d/d1e/d24/d59/cd1 0 2024-04-26T04:42:16.899 INFO:tasks.workunit.client.1.smithi104.stdout:4/583: write d4/db/f4b [2018669,104923] 0 2024-04-26T04:42:16.900 INFO:tasks.workunit.client.0.smithi090.stdout:8/142: creat dc/f30 x:0 0 0 2024-04-26T04:42:16.901 INFO:tasks.workunit.client.1.smithi104.stdout:4/584: mknod d4/d20/cd2 0 2024-04-26T04:42:16.903 INFO:tasks.workunit.client.1.smithi104.stdout:4/585: mkdir d4/d7/d1d/d2e/d74/dbd/dd3 0 2024-04-26T04:42:16.905 INFO:tasks.workunit.client.1.smithi104.stdout:4/586: write d4/d7/d1d/d1e/d8f/db6/d7d/fa1 [638701,60617] 0 2024-04-26T04:42:16.905 INFO:tasks.workunit.client.1.smithi104.stdout:4/587: dread - d4/db/d6b/d7f/fcc zero size 2024-04-26T04:42:16.905 INFO:tasks.workunit.client.1.smithi104.stdout:4/588: write d4/d7/d1d/d1e/d24/d48/f5c [2564145,45546] 0 2024-04-26T04:42:16.907 INFO:tasks.workunit.client.0.smithi090.stdout:0/230: symlink d0/d26/l4c 0 2024-04-26T04:42:16.907 INFO:tasks.workunit.client.0.smithi090.stdout:0/231: fdatasync d0/d9/f17 0 2024-04-26T04:42:16.907 INFO:tasks.workunit.client.0.smithi090.stdout:0/232: write d0/de/f2c [9339986,8012] 0 2024-04-26T04:42:16.911 INFO:tasks.workunit.client.0.smithi090.stdout:7/216: dread d0/db/df/d1c/d23/f26 [0,4194304] 0 2024-04-26T04:42:16.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/159: mkdir db/dd/d18/d38 0 2024-04-26T04:42:16.919 INFO:tasks.workunit.client.0.smithi090.stdout:6/273: rename d9/d1c/d31/d3d to d9/d21/d40/d4e/d66 0 2024-04-26T04:42:16.921 INFO:tasks.workunit.client.0.smithi090.stdout:8/143: symlink dc/d1f/l31 0 2024-04-26T04:42:16.921 INFO:tasks.workunit.client.0.smithi090.stdout:8/144: fdatasync dc/f30 0 2024-04-26T04:42:16.921 INFO:tasks.workunit.client.0.smithi090.stdout:1/182: dwrite d0/d4/d7/d17/f21 [0,4194304] 0 2024-04-26T04:42:16.921 INFO:tasks.workunit.client.0.smithi090.stdout:1/183: stat d0/d4/c1d 0 2024-04-26T04:42:16.921 INFO:tasks.workunit.client.0.smithi090.stdout:0/233: rmdir d0 39 2024-04-26T04:42:16.921 INFO:tasks.workunit.client.1.smithi104.stdout:8/379: dwrite d0/d1c/d27/f72 [0,4194304] 0 2024-04-26T04:42:16.923 INFO:tasks.workunit.client.0.smithi090.stdout:7/217: symlink d0/db/d15/l50 0 2024-04-26T04:42:16.923 INFO:tasks.workunit.client.0.smithi090.stdout:7/218: chown d0/db/d15/d30/f32 5 1 2024-04-26T04:42:16.923 INFO:tasks.workunit.client.0.smithi090.stdout:7/219: fdatasync d0/db/df/f42 0 2024-04-26T04:42:16.923 INFO:tasks.workunit.client.0.smithi090.stdout:5/275: dwrite d4/d1a/d3f/f52 [0,4194304] 0 2024-04-26T04:42:16.924 INFO:tasks.workunit.client.0.smithi090.stdout:7/220: write d0/db/df/d1c/d23/f31 [937455,93612] 0 2024-04-26T04:42:16.924 INFO:tasks.workunit.client.0.smithi090.stdout:7/221: fsync d0/db/df/f42 0 2024-04-26T04:42:16.924 INFO:tasks.workunit.client.0.smithi090.stdout:7/222: read d0/d1/d4/f10 [729564,93889] 0 2024-04-26T04:42:16.924 INFO:tasks.workunit.client.1.smithi104.stdout:4/589: dwrite d4/d20/d55/d56/da3/fc4 [0,4194304] 0 2024-04-26T04:42:16.924 INFO:tasks.workunit.client.0.smithi090.stdout:9/160: mkdir db/dd/d18/d38/d39 0 2024-04-26T04:42:16.928 INFO:tasks.workunit.client.0.smithi090.stdout:4/261: rename d3/db/f2f to d3/db/d12/d20/d24/f52 0 2024-04-26T04:42:16.928 INFO:tasks.workunit.client.0.smithi090.stdout:6/274: symlink d9/d21/d40/d4e/l67 0 2024-04-26T04:42:16.928 INFO:tasks.workunit.client.0.smithi090.stdout:1/184: write d0/f1c [864486,100868] 0 2024-04-26T04:42:16.929 INFO:tasks.workunit.client.0.smithi090.stdout:8/145: mknod dc/c32 0 2024-04-26T04:42:16.930 INFO:tasks.workunit.client.0.smithi090.stdout:7/223: dread d0/d1/d4/f2e [8388608,4194304] 0 2024-04-26T04:42:16.930 INFO:tasks.workunit.client.0.smithi090.stdout:7/224: chown d0/db/d15/d30/f32 30185 1 2024-04-26T04:42:16.931 INFO:tasks.workunit.client.1.smithi104.stdout:1/526: sync 2024-04-26T04:42:16.931 INFO:tasks.workunit.client.1.smithi104.stdout:0/395: sync 2024-04-26T04:42:16.932 INFO:tasks.workunit.client.1.smithi104.stdout:1/527: write d0/d6/d2e/d32/f62 [745818,85221] 0 2024-04-26T04:42:16.932 INFO:tasks.workunit.client.1.smithi104.stdout:0/396: mkdir d0/d4/d3c/d53/d5d/d80 0 2024-04-26T04:42:16.934 INFO:tasks.workunit.client.1.smithi104.stdout:1/528: link d0/d6/d19/d21/d66/l3d d0/d6/d19/d21/d66/d20/d2a/lab 0 2024-04-26T04:42:16.935 INFO:tasks.workunit.client.0.smithi090.stdout:9/161: creat db/dd/d18/f3a x:0 0 0 2024-04-26T04:42:16.939 INFO:tasks.workunit.client.0.smithi090.stdout:4/262: mkdir d3/db/d12/d20/d45/d53 0 2024-04-26T04:42:16.940 INFO:tasks.workunit.client.0.smithi090.stdout:6/275: truncate d9/d1c/f2b 6331035 0 2024-04-26T04:42:16.940 INFO:tasks.workunit.client.0.smithi090.stdout:6/276: chown d9/d1c/d4d 0 1 2024-04-26T04:42:16.942 INFO:tasks.workunit.client.0.smithi090.stdout:1/185: symlink d0/d4/d7/l37 0 2024-04-26T04:42:16.942 INFO:tasks.workunit.client.0.smithi090.stdout:1/186: truncate d0/d4/d7/d17/f36 399009 0 2024-04-26T04:42:16.942 INFO:tasks.workunit.client.0.smithi090.stdout:1/187: stat d0/d4/f15 0 2024-04-26T04:42:16.942 INFO:tasks.workunit.client.0.smithi090.stdout:1/188: creat d0/d4/d7/d17/f38 x:0 0 0 2024-04-26T04:42:16.943 INFO:tasks.workunit.client.0.smithi090.stdout:1/189: creat d0/d4/d7/d17/d22/d31/f39 x:0 0 0 2024-04-26T04:42:16.944 INFO:tasks.workunit.client.1.smithi104.stdout:1/529: dread d0/f14 [4194304,4194304] 0 2024-04-26T04:42:16.944 INFO:tasks.workunit.client.1.smithi104.stdout:1/530: dread - d0/f80 zero size 2024-04-26T04:42:16.944 INFO:tasks.workunit.client.1.smithi104.stdout:1/531: readlink d0/d6/d19/d1b/l26 0 2024-04-26T04:42:16.944 INFO:tasks.workunit.client.1.smithi104.stdout:1/532: creat d0/d6/d2e/d32/fac x:0 0 0 2024-04-26T04:42:16.945 INFO:tasks.workunit.client.0.smithi090.stdout:8/146: creat dc/d1f/f33 x:0 0 0 2024-04-26T04:42:16.946 INFO:tasks.workunit.client.1.smithi104.stdout:8/380: dwrite d0/d1c/d27/f5b [0,4194304] 0 2024-04-26T04:42:16.946 INFO:tasks.workunit.client.1.smithi104.stdout:0/397: write d0/fe [5677092,116758] 0 2024-04-26T04:42:16.947 INFO:tasks.workunit.client.1.smithi104.stdout:8/381: symlink d0/d50/l73 0 2024-04-26T04:42:16.948 INFO:tasks.workunit.client.0.smithi090.stdout:0/234: mknod d0/df/c4d 0 2024-04-26T04:42:16.948 INFO:tasks.workunit.client.1.smithi104.stdout:0/398: mkdir d0/d81 0 2024-04-26T04:42:16.948 INFO:tasks.workunit.client.1.smithi104.stdout:0/399: chown d0/d2/d10 10315954 1 2024-04-26T04:42:16.949 INFO:tasks.workunit.client.0.smithi090.stdout:9/162: rename f6 to db/dd/d18/d38/d39/f3b 0 2024-04-26T04:42:16.956 INFO:tasks.workunit.client.0.smithi090.stdout:4/263: creat d3/d13/d1d/f54 x:0 0 0 2024-04-26T04:42:16.956 INFO:tasks.workunit.client.0.smithi090.stdout:4/264: fsync d3/db/d12/d20/f26 0 2024-04-26T04:42:16.956 INFO:tasks.workunit.client.0.smithi090.stdout:4/265: write d3/f36 [4599302,37248] 0 2024-04-26T04:42:16.956 INFO:tasks.workunit.client.1.smithi104.stdout:8/382: dread d0/d1c/d27/f5b [0,4194304] 0 2024-04-26T04:42:16.956 INFO:tasks.workunit.client.1.smithi104.stdout:8/383: write d0/f4e [4858335,67167] 0 2024-04-26T04:42:16.957 INFO:tasks.workunit.client.1.smithi104.stdout:8/384: creat d0/d33/d35/d6d/f74 x:0 0 0 2024-04-26T04:42:16.957 INFO:tasks.workunit.client.0.smithi090.stdout:6/277: creat d9/f68 x:0 0 0 2024-04-26T04:42:16.957 INFO:tasks.workunit.client.1.smithi104.stdout:8/385: getdents d0/d1c/d27/d47 0 2024-04-26T04:42:16.958 INFO:tasks.workunit.client.1.smithi104.stdout:8/386: mknod d0/d33/c75 0 2024-04-26T04:42:16.959 INFO:tasks.workunit.client.1.smithi104.stdout:8/387: mkdir d0/d1c/d27/d47/d67/d76 0 2024-04-26T04:42:16.960 INFO:tasks.workunit.client.0.smithi090.stdout:8/147: mkdir dc/d34 0 2024-04-26T04:42:16.960 INFO:tasks.workunit.client.0.smithi090.stdout:8/148: creat dc/d1f/f35 x:0 0 0 2024-04-26T04:42:16.960 INFO:tasks.workunit.client.0.smithi090.stdout:8/149: chown dc/d1f/l2a 27301 1 2024-04-26T04:42:16.960 INFO:tasks.workunit.client.0.smithi090.stdout:8/150: chown dc/f14 1618417 1 2024-04-26T04:42:16.961 INFO:tasks.workunit.client.1.smithi104.stdout:8/388: truncate d0/f5 3785434 0 2024-04-26T04:42:16.962 INFO:tasks.workunit.client.0.smithi090.stdout:7/225: mknod d0/db/c51 0 2024-04-26T04:42:16.963 INFO:tasks.workunit.client.1.smithi104.stdout:0/400: dwrite d0/d2/d21/d49/d5e/f78 [0,4194304] 0 2024-04-26T04:42:16.963 INFO:tasks.workunit.client.1.smithi104.stdout:0/401: chown d0/d2/d10/d3d/f72 2 1 2024-04-26T04:42:16.964 INFO:tasks.workunit.client.0.smithi090.stdout:0/235: mkdir d0/d26/d48/d4e 0 2024-04-26T04:42:16.966 INFO:tasks.workunit.client.0.smithi090.stdout:2/156: sync 2024-04-26T04:42:16.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/141: sync 2024-04-26T04:42:16.968 INFO:tasks.workunit.client.0.smithi090.stdout:8/151: rename f4 to dc/d1f/f36 0 2024-04-26T04:42:16.968 INFO:tasks.workunit.client.0.smithi090.stdout:8/152: fdatasync dc/d1f/f24 0 2024-04-26T04:42:16.968 INFO:tasks.workunit.client.0.smithi090.stdout:8/153: chown dc/d12/f17 498 1 2024-04-26T04:42:16.971 INFO:tasks.workunit.client.0.smithi090.stdout:7/226: link d0/f4e d0/d1/f52 0 2024-04-26T04:42:16.974 INFO:tasks.workunit.client.0.smithi090.stdout:0/236: creat d0/f4f x:0 0 0 2024-04-26T04:42:16.977 INFO:tasks.workunit.client.0.smithi090.stdout:3/142: mkdir d1/d6/d2c 0 2024-04-26T04:42:16.977 INFO:tasks.workunit.client.0.smithi090.stdout:8/154: symlink dc/d1f/l37 0 2024-04-26T04:42:16.977 INFO:tasks.workunit.client.0.smithi090.stdout:8/155: chown dc/d12 444 1 2024-04-26T04:42:16.978 INFO:tasks.workunit.client.0.smithi090.stdout:6/278: dread d9/fd [0,4194304] 0 2024-04-26T04:42:16.978 INFO:tasks.workunit.client.0.smithi090.stdout:6/279: fdatasync d9/d21/d3b/f46 0 2024-04-26T04:42:16.978 INFO:tasks.workunit.client.0.smithi090.stdout:4/266: dwrite f1 [4194304,4194304] 0 2024-04-26T04:42:16.978 INFO:tasks.workunit.client.0.smithi090.stdout:4/267: readlink d3/db/d12/d20/l44 0 2024-04-26T04:42:16.978 INFO:tasks.workunit.client.0.smithi090.stdout:4/268: chown d3/db/d12/f10 7236 1 2024-04-26T04:42:16.978 INFO:tasks.workunit.client.0.smithi090.stdout:4/269: chown d3/db/f23 4 1 2024-04-26T04:42:16.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/157: read db/d11/d14/d1b/d20/f24 [2140323,40406] 0 2024-04-26T04:42:16.984 INFO:tasks.workunit.client.0.smithi090.stdout:3/143: rmdir d1/d6/d1b 39 2024-04-26T04:42:16.984 INFO:tasks.workunit.client.0.smithi090.stdout:2/158: chown db/d11/d14/d25/l31 31470 1 2024-04-26T04:42:16.984 INFO:tasks.workunit.client.0.smithi090.stdout:2/159: read - db/dc/f1f zero size 2024-04-26T04:42:16.986 INFO:tasks.workunit.client.0.smithi090.stdout:7/227: rename d0/d1/d1f to d0/db/d53 0 2024-04-26T04:42:16.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/237: symlink d0/l50 0 2024-04-26T04:42:16.987 INFO:tasks.workunit.client.0.smithi090.stdout:3/144: symlink d1/d6/d1b/d1e/d2a/l2d 0 2024-04-26T04:42:16.987 INFO:tasks.workunit.client.0.smithi090.stdout:3/145: dread - d1/d6/f21 zero size 2024-04-26T04:42:16.988 INFO:tasks.workunit.client.0.smithi090.stdout:7/228: symlink d0/db/d15/d30/l54 0 2024-04-26T04:42:16.988 INFO:tasks.workunit.client.0.smithi090.stdout:7/229: truncate d0/db/d15/d30/f3a 889794 0 2024-04-26T04:42:16.988 INFO:tasks.workunit.client.0.smithi090.stdout:7/230: stat d0/f45 0 2024-04-26T04:42:16.993 INFO:tasks.workunit.client.0.smithi090.stdout:3/146: getdents d1/d17 0 2024-04-26T04:42:16.994 INFO:tasks.workunit.client.0.smithi090.stdout:7/231: mknod d0/c55 0 2024-04-26T04:42:16.994 INFO:tasks.workunit.client.0.smithi090.stdout:0/238: rmdir d0/de 39 2024-04-26T04:42:16.994 INFO:tasks.workunit.client.0.smithi090.stdout:0/239: write d0/de/f47 [551108,113971] 0 2024-04-26T04:42:16.997 INFO:tasks.workunit.client.0.smithi090.stdout:3/147: mknod d1/d6/c2e 0 2024-04-26T04:42:16.997 INFO:tasks.workunit.client.0.smithi090.stdout:3/148: chown d1 436425 1 2024-04-26T04:42:16.998 INFO:tasks.workunit.client.0.smithi090.stdout:7/232: truncate d0/db/d15/d29/f18 244102 0 2024-04-26T04:42:16.998 INFO:tasks.workunit.client.0.smithi090.stdout:7/233: fdatasync d0/db/d15/d30/f3a 0 2024-04-26T04:42:16.999 INFO:tasks.workunit.client.0.smithi090.stdout:7/234: write d0/db/df/f42 [34598,40930] 0 2024-04-26T04:42:16.999 INFO:tasks.workunit.client.0.smithi090.stdout:2/160: write db/d11/d14/d25/f2a [2457462,39002] 0 2024-04-26T04:42:16.999 INFO:tasks.workunit.client.0.smithi090.stdout:2/161: write db/d11/f17 [586993,39777] 0 2024-04-26T04:42:16.999 INFO:tasks.workunit.client.1.smithi104.stdout:8/389: dwrite d0/d33/f1f [0,4194304] 0 2024-04-26T04:42:17.002 INFO:tasks.workunit.client.0.smithi090.stdout:2/162: creat db/d11/d14/d27/f38 x:0 0 0 2024-04-26T04:42:17.006 INFO:tasks.workunit.client.0.smithi090.stdout:2/163: mkdir db/d11/d14/d1b/d39 0 2024-04-26T04:42:17.008 INFO:tasks.workunit.client.0.smithi090.stdout:2/164: rename db/d11/f30 to db/d11/d14/d25/f3a 0 2024-04-26T04:42:17.008 INFO:tasks.workunit.client.0.smithi090.stdout:2/165: write db/dc/f1e [3310878,109170] 0 2024-04-26T04:42:17.009 INFO:tasks.workunit.client.0.smithi090.stdout:2/166: mknod db/d11/d14/d1b/d20/c3b 0 2024-04-26T04:42:17.009 INFO:tasks.workunit.client.0.smithi090.stdout:2/167: readlink db/d11/l2e 0 2024-04-26T04:42:17.011 INFO:tasks.workunit.client.0.smithi090.stdout:2/168: rename db/f16 to db/dc/f3c 0 2024-04-26T04:42:17.011 INFO:tasks.workunit.client.0.smithi090.stdout:2/169: write db/f33 [377368,28803] 0 2024-04-26T04:42:17.011 INFO:tasks.workunit.client.0.smithi090.stdout:2/170: readlink db/d11/l2e 0 2024-04-26T04:42:17.015 INFO:tasks.workunit.client.1.smithi104.stdout:8/390: read d0/d33/f1f [98934,71101] 0 2024-04-26T04:42:17.016 INFO:tasks.workunit.client.1.smithi104.stdout:8/391: mknod d0/d1c/d27/d47/d67/d76/c77 0 2024-04-26T04:42:17.016 INFO:tasks.workunit.client.1.smithi104.stdout:8/392: creat d0/d1c/d27/d46/f78 x:0 0 0 2024-04-26T04:42:17.017 INFO:tasks.workunit.client.1.smithi104.stdout:8/393: fsync d0/d1c/d27/f66 0 2024-04-26T04:42:17.017 INFO:tasks.workunit.client.1.smithi104.stdout:8/394: truncate d0/f9 1081162 0 2024-04-26T04:42:17.017 INFO:tasks.workunit.client.1.smithi104.stdout:8/395: mknod d0/d1c/c79 0 2024-04-26T04:42:17.018 INFO:tasks.workunit.client.0.smithi090.stdout:2/171: unlink db/l1d 0 2024-04-26T04:42:17.019 INFO:tasks.workunit.client.1.smithi104.stdout:0/402: dwrite d0/d2/d21/d49/f7e [0,4194304] 0 2024-04-26T04:42:17.020 INFO:tasks.workunit.client.0.smithi090.stdout:5/276: dwrite d4/d1a/d3f/f52 [4194304,4194304] 0 2024-04-26T04:42:17.021 INFO:tasks.workunit.client.0.smithi090.stdout:8/156: dwrite dc/d12/f20 [0,4194304] 0 2024-04-26T04:42:17.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/240: fsync d0/de/f47 0 2024-04-26T04:42:17.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/241: stat d0/f2d 0 2024-04-26T04:42:17.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/242: truncate d0/d26/f42 832138 0 2024-04-26T04:42:17.021 INFO:tasks.workunit.client.0.smithi090.stdout:3/149: dwrite d1/d17/f25 [0,4194304] 0 2024-04-26T04:42:17.021 INFO:tasks.workunit.client.0.smithi090.stdout:2/172: creat db/d11/d14/d27/d28/f3d x:0 0 0 2024-04-26T04:42:17.022 INFO:tasks.workunit.client.0.smithi090.stdout:2/173: write db/d11/d14/d25/f37 [547122,69204] 0 2024-04-26T04:42:17.024 INFO:tasks.workunit.client.1.smithi104.stdout:0/403: creat d0/d2/d21/d27/f82 x:0 0 0 2024-04-26T04:42:17.024 INFO:tasks.workunit.client.1.smithi104.stdout:0/404: write d0/d2/d21/d27/d1e/d57/d6c/f56 [471933,34569] 0 2024-04-26T04:42:17.025 INFO:tasks.workunit.client.0.smithi090.stdout:6/280: dwrite d9/f14 [4194304,4194304] 0 2024-04-26T04:42:17.025 INFO:tasks.workunit.client.0.smithi090.stdout:6/281: fsync d9/f23 0 2024-04-26T04:42:17.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/243: rmdir d0/de 39 2024-04-26T04:42:17.027 INFO:tasks.workunit.client.1.smithi104.stdout:8/396: write d0/d33/f28 [1810420,106689] 0 2024-04-26T04:42:17.028 INFO:tasks.workunit.client.0.smithi090.stdout:6/282: dread d9/d21/d3b/d62/f29 [4194304,4194304] 0 2024-04-26T04:42:17.030 INFO:tasks.workunit.client.0.smithi090.stdout:5/277: dread d4/d1a/df/f22 [0,4194304] 0 2024-04-26T04:42:17.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/283: dread d9/d1c/f57 [0,4194304] 0 2024-04-26T04:42:17.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/150: link d1/f27 d1/d17/f2f 0 2024-04-26T04:42:17.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/151: stat d1/d9/fb 0 2024-04-26T04:42:17.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/152: truncate d1/d6/f21 431888 0 2024-04-26T04:42:17.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/153: truncate d1/fc 1391137 0 2024-04-26T04:42:17.035 INFO:tasks.workunit.client.1.smithi104.stdout:0/405: read d0/fa [933375,17707] 0 2024-04-26T04:42:17.037 INFO:tasks.workunit.client.0.smithi090.stdout:0/244: rename d0/c30 to d0/d26/c51 0 2024-04-26T04:42:17.037 INFO:tasks.workunit.client.0.smithi090.stdout:5/278: write d4/d1a/d45/f32 [355967,124347] 0 2024-04-26T04:42:17.038 INFO:tasks.workunit.client.1.smithi104.stdout:0/406: rename d0/d2/d21/d27/d1e/d57/f68 to d0/d2/d21/d27/f83 0 2024-04-26T04:42:17.038 INFO:tasks.workunit.client.1.smithi104.stdout:0/407: write d0/d2/d21/d27/f7b [1596244,129533] 0 2024-04-26T04:42:17.038 INFO:tasks.workunit.client.1.smithi104.stdout:0/408: truncate d0/d2/d21/d27/d1e/f2c 1775448 0 2024-04-26T04:42:17.039 INFO:tasks.workunit.client.0.smithi090.stdout:6/284: mkdir d9/d1c/d31/d69 0 2024-04-26T04:42:17.039 INFO:tasks.workunit.client.0.smithi090.stdout:8/157: dwrite dc/ff [0,4194304] 0 2024-04-26T04:42:17.040 INFO:tasks.workunit.client.1.smithi104.stdout:0/409: dread d0/d2/d21/f45 [0,4194304] 0 2024-04-26T04:42:17.040 INFO:tasks.workunit.client.1.smithi104.stdout:0/410: truncate d0/d2/d21/d38/f6b 1306624 0 2024-04-26T04:42:17.041 INFO:tasks.workunit.client.0.smithi090.stdout:0/245: creat d0/d4b/f52 x:0 0 0 2024-04-26T04:42:17.041 INFO:tasks.workunit.client.0.smithi090.stdout:0/246: write d0/d26/f42 [876108,47415] 0 2024-04-26T04:42:17.043 INFO:tasks.workunit.client.0.smithi090.stdout:5/279: truncate d4/d1a/df/f28 40524 0 2024-04-26T04:42:17.043 INFO:tasks.workunit.client.0.smithi090.stdout:5/280: creat d4/d1a/df/d36/f5c x:0 0 0 2024-04-26T04:42:17.043 INFO:tasks.workunit.client.0.smithi090.stdout:5/281: read - d4/d1a/df/d36/f54 zero size 2024-04-26T04:42:17.043 INFO:tasks.workunit.client.0.smithi090.stdout:5/282: read d4/fe [1373975,30616] 0 2024-04-26T04:42:17.045 INFO:tasks.workunit.client.0.smithi090.stdout:6/285: symlink d9/d21/l6a 0 2024-04-26T04:42:17.046 INFO:tasks.workunit.client.0.smithi090.stdout:7/235: rmdir d0 39 2024-04-26T04:42:17.050 INFO:tasks.workunit.client.1.smithi104.stdout:0/411: dread d0/d2/d21/d38/f36 [0,4194304] 0 2024-04-26T04:42:17.051 INFO:tasks.workunit.client.0.smithi090.stdout:8/158: unlink dc/d1f/f22 0 2024-04-26T04:42:17.051 INFO:tasks.workunit.client.0.smithi090.stdout:8/159: chown dc/d12/f2e 0 1 2024-04-26T04:42:17.051 INFO:tasks.workunit.client.1.smithi104.stdout:0/412: unlink d0/d2/d21/d38/f5b 0 2024-04-26T04:42:17.052 INFO:tasks.workunit.client.0.smithi090.stdout:0/247: mkdir d0/d9/d45/d53 0 2024-04-26T04:42:17.053 INFO:tasks.workunit.client.1.smithi104.stdout:0/413: mkdir d0/d2/d10/d84 0 2024-04-26T04:42:17.053 INFO:tasks.workunit.client.1.smithi104.stdout:0/414: stat d0/f7f 0 2024-04-26T04:42:17.053 INFO:tasks.workunit.client.1.smithi104.stdout:0/415: truncate d0/d4/d3b/f77 385189 0 2024-04-26T04:42:17.054 INFO:tasks.workunit.client.1.smithi104.stdout:0/416: mkdir d0/d2/d21/d27/d1e/d57/d6c/d85 0 2024-04-26T04:42:17.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/286: mknod d9/d21/c6b 0 2024-04-26T04:42:17.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/287: write d9/d21/d40/d4c/f59 [1505516,70662] 0 2024-04-26T04:42:17.055 INFO:tasks.workunit.client.0.smithi090.stdout:6/288: write d9/d21/d3b/d62/f26 [4614244,123992] 0 2024-04-26T04:42:17.056 INFO:tasks.workunit.client.0.smithi090.stdout:7/236: creat d0/db/d53/f56 x:0 0 0 2024-04-26T04:42:17.057 INFO:tasks.workunit.client.1.smithi104.stdout:0/417: link d0/d2/d21/d27/f6d d0/d4/d3c/f86 0 2024-04-26T04:42:17.058 INFO:tasks.workunit.client.0.smithi090.stdout:0/248: rmdir d0/d26/d48/d4e 0 2024-04-26T04:42:17.059 INFO:tasks.workunit.client.1.smithi104.stdout:5/476: sync 2024-04-26T04:42:17.059 INFO:tasks.workunit.client.0.smithi090.stdout:3/154: dwrite d1/d9/fb [0,4194304] 0 2024-04-26T04:42:17.060 INFO:tasks.workunit.client.1.smithi104.stdout:9/346: sync 2024-04-26T04:42:17.060 INFO:tasks.workunit.client.1.smithi104.stdout:9/347: chown d11/d15/d32/d35/l5b 815816056 1 2024-04-26T04:42:17.060 INFO:tasks.workunit.client.1.smithi104.stdout:9/348: readlink l0 0 2024-04-26T04:42:17.060 INFO:tasks.workunit.client.1.smithi104.stdout:5/477: creat d0/d1/d64/d26/d36/d72/fa3 x:0 0 0 2024-04-26T04:42:17.061 INFO:tasks.workunit.client.0.smithi090.stdout:6/289: rename d9/f16 to d9/d1c/d4d/d58/f6c 0 2024-04-26T04:42:17.062 INFO:tasks.workunit.client.1.smithi104.stdout:8/397: dwrite d0/d1c/d27/f5b [0,4194304] 0 2024-04-26T04:42:17.062 INFO:tasks.workunit.client.1.smithi104.stdout:9/349: truncate d11/d15/d32/d35/f3a 3141165 0 2024-04-26T04:42:17.062 INFO:tasks.workunit.client.0.smithi090.stdout:0/249: symlink d0/d9/d45/d53/l54 0 2024-04-26T04:42:17.063 INFO:tasks.workunit.client.1.smithi104.stdout:5/478: mkdir d0/d1/d4/d2a/d6b/d73/da4 0 2024-04-26T04:42:17.064 INFO:tasks.workunit.client.1.smithi104.stdout:8/398: mkdir d0/d1c/d2d/d7a 0 2024-04-26T04:42:17.065 INFO:tasks.workunit.client.1.smithi104.stdout:8/399: write d0/d1c/d27/f66 [111209,84175] 0 2024-04-26T04:42:17.066 INFO:tasks.workunit.client.1.smithi104.stdout:9/350: creat d11/d15/f6a x:0 0 0 2024-04-26T04:42:17.066 INFO:tasks.workunit.client.1.smithi104.stdout:9/351: readlink d11/d15/d32/l3f 0 2024-04-26T04:42:17.066 INFO:tasks.workunit.client.1.smithi104.stdout:9/352: stat d11/d15/d32/d35 0 2024-04-26T04:42:17.066 INFO:tasks.workunit.client.1.smithi104.stdout:9/353: fsync d11/d15/d32/d35/f5e 0 2024-04-26T04:42:17.068 INFO:tasks.workunit.client.0.smithi090.stdout:3/155: symlink d1/d6/d1b/l30 0 2024-04-26T04:42:17.068 INFO:tasks.workunit.client.0.smithi090.stdout:2/174: getdents db 0 2024-04-26T04:42:17.068 INFO:tasks.workunit.client.0.smithi090.stdout:2/175: creat db/d11/d14/d25/f3e x:0 0 0 2024-04-26T04:42:17.071 INFO:tasks.workunit.client.1.smithi104.stdout:8/400: creat d0/d33/f7b x:0 0 0 2024-04-26T04:42:17.071 INFO:tasks.workunit.client.1.smithi104.stdout:5/479: creat d0/d1/d4/d2a/d6b/d73/d95/fa5 x:0 0 0 2024-04-26T04:42:17.071 INFO:tasks.workunit.client.1.smithi104.stdout:9/354: fdatasync d11/d1d/f31 0 2024-04-26T04:42:17.071 INFO:tasks.workunit.client.1.smithi104.stdout:8/401: creat d0/d1c/d27/f7c x:0 0 0 2024-04-26T04:42:17.071 INFO:tasks.workunit.client.0.smithi090.stdout:6/290: rename d9/d21/d40/d4e to d9/d1c/d4d/d6d 0 2024-04-26T04:42:17.074 INFO:tasks.workunit.client.0.smithi090.stdout:0/250: creat d0/d9/d45/f55 x:0 0 0 2024-04-26T04:42:17.074 INFO:tasks.workunit.client.0.smithi090.stdout:8/160: dwrite dc/d1f/f27 [0,4194304] 0 2024-04-26T04:42:17.074 INFO:tasks.workunit.client.1.smithi104.stdout:5/480: dread d0/d1/f3d [0,4194304] 0 2024-04-26T04:42:17.076 INFO:tasks.workunit.client.1.smithi104.stdout:5/481: readlink d0/d1/d1e/l94 0 2024-04-26T04:42:17.077 INFO:tasks.workunit.client.0.smithi090.stdout:6/291: mknod d9/d21/d40/c6e 0 2024-04-26T04:42:17.077 INFO:tasks.workunit.client.0.smithi090.stdout:6/292: chown d9/d21/d3b/d62/f29 2 1 2024-04-26T04:42:17.078 INFO:tasks.workunit.client.1.smithi104.stdout:5/482: truncate d0/f58 2862372 0 2024-04-26T04:42:17.087 INFO:tasks.workunit.client.0.smithi090.stdout:7/237: dwrite d0/f8 [0,4194304] 0 2024-04-26T04:42:17.088 INFO:tasks.workunit.client.0.smithi090.stdout:7/238: chown d0/d1/d4/f47 59225 1 2024-04-26T04:42:17.091 INFO:tasks.workunit.client.1.smithi104.stdout:9/355: dread d11/d1d/d34/f5d [0,4194304] 0 2024-04-26T04:42:17.092 INFO:tasks.workunit.client.1.smithi104.stdout:9/356: chown d11/d15/l21 19311248 1 2024-04-26T04:42:17.094 INFO:tasks.workunit.client.1.smithi104.stdout:9/357: creat d11/d1d/f6b x:0 0 0 2024-04-26T04:42:17.106 INFO:tasks.workunit.client.0.smithi090.stdout:8/161: symlink dc/d34/l38 0 2024-04-26T04:42:17.107 INFO:tasks.workunit.client.1.smithi104.stdout:9/358: read d11/f17 [3025790,17052] 0 2024-04-26T04:42:17.107 INFO:tasks.workunit.client.1.smithi104.stdout:9/359: creat d11/d1d/d34/f6c x:0 0 0 2024-04-26T04:42:17.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/293: mkdir d9/d21/d40/d4c/d6f 0 2024-04-26T04:42:17.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/294: read - d9/d1c/f47 zero size 2024-04-26T04:42:17.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/295: creat d9/d21/f70 x:0 0 0 2024-04-26T04:42:17.109 INFO:tasks.workunit.client.1.smithi104.stdout:9/360: stat d11/d1d/l1e 0 2024-04-26T04:42:17.110 INFO:tasks.workunit.client.1.smithi104.stdout:9/361: mkdir d11/d15/d1b/d6d 0 2024-04-26T04:42:17.110 INFO:tasks.workunit.client.1.smithi104.stdout:9/362: write d11/d1d/d34/f3c [411578,49121] 0 2024-04-26T04:42:17.110 INFO:tasks.workunit.client.1.smithi104.stdout:9/363: creat d11/d15/d32/d4c/d54/f6e x:0 0 0 2024-04-26T04:42:17.110 INFO:tasks.workunit.client.1.smithi104.stdout:9/364: fdatasync d11/d15/d32/d35/f5e 0 2024-04-26T04:42:17.111 INFO:tasks.workunit.client.1.smithi104.stdout:0/418: dwrite d0/d2/d21/d27/d1e/d57/d6c/f56 [0,4194304] 0 2024-04-26T04:42:17.112 INFO:tasks.workunit.client.1.smithi104.stdout:9/365: symlink d11/d53/l6f 0 2024-04-26T04:42:17.114 INFO:tasks.workunit.client.0.smithi090.stdout:5/283: truncate d4/d1a/df/f28 262163 0 2024-04-26T04:42:17.116 INFO:tasks.workunit.client.1.smithi104.stdout:0/419: unlink d0/d2/d10/d3d/f4c 0 2024-04-26T04:42:17.117 INFO:tasks.workunit.client.1.smithi104.stdout:9/366: mkdir d11/d1d/d34/d70 0 2024-04-26T04:42:17.117 INFO:tasks.workunit.client.1.smithi104.stdout:9/367: write d11/d1d/d34/f6c [96826,38634] 0 2024-04-26T04:42:17.117 INFO:tasks.workunit.client.1.smithi104.stdout:9/368: fsync d11/f16 0 2024-04-26T04:42:17.118 INFO:tasks.workunit.client.1.smithi104.stdout:0/420: truncate d0/d2/f23 795401 0 2024-04-26T04:42:17.118 INFO:tasks.workunit.client.1.smithi104.stdout:9/369: chown d11/d15/l2b 1497241 1 2024-04-26T04:42:17.119 INFO:tasks.workunit.client.1.smithi104.stdout:9/370: truncate d11/f17 5048736 0 2024-04-26T04:42:17.119 INFO:tasks.workunit.client.1.smithi104.stdout:9/371: truncate d11/d15/d32/d35/f5e 833627 0 2024-04-26T04:42:17.121 INFO:tasks.workunit.client.1.smithi104.stdout:9/372: mkdir d11/d1d/d71 0 2024-04-26T04:42:17.121 INFO:tasks.workunit.client.1.smithi104.stdout:9/373: fdatasync d11/d15/d32/d35/f5e 0 2024-04-26T04:42:17.121 INFO:tasks.workunit.client.1.smithi104.stdout:9/374: read - d11/d1d/f6b zero size 2024-04-26T04:42:17.121 INFO:tasks.workunit.client.1.smithi104.stdout:9/375: fsync d11/d1d/d34/f5d 0 2024-04-26T04:42:17.122 INFO:tasks.workunit.client.0.smithi090.stdout:2/176: dwrite db/dc/f19 [0,4194304] 0 2024-04-26T04:42:17.123 INFO:tasks.workunit.client.1.smithi104.stdout:9/376: creat d11/d15/d1b/f72 x:0 0 0 2024-04-26T04:42:17.123 INFO:tasks.workunit.client.1.smithi104.stdout:9/377: readlink d11/d1d/l4d 0 2024-04-26T04:42:17.124 INFO:tasks.workunit.client.0.smithi090.stdout:3/156: dwrite d1/d6/f21 [0,4194304] 0 2024-04-26T04:42:17.125 INFO:tasks.workunit.client.0.smithi090.stdout:7/239: dwrite d0/d1/f38 [4194304,4194304] 0 2024-04-26T04:42:17.127 INFO:tasks.workunit.client.1.smithi104.stdout:0/421: dread d0/d2/d21/d49/f55 [0,4194304] 0 2024-04-26T04:42:17.128 INFO:tasks.workunit.client.1.smithi104.stdout:0/422: symlink d0/d2/d10/d3d/l87 0 2024-04-26T04:42:17.130 INFO:tasks.workunit.client.1.smithi104.stdout:3/531: sync 2024-04-26T04:42:17.130 INFO:tasks.workunit.client.1.smithi104.stdout:7/403: sync 2024-04-26T04:42:17.130 INFO:tasks.workunit.client.1.smithi104.stdout:7/404: truncate d2/d4/d32/d3d/f50 706383 0 2024-04-26T04:42:17.133 INFO:tasks.workunit.client.1.smithi104.stdout:7/405: rename d2/d4/d32/d3d/d41/c6b to d2/d4/d32/d3d/d41/d7e/c85 0 2024-04-26T04:42:17.133 INFO:tasks.workunit.client.0.smithi090.stdout:8/162: mkdir dc/d12/d39 0 2024-04-26T04:42:17.133 INFO:tasks.workunit.client.0.smithi090.stdout:8/163: stat dc/ff 0 2024-04-26T04:42:17.139 INFO:tasks.workunit.client.0.smithi090.stdout:3/157: unlink d1/fc 0 2024-04-26T04:42:17.139 INFO:tasks.workunit.client.1.smithi104.stdout:7/406: creat d2/d4/d32/d35/d37/f86 x:0 0 0 2024-04-26T04:42:17.139 INFO:tasks.workunit.client.1.smithi104.stdout:7/407: creat d2/d4/d32/d35/f87 x:0 0 0 2024-04-26T04:42:17.139 INFO:tasks.workunit.client.1.smithi104.stdout:7/408: fdatasync d2/f2b 0 2024-04-26T04:42:17.141 INFO:tasks.workunit.client.0.smithi090.stdout:7/240: creat d0/db/d15/f57 x:0 0 0 2024-04-26T04:42:17.141 INFO:tasks.workunit.client.0.smithi090.stdout:7/241: write d0/f4e [506179,11068] 0 2024-04-26T04:42:17.141 INFO:tasks.workunit.client.1.smithi104.stdout:7/409: rmdir d2/d4/d5a 39 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:3/532: dread d5/d9/dc/f16 [0,4194304] 0 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:3/533: stat d5/d9/dc/d69/f6e 0 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:0/423: read d0/d4/f29 [2380663,49667] 0 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:0/424: stat d0/d2/d10/l39 0 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:3/534: mkdir d5/db2/d46/db7 0 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:9/378: dwrite d11/d15/d32/d35/f57 [0,4194304] 0 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:9/379: dread - d11/d15/f6a zero size 2024-04-26T04:42:17.144 INFO:tasks.workunit.client.1.smithi104.stdout:9/380: creat d11/d1d/d34/f73 x:0 0 0 2024-04-26T04:42:17.145 INFO:tasks.workunit.client.1.smithi104.stdout:9/381: write d11/d15/f6a [787814,52635] 0 2024-04-26T04:42:17.145 INFO:tasks.workunit.client.0.smithi090.stdout:7/242: write d0/f4e [213761,130821] 0 2024-04-26T04:42:17.147 INFO:tasks.workunit.client.0.smithi090.stdout:3/158: symlink d1/d9/l31 0 2024-04-26T04:42:17.149 INFO:tasks.workunit.client.1.smithi104.stdout:3/535: creat d5/d9/dc/d3a/fb8 x:0 0 0 2024-04-26T04:42:17.151 INFO:tasks.workunit.client.1.smithi104.stdout:3/536: symlink d5/db2/da3/lb9 0 2024-04-26T04:42:17.152 INFO:tasks.workunit.client.1.smithi104.stdout:3/537: mknod d5/d9/dc/d32/d47/cba 0 2024-04-26T04:42:17.152 INFO:tasks.workunit.client.1.smithi104.stdout:3/538: creat d5/fbb x:0 0 0 2024-04-26T04:42:17.154 INFO:tasks.workunit.client.1.smithi104.stdout:3/539: mkdir d5/d9/d80/dbc 0 2024-04-26T04:42:17.156 INFO:tasks.workunit.client.1.smithi104.stdout:3/540: creat d5/d9/dc/d29/fbd x:0 0 0 2024-04-26T04:42:17.157 INFO:tasks.workunit.client.1.smithi104.stdout:3/541: getdents d5/d9/dc/d32/d48 0 2024-04-26T04:42:17.158 INFO:tasks.workunit.client.1.smithi104.stdout:3/542: creat d5/d9/dc/d29/d7d/d8e/fbe x:0 0 0 2024-04-26T04:42:17.160 INFO:tasks.workunit.client.1.smithi104.stdout:3/543: rename d5/d9/dc/f42 to d5/d9/dc/d69/fbf 0 2024-04-26T04:42:17.161 INFO:tasks.workunit.client.0.smithi090.stdout:3/159: dread d1/d9/fb [0,4194304] 0 2024-04-26T04:42:17.161 INFO:tasks.workunit.client.0.smithi090.stdout:3/160: creat d1/d6/d1b/d1e/f32 x:0 0 0 2024-04-26T04:42:17.162 INFO:tasks.workunit.client.1.smithi104.stdout:3/544: link d5/d9/dc/d29/d7d/d8e/la1 d5/d9/dc/d32/d48/lc0 0 2024-04-26T04:42:17.162 INFO:tasks.workunit.client.1.smithi104.stdout:3/545: write d5/db2/f2a [872367,36261] 0 2024-04-26T04:42:17.162 INFO:tasks.workunit.client.1.smithi104.stdout:3/546: fsync d5/d9/dc/d29/d49/f82 0 2024-04-26T04:42:17.162 INFO:tasks.workunit.client.1.smithi104.stdout:3/547: chown d5/d9/dc/d32/d48/dac 56347856 1 2024-04-26T04:42:17.165 INFO:tasks.workunit.client.0.smithi090.stdout:3/161: mkdir d1/d6/d1b/d1e/d33 0 2024-04-26T04:42:17.171 INFO:tasks.workunit.client.1.smithi104.stdout:9/382: dread d11/d1d/d34/f6c [0,4194304] 0 2024-04-26T04:42:17.172 INFO:tasks.workunit.client.1.smithi104.stdout:9/383: mknod d11/d15/d32/d4c/d54/c74 0 2024-04-26T04:42:17.174 INFO:tasks.workunit.client.1.smithi104.stdout:9/384: rmdir d11/d15/d32/d4c 39 2024-04-26T04:42:17.174 INFO:tasks.workunit.client.1.smithi104.stdout:9/385: fdatasync d11/f16 0 2024-04-26T04:42:17.174 INFO:tasks.workunit.client.1.smithi104.stdout:9/386: chown d11/d1d/d34/c48 7 1 2024-04-26T04:42:17.174 INFO:tasks.workunit.client.1.smithi104.stdout:9/387: creat d11/d15/d32/d4c/d54/f75 x:0 0 0 2024-04-26T04:42:17.175 INFO:tasks.workunit.client.1.smithi104.stdout:7/410: dwrite d2/d4/f13 [0,4194304] 0 2024-04-26T04:42:17.175 INFO:tasks.workunit.client.0.smithi090.stdout:8/164: dwrite dc/f14 [0,4194304] 0 2024-04-26T04:42:17.176 INFO:tasks.workunit.client.1.smithi104.stdout:0/425: dwrite d0/d4/d3c/d53/d5d/f75 [0,4194304] 0 2024-04-26T04:42:17.182 INFO:tasks.workunit.client.1.smithi104.stdout:0/426: rmdir d0/d2/d21/d49/d5e 39 2024-04-26T04:42:17.183 INFO:tasks.workunit.client.0.smithi090.stdout:8/165: mkdir dc/d1f/d3a 0 2024-04-26T04:42:17.183 INFO:tasks.workunit.client.0.smithi090.stdout:8/166: fsync dc/f1e 0 2024-04-26T04:42:17.184 INFO:tasks.workunit.client.0.smithi090.stdout:7/243: dwrite d0/d1/d4/f3b [4194304,4194304] 0 2024-04-26T04:42:17.187 INFO:tasks.workunit.client.0.smithi090.stdout:8/167: mkdir dc/d12/d3b 0 2024-04-26T04:42:17.195 INFO:tasks.workunit.client.0.smithi090.stdout:7/244: write d0/db/d15/d29/f14 [2021806,64130] 0 2024-04-26T04:42:17.196 INFO:tasks.workunit.client.0.smithi090.stdout:7/245: symlink d0/d1/d4/l58 0 2024-04-26T04:42:17.200 INFO:tasks.workunit.client.1.smithi104.stdout:3/548: dwrite d5/db2/d60/f7e [0,4194304] 0 2024-04-26T04:42:17.201 INFO:tasks.workunit.client.1.smithi104.stdout:3/549: chown d5/d9/dc/d29/fbd 6 1 2024-04-26T04:42:17.201 INFO:tasks.workunit.client.1.smithi104.stdout:3/550: chown d5/d9/dc/d29/d7d/d8e/la1 3801721 1 2024-04-26T04:42:17.201 INFO:tasks.workunit.client.1.smithi104.stdout:3/551: creat d5/d9/dc/d29/fc1 x:0 0 0 2024-04-26T04:42:17.202 INFO:tasks.workunit.client.1.smithi104.stdout:9/388: getdents d11/d15/d32/d4c/d54 0 2024-04-26T04:42:17.203 INFO:tasks.workunit.client.1.smithi104.stdout:9/389: truncate d11/f68 953599 0 2024-04-26T04:42:17.206 INFO:tasks.workunit.client.1.smithi104.stdout:3/552: mkdir d5/d9/dc/d29/d7d/d8e/dc2 0 2024-04-26T04:42:17.206 INFO:tasks.workunit.client.1.smithi104.stdout:3/553: dread - d5/d9/d80/da2/fad zero size 2024-04-26T04:42:17.207 INFO:tasks.workunit.client.1.smithi104.stdout:9/390: rmdir d11/d1d/d34/d70 0 2024-04-26T04:42:17.207 INFO:tasks.workunit.client.1.smithi104.stdout:9/391: read - d11/d15/d32/d4c/d54/f61 zero size 2024-04-26T04:42:17.207 INFO:tasks.workunit.client.0.smithi090.stdout:8/168: dread dc/d1f/f36 [0,4194304] 0 2024-04-26T04:42:17.208 INFO:tasks.workunit.client.1.smithi104.stdout:7/411: dwrite d2/d4/d32/d60/f63 [0,4194304] 0 2024-04-26T04:42:17.208 INFO:tasks.workunit.client.1.smithi104.stdout:3/554: creat d5/db2/da3/fc3 x:0 0 0 2024-04-26T04:42:17.208 INFO:tasks.workunit.client.1.smithi104.stdout:3/555: creat d5/db2/da3/fc4 x:0 0 0 2024-04-26T04:42:17.209 INFO:tasks.workunit.client.1.smithi104.stdout:3/556: write d5/f25 [130922,61873] 0 2024-04-26T04:42:17.209 INFO:tasks.workunit.client.0.smithi090.stdout:6/296: rename d9/cb to d9/d3c/c71 0 2024-04-26T04:42:17.210 INFO:tasks.workunit.client.0.smithi090.stdout:5/284: rename d4/d1a/d1f/f2f to d4/d1a/d45/d5b/f5d 0 2024-04-26T04:42:17.213 INFO:tasks.workunit.client.0.smithi090.stdout:6/297: mknod d9/d21/d40/c72 0 2024-04-26T04:42:17.214 INFO:tasks.workunit.client.1.smithi104.stdout:7/412: unlink d2/d4/d32/d35/l78 0 2024-04-26T04:42:17.214 INFO:tasks.workunit.client.1.smithi104.stdout:7/413: fsync d2/f42 0 2024-04-26T04:42:17.217 INFO:tasks.workunit.client.1.smithi104.stdout:9/392: rename d11/d1d/c25 to d11/d1d/d34/c76 0 2024-04-26T04:42:17.217 INFO:tasks.workunit.client.1.smithi104.stdout:9/393: chown d11/f12 54572789 1 2024-04-26T04:42:17.217 INFO:tasks.workunit.client.1.smithi104.stdout:9/394: write d11/d15/d1b/f60 [4751823,60798] 0 2024-04-26T04:42:17.217 INFO:tasks.workunit.client.1.smithi104.stdout:9/395: write d11/d15/d32/d4c/d54/f61 [935349,42175] 0 2024-04-26T04:42:17.217 INFO:tasks.workunit.client.1.smithi104.stdout:9/396: chown d11/d1d/d34/c45 43992 1 2024-04-26T04:42:17.218 INFO:tasks.workunit.client.1.smithi104.stdout:9/397: chown d11/d15/d1b/c2e 19918 1 2024-04-26T04:42:17.218 INFO:tasks.workunit.client.0.smithi090.stdout:2/177: rename db/d11/d14/d34 to db/d3f 0 2024-04-26T04:42:17.218 INFO:tasks.workunit.client.0.smithi090.stdout:2/178: chown db/d11/d14/d1b/d39 1172788956 1 2024-04-26T04:42:17.220 INFO:tasks.workunit.client.1.smithi104.stdout:4/590: sync 2024-04-26T04:42:17.220 INFO:tasks.workunit.client.1.smithi104.stdout:2/646: sync 2024-04-26T04:42:17.220 INFO:tasks.workunit.client.1.smithi104.stdout:6/635: sync 2024-04-26T04:42:17.220 INFO:tasks.workunit.client.1.smithi104.stdout:6/636: fdatasync d0/d9/d13/d41/d96/fb0 0 2024-04-26T04:42:17.220 INFO:tasks.workunit.client.1.smithi104.stdout:1/533: sync 2024-04-26T04:42:17.221 INFO:tasks.workunit.client.1.smithi104.stdout:1/534: truncate d0/d6/d2e/f86 607553 0 2024-04-26T04:42:17.221 INFO:tasks.workunit.client.1.smithi104.stdout:2/647: symlink d6/de/d1d/dd7/lde 0 2024-04-26T04:42:17.222 INFO:tasks.workunit.client.1.smithi104.stdout:4/591: symlink d4/db/d6b/ld4 0 2024-04-26T04:42:17.222 INFO:tasks.workunit.client.1.smithi104.stdout:4/592: stat d4/d7/d1d/d1e/d24/d59 0 2024-04-26T04:42:17.225 INFO:tasks.workunit.client.1.smithi104.stdout:6/637: creat d0/d9/d13/d41/d82/dc8/fd6 x:0 0 0 2024-04-26T04:42:17.225 INFO:tasks.workunit.client.1.smithi104.stdout:6/638: write d0/d9/d13/d41/f73 [609420,92637] 0 2024-04-26T04:42:17.225 INFO:tasks.workunit.client.1.smithi104.stdout:1/535: symlink d0/d6/d2e/lad 0 2024-04-26T04:42:17.228 INFO:tasks.workunit.client.0.smithi090.stdout:7/246: rename d0/db/df/c34 to d0/c59 0 2024-04-26T04:42:17.228 INFO:tasks.workunit.client.0.smithi090.stdout:8/169: rename dc/d1f/d3a to dc/d1f/d3a/d3c 22 2024-04-26T04:42:17.228 INFO:tasks.workunit.client.0.smithi090.stdout:2/179: getdents db/d11/d14/d1b 0 2024-04-26T04:42:17.229 INFO:tasks.workunit.client.0.smithi090.stdout:2/180: dread - db/d11/f26 zero size 2024-04-26T04:42:17.229 INFO:tasks.workunit.client.0.smithi090.stdout:2/181: fdatasync db/d11/d14/d25/f37 0 2024-04-26T04:42:17.229 INFO:tasks.workunit.client.0.smithi090.stdout:2/182: write db/d11/d14/d25/d2d/f36 [468955,1838] 0 2024-04-26T04:42:17.229 INFO:tasks.workunit.client.1.smithi104.stdout:1/536: rename d0/d6/d2e/f95 to d0/d6/d19/fae 0 2024-04-26T04:42:17.229 INFO:tasks.workunit.client.1.smithi104.stdout:2/648: getdents d6/d8/d3e 0 2024-04-26T04:42:17.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/298: getdents d9/d21/d3b 0 2024-04-26T04:42:17.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/299: fdatasync d9/d21/d3b/d62/f29 0 2024-04-26T04:42:17.231 INFO:tasks.workunit.client.1.smithi104.stdout:2/649: write d6/de/d1d/f6d [284093,20262] 0 2024-04-26T04:42:17.232 INFO:tasks.workunit.client.0.smithi090.stdout:2/183: mkdir db/d11/d14/d1b/d39/d40 0 2024-04-26T04:42:17.234 INFO:tasks.workunit.client.0.smithi090.stdout:7/247: dread d0/f4e [0,4194304] 0 2024-04-26T04:42:17.235 INFO:tasks.workunit.client.0.smithi090.stdout:6/300: symlink d9/d21/d40/d4c/d6f/l73 0 2024-04-26T04:42:17.245 INFO:tasks.workunit.client.1.smithi104.stdout:2/650: dread d6/de/f40 [0,4194304] 0 2024-04-26T04:42:17.247 INFO:tasks.workunit.client.1.smithi104.stdout:2/651: rename d6/d1b/d51/d6a/la7 to d6/d1b/d51/d6a/ldf 0 2024-04-26T04:42:17.249 INFO:tasks.workunit.client.1.smithi104.stdout:3/557: dwrite d5/d9/dc/d29/d7d/d8e/fbe [0,4194304] 0 2024-04-26T04:42:17.250 INFO:tasks.workunit.client.0.smithi090.stdout:8/170: write dc/f14 [749812,90047] 0 2024-04-26T04:42:17.251 INFO:tasks.workunit.client.0.smithi090.stdout:8/171: write dc/d1f/f36 [1123312,45305] 0 2024-04-26T04:42:17.251 INFO:tasks.workunit.client.0.smithi090.stdout:8/172: chown dc/f1d 6262 1 2024-04-26T04:42:17.251 INFO:tasks.workunit.client.1.smithi104.stdout:3/558: link d5/db2/f34 d5/d9/dc/d29/fc5 0 2024-04-26T04:42:17.252 INFO:tasks.workunit.client.1.smithi104.stdout:3/559: link d5/d9/dc/d69/fb1 d5/d9/dc/d29/fc6 0 2024-04-26T04:42:17.263 INFO:tasks.workunit.client.0.smithi090.stdout:5/285: dwrite d4/d1a/df/d36/f54 [0,4194304] 0 2024-04-26T04:42:17.264 INFO:tasks.workunit.client.1.smithi104.stdout:7/414: dwrite d2/d4/d32/f69 [0,4194304] 0 2024-04-26T04:42:17.266 INFO:tasks.workunit.client.0.smithi090.stdout:8/173: write dc/d12/f20 [2089340,38613] 0 2024-04-26T04:42:17.268 INFO:tasks.workunit.client.0.smithi090.stdout:5/286: rename d4/d1a/d45/d5b to d4/d1a/d1f/d53/d59/d5e 0 2024-04-26T04:42:17.268 INFO:tasks.workunit.client.1.smithi104.stdout:7/415: symlink d2/d4/d32/d35/d44/l88 0 2024-04-26T04:42:17.269 INFO:tasks.workunit.client.1.smithi104.stdout:7/416: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d89 0 2024-04-26T04:42:17.270 INFO:tasks.workunit.client.1.smithi104.stdout:7/417: creat d2/d4/d32/d3d/d47/f8a x:0 0 0 2024-04-26T04:42:17.271 INFO:tasks.workunit.client.1.smithi104.stdout:7/418: stat d2/le 0 2024-04-26T04:42:17.271 INFO:tasks.workunit.client.1.smithi104.stdout:7/419: truncate d2/d4/d32/d3d/d47/f8a 665718 0 2024-04-26T04:42:17.271 INFO:tasks.workunit.client.1.smithi104.stdout:7/420: truncate d2/f42 1275125 0 2024-04-26T04:42:17.271 INFO:tasks.workunit.client.1.smithi104.stdout:7/421: fsync d2/d4/d32/d60/f63 0 2024-04-26T04:42:17.284 INFO:tasks.workunit.client.0.smithi090.stdout:5/287: dread d4/d1a/d1f/f2b [4194304,4194304] 0 2024-04-26T04:42:17.285 INFO:tasks.workunit.client.1.smithi104.stdout:9/398: dwrite d11/d15/d32/d35/f5e [0,4194304] 0 2024-04-26T04:42:17.286 INFO:tasks.workunit.client.0.smithi090.stdout:5/288: rmdir d4/d1a/df/d4a 39 2024-04-26T04:42:17.286 INFO:tasks.workunit.client.1.smithi104.stdout:1/537: dwrite d0/d6/d19/d21/d66/d20/d2a/f61 [4194304,4194304] 0 2024-04-26T04:42:17.286 INFO:tasks.workunit.client.1.smithi104.stdout:6/639: dwrite d0/d9/d13/d44/f97 [0,4194304] 0 2024-04-26T04:42:17.287 INFO:tasks.workunit.client.0.smithi090.stdout:5/289: getdents d4/d1a/df/d4a 0 2024-04-26T04:42:17.287 INFO:tasks.workunit.client.1.smithi104.stdout:2/652: rename d6/de/d1d/dd7 to d6/d1b/d68/de0 0 2024-04-26T04:42:17.288 INFO:tasks.workunit.client.0.smithi090.stdout:5/290: symlink d4/d1a/df/d36/d42/d49/l5f 0 2024-04-26T04:42:17.288 INFO:tasks.workunit.client.1.smithi104.stdout:3/560: dwrite d5/d9/dc/d29/d7d/d8e/f98 [0,4194304] 0 2024-04-26T04:42:17.289 INFO:tasks.workunit.client.1.smithi104.stdout:9/399: creat d11/d15/d32/d4c/d54/f77 x:0 0 0 2024-04-26T04:42:17.290 INFO:tasks.workunit.client.0.smithi090.stdout:8/174: fdatasync dc/d12/f20 0 2024-04-26T04:42:17.290 INFO:tasks.workunit.client.0.smithi090.stdout:6/301: dwrite d9/d1c/f57 [0,4194304] 0 2024-04-26T04:42:17.290 INFO:tasks.workunit.client.0.smithi090.stdout:7/248: dwrite d0/db/d15/d29/fc [8388608,4194304] 0 2024-04-26T04:42:17.290 INFO:tasks.workunit.client.0.smithi090.stdout:6/302: chown d9/d21/d3b/d62/c4f 100182 1 2024-04-26T04:42:17.291 INFO:tasks.workunit.client.1.smithi104.stdout:7/422: dwrite d2/d4/d32/f76 [0,4194304] 0 2024-04-26T04:42:17.292 INFO:tasks.workunit.client.1.smithi104.stdout:7/423: chown d2/d4/d1d/l2c 54 1 2024-04-26T04:42:17.292 INFO:tasks.workunit.client.1.smithi104.stdout:7/424: write d2/f2b [4680375,64307] 0 2024-04-26T04:42:17.292 INFO:tasks.workunit.client.0.smithi090.stdout:8/175: mkdir dc/d12/d3b/d3d 0 2024-04-26T04:42:17.292 INFO:tasks.workunit.client.0.smithi090.stdout:8/176: readlink dc/d1f/l37 0 2024-04-26T04:42:17.292 INFO:tasks.workunit.client.0.smithi090.stdout:2/184: dwrite db/dc/f1e [4194304,4194304] 0 2024-04-26T04:42:17.293 INFO:tasks.workunit.client.1.smithi104.stdout:1/538: rmdir d0/d6/d19/d1b/d35 39 2024-04-26T04:42:17.296 INFO:tasks.workunit.client.1.smithi104.stdout:2/653: mknod d6/ce1 0 2024-04-26T04:42:17.297 INFO:tasks.workunit.client.1.smithi104.stdout:3/561: creat d5/d9/dc/d32/d48/fc7 x:0 0 0 2024-04-26T04:42:17.297 INFO:tasks.workunit.client.0.smithi090.stdout:7/249: creat d0/db/df/d4d/f5a x:0 0 0 2024-04-26T04:42:17.300 INFO:tasks.workunit.client.1.smithi104.stdout:9/400: creat d11/d1d/d34/d69/f78 x:0 0 0 2024-04-26T04:42:17.300 INFO:tasks.workunit.client.1.smithi104.stdout:9/401: read d11/f17 [4066484,119194] 0 2024-04-26T04:42:17.301 INFO:tasks.workunit.client.0.smithi090.stdout:8/177: link dc/d12/f17 dc/d1f/d3a/f3e 0 2024-04-26T04:42:17.302 INFO:tasks.workunit.client.1.smithi104.stdout:7/425: rename d2/f28 to d2/d4/d32/d3d/d41/d5f/d6d/f8b 0 2024-04-26T04:42:17.304 INFO:tasks.workunit.client.1.smithi104.stdout:1/539: symlink d0/d6/d19/d21/laf 0 2024-04-26T04:42:17.307 INFO:tasks.workunit.client.1.smithi104.stdout:7/426: getdents d2/d4/d1d/d34 0 2024-04-26T04:42:17.307 INFO:tasks.workunit.client.1.smithi104.stdout:7/427: fdatasync d2/d4/d32/d35/d37/f86 0 2024-04-26T04:42:17.308 INFO:tasks.workunit.client.0.smithi090.stdout:2/185: rename db/d11/d14/d1b/d39/d40 to db/d11/d14/d25/d41 0 2024-04-26T04:42:17.308 INFO:tasks.workunit.client.1.smithi104.stdout:1/540: rmdir d0/d6/d2e/d32/d4d 39 2024-04-26T04:42:17.309 INFO:tasks.workunit.client.1.smithi104.stdout:7/428: creat d2/d4/d32/d60/f8c x:0 0 0 2024-04-26T04:42:17.310 INFO:tasks.workunit.client.1.smithi104.stdout:2/654: dread d6/d8/d5a/fb0 [0,4194304] 0 2024-04-26T04:42:17.310 INFO:tasks.workunit.client.1.smithi104.stdout:7/429: read d2/d4/d1d/f3e [111388,59795] 0 2024-04-26T04:42:17.310 INFO:tasks.workunit.client.0.smithi090.stdout:8/178: truncate dc/d12/f2e 1556364 0 2024-04-26T04:42:17.310 INFO:tasks.workunit.client.1.smithi104.stdout:1/541: creat d0/d6/d2e/d32/d4c/fb0 x:0 0 0 2024-04-26T04:42:17.311 INFO:tasks.workunit.client.0.smithi090.stdout:2/186: truncate db/dc/f12 594216 0 2024-04-26T04:42:17.311 INFO:tasks.workunit.client.1.smithi104.stdout:2/655: creat d6/de/fe2 x:0 0 0 2024-04-26T04:42:17.311 INFO:tasks.workunit.client.1.smithi104.stdout:2/656: truncate d6/d1b/f38 673672 0 2024-04-26T04:42:17.312 INFO:tasks.workunit.client.1.smithi104.stdout:1/542: mkdir d0/d6/d2e/d32/d4c/db1 0 2024-04-26T04:42:17.312 INFO:tasks.workunit.client.1.smithi104.stdout:2/657: symlink d6/de/le3 0 2024-04-26T04:42:17.312 INFO:tasks.workunit.client.1.smithi104.stdout:2/658: write d6/de/d1d/f4a [615465,39342] 0 2024-04-26T04:42:17.313 INFO:tasks.workunit.client.0.smithi090.stdout:8/179: link dc/d1f/l2f dc/d12/d3b/l3f 0 2024-04-26T04:42:17.313 INFO:tasks.workunit.client.0.smithi090.stdout:8/180: write dc/f1e [1307609,103014] 0 2024-04-26T04:42:17.314 INFO:tasks.workunit.client.0.smithi090.stdout:2/187: mkdir db/d11/d14/d27/d2f/d42 0 2024-04-26T04:42:17.314 INFO:tasks.workunit.client.0.smithi090.stdout:2/188: write db/d11/f26 [437008,78587] 0 2024-04-26T04:42:17.314 INFO:tasks.workunit.client.0.smithi090.stdout:2/189: readlink db/l13 0 2024-04-26T04:42:17.317 INFO:tasks.workunit.client.0.smithi090.stdout:8/181: creat dc/d34/f40 x:0 0 0 2024-04-26T04:42:17.317 INFO:tasks.workunit.client.1.smithi104.stdout:6/640: dwrite d0/d9/dc6/f2c [0,4194304] 0 2024-04-26T04:42:17.320 INFO:tasks.workunit.client.0.smithi090.stdout:7/250: dwrite d0/db/d15/d29/f44 [0,4194304] 0 2024-04-26T04:42:17.321 INFO:tasks.workunit.client.0.smithi090.stdout:7/251: stat d0/db/d15/c2a 0 2024-04-26T04:42:17.321 INFO:tasks.workunit.client.1.smithi104.stdout:6/641: creat d0/d9/d25/fd7 x:0 0 0 2024-04-26T04:42:17.322 INFO:tasks.workunit.client.0.smithi090.stdout:5/291: dwrite d4/d1a/d1f/f3a [0,4194304] 0 2024-04-26T04:42:17.324 INFO:tasks.workunit.client.1.smithi104.stdout:6/642: mkdir d0/dd8 0 2024-04-26T04:42:17.327 INFO:tasks.workunit.client.1.smithi104.stdout:6/643: unlink d0/d9/d13/d41/d96/dac/c31 0 2024-04-26T04:42:17.327 INFO:tasks.workunit.client.1.smithi104.stdout:7/430: read d2/f75 [503356,82235] 0 2024-04-26T04:42:17.329 INFO:tasks.workunit.client.0.smithi090.stdout:4/270: sync 2024-04-26T04:42:17.329 INFO:tasks.workunit.client.0.smithi090.stdout:9/163: sync 2024-04-26T04:42:17.329 INFO:tasks.workunit.client.0.smithi090.stdout:0/251: sync 2024-04-26T04:42:17.330 INFO:tasks.workunit.client.0.smithi090.stdout:7/252: write d0/db/d53/fa [7816136,61080] 0 2024-04-26T04:42:17.330 INFO:tasks.workunit.client.0.smithi090.stdout:7/253: write d0/f3c [236311,90215] 0 2024-04-26T04:42:17.330 INFO:tasks.workunit.client.0.smithi090.stdout:7/254: chown d0/d1/d4/d43/d4a 767 1 2024-04-26T04:42:17.330 INFO:tasks.workunit.client.0.smithi090.stdout:7/255: chown d0/f45 21 1 2024-04-26T04:42:17.331 INFO:tasks.workunit.client.1.smithi104.stdout:7/431: unlink d2/f2b 0 2024-04-26T04:42:17.332 INFO:tasks.workunit.client.1.smithi104.stdout:6/644: mkdir d0/d9/d13/dcd/dd9 0 2024-04-26T04:42:17.332 INFO:tasks.workunit.client.0.smithi090.stdout:9/164: rmdir db/dd/d18/d38/d39 39 2024-04-26T04:42:17.332 INFO:tasks.workunit.client.0.smithi090.stdout:9/165: write db/dd/d18/f2a [588107,3384] 0 2024-04-26T04:42:17.333 INFO:tasks.workunit.client.0.smithi090.stdout:9/166: dread - db/dd/d18/f36 zero size 2024-04-26T04:42:17.334 INFO:tasks.workunit.client.0.smithi090.stdout:3/162: sync 2024-04-26T04:42:17.334 INFO:tasks.workunit.client.0.smithi090.stdout:1/190: sync 2024-04-26T04:42:17.335 INFO:tasks.workunit.client.1.smithi104.stdout:6/645: mkdir d0/d9/d13/d41/d82/dc8/dba/daa/dda 0 2024-04-26T04:42:17.338 INFO:tasks.workunit.client.0.smithi090.stdout:0/252: creat d0/d26/d48/f56 x:0 0 0 2024-04-26T04:42:17.341 INFO:tasks.workunit.client.0.smithi090.stdout:9/167: creat db/dd/d18/d31/f3c x:0 0 0 2024-04-26T04:42:17.342 INFO:tasks.workunit.client.0.smithi090.stdout:9/168: fdatasync db/dd/ff 0 2024-04-26T04:42:17.342 INFO:tasks.workunit.client.0.smithi090.stdout:4/271: getdents d3/d13/d1d 0 2024-04-26T04:42:17.344 INFO:tasks.workunit.client.0.smithi090.stdout:3/163: link d1/d6/d1b/l30 d1/d6/d1b/l34 0 2024-04-26T04:42:17.345 INFO:tasks.workunit.client.0.smithi090.stdout:7/256: rmdir d0/db/d15 39 2024-04-26T04:42:17.346 INFO:tasks.workunit.client.0.smithi090.stdout:4/272: unlink d3/db/d12/d20/f26 0 2024-04-26T04:42:17.348 INFO:tasks.workunit.client.0.smithi090.stdout:3/164: link d1/d6/f14 d1/d6/d1b/f35 0 2024-04-26T04:42:17.349 INFO:tasks.workunit.client.0.smithi090.stdout:4/273: mknod d3/db/d12/d20/d24/c55 0 2024-04-26T04:42:17.349 INFO:tasks.workunit.client.0.smithi090.stdout:1/191: dread d0/d4/f14 [0,4194304] 0 2024-04-26T04:42:17.351 INFO:tasks.workunit.client.0.smithi090.stdout:4/274: write d3/db/d12/d20/d24/f32 [351786,24752] 0 2024-04-26T04:42:17.351 INFO:tasks.workunit.client.0.smithi090.stdout:4/275: write d3/db/d12/d20/d2d/d2e/f3a [1146130,9905] 0 2024-04-26T04:42:17.351 INFO:tasks.workunit.client.0.smithi090.stdout:0/253: write d0/f38 [3319423,88467] 0 2024-04-26T04:42:17.353 INFO:tasks.workunit.client.0.smithi090.stdout:9/169: dread db/f13 [0,4194304] 0 2024-04-26T04:42:17.353 INFO:tasks.workunit.client.0.smithi090.stdout:4/276: rename d3/db/d12/d20/d24/c4a to d3/d13/d1d/d50/c56 0 2024-04-26T04:42:17.353 INFO:tasks.workunit.client.0.smithi090.stdout:1/192: write d0/f2e [259692,73964] 0 2024-04-26T04:42:17.353 INFO:tasks.workunit.client.0.smithi090.stdout:9/170: write db/dd/d18/f36 [954795,110884] 0 2024-04-26T04:42:17.354 INFO:tasks.workunit.client.0.smithi090.stdout:9/171: dread - db/f1b zero size 2024-04-26T04:42:17.354 INFO:tasks.workunit.client.0.smithi090.stdout:9/172: creat db/f3d x:0 0 0 2024-04-26T04:42:17.354 INFO:tasks.workunit.client.0.smithi090.stdout:9/173: write db/f1b [928175,22045] 0 2024-04-26T04:42:17.356 INFO:tasks.workunit.client.0.smithi090.stdout:9/174: mknod db/d1e/c3e 0 2024-04-26T04:42:17.356 INFO:tasks.workunit.client.0.smithi090.stdout:0/254: symlink d0/df/l57 0 2024-04-26T04:42:17.357 INFO:tasks.workunit.client.0.smithi090.stdout:7/257: write d0/d1/d4/f3b [7198897,26735] 0 2024-04-26T04:42:17.357 INFO:tasks.workunit.client.0.smithi090.stdout:1/193: symlink d0/d4/d7/d17/d22/l3a 0 2024-04-26T04:42:17.357 INFO:tasks.workunit.client.0.smithi090.stdout:1/194: readlink d0/d4/d7/d17/d22/l3a 0 2024-04-26T04:42:17.358 INFO:tasks.workunit.client.0.smithi090.stdout:9/175: mkdir db/dd/d18/d38/d39/d3f 0 2024-04-26T04:42:17.359 INFO:tasks.workunit.client.0.smithi090.stdout:7/258: symlink d0/d1/d4/l5b 0 2024-04-26T04:42:17.360 INFO:tasks.workunit.client.0.smithi090.stdout:1/195: symlink d0/d4/d7/l3b 0 2024-04-26T04:42:17.360 INFO:tasks.workunit.client.0.smithi090.stdout:1/196: creat d0/d4/d7/d17/f3c x:0 0 0 2024-04-26T04:42:17.360 INFO:tasks.workunit.client.0.smithi090.stdout:1/197: creat d0/d4/d7/d17/f3d x:0 0 0 2024-04-26T04:42:17.362 INFO:tasks.workunit.client.0.smithi090.stdout:1/198: fsync d0/d4/d7/f30 0 2024-04-26T04:42:17.363 INFO:tasks.workunit.client.0.smithi090.stdout:0/255: getdents d0/d9 0 2024-04-26T04:42:17.364 INFO:tasks.workunit.client.1.smithi104.stdout:1/543: dwrite d0/d6/d19/d21/d66/f83 [0,4194304] 0 2024-04-26T04:42:17.368 INFO:tasks.workunit.client.1.smithi104.stdout:1/544: symlink d0/d6/d19/d21/d58/lb2 0 2024-04-26T04:42:17.370 INFO:tasks.workunit.client.1.smithi104.stdout:3/562: dwrite d5/fbb [0,4194304] 0 2024-04-26T04:42:17.371 INFO:tasks.workunit.client.0.smithi090.stdout:0/256: mknod d0/c58 0 2024-04-26T04:42:17.371 INFO:tasks.workunit.client.0.smithi090.stdout:0/257: readlink d0/df/l57 0 2024-04-26T04:42:17.373 INFO:tasks.workunit.client.1.smithi104.stdout:7/432: dwrite d2/d4/d32/d35/f7a [0,4194304] 0 2024-04-26T04:42:17.374 INFO:tasks.workunit.client.1.smithi104.stdout:2/659: dwrite d6/d1b/d93/faa [0,4194304] 0 2024-04-26T04:42:17.374 INFO:tasks.workunit.client.1.smithi104.stdout:2/660: stat d6/de/d8e/l57 0 2024-04-26T04:42:17.374 INFO:tasks.workunit.client.1.smithi104.stdout:2/661: chown d6/d8/d3e/f2a 177973 1 2024-04-26T04:42:17.374 INFO:tasks.workunit.client.1.smithi104.stdout:2/662: dread - d6/d8/fcd zero size 2024-04-26T04:42:17.374 INFO:tasks.workunit.client.1.smithi104.stdout:2/663: fsync d6/d1b/d93/faa 0 2024-04-26T04:42:17.374 INFO:tasks.workunit.client.1.smithi104.stdout:7/433: dread d2/d4/d1d/f3e [0,4194304] 0 2024-04-26T04:42:17.375 INFO:tasks.workunit.client.0.smithi090.stdout:0/258: rename d0/f2d to d0/d9/d45/f59 0 2024-04-26T04:42:17.377 INFO:tasks.workunit.client.1.smithi104.stdout:7/434: creat d2/d4/d32/d3d/d41/d5f/f8d x:0 0 0 2024-04-26T04:42:17.378 INFO:tasks.workunit.client.0.smithi090.stdout:2/190: dwrite db/f10 [0,4194304] 0 2024-04-26T04:42:17.378 INFO:tasks.workunit.client.0.smithi090.stdout:2/191: read - db/d11/d14/d25/f3a zero size 2024-04-26T04:42:17.379 INFO:tasks.workunit.client.1.smithi104.stdout:7/435: mknod d2/c8e 0 2024-04-26T04:42:17.379 INFO:tasks.workunit.client.1.smithi104.stdout:7/436: chown d2/d4/d5a/c79 19308422 1 2024-04-26T04:42:17.381 INFO:tasks.workunit.client.1.smithi104.stdout:7/437: mkdir d2/d4/d1d/d34/d7f/d8f 0 2024-04-26T04:42:17.382 INFO:tasks.workunit.client.1.smithi104.stdout:2/664: dread d6/de/f40 [0,4194304] 0 2024-04-26T04:42:17.382 INFO:tasks.workunit.client.1.smithi104.stdout:2/665: write d6/d1b/f71 [1511185,85915] 0 2024-04-26T04:42:17.382 INFO:tasks.workunit.client.1.smithi104.stdout:7/438: read d2/f42 [1069980,77930] 0 2024-04-26T04:42:17.383 INFO:tasks.workunit.client.0.smithi090.stdout:8/182: dwrite dc/d1f/f21 [0,4194304] 0 2024-04-26T04:42:17.385 INFO:tasks.workunit.client.1.smithi104.stdout:7/439: unlink d2/d4/d32/d3d/d41/d5f/l81 0 2024-04-26T04:42:17.386 INFO:tasks.workunit.client.0.smithi090.stdout:0/259: truncate d0/d9/f41 686808 0 2024-04-26T04:42:17.386 INFO:tasks.workunit.client.1.smithi104.stdout:7/440: truncate d2/d4/d32/d35/d37/f3a 934769 0 2024-04-26T04:42:17.388 INFO:tasks.workunit.client.1.smithi104.stdout:7/441: rename d2/d4/d1d/f29 to d2/d4/d1d/f90 0 2024-04-26T04:42:17.391 INFO:tasks.workunit.client.0.smithi090.stdout:8/183: link dc/c19 dc/c41 0 2024-04-26T04:42:17.391 INFO:tasks.workunit.client.1.smithi104.stdout:7/442: truncate d2/d4/d32/f69 2269098 0 2024-04-26T04:42:17.392 INFO:tasks.workunit.client.0.smithi090.stdout:8/184: mknod dc/d1f/c42 0 2024-04-26T04:42:17.394 INFO:tasks.workunit.client.1.smithi104.stdout:7/443: creat d2/d4/d32/d35/d44/f91 x:0 0 0 2024-04-26T04:42:17.395 INFO:tasks.workunit.client.1.smithi104.stdout:6/646: dwrite d0/d9/d25/f39 [0,4194304] 0 2024-04-26T04:42:17.396 INFO:tasks.workunit.client.1.smithi104.stdout:2/666: dread d6/d8/d5a/f2e [0,4194304] 0 2024-04-26T04:42:17.396 INFO:tasks.workunit.client.1.smithi104.stdout:2/667: chown d6/de/d8e/f4c 23927774 1 2024-04-26T04:42:17.396 INFO:tasks.workunit.client.1.smithi104.stdout:7/444: symlink d2/d4/d32/d3d/l92 0 2024-04-26T04:42:17.400 INFO:tasks.workunit.client.1.smithi104.stdout:6/647: creat d0/d9/d13/d41/d82/dc8/dba/fdb x:0 0 0 2024-04-26T04:42:17.402 INFO:tasks.workunit.client.1.smithi104.stdout:2/668: creat d6/d1b/fe4 x:0 0 0 2024-04-26T04:42:17.402 INFO:tasks.workunit.client.1.smithi104.stdout:2/669: fsync d6/f33 0 2024-04-26T04:42:17.403 INFO:tasks.workunit.client.1.smithi104.stdout:2/670: fsync d6/d1b/d51/d6a/dd8/fdc 0 2024-04-26T04:42:17.403 INFO:tasks.workunit.client.1.smithi104.stdout:2/671: creat d6/d1b/d93/fe5 x:0 0 0 2024-04-26T04:42:17.408 INFO:tasks.workunit.client.0.smithi090.stdout:9/176: dwrite db/f3d [0,4194304] 0 2024-04-26T04:42:17.408 INFO:tasks.workunit.client.0.smithi090.stdout:9/177: readlink db/dd/l22 0 2024-04-26T04:42:17.408 INFO:tasks.workunit.client.0.smithi090.stdout:9/178: truncate f3 2416467 0 2024-04-26T04:42:17.408 INFO:tasks.workunit.client.0.smithi090.stdout:9/179: fdatasync db/d1e/f27 0 2024-04-26T04:42:17.408 INFO:tasks.workunit.client.1.smithi104.stdout:6/648: dread d0/d9/d13/d41/d82/dc8/f5e [0,4194304] 0 2024-04-26T04:42:17.408 INFO:tasks.workunit.client.1.smithi104.stdout:6/649: readlink d0/l14 0 2024-04-26T04:42:17.409 INFO:tasks.workunit.client.1.smithi104.stdout:6/650: dread d0/d9/d13/d44/d71/fb9 [0,4194304] 0 2024-04-26T04:42:17.410 INFO:tasks.workunit.client.1.smithi104.stdout:6/651: creat d0/d9/d13/d41/d82/dc8/dba/daa/fdc x:0 0 0 2024-04-26T04:42:17.416 INFO:tasks.workunit.client.0.smithi090.stdout:7/259: dwrite d0/db/df/d4d/f5a [0,4194304] 0 2024-04-26T04:42:17.416 INFO:tasks.workunit.client.0.smithi090.stdout:7/260: write d0/f45 [742363,109664] 0 2024-04-26T04:42:17.416 INFO:tasks.workunit.client.0.smithi090.stdout:7/261: write d0/f4e [955356,91771] 0 2024-04-26T04:42:17.416 INFO:tasks.workunit.client.0.smithi090.stdout:4/277: dwrite d3/db/d12/d20/d24/f52 [0,4194304] 0 2024-04-26T04:42:17.418 INFO:tasks.workunit.client.0.smithi090.stdout:4/278: dread d3/db/d12/d20/d24/d30/f37 [0,4194304] 0 2024-04-26T04:42:17.421 INFO:tasks.workunit.client.0.smithi090.stdout:7/262: dread d0/f3c [0,4194304] 0 2024-04-26T04:42:17.423 INFO:tasks.workunit.client.0.smithi090.stdout:7/263: symlink d0/db/d15/d30/d4f/l5c 0 2024-04-26T04:42:17.424 INFO:tasks.workunit.client.0.smithi090.stdout:4/279: dread d3/db/d12/d20/f41 [0,4194304] 0 2024-04-26T04:42:17.424 INFO:tasks.workunit.client.0.smithi090.stdout:7/264: rename d0/db/d15/l50 to d0/d1/l5d 0 2024-04-26T04:42:17.426 INFO:tasks.workunit.client.0.smithi090.stdout:4/280: mkdir d3/d13/d57 0 2024-04-26T04:42:17.426 INFO:tasks.workunit.client.0.smithi090.stdout:4/281: stat d3/db/d12 0 2024-04-26T04:42:17.427 INFO:tasks.workunit.client.0.smithi090.stdout:7/265: mkdir d0/db/d53/d5e 0 2024-04-26T04:42:17.428 INFO:tasks.workunit.client.0.smithi090.stdout:7/266: mknod d0/c5f 0 2024-04-26T04:42:17.428 INFO:tasks.workunit.client.0.smithi090.stdout:7/267: write d0/db/df/d1c/d23/f31 [1418057,36030] 0 2024-04-26T04:42:17.429 INFO:tasks.workunit.client.1.smithi104.stdout:6/652: write d0/d9/d13/d41/d82/dc8/f2b [3075545,59985] 0 2024-04-26T04:42:17.429 INFO:tasks.workunit.client.1.smithi104.stdout:6/653: write d0/d9/d13/d44/d71/f4b [2004436,69382] 0 2024-04-26T04:42:17.431 INFO:tasks.workunit.client.1.smithi104.stdout:6/654: creat d0/d9/d13/d44/d47/fdd x:0 0 0 2024-04-26T04:42:17.431 INFO:tasks.workunit.client.0.smithi090.stdout:7/268: creat d0/db/df/d1c/d23/f60 x:0 0 0 2024-04-26T04:42:17.435 INFO:tasks.workunit.client.1.smithi104.stdout:6/655: dread d0/d9/d13/d41/d82/dc8/f84 [0,4194304] 0 2024-04-26T04:42:17.435 INFO:tasks.workunit.client.1.smithi104.stdout:6/656: truncate d0/d9/d13/d41/d82/dc8/d80/fc2 666216 0 2024-04-26T04:42:17.437 INFO:tasks.workunit.client.1.smithi104.stdout:3/563: dwrite d5/f24 [0,4194304] 0 2024-04-26T04:42:17.439 INFO:tasks.workunit.client.0.smithi090.stdout:0/260: dwrite d0/de/f2f [0,4194304] 0 2024-04-26T04:42:17.439 INFO:tasks.workunit.client.0.smithi090.stdout:1/199: dwrite d0/d4/d7/d17/f18 [0,4194304] 0 2024-04-26T04:42:17.439 INFO:tasks.workunit.client.0.smithi090.stdout:1/200: chown d0/d4/f15 63897 1 2024-04-26T04:42:17.440 INFO:tasks.workunit.client.0.smithi090.stdout:8/185: dwrite dc/f14 [0,4194304] 0 2024-04-26T04:42:17.440 INFO:tasks.workunit.client.0.smithi090.stdout:8/186: creat dc/d1f/f43 x:0 0 0 2024-04-26T04:42:17.440 INFO:tasks.workunit.client.0.smithi090.stdout:8/187: readlink dc/d1f/l37 0 2024-04-26T04:42:17.440 INFO:tasks.workunit.client.0.smithi090.stdout:2/192: dwrite db/d11/d14/d25/f37 [0,4194304] 0 2024-04-26T04:42:17.440 INFO:tasks.workunit.client.0.smithi090.stdout:2/193: write db/d11/d14/d25/f3a [138612,89638] 0 2024-04-26T04:42:17.440 INFO:tasks.workunit.client.0.smithi090.stdout:2/194: chown db/d11/f26 0 1 2024-04-26T04:42:17.441 INFO:tasks.workunit.client.1.smithi104.stdout:3/564: dread d5/d9/dc/d32/d48/dac/f89 [0,4194304] 0 2024-04-26T04:42:17.442 INFO:tasks.workunit.client.1.smithi104.stdout:7/445: dwrite d2/d4/d32/d3d/d47/f8a [0,4194304] 0 2024-04-26T04:42:17.442 INFO:tasks.workunit.client.1.smithi104.stdout:7/446: creat d2/d4/d32/d3d/d47/f93 x:0 0 0 2024-04-26T04:42:17.442 INFO:tasks.workunit.client.1.smithi104.stdout:6/657: rename d0/d9/d13/f43 to d0/d9/d13/d44/d47/da4/da6/fde 0 2024-04-26T04:42:17.443 INFO:tasks.workunit.client.0.smithi090.stdout:0/261: symlink d0/d9/d45/l5a 0 2024-04-26T04:42:17.443 INFO:tasks.workunit.client.1.smithi104.stdout:2/672: dwrite d6/de/f40 [0,4194304] 0 2024-04-26T04:42:17.446 INFO:tasks.workunit.client.0.smithi090.stdout:1/201: symlink d0/l3e 0 2024-04-26T04:42:17.446 INFO:tasks.workunit.client.0.smithi090.stdout:1/202: fsync d0/d4/d7/d17/f18 0 2024-04-26T04:42:17.446 INFO:tasks.workunit.client.0.smithi090.stdout:9/180: dwrite db/f35 [0,4194304] 0 2024-04-26T04:42:17.446 INFO:tasks.workunit.client.0.smithi090.stdout:9/181: readlink db/d16/l23 0 2024-04-26T04:42:17.447 INFO:tasks.workunit.client.1.smithi104.stdout:7/447: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d94 0 2024-04-26T04:42:17.448 INFO:tasks.workunit.client.0.smithi090.stdout:4/282: dwrite d3/db/d12/f17 [0,4194304] 0 2024-04-26T04:42:17.448 INFO:tasks.workunit.client.0.smithi090.stdout:4/283: write d3/db/d12/d20/d24/d30/f49 [1561795,44719] 0 2024-04-26T04:42:17.449 INFO:tasks.workunit.client.1.smithi104.stdout:6/658: write d0/d9/d13/d44/d47/f4c [202159,84518] 0 2024-04-26T04:42:17.449 INFO:tasks.workunit.client.1.smithi104.stdout:6/659: write d0/d9/d13/d41/d96/fb0 [836659,6188] 0 2024-04-26T04:42:17.449 INFO:tasks.workunit.client.1.smithi104.stdout:6/660: creat d0/d9/d13/d41/d82/dc8/dba/daa/fdf x:0 0 0 2024-04-26T04:42:17.450 INFO:tasks.workunit.client.0.smithi090.stdout:8/188: creat dc/d1f/d3a/f44 x:0 0 0 2024-04-26T04:42:17.450 INFO:tasks.workunit.client.0.smithi090.stdout:8/189: write dc/d1f/f43 [644617,28188] 0 2024-04-26T04:42:17.450 INFO:tasks.workunit.client.0.smithi090.stdout:8/190: fdatasync dc/f1a 0 2024-04-26T04:42:17.451 INFO:tasks.workunit.client.1.smithi104.stdout:7/448: unlink d2/c8e 0 2024-04-26T04:42:17.451 INFO:tasks.workunit.client.1.smithi104.stdout:7/449: read d2/f75 [3955759,82510] 0 2024-04-26T04:42:17.452 INFO:tasks.workunit.client.0.smithi090.stdout:8/191: write dc/d12/f17 [722165,22786] 0 2024-04-26T04:42:17.452 INFO:tasks.workunit.client.0.smithi090.stdout:8/192: truncate dc/d1f/f24 1048129 0 2024-04-26T04:42:17.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/193: write dc/d12/f17 [762435,50] 0 2024-04-26T04:42:17.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/194: dread - dc/d1f/f33 zero size 2024-04-26T04:42:17.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/195: fdatasync dc/f1a 0 2024-04-26T04:42:17.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/196: chown dc/d1f/d3a 10302594 1 2024-04-26T04:42:17.454 INFO:tasks.workunit.client.0.smithi090.stdout:8/197: dread - dc/f1d zero size 2024-04-26T04:42:17.454 INFO:tasks.workunit.client.0.smithi090.stdout:8/198: fsync dc/d1f/f35 0 2024-04-26T04:42:17.455 INFO:tasks.workunit.client.1.smithi104.stdout:6/661: mkdir d0/dd8/de0 0 2024-04-26T04:42:17.455 INFO:tasks.workunit.client.1.smithi104.stdout:6/662: fdatasync d0/d9/d13/d44/d47/da4/da6/fde 0 2024-04-26T04:42:17.456 INFO:tasks.workunit.client.0.smithi090.stdout:2/195: creat db/d11/d14/d1b/f43 x:0 0 0 2024-04-26T04:42:17.456 INFO:tasks.workunit.client.0.smithi090.stdout:2/196: chown db/l13 2 1 2024-04-26T04:42:17.457 INFO:tasks.workunit.client.1.smithi104.stdout:7/450: truncate d2/d4/f26 1950458 0 2024-04-26T04:42:17.457 INFO:tasks.workunit.client.1.smithi104.stdout:6/663: mknod d0/ce1 0 2024-04-26T04:42:17.458 INFO:tasks.workunit.client.0.smithi090.stdout:0/262: rename d0/l15 to d0/d9/d45/d53/l5b 0 2024-04-26T04:42:17.458 INFO:tasks.workunit.client.0.smithi090.stdout:0/263: getdents d0/d9 0 2024-04-26T04:42:17.458 INFO:tasks.workunit.client.0.smithi090.stdout:0/264: write d0/f4f [347128,86263] 0 2024-04-26T04:42:17.459 INFO:tasks.workunit.client.1.smithi104.stdout:7/451: mknod d2/d4/d32/d35/d37/c95 0 2024-04-26T04:42:17.459 INFO:tasks.workunit.client.1.smithi104.stdout:7/452: chown d2/d4/d32/d35/d37/c4c 0 1 2024-04-26T04:42:17.460 INFO:tasks.workunit.client.0.smithi090.stdout:1/203: chown d0/d4/d7/d17/d22/f23 856 1 2024-04-26T04:42:17.462 INFO:tasks.workunit.client.1.smithi104.stdout:7/453: rename d2/c31 to d2/d4/d32/d3d/d41/d5f/d6c/d89/c96 0 2024-04-26T04:42:17.463 INFO:tasks.workunit.client.0.smithi090.stdout:4/284: dread d3/db/d12/ff [4194304,4194304] 0 2024-04-26T04:42:17.464 INFO:tasks.workunit.client.0.smithi090.stdout:8/199: symlink dc/d1f/d3a/l45 0 2024-04-26T04:42:17.465 INFO:tasks.workunit.client.0.smithi090.stdout:2/197: getdents db/d11/d14 0 2024-04-26T04:42:17.465 INFO:tasks.workunit.client.0.smithi090.stdout:2/198: write db/d11/d14/d1b/d20/f24 [1891786,33836] 0 2024-04-26T04:42:17.465 INFO:tasks.workunit.client.0.smithi090.stdout:2/199: creat db/d11/f44 x:0 0 0 2024-04-26T04:42:17.466 INFO:tasks.workunit.client.1.smithi104.stdout:3/565: dread d5/db2/f5c [0,4194304] 0 2024-04-26T04:42:17.466 INFO:tasks.workunit.client.1.smithi104.stdout:3/566: fsync d5/d9/dc/d32/d48/fc7 0 2024-04-26T04:42:17.468 INFO:tasks.workunit.client.0.smithi090.stdout:0/265: mkdir d0/d26/d5c 0 2024-04-26T04:42:17.471 INFO:tasks.workunit.client.0.smithi090.stdout:4/285: mkdir d3/db/d12/d20/d2d/d58 0 2024-04-26T04:42:17.473 INFO:tasks.workunit.client.1.smithi104.stdout:5/483: sync 2024-04-26T04:42:17.473 INFO:tasks.workunit.client.1.smithi104.stdout:5/484: write d0/d1/d64/d26/d36/f9f [404313,45278] 0 2024-04-26T04:42:17.473 INFO:tasks.workunit.client.1.smithi104.stdout:8/402: sync 2024-04-26T04:42:17.474 INFO:tasks.workunit.client.1.smithi104.stdout:0/427: sync 2024-04-26T04:42:17.474 INFO:tasks.workunit.client.1.smithi104.stdout:0/428: fsync d0/d2/d21/d27/f7b 0 2024-04-26T04:42:17.474 INFO:tasks.workunit.client.1.smithi104.stdout:0/429: creat d0/d4/d3c/d53/d5d/f88 x:0 0 0 2024-04-26T04:42:17.474 INFO:tasks.workunit.client.1.smithi104.stdout:0/430: readlink d0/d4/l35 0 2024-04-26T04:42:17.475 INFO:tasks.workunit.client.1.smithi104.stdout:3/567: link d5/d9/dc/d29/c2e d5/d9/dc/d29/d7d/d8e/dc2/cc8 0 2024-04-26T04:42:17.476 INFO:tasks.workunit.client.1.smithi104.stdout:5/485: rmdir d0/d7b 39 2024-04-26T04:42:17.476 INFO:tasks.workunit.client.1.smithi104.stdout:5/486: fdatasync d0/d1/d3e/f59 0 2024-04-26T04:42:17.477 INFO:tasks.workunit.client.1.smithi104.stdout:8/403: symlink d0/d1c/d27/d47/l7d 0 2024-04-26T04:42:17.477 INFO:tasks.workunit.client.1.smithi104.stdout:8/404: creat d0/d1c/f7e x:0 0 0 2024-04-26T04:42:17.479 INFO:tasks.workunit.client.1.smithi104.stdout:0/431: creat d0/d4/f89 x:0 0 0 2024-04-26T04:42:17.479 INFO:tasks.workunit.client.1.smithi104.stdout:0/432: fsync d0/d4/d3c/d53/f60 0 2024-04-26T04:42:17.479 INFO:tasks.workunit.client.1.smithi104.stdout:0/433: fdatasync d0/d2/f23 0 2024-04-26T04:42:17.479 INFO:tasks.workunit.client.1.smithi104.stdout:0/434: stat d0/d2/d21/d27/c79 0 2024-04-26T04:42:17.480 INFO:tasks.workunit.client.1.smithi104.stdout:3/568: symlink d5/db2/d46/db7/lc9 0 2024-04-26T04:42:17.481 INFO:tasks.workunit.client.1.smithi104.stdout:0/435: creat d0/d81/f8a x:0 0 0 2024-04-26T04:42:17.486 INFO:tasks.workunit.client.0.smithi090.stdout:8/200: dread dc/f1a [0,4194304] 0 2024-04-26T04:42:17.487 INFO:tasks.workunit.client.1.smithi104.stdout:0/436: symlink d0/d2/d21/d38/d7c/l8b 0 2024-04-26T04:42:17.487 INFO:tasks.workunit.client.1.smithi104.stdout:3/569: rename d5/d9/dc/d29/d7d/fa0 to d5/db2/d60/fca 0 2024-04-26T04:42:17.488 INFO:tasks.workunit.client.1.smithi104.stdout:3/570: write d5/d9/dc/d32/d48/f78 [593434,27966] 0 2024-04-26T04:42:17.488 INFO:tasks.workunit.client.0.smithi090.stdout:9/182: dwrite db/d16/f33 [0,4194304] 0 2024-04-26T04:42:17.489 INFO:tasks.workunit.client.0.smithi090.stdout:7/269: getdents d0/db/d53 0 2024-04-26T04:42:17.489 INFO:tasks.workunit.client.1.smithi104.stdout:7/454: read d2/d4/d32/f76 [8134862,107611] 0 2024-04-26T04:42:17.489 INFO:tasks.workunit.client.1.smithi104.stdout:2/673: dwrite d6/d8/d3e/f49 [0,4194304] 0 2024-04-26T04:42:17.492 INFO:tasks.workunit.client.0.smithi090.stdout:8/201: creat dc/d12/f46 x:0 0 0 2024-04-26T04:42:17.492 INFO:tasks.workunit.client.0.smithi090.stdout:8/202: write dc/d34/f40 [110369,60646] 0 2024-04-26T04:42:17.492 INFO:tasks.workunit.client.0.smithi090.stdout:1/204: dwrite d0/d4/d7/d17/f36 [0,4194304] 0 2024-04-26T04:42:17.498 INFO:tasks.workunit.client.1.smithi104.stdout:7/455: rename d2/d4/d32/d3d/d41/d5f/d6c/c73 to d2/d4/d32/d3d/d41/d5f/d6c/c97 0 2024-04-26T04:42:17.498 INFO:tasks.workunit.client.1.smithi104.stdout:6/664: dwrite d0/d9/f2a [0,4194304] 0 2024-04-26T04:42:17.498 INFO:tasks.workunit.client.1.smithi104.stdout:3/571: dread d5/d9/dc/f21 [0,4194304] 0 2024-04-26T04:42:17.499 INFO:tasks.workunit.client.1.smithi104.stdout:2/674: getdents d6 0 2024-04-26T04:42:17.499 INFO:tasks.workunit.client.1.smithi104.stdout:2/675: write d6/d8/d3e/f7a [4583104,44854] 0 2024-04-26T04:42:17.499 INFO:tasks.workunit.client.1.smithi104.stdout:2/676: stat d6/de/d1d/f5f 0 2024-04-26T04:42:17.500 INFO:tasks.workunit.client.0.smithi090.stdout:9/183: mknod db/dd/d18/d38/d39/d3f/c40 0 2024-04-26T04:42:17.504 INFO:tasks.workunit.client.1.smithi104.stdout:6/665: mknod d0/d9/d13/d41/d96/dac/d8e/dbd/ce2 0 2024-04-26T04:42:17.507 INFO:tasks.workunit.client.0.smithi090.stdout:1/205: dread d0/d4/d7/f2c [0,4194304] 0 2024-04-26T04:42:17.507 INFO:tasks.workunit.client.1.smithi104.stdout:3/572: creat d5/d9/d80/dbc/fcb x:0 0 0 2024-04-26T04:42:17.508 INFO:tasks.workunit.client.1.smithi104.stdout:6/666: link d0/d9/l4d d0/d9/d25/le3 0 2024-04-26T04:42:17.509 INFO:tasks.workunit.client.0.smithi090.stdout:1/206: truncate d0/d4/f20 376034 0 2024-04-26T04:42:17.509 INFO:tasks.workunit.client.1.smithi104.stdout:7/456: dread d2/d4/d32/f76 [4194304,4194304] 0 2024-04-26T04:42:17.509 INFO:tasks.workunit.client.1.smithi104.stdout:7/457: write d2/f42 [2307042,10018] 0 2024-04-26T04:42:17.509 INFO:tasks.workunit.client.1.smithi104.stdout:7/458: chown d2/c23 118336735 1 2024-04-26T04:42:17.510 INFO:tasks.workunit.client.1.smithi104.stdout:7/459: truncate d2/d4/d32/d35/d37/f6f 4496795 0 2024-04-26T04:42:17.510 INFO:tasks.workunit.client.1.smithi104.stdout:6/667: creat d0/d9/d13/d41/d96/dac/db3/fe4 x:0 0 0 2024-04-26T04:42:17.512 INFO:tasks.workunit.client.1.smithi104.stdout:7/460: creat d2/d4/d32/d3d/d41/d5f/d6c/f98 x:0 0 0 2024-04-26T04:42:17.512 INFO:tasks.workunit.client.1.smithi104.stdout:7/461: chown d2/d4/d32/d3d/d41/d7e/c85 0 1 2024-04-26T04:42:17.512 INFO:tasks.workunit.client.1.smithi104.stdout:7/462: fdatasync d2/d4/d32/d35/d37/f3a 0 2024-04-26T04:42:17.515 INFO:tasks.workunit.client.1.smithi104.stdout:7/463: rename d2/d4/f10 to d2/d4/d5a/f99 0 2024-04-26T04:42:17.521 INFO:tasks.workunit.client.1.smithi104.stdout:7/464: dread d2/d4/d32/d3d/d41/d5f/d6d/f8b [4194304,4194304] 0 2024-04-26T04:42:17.521 INFO:tasks.workunit.client.1.smithi104.stdout:7/465: stat d2/d4/f33 0 2024-04-26T04:42:17.524 INFO:tasks.workunit.client.1.smithi104.stdout:6/668: dread d0/d9/d13/d41/d82/dc8/f60 [0,4194304] 0 2024-04-26T04:42:17.526 INFO:tasks.workunit.client.1.smithi104.stdout:3/573: dread d5/d9/dc/d69/f3e [0,4194304] 0 2024-04-26T04:42:17.533 INFO:tasks.workunit.client.1.smithi104.stdout:2/677: dread d6/f10 [0,4194304] 0 2024-04-26T04:42:17.535 INFO:tasks.workunit.client.1.smithi104.stdout:2/678: creat d6/d1b/d51/fe6 x:0 0 0 2024-04-26T04:42:17.535 INFO:tasks.workunit.client.1.smithi104.stdout:2/679: write d6/d8/d3e/f2a [593625,124358] 0 2024-04-26T04:42:17.536 INFO:tasks.workunit.client.1.smithi104.stdout:2/680: mkdir d6/de/d1d/dcf/de7 0 2024-04-26T04:42:17.538 INFO:tasks.workunit.client.1.smithi104.stdout:2/681: rename d6/ce1 to d6/de/d85/ce8 0 2024-04-26T04:42:17.538 INFO:tasks.workunit.client.1.smithi104.stdout:2/682: chown d6/de/d85/ce8 505361 1 2024-04-26T04:42:17.539 INFO:tasks.workunit.client.1.smithi104.stdout:2/683: symlink d6/d1b/d68/le9 0 2024-04-26T04:42:17.539 INFO:tasks.workunit.client.1.smithi104.stdout:2/684: fdatasync d6/d1b/d51/d6a/fc5 0 2024-04-26T04:42:17.540 INFO:tasks.workunit.client.1.smithi104.stdout:2/685: creat d6/d8/fea x:0 0 0 2024-04-26T04:42:17.540 INFO:tasks.workunit.client.1.smithi104.stdout:2/686: readlink d6/de/lc3 0 2024-04-26T04:42:17.550 INFO:tasks.workunit.client.1.smithi104.stdout:8/405: dwrite d0/f2f [0,4194304] 0 2024-04-26T04:42:17.551 INFO:tasks.workunit.client.1.smithi104.stdout:8/406: mkdir d0/d1c/d27/d47/d67/d76/d7f 0 2024-04-26T04:42:17.553 INFO:tasks.workunit.client.1.smithi104.stdout:2/687: dread d6/de/d8e/f69 [0,4194304] 0 2024-04-26T04:42:17.554 INFO:tasks.workunit.client.1.smithi104.stdout:2/688: fdatasync d6/d1b/d75/fc9 0 2024-04-26T04:42:17.554 INFO:tasks.workunit.client.1.smithi104.stdout:2/689: write d6/d8/d5a/fb0 [1359640,42588] 0 2024-04-26T04:42:17.554 INFO:tasks.workunit.client.1.smithi104.stdout:2/690: creat d6/d1b/d68/feb x:0 0 0 2024-04-26T04:42:17.555 INFO:tasks.workunit.client.1.smithi104.stdout:2/691: mkdir d6/d1b/d51/dec 0 2024-04-26T04:42:17.556 INFO:tasks.workunit.client.1.smithi104.stdout:2/692: symlink d6/led 0 2024-04-26T04:42:17.564 INFO:tasks.workunit.client.0.smithi090.stdout:2/200: dwrite db/dc/f19 [0,4194304] 0 2024-04-26T04:42:17.565 INFO:tasks.workunit.client.0.smithi090.stdout:2/201: write db/dc/f12 [957310,101798] 0 2024-04-26T04:42:17.565 INFO:tasks.workunit.client.1.smithi104.stdout:5/487: dwrite d0/f6 [0,4194304] 0 2024-04-26T04:42:17.565 INFO:tasks.workunit.client.0.smithi090.stdout:4/286: dwrite d3/db/d12/d20/d2d/d2e/f35 [0,4194304] 0 2024-04-26T04:42:17.569 INFO:tasks.workunit.client.1.smithi104.stdout:8/407: getdents d0/d1c/d27/d47 0 2024-04-26T04:42:17.573 INFO:tasks.workunit.client.0.smithi090.stdout:1/207: dwrite d0/d4/d7/d17/d22/d31/f39 [0,4194304] 0 2024-04-26T04:42:17.574 INFO:tasks.workunit.client.1.smithi104.stdout:2/693: dwrite d6/d8/d5a/f74 [0,4194304] 0 2024-04-26T04:42:17.575 INFO:tasks.workunit.client.0.smithi090.stdout:7/270: dwrite d0/db/d15/d30/f40 [0,4194304] 0 2024-04-26T04:42:17.576 INFO:tasks.workunit.client.0.smithi090.stdout:8/203: dwrite dc/d1f/f21 [0,4194304] 0 2024-04-26T04:42:17.577 INFO:tasks.workunit.client.1.smithi104.stdout:5/488: write d0/d1/d64/f31 [612357,45774] 0 2024-04-26T04:42:17.577 INFO:tasks.workunit.client.1.smithi104.stdout:0/437: dwrite d0/d2/d21/d38/f6b [0,4194304] 0 2024-04-26T04:42:17.577 INFO:tasks.workunit.client.1.smithi104.stdout:0/438: stat d0/d2/d21 0 2024-04-26T04:42:17.577 INFO:tasks.workunit.client.0.smithi090.stdout:1/208: link d0/c19 d0/d4/d7/c3f 0 2024-04-26T04:42:17.580 INFO:tasks.workunit.client.0.smithi090.stdout:2/202: dread db/dc/f3c [0,4194304] 0 2024-04-26T04:42:17.580 INFO:tasks.workunit.client.1.smithi104.stdout:1/545: sync 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:4/593: sync 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:4/594: truncate d4/f71 529582 0 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:3/574: dwrite d5/db2/f73 [0,4194304] 0 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:3/575: write d5/db2/d60/fca [316431,13754] 0 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:3/576: creat d5/d9/dc/d29/fcc x:0 0 0 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:3/577: write d5/d9/dc/f40 [1202748,102080] 0 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:3/578: stat d5/d9/dc/d29/fcc 0 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:3/579: fdatasync d5/f24 0 2024-04-26T04:42:17.581 INFO:tasks.workunit.client.1.smithi104.stdout:3/580: fdatasync d5/d9/fe 0 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/581: truncate d5/f23 4645340 0 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/582: write d5/d9/d80/f9e [12028,74840] 0 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/583: dread - d5/d9/dc/d29/fbd zero size 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/584: dread - d5/db2/da3/fc3 zero size 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/585: write d5/d9/dc/d32/f77 [738415,1498] 0 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/586: chown d5/d9/dc/d29/l71 133 1 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:3/587: chown d5/db2/d60/c97 911912 1 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:9/402: sync 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:7/466: dwrite d2/d4/d32/d60/f63 [0,4194304] 0 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:7/467: readlink d2/d4/d32/d3d/d41/l4b 0 2024-04-26T04:42:17.582 INFO:tasks.workunit.client.1.smithi104.stdout:6/669: rmdir d0/d9/d13/d41/d96 39 2024-04-26T04:42:17.583 INFO:tasks.workunit.client.0.smithi090.stdout:7/271: unlink d0/db/d15/d30/f40 0 2024-04-26T04:42:17.583 INFO:tasks.workunit.client.1.smithi104.stdout:2/694: mknod d6/d1b/d51/d6a/cee 0 2024-04-26T04:42:17.583 INFO:tasks.workunit.client.1.smithi104.stdout:2/695: write d6/d8/d5a/fb0 [1678825,86169] 0 2024-04-26T04:42:17.584 INFO:tasks.workunit.client.1.smithi104.stdout:5/489: dread d0/f58 [0,4194304] 0 2024-04-26T04:42:17.588 INFO:tasks.workunit.client.1.smithi104.stdout:1/546: rename d0/d6/d19/f75 to d0/d6/d2e/fb3 0 2024-04-26T04:42:17.588 INFO:tasks.workunit.client.1.smithi104.stdout:1/547: getdents d0/d6/d9e 0 2024-04-26T04:42:17.589 INFO:tasks.workunit.client.0.smithi090.stdout:4/287: dread d3/db/d12/f17 [0,4194304] 0 2024-04-26T04:42:17.589 INFO:tasks.workunit.client.0.smithi090.stdout:4/288: chown d3/db/d12/d20/d24/d30/c42 7666 1 2024-04-26T04:42:17.589 INFO:tasks.workunit.client.0.smithi090.stdout:4/289: fdatasync d3/db/f19 0 2024-04-26T04:42:17.592 INFO:tasks.workunit.client.0.smithi090.stdout:0/266: rename d0/d26 to d0/d4b/d5d 0 2024-04-26T04:42:17.592 INFO:tasks.workunit.client.0.smithi090.stdout:0/267: readlink d0/l3a 0 2024-04-26T04:42:17.592 INFO:tasks.workunit.client.0.smithi090.stdout:1/209: write d0/d4/d7/d17/f36 [4516754,88026] 0 2024-04-26T04:42:17.593 INFO:tasks.workunit.client.1.smithi104.stdout:8/408: dwrite d0/d1c/f24 [0,4194304] 0 2024-04-26T04:42:17.593 INFO:tasks.workunit.client.1.smithi104.stdout:8/409: truncate d0/d1c/d27/d46/f78 285748 0 2024-04-26T04:42:17.593 INFO:tasks.workunit.client.1.smithi104.stdout:8/410: write d0/f5 [4262426,99497] 0 2024-04-26T04:42:17.594 INFO:tasks.workunit.client.0.smithi090.stdout:8/204: mknod dc/d12/c47 0 2024-04-26T04:42:17.594 INFO:tasks.workunit.client.0.smithi090.stdout:8/205: dread - dc/f1d zero size 2024-04-26T04:42:17.594 INFO:tasks.workunit.client.0.smithi090.stdout:7/272: write d0/db/d15/d29/f44 [1477722,89546] 0 2024-04-26T04:42:17.595 INFO:tasks.workunit.client.1.smithi104.stdout:3/588: mknod d5/d9/dc/d29/ccd 0 2024-04-26T04:42:17.596 INFO:tasks.workunit.client.1.smithi104.stdout:9/403: getdents d11/d53 0 2024-04-26T04:42:17.596 INFO:tasks.workunit.client.1.smithi104.stdout:9/404: dread - d11/d1d/f31 zero size 2024-04-26T04:42:17.597 INFO:tasks.workunit.client.0.smithi090.stdout:0/268: creat d0/d4b/d5d/d5c/f5e x:0 0 0 2024-04-26T04:42:17.598 INFO:tasks.workunit.client.0.smithi090.stdout:0/269: write d0/df/f3c [579862,95784] 0 2024-04-26T04:42:17.599 INFO:tasks.workunit.client.1.smithi104.stdout:7/468: symlink d2/d4/d32/d35/d44/d5b/l9a 0 2024-04-26T04:42:17.601 INFO:tasks.workunit.client.0.smithi090.stdout:2/203: rmdir db/d11/d14/d1b/d39 0 2024-04-26T04:42:17.601 INFO:tasks.workunit.client.0.smithi090.stdout:2/204: creat db/d11/d14/d27/d28/f45 x:0 0 0 2024-04-26T04:42:17.602 INFO:tasks.workunit.client.1.smithi104.stdout:1/548: mkdir d0/d6/d19/d21/d58/db4 0 2024-04-26T04:42:17.602 INFO:tasks.workunit.client.1.smithi104.stdout:1/549: write d0/d6/d2e/f84 [1442693,15626] 0 2024-04-26T04:42:17.602 INFO:tasks.workunit.client.1.smithi104.stdout:6/670: link d0/d9/f93 d0/d9/d13/d41/d96/dac/db3/fe5 0 2024-04-26T04:42:17.602 INFO:tasks.workunit.client.1.smithi104.stdout:6/671: chown d0/ce1 30852 1 2024-04-26T04:42:17.603 INFO:tasks.workunit.client.1.smithi104.stdout:4/595: dread d4/d7/d1d/d8e/fc3 [0,4194304] 0 2024-04-26T04:42:17.603 INFO:tasks.workunit.client.1.smithi104.stdout:4/596: chown d4/d7/d1d/d2e/l79 30904 1 2024-04-26T04:42:17.604 INFO:tasks.workunit.client.0.smithi090.stdout:7/273: creat d0/d1/f61 x:0 0 0 2024-04-26T04:42:17.606 INFO:tasks.workunit.client.1.smithi104.stdout:8/411: creat d0/d1c/d27/d47/d67/f80 x:0 0 0 2024-04-26T04:42:17.606 INFO:tasks.workunit.client.1.smithi104.stdout:8/412: stat d0/d1c/d27/d47/f38 0 2024-04-26T04:42:17.608 INFO:tasks.workunit.client.1.smithi104.stdout:9/405: rename d11/d1d/d34/f73 to d11/d1d/d34/d69/f79 0 2024-04-26T04:42:17.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/210: rmdir d0/d4/d7/d17 39 2024-04-26T04:42:17.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/211: chown d0/f2e 0 1 2024-04-26T04:42:17.609 INFO:tasks.workunit.client.1.smithi104.stdout:4/597: truncate d4/d7/d1d/d1e/d24/d48/f5c 2308352 0 2024-04-26T04:42:17.609 INFO:tasks.workunit.client.0.smithi090.stdout:0/270: symlink d0/d4b/d5d/l5f 0 2024-04-26T04:42:17.611 INFO:tasks.workunit.client.1.smithi104.stdout:2/696: getdents d6 0 2024-04-26T04:42:17.612 INFO:tasks.workunit.client.1.smithi104.stdout:9/406: rmdir d11/d15/d32/d35 39 2024-04-26T04:42:17.612 INFO:tasks.workunit.client.1.smithi104.stdout:9/407: creat d11/d15/d32/d4c/d54/f7a x:0 0 0 2024-04-26T04:42:17.615 INFO:tasks.workunit.client.0.smithi090.stdout:2/205: rename db/d11/d14/d27/d28/l2b to db/d11/l46 0 2024-04-26T04:42:17.615 INFO:tasks.workunit.client.0.smithi090.stdout:2/206: creat db/d11/d14/d27/d28/f47 x:0 0 0 2024-04-26T04:42:17.615 INFO:tasks.workunit.client.0.smithi090.stdout:2/207: read db/dc/f1e [664251,120685] 0 2024-04-26T04:42:17.615 INFO:tasks.workunit.client.0.smithi090.stdout:2/208: readlink db/dc/l1a 0 2024-04-26T04:42:17.615 INFO:tasks.workunit.client.0.smithi090.stdout:7/274: symlink d0/l62 0 2024-04-26T04:42:17.616 INFO:tasks.workunit.client.1.smithi104.stdout:1/550: rename d0/c13 to d0/d6/d9e/cb5 0 2024-04-26T04:42:17.616 INFO:tasks.workunit.client.1.smithi104.stdout:8/413: dread d0/f49 [0,4194304] 0 2024-04-26T04:42:17.616 INFO:tasks.workunit.client.1.smithi104.stdout:2/697: symlink d6/d1b/d68/d81/d97/lef 0 2024-04-26T04:42:17.616 INFO:tasks.workunit.client.1.smithi104.stdout:0/439: dwrite d0/d2/d21/d49/f55 [0,4194304] 0 2024-04-26T04:42:17.616 INFO:tasks.workunit.client.1.smithi104.stdout:0/440: fdatasync d0/d4/d3c/f86 0 2024-04-26T04:42:17.617 INFO:tasks.workunit.client.0.smithi090.stdout:1/212: creat d0/d4/d7/d17/d22/d31/f40 x:0 0 0 2024-04-26T04:42:17.617 INFO:tasks.workunit.client.0.smithi090.stdout:1/213: chown d0/d4/d7/d17/d22/f23 7290 1 2024-04-26T04:42:17.618 INFO:tasks.workunit.client.1.smithi104.stdout:0/441: dread d0/d4/d3b/f77 [0,4194304] 0 2024-04-26T04:42:17.618 INFO:tasks.workunit.client.1.smithi104.stdout:9/408: mkdir d11/d15/d32/d35/d7b 0 2024-04-26T04:42:17.618 INFO:tasks.workunit.client.1.smithi104.stdout:9/409: readlink d11/d15/d32/l3f 0 2024-04-26T04:42:17.620 INFO:tasks.workunit.client.1.smithi104.stdout:6/672: rename d0/d9/d13/d41/f73 to d0/d9/d13/d44/d71/d3f/fe6 0 2024-04-26T04:42:17.623 INFO:tasks.workunit.client.0.smithi090.stdout:1/214: dread d0/d4/f15 [0,4194304] 0 2024-04-26T04:42:17.623 INFO:tasks.workunit.client.0.smithi090.stdout:0/271: read d0/d9/f3e [6674840,102153] 0 2024-04-26T04:42:17.623 INFO:tasks.workunit.client.1.smithi104.stdout:1/551: creat d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/fb6 x:0 0 0 2024-04-26T04:42:17.624 INFO:tasks.workunit.client.0.smithi090.stdout:1/215: symlink d0/d4/l41 0 2024-04-26T04:42:17.624 INFO:tasks.workunit.client.0.smithi090.stdout:1/216: truncate d0/d4/d7/f2c 4696068 0 2024-04-26T04:42:17.624 INFO:tasks.workunit.client.0.smithi090.stdout:1/217: chown d0/l5 135039 1 2024-04-26T04:42:17.624 INFO:tasks.workunit.client.0.smithi090.stdout:0/272: mknod d0/d9/d45/c60 0 2024-04-26T04:42:17.625 INFO:tasks.workunit.client.1.smithi104.stdout:8/414: unlink d0/d33/f28 0 2024-04-26T04:42:17.627 INFO:tasks.workunit.client.1.smithi104.stdout:2/698: unlink d6/d1b/d51/lca 0 2024-04-26T04:42:17.627 INFO:tasks.workunit.client.1.smithi104.stdout:2/699: read - d6/d8/d5a/fb3 zero size 2024-04-26T04:42:17.627 INFO:tasks.workunit.client.1.smithi104.stdout:2/700: creat d6/d1b/d51/d6a/ff0 x:0 0 0 2024-04-26T04:42:17.630 INFO:tasks.workunit.client.1.smithi104.stdout:9/410: mkdir d11/d15/d32/d4c/d7c 0 2024-04-26T04:42:17.630 INFO:tasks.workunit.client.1.smithi104.stdout:9/411: chown d11/f20 58 1 2024-04-26T04:42:17.630 INFO:tasks.workunit.client.1.smithi104.stdout:9/412: chown d11/d15/l21 267369739 1 2024-04-26T04:42:17.631 INFO:tasks.workunit.client.0.smithi090.stdout:2/209: write db/d11/d14/d27/d2f/f32 [457240,8308] 0 2024-04-26T04:42:17.632 INFO:tasks.workunit.client.1.smithi104.stdout:5/490: dwrite d0/f22 [0,4194304] 0 2024-04-26T04:42:17.632 INFO:tasks.workunit.client.1.smithi104.stdout:5/491: chown d0/d7b/c9e 3797 1 2024-04-26T04:42:17.632 INFO:tasks.workunit.client.1.smithi104.stdout:5/492: write d0/d1/d4/d2a/d2b/d37/f9c [1176524,108824] 0 2024-04-26T04:42:17.632 INFO:tasks.workunit.client.1.smithi104.stdout:8/415: rename d0/d1c/d27/d47/d67/f80 to d0/d1c/d2d/f81 0 2024-04-26T04:42:17.633 INFO:tasks.workunit.client.0.smithi090.stdout:2/210: dread db/d11/f17 [0,4194304] 0 2024-04-26T04:42:17.633 INFO:tasks.workunit.client.0.smithi090.stdout:2/211: write db/d11/d14/d27/d28/f3d [881922,74250] 0 2024-04-26T04:42:17.633 INFO:tasks.workunit.client.0.smithi090.stdout:2/212: readlink db/d11/d14/l18 0 2024-04-26T04:42:17.633 INFO:tasks.workunit.client.0.smithi090.stdout:2/213: truncate db/d11/d14/d27/d28/f47 489992 0 2024-04-26T04:42:17.633 INFO:tasks.workunit.client.0.smithi090.stdout:2/214: chown db/d11/d14/d1b/l35 217827284 1 2024-04-26T04:42:17.633 INFO:tasks.workunit.client.0.smithi090.stdout:2/215: stat db/d11/d14/d1b/f43 0 2024-04-26T04:42:17.634 INFO:tasks.workunit.client.0.smithi090.stdout:2/216: mkdir db/d11/d14/d25/d2d/d48 0 2024-04-26T04:42:17.634 INFO:tasks.workunit.client.0.smithi090.stdout:2/217: creat db/d11/d14/d25/f49 x:0 0 0 2024-04-26T04:42:17.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/413: symlink d11/d1d/d34/d69/l7d 0 2024-04-26T04:42:17.636 INFO:tasks.workunit.client.0.smithi090.stdout:2/218: write db/d11/f26 [179200,49906] 0 2024-04-26T04:42:17.636 INFO:tasks.workunit.client.0.smithi090.stdout:2/219: write db/f10 [4967406,52464] 0 2024-04-26T04:42:17.638 INFO:tasks.workunit.client.1.smithi104.stdout:6/673: write d0/d9/f2a [50551,71533] 0 2024-04-26T04:42:17.639 INFO:tasks.workunit.client.1.smithi104.stdout:5/493: mknod d0/d1/d4/d2a/d2b/d37/d48/ca6 0 2024-04-26T04:42:17.640 INFO:tasks.workunit.client.1.smithi104.stdout:9/414: symlink d11/d15/d1b/d6d/l7e 0 2024-04-26T04:42:17.640 INFO:tasks.workunit.client.1.smithi104.stdout:9/415: write d11/d15/f30 [270606,128864] 0 2024-04-26T04:42:17.641 INFO:tasks.workunit.client.1.smithi104.stdout:6/674: creat d0/d9/d13/d44/d47/fe7 x:0 0 0 2024-04-26T04:42:17.641 INFO:tasks.workunit.client.0.smithi090.stdout:8/206: dwrite dc/f1e [0,4194304] 0 2024-04-26T04:42:17.642 INFO:tasks.workunit.client.1.smithi104.stdout:5/494: write d0/d1/d1e/f21 [217092,95192] 0 2024-04-26T04:42:17.644 INFO:tasks.workunit.client.1.smithi104.stdout:6/675: creat d0/d9/d13/d44/d47/da4/da6/fe8 x:0 0 0 2024-04-26T04:42:17.646 INFO:tasks.workunit.client.1.smithi104.stdout:2/701: read d6/de/d1d/f83 [2878366,88532] 0 2024-04-26T04:42:17.647 INFO:tasks.workunit.client.1.smithi104.stdout:2/702: dread - d6/d1b/d68/feb zero size 2024-04-26T04:42:17.647 INFO:tasks.workunit.client.1.smithi104.stdout:2/703: fsync d6/f33 0 2024-04-26T04:42:17.647 INFO:tasks.workunit.client.1.smithi104.stdout:5/495: mkdir d0/d1/d64/d26/d36/da7 0 2024-04-26T04:42:17.647 INFO:tasks.workunit.client.1.smithi104.stdout:5/496: truncate d0/d41/f74 122466 0 2024-04-26T04:42:17.647 INFO:tasks.workunit.client.1.smithi104.stdout:5/497: chown d0/f1d 2284 1 2024-04-26T04:42:17.647 INFO:tasks.workunit.client.1.smithi104.stdout:2/704: write d6/de/d1d/f6d [1765276,120555] 0 2024-04-26T04:42:17.647 INFO:tasks.workunit.client.1.smithi104.stdout:9/416: symlink d11/d15/l7f 0 2024-04-26T04:42:17.652 INFO:tasks.workunit.client.1.smithi104.stdout:2/705: dread d6/de/d1d/f83 [0,4194304] 0 2024-04-26T04:42:17.652 INFO:tasks.workunit.client.1.smithi104.stdout:2/706: dread - d6/d1b/d68/d81/d97/d99/fa8 zero size 2024-04-26T04:42:17.653 INFO:tasks.workunit.client.0.smithi090.stdout:4/290: dwrite d3/db/d12/d20/d2d/d2e/f35 [0,4194304] 0 2024-04-26T04:42:17.654 INFO:tasks.workunit.client.1.smithi104.stdout:3/589: dwrite d5/f3f [0,4194304] 0 2024-04-26T04:42:17.654 INFO:tasks.workunit.client.1.smithi104.stdout:0/442: dwrite d0/d4/d3c/f86 [0,4194304] 0 2024-04-26T04:42:17.654 INFO:tasks.workunit.client.1.smithi104.stdout:3/590: fsync d5/db2/f1b 0 2024-04-26T04:42:17.654 INFO:tasks.workunit.client.0.smithi090.stdout:8/207: write dc/d1f/f27 [3987703,80478] 0 2024-04-26T04:42:17.656 INFO:tasks.workunit.client.1.smithi104.stdout:7/469: dwrite d2/d4/d32/d60/f82 [0,4194304] 0 2024-04-26T04:42:17.657 INFO:tasks.workunit.client.1.smithi104.stdout:6/676: dread d0/d9/f4a [0,4194304] 0 2024-04-26T04:42:17.657 INFO:tasks.workunit.client.1.smithi104.stdout:4/598: dwrite d4/d7/d1d/d1e/d15/f1a [4194304,4194304] 0 2024-04-26T04:42:17.657 INFO:tasks.workunit.client.1.smithi104.stdout:4/599: stat d4/d7/d1d/d2e 0 2024-04-26T04:42:17.657 INFO:tasks.workunit.client.1.smithi104.stdout:6/677: write d0/d9/f49 [679855,51619] 0 2024-04-26T04:42:17.659 INFO:tasks.workunit.client.1.smithi104.stdout:0/443: creat d0/d4/d3c/d53/d5d/d6f/f8c x:0 0 0 2024-04-26T04:42:17.660 INFO:tasks.workunit.client.0.smithi090.stdout:3/165: sync 2024-04-26T04:42:17.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/292: sync 2024-04-26T04:42:17.660 INFO:tasks.workunit.client.0.smithi090.stdout:6/303: sync 2024-04-26T04:42:17.660 INFO:tasks.workunit.client.0.smithi090.stdout:6/304: readlink d9/d1c/d4d/d6d/l67 0 2024-04-26T04:42:17.660 INFO:tasks.workunit.client.0.smithi090.stdout:6/305: creat d9/d21/d40/f74 x:0 0 0 2024-04-26T04:42:17.661 INFO:tasks.workunit.client.0.smithi090.stdout:2/220: fsync db/dc/f12 0 2024-04-26T04:42:17.661 INFO:tasks.workunit.client.0.smithi090.stdout:2/221: readlink l5 0 2024-04-26T04:42:17.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/275: dread d0/db/d15/d29/f44 [0,4194304] 0 2024-04-26T04:42:17.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/276: creat d0/db/df/d4d/f63 x:0 0 0 2024-04-26T04:42:17.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/277: fsync d0/db/d15/d29/f18 0 2024-04-26T04:42:17.662 INFO:tasks.workunit.client.0.smithi090.stdout:7/278: truncate d0/db/d53/f17 163940 0 2024-04-26T04:42:17.662 INFO:tasks.workunit.client.1.smithi104.stdout:1/552: dread d0/d6/d2e/f84 [0,4194304] 0 2024-04-26T04:42:17.663 INFO:tasks.workunit.client.1.smithi104.stdout:3/591: symlink d5/d9/dc/d32/d47/lce 0 2024-04-26T04:42:17.666 INFO:tasks.workunit.client.1.smithi104.stdout:4/600: getdents d4/db/d6b/d86 0 2024-04-26T04:42:17.668 INFO:tasks.workunit.client.0.smithi090.stdout:6/306: dread d9/d21/d3b/f46 [0,4194304] 0 2024-04-26T04:42:17.669 INFO:tasks.workunit.client.1.smithi104.stdout:6/678: rename d0/d9/d13/d41/d82/dc8/d80/faf to d0/dd8/fe9 0 2024-04-26T04:42:17.669 INFO:tasks.workunit.client.0.smithi090.stdout:2/222: creat db/d11/d14/f4a x:0 0 0 2024-04-26T04:42:17.670 INFO:tasks.workunit.client.1.smithi104.stdout:1/553: symlink d0/d6/d19/d21/d66/d30/d81/lb7 0 2024-04-26T04:42:17.673 INFO:tasks.workunit.client.1.smithi104.stdout:4/601: symlink d4/ld5 0 2024-04-26T04:42:17.675 INFO:tasks.workunit.client.0.smithi090.stdout:5/293: rmdir d4/d1a/d1f 39 2024-04-26T04:42:17.676 INFO:tasks.workunit.client.1.smithi104.stdout:1/554: mkdir d0/d6/d19/d1b/da7/db8 0 2024-04-26T04:42:17.676 INFO:tasks.workunit.client.1.smithi104.stdout:1/555: read d0/d6/d19/d21/d66/f41 [1104943,118077] 0 2024-04-26T04:42:17.677 INFO:tasks.workunit.client.1.smithi104.stdout:1/556: symlink d0/d6/d19/d21/d58/lb9 0 2024-04-26T04:42:17.677 INFO:tasks.workunit.client.1.smithi104.stdout:1/557: dread - d0/f80 zero size 2024-04-26T04:42:17.678 INFO:tasks.workunit.client.1.smithi104.stdout:1/558: symlink d0/d6/lba 0 2024-04-26T04:42:17.678 INFO:tasks.workunit.client.1.smithi104.stdout:1/559: write d0/d6/d2e/f84 [1572050,17037] 0 2024-04-26T04:42:17.678 INFO:tasks.workunit.client.0.smithi090.stdout:6/307: creat d9/d1c/d4d/d6d/d66/f75 x:0 0 0 2024-04-26T04:42:17.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/592: dread d5/d9/dc/d69/fbf [0,4194304] 0 2024-04-26T04:42:17.680 INFO:tasks.workunit.client.1.smithi104.stdout:1/560: link d0/d6/lba d0/d6/d19/d21/d59/lbb 0 2024-04-26T04:42:17.681 INFO:tasks.workunit.client.1.smithi104.stdout:1/561: fdatasync d0/d6/d19/d21/d66/d20/d2a/d56/d8c/f99 0 2024-04-26T04:42:17.681 INFO:tasks.workunit.client.0.smithi090.stdout:2/223: mkdir db/d11/d14/d27/d4b 0 2024-04-26T04:42:17.683 INFO:tasks.workunit.client.0.smithi090.stdout:4/291: dread d3/db/d12/d20/d24/f52 [0,4194304] 0 2024-04-26T04:42:17.683 INFO:tasks.workunit.client.0.smithi090.stdout:6/308: rmdir d9/d1c/d4d/d6d 39 2024-04-26T04:42:17.684 INFO:tasks.workunit.client.1.smithi104.stdout:3/593: rename d5/d9/dc/d69/c30 to d5/d9/dc/d29/d49/d59/ccf 0 2024-04-26T04:42:17.685 INFO:tasks.workunit.client.0.smithi090.stdout:3/166: rename d1/d6/d1b/d1e to d1/d17/d36 0 2024-04-26T04:42:17.685 INFO:tasks.workunit.client.0.smithi090.stdout:3/167: rename d1/d6/d1b to d1/d6/d1b/d37 22 2024-04-26T04:42:17.685 INFO:tasks.workunit.client.0.smithi090.stdout:3/168: creat d1/d9/f38 x:0 0 0 2024-04-26T04:42:17.686 INFO:tasks.workunit.client.0.smithi090.stdout:6/309: creat d9/d21/d3b/f76 x:0 0 0 2024-04-26T04:42:17.687 INFO:tasks.workunit.client.1.smithi104.stdout:3/594: mkdir d5/db2/da3/dd0 0 2024-04-26T04:42:17.688 INFO:tasks.workunit.client.0.smithi090.stdout:1/218: truncate d0/d4/d7/f2c 1071032 0 2024-04-26T04:42:17.689 INFO:tasks.workunit.client.0.smithi090.stdout:4/292: dread d3/db/d12/d20/d24/f52 [0,4194304] 0 2024-04-26T04:42:17.689 INFO:tasks.workunit.client.0.smithi090.stdout:4/293: chown d3/d13/d1d/f38 895 1 2024-04-26T04:42:17.690 INFO:tasks.workunit.client.0.smithi090.stdout:4/294: dread d3/db/d12/d20/f2a [0,4194304] 0 2024-04-26T04:42:17.690 INFO:tasks.workunit.client.0.smithi090.stdout:1/219: rename d0/l1b to d0/d4/d7/d17/l42 0 2024-04-26T04:42:17.691 INFO:tasks.workunit.client.0.smithi090.stdout:6/310: mkdir d9/d77 0 2024-04-26T04:42:17.691 INFO:tasks.workunit.client.0.smithi090.stdout:6/311: readlink d9/d21/d40/d4c/d6f/l73 0 2024-04-26T04:42:17.692 INFO:tasks.workunit.client.0.smithi090.stdout:4/295: symlink d3/d13/d1d/d50/d51/l59 0 2024-04-26T04:42:17.692 INFO:tasks.workunit.client.0.smithi090.stdout:5/294: dread d4/d1a/d1f/d53/d59/d5e/f5d [0,4194304] 0 2024-04-26T04:42:17.692 INFO:tasks.workunit.client.0.smithi090.stdout:5/295: readlink d4/d1a/d1f/l24 0 2024-04-26T04:42:17.692 INFO:tasks.workunit.client.0.smithi090.stdout:5/296: rename d4/d1a/d1f/d53/d59 to d4/d1a/d1f/d53/d59/d5e/d60 22 2024-04-26T04:42:17.692 INFO:tasks.workunit.client.0.smithi090.stdout:5/297: read d4/fe [1607149,52599] 0 2024-04-26T04:42:17.692 INFO:tasks.workunit.client.0.smithi090.stdout:6/312: mknod d9/d77/c78 0 2024-04-26T04:42:17.695 INFO:tasks.workunit.client.0.smithi090.stdout:5/298: mkdir d4/d1a/df/d36/d61 0 2024-04-26T04:42:17.695 INFO:tasks.workunit.client.0.smithi090.stdout:6/313: mkdir d9/d1c/d31/d79 0 2024-04-26T04:42:17.695 INFO:tasks.workunit.client.1.smithi104.stdout:3/595: mkdir d5/d9/dd1 0 2024-04-26T04:42:17.696 INFO:tasks.workunit.client.1.smithi104.stdout:3/596: chown d5/d9/dc/c65 3 1 2024-04-26T04:42:17.705 INFO:tasks.workunit.client.1.smithi104.stdout:2/707: dread d6/de/d1d/f6d [0,4194304] 0 2024-04-26T04:42:17.705 INFO:tasks.workunit.client.1.smithi104.stdout:2/708: chown d6/d8/lae 4033646 1 2024-04-26T04:42:17.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/169: dread d1/d17/d36/f24 [0,4194304] 0 2024-04-26T04:42:17.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/170: rename d1 to d1/d6/d1b/d39 22 2024-04-26T04:42:17.707 INFO:tasks.workunit.client.1.smithi104.stdout:5/498: dwrite d0/f58 [0,4194304] 0 2024-04-26T04:42:17.707 INFO:tasks.workunit.client.1.smithi104.stdout:5/499: write d0/d1/d4/f8c [4891074,44825] 0 2024-04-26T04:42:17.708 INFO:tasks.workunit.client.1.smithi104.stdout:2/709: rename d6/d1b/d68/d81/lbf to d6/d1b/d68/d81/dce/lf1 0 2024-04-26T04:42:17.709 INFO:tasks.workunit.client.1.smithi104.stdout:2/710: chown d6/de/d1d/dcf/de7 382 1 2024-04-26T04:42:17.709 INFO:tasks.workunit.client.1.smithi104.stdout:2/711: readlink d6/de/d1d/la9 0 2024-04-26T04:42:17.712 INFO:tasks.workunit.client.1.smithi104.stdout:5/500: rename d0/d1/d4/f17 to d0/d1/d64/d26/d36/da7/fa8 0 2024-04-26T04:42:17.717 INFO:tasks.workunit.client.1.smithi104.stdout:0/444: dwrite d0/d2/d21/d38/f6b [0,4194304] 0 2024-04-26T04:42:17.718 INFO:tasks.workunit.client.1.smithi104.stdout:9/417: dwrite d11/d1d/d34/d69/f79 [0,4194304] 0 2024-04-26T04:42:17.719 INFO:tasks.workunit.client.1.smithi104.stdout:9/418: dread d11/f27 [0,4194304] 0 2024-04-26T04:42:17.722 INFO:tasks.workunit.client.0.smithi090.stdout:3/171: dread d1/d9/f12 [4194304,4194304] 0 2024-04-26T04:42:17.726 INFO:tasks.workunit.client.0.smithi090.stdout:3/172: mkdir d1/d17/d36/d2a/d3a 0 2024-04-26T04:42:17.726 INFO:tasks.workunit.client.1.smithi104.stdout:0/445: rmdir d0/d2/d10 39 2024-04-26T04:42:17.728 INFO:tasks.workunit.client.1.smithi104.stdout:0/446: link d0/d4/f11 d0/d2/d10/d3d/f8d 0 2024-04-26T04:42:17.728 INFO:tasks.workunit.client.1.smithi104.stdout:0/447: write d0/d2/f23 [1376950,11636] 0 2024-04-26T04:42:17.729 INFO:tasks.workunit.client.1.smithi104.stdout:0/448: mknod d0/d2/d21/d27/c8e 0 2024-04-26T04:42:17.729 INFO:tasks.workunit.client.1.smithi104.stdout:0/449: read d0/d2/d21/f52 [857239,116983] 0 2024-04-26T04:42:17.734 INFO:tasks.workunit.client.1.smithi104.stdout:0/450: dread d0/d2/d21/d49/f7e [0,4194304] 0 2024-04-26T04:42:17.736 INFO:tasks.workunit.client.0.smithi090.stdout:8/208: dwrite dc/d1f/f43 [0,4194304] 0 2024-04-26T04:42:17.737 INFO:tasks.workunit.client.0.smithi090.stdout:7/279: dwrite d0/d1/d4/d3e/f4b [0,4194304] 0 2024-04-26T04:42:17.737 INFO:tasks.workunit.client.1.smithi104.stdout:4/602: dwrite d4/d20/fab [0,4194304] 0 2024-04-26T04:42:17.738 INFO:tasks.workunit.client.1.smithi104.stdout:4/603: chown d4/d20/d55/d56/da3/fc4 37 1 2024-04-26T04:42:17.738 INFO:tasks.workunit.client.1.smithi104.stdout:4/604: chown d4/d7/f2a 2257779 1 2024-04-26T04:42:17.739 INFO:tasks.workunit.client.1.smithi104.stdout:2/712: dread d6/d1b/d51/d6a/fc5 [0,4194304] 0 2024-04-26T04:42:17.739 INFO:tasks.workunit.client.1.smithi104.stdout:4/605: mknod d4/d7/d1d/d1e/d15/cd6 0 2024-04-26T04:42:17.739 INFO:tasks.workunit.client.1.smithi104.stdout:2/713: fsync d6/d8/d5a/f54 0 2024-04-26T04:42:17.742 INFO:tasks.workunit.client.0.smithi090.stdout:7/280: rmdir d0/d1/d4/d43 39 2024-04-26T04:42:17.743 INFO:tasks.workunit.client.0.smithi090.stdout:8/209: getdents dc/d1f/d3a 0 2024-04-26T04:42:17.743 INFO:tasks.workunit.client.1.smithi104.stdout:3/597: dwrite d5/d9/dc/d29/fc1 [0,4194304] 0 2024-04-26T04:42:17.744 INFO:tasks.workunit.client.0.smithi090.stdout:7/281: mkdir d0/db/df/d4d/d64 0 2024-04-26T04:42:17.744 INFO:tasks.workunit.client.0.smithi090.stdout:7/282: chown d0 49 1 2024-04-26T04:42:17.745 INFO:tasks.workunit.client.0.smithi090.stdout:8/210: symlink dc/d1f/l48 0 2024-04-26T04:42:17.745 INFO:tasks.workunit.client.0.smithi090.stdout:8/211: getdents dc/d12/d3b/d3d 0 2024-04-26T04:42:17.745 INFO:tasks.workunit.client.0.smithi090.stdout:7/283: mkdir d0/d1/d4/d43/d65 0 2024-04-26T04:42:17.746 INFO:tasks.workunit.client.0.smithi090.stdout:1/220: dwrite d0/f12 [0,4194304] 0 2024-04-26T04:42:17.750 INFO:tasks.workunit.client.0.smithi090.stdout:1/221: read d0/d4/d7/f2c [850912,3728] 0 2024-04-26T04:42:17.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/222: truncate d0/d4/d7/d17/f2a 774106 0 2024-04-26T04:42:17.754 INFO:tasks.workunit.client.0.smithi090.stdout:1/223: mkdir d0/d4/d43 0 2024-04-26T04:42:17.754 INFO:tasks.workunit.client.0.smithi090.stdout:5/299: dwrite d4/f37 [0,4194304] 0 2024-04-26T04:42:17.754 INFO:tasks.workunit.client.0.smithi090.stdout:4/296: dwrite d3/db/d12/f4e [0,4194304] 0 2024-04-26T04:42:17.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/314: dwrite d9/d21/f48 [0,4194304] 0 2024-04-26T04:42:17.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/315: readlink d9/d1c/d4d/d6d/l67 0 2024-04-26T04:42:17.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/316: readlink d9/d21/l6a 0 2024-04-26T04:42:17.755 INFO:tasks.workunit.client.0.smithi090.stdout:6/317: dread - d9/d21/f70 zero size 2024-04-26T04:42:17.756 INFO:tasks.workunit.client.0.smithi090.stdout:2/224: dwrite db/d11/f17 [0,4194304] 0 2024-04-26T04:42:17.756 INFO:tasks.workunit.client.0.smithi090.stdout:2/225: fdatasync db/d11/f44 0 2024-04-26T04:42:17.758 INFO:tasks.workunit.client.0.smithi090.stdout:1/224: unlink d0/d4/d7/d17/f36 0 2024-04-26T04:42:17.761 INFO:tasks.workunit.client.1.smithi104.stdout:3/598: dread d5/f3f [0,4194304] 0 2024-04-26T04:42:17.761 INFO:tasks.workunit.client.1.smithi104.stdout:2/714: dread d6/d1b/d68/fa1 [0,4194304] 0 2024-04-26T04:42:17.761 INFO:tasks.workunit.client.1.smithi104.stdout:5/501: dwrite d0/f22 [4194304,4194304] 0 2024-04-26T04:42:17.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/318: rename d9/d21/d3b/d62/f43 to d9/d21/d40/d4c/f7a 0 2024-04-26T04:42:17.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/319: creat d9/d21/d40/d4c/f7b x:0 0 0 2024-04-26T04:42:17.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/320: readlink d9/l63 0 2024-04-26T04:42:17.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/321: creat d9/d21/d3b/f7c x:0 0 0 2024-04-26T04:42:17.767 INFO:tasks.workunit.client.1.smithi104.stdout:2/715: rename d6/de/d8e/c77 to d6/d1b/d51/d6a/cf2 0 2024-04-26T04:42:17.767 INFO:tasks.workunit.client.0.smithi090.stdout:2/226: dread db/d11/f26 [0,4194304] 0 2024-04-26T04:42:17.767 INFO:tasks.workunit.client.0.smithi090.stdout:2/227: chown db/d11/d14/d25/l31 36 1 2024-04-26T04:42:17.768 INFO:tasks.workunit.client.0.smithi090.stdout:7/284: dwrite d0/d1/d4/f10 [0,4194304] 0 2024-04-26T04:42:17.768 INFO:tasks.workunit.client.1.smithi104.stdout:5/502: mkdir d0/d1/d64/d26/d36/d72/da9 0 2024-04-26T04:42:17.768 INFO:tasks.workunit.client.1.smithi104.stdout:5/503: read d0/d1/d4/d2a/d2b/d37/f9c [824573,22606] 0 2024-04-26T04:42:17.769 INFO:tasks.workunit.client.1.smithi104.stdout:2/716: symlink d6/de/d1d/dcf/lf3 0 2024-04-26T04:42:17.769 INFO:tasks.workunit.client.1.smithi104.stdout:2/717: truncate d6/d1b/d51/d6a/ff0 278119 0 2024-04-26T04:42:17.769 INFO:tasks.workunit.client.1.smithi104.stdout:2/718: chown d6/d1b/d93 1554 1 2024-04-26T04:42:17.769 INFO:tasks.workunit.client.1.smithi104.stdout:2/719: dread - d6/d8/fea zero size 2024-04-26T04:42:17.771 INFO:tasks.workunit.client.0.smithi090.stdout:1/225: mknod d0/d4/d7/d17/d22/d31/c44 0 2024-04-26T04:42:17.772 INFO:tasks.workunit.client.0.smithi090.stdout:1/226: readlink d0/d4/l41 0 2024-04-26T04:42:17.772 INFO:tasks.workunit.client.1.smithi104.stdout:2/720: symlink d6/d1b/d68/d81/d97/d99/lf4 0 2024-04-26T04:42:17.772 INFO:tasks.workunit.client.1.smithi104.stdout:2/721: write d6/d1b/d68/d81/d97/d99/fa8 [1048192,73942] 0 2024-04-26T04:42:17.774 INFO:tasks.workunit.client.1.smithi104.stdout:0/451: fsync d0/d4/d3c/d53/d5d/d6f/f8c 0 2024-04-26T04:42:17.776 INFO:tasks.workunit.client.1.smithi104.stdout:2/722: dread d6/de/d8e/f4c [0,4194304] 0 2024-04-26T04:42:17.776 INFO:tasks.workunit.client.1.smithi104.stdout:2/723: chown d6/d1b/d51 66 1 2024-04-26T04:42:17.776 INFO:tasks.workunit.client.0.smithi090.stdout:6/322: creat d9/d77/f7d x:0 0 0 2024-04-26T04:42:17.777 INFO:tasks.workunit.client.0.smithi090.stdout:6/323: fsync d9/f11 0 2024-04-26T04:42:17.778 INFO:tasks.workunit.client.1.smithi104.stdout:2/724: truncate d6/d8/f70 3851778 0 2024-04-26T04:42:17.778 INFO:tasks.workunit.client.0.smithi090.stdout:2/228: mkdir db/d11/d14/d1b/d20/d4c 0 2024-04-26T04:42:17.780 INFO:tasks.workunit.client.0.smithi090.stdout:5/300: dwrite d4/d1a/d1f/f2b [0,4194304] 0 2024-04-26T04:42:17.780 INFO:tasks.workunit.client.0.smithi090.stdout:5/301: getdents d4/d1a/df/d4a 0 2024-04-26T04:42:17.780 INFO:tasks.workunit.client.0.smithi090.stdout:5/302: write d4/d1a/d45/f32 [2587052,102391] 0 2024-04-26T04:42:17.781 INFO:tasks.workunit.client.0.smithi090.stdout:1/227: mknod d0/d4/d43/c45 0 2024-04-26T04:42:17.781 INFO:tasks.workunit.client.0.smithi090.stdout:1/228: write d0/d4/d7/d17/f3c [932199,22939] 0 2024-04-26T04:42:17.782 INFO:tasks.workunit.client.0.smithi090.stdout:8/212: truncate dc/d1f/f43 1860154 0 2024-04-26T04:42:17.782 INFO:tasks.workunit.client.0.smithi090.stdout:8/213: stat dc/d34/f40 0 2024-04-26T04:42:17.782 INFO:tasks.workunit.client.1.smithi104.stdout:0/452: rmdir d0/d2/d21/d49/d62 0 2024-04-26T04:42:17.785 INFO:tasks.workunit.client.0.smithi090.stdout:6/324: creat d9/d1c/d31/d79/f7e x:0 0 0 2024-04-26T04:42:17.788 INFO:tasks.workunit.client.0.smithi090.stdout:1/229: unlink d0/lf 0 2024-04-26T04:42:17.794 INFO:tasks.workunit.client.1.smithi104.stdout:2/725: dread d6/fc [0,4194304] 0 2024-04-26T04:42:17.794 INFO:tasks.workunit.client.1.smithi104.stdout:2/726: readlink d6/l18 0 2024-04-26T04:42:17.796 INFO:tasks.workunit.client.1.smithi104.stdout:0/453: getdents d0/d2/d10/d3d 0 2024-04-26T04:42:17.797 INFO:tasks.workunit.client.0.smithi090.stdout:8/214: rename dc/d12 to dc/d1f/d49 0 2024-04-26T04:42:17.797 INFO:tasks.workunit.client.0.smithi090.stdout:6/325: rename d9/d1c to d9/d1c/d31/d69/d7f 22 2024-04-26T04:42:17.797 INFO:tasks.workunit.client.0.smithi090.stdout:6/326: creat d9/d1c/f80 x:0 0 0 2024-04-26T04:42:17.798 INFO:tasks.workunit.client.1.smithi104.stdout:5/504: dwrite d0/f4e [0,4194304] 0 2024-04-26T04:42:17.799 INFO:tasks.workunit.client.1.smithi104.stdout:0/454: link d0/d4/f29 d0/d2/d21/d27/d1e/d57/f8f 0 2024-04-26T04:42:17.801 INFO:tasks.workunit.client.0.smithi090.stdout:6/327: getdents d9/d21/d3b/d62 0 2024-04-26T04:42:17.804 INFO:tasks.workunit.client.0.smithi090.stdout:6/328: truncate d9/d21/f48 1624606 0 2024-04-26T04:42:17.804 INFO:tasks.workunit.client.1.smithi104.stdout:0/455: unlink d0/d4/cf 0 2024-04-26T04:42:17.805 INFO:tasks.workunit.client.0.smithi090.stdout:6/329: truncate d9/d21/d40/d4c/f7a 3268481 0 2024-04-26T04:42:17.806 INFO:tasks.workunit.client.0.smithi090.stdout:6/330: stat d9/d21/d40/d4c/d6f 0 2024-04-26T04:42:17.807 INFO:tasks.workunit.client.0.smithi090.stdout:6/331: mknod d9/d21/d3b/c81 0 2024-04-26T04:42:17.811 INFO:tasks.workunit.client.0.smithi090.stdout:7/285: dwrite d0/d1/f61 [0,4194304] 0 2024-04-26T04:42:17.815 INFO:tasks.workunit.client.0.smithi090.stdout:5/303: dwrite d4/d1a/d45/f4f [0,4194304] 0 2024-04-26T04:42:17.816 INFO:tasks.workunit.client.0.smithi090.stdout:2/229: dwrite db/f33 [0,4194304] 0 2024-04-26T04:42:17.816 INFO:tasks.workunit.client.0.smithi090.stdout:7/286: mkdir d0/db/df/d4d/d64/d66 0 2024-04-26T04:42:17.819 INFO:tasks.workunit.client.1.smithi104.stdout:0/456: dread d0/d4/d3c/f4b [0,4194304] 0 2024-04-26T04:42:17.819 INFO:tasks.workunit.client.1.smithi104.stdout:5/505: dread d0/d1/d1e/f30 [0,4194304] 0 2024-04-26T04:42:17.819 INFO:tasks.workunit.client.1.smithi104.stdout:5/506: stat d0/d1/d1e 0 2024-04-26T04:42:17.820 INFO:tasks.workunit.client.1.smithi104.stdout:5/507: chown d0/d1/d4/d2a/d2b/d5c/c84 46166 1 2024-04-26T04:42:17.820 INFO:tasks.workunit.client.0.smithi090.stdout:5/304: mknod d4/d1a/df/d36/d42/d49/c62 0 2024-04-26T04:42:17.820 INFO:tasks.workunit.client.0.smithi090.stdout:8/215: write dc/d1f/f43 [794984,73830] 0 2024-04-26T04:42:17.822 INFO:tasks.workunit.client.1.smithi104.stdout:0/457: rmdir d0/d4/d3c 39 2024-04-26T04:42:17.825 INFO:tasks.workunit.client.1.smithi104.stdout:0/458: creat d0/d2/d10/d84/f90 x:0 0 0 2024-04-26T04:42:17.825 INFO:tasks.workunit.client.1.smithi104.stdout:0/459: creat d0/f91 x:0 0 0 2024-04-26T04:42:17.825 INFO:tasks.workunit.client.1.smithi104.stdout:2/727: dwrite d6/d8/d3e/f7a [0,4194304] 0 2024-04-26T04:42:17.825 INFO:tasks.workunit.client.1.smithi104.stdout:2/728: fsync d6/d8/f39 0 2024-04-26T04:42:17.825 INFO:tasks.workunit.client.1.smithi104.stdout:2/729: dread - d6/d8/d5a/fb3 zero size 2024-04-26T04:42:17.825 INFO:tasks.workunit.client.1.smithi104.stdout:2/730: chown d6/d8/f39 5 1 2024-04-26T04:42:17.826 INFO:tasks.workunit.client.0.smithi090.stdout:9/184: sync 2024-04-26T04:42:17.826 INFO:tasks.workunit.client.0.smithi090.stdout:9/185: chown db/dd/l12 128 1 2024-04-26T04:42:17.826 INFO:tasks.workunit.client.0.smithi090.stdout:2/230: rename db/d11/d14/d27/d28/f47 to db/d3f/f4d 0 2024-04-26T04:42:17.827 INFO:tasks.workunit.client.0.smithi090.stdout:2/231: truncate db/d3f/f4d 770578 0 2024-04-26T04:42:17.827 INFO:tasks.workunit.client.0.smithi090.stdout:5/305: rename d4/d1a/d1f to d4/d1a/d1f/d63 22 2024-04-26T04:42:17.827 INFO:tasks.workunit.client.0.smithi090.stdout:8/216: dread dc/d1f/d49/f2e [0,4194304] 0 2024-04-26T04:42:17.827 INFO:tasks.workunit.client.0.smithi090.stdout:1/230: dwrite d0/d4/d7/d17/d22/f2f [0,4194304] 0 2024-04-26T04:42:17.827 INFO:tasks.workunit.client.0.smithi090.stdout:1/231: dread - d0/d4/d7/f33 zero size 2024-04-26T04:42:17.827 INFO:tasks.workunit.client.0.smithi090.stdout:7/287: symlink d0/db/d15/d30/d4f/l67 0 2024-04-26T04:42:17.830 INFO:tasks.workunit.client.1.smithi104.stdout:0/460: symlink d0/d4/l92 0 2024-04-26T04:42:17.830 INFO:tasks.workunit.client.1.smithi104.stdout:0/461: fdatasync d0/d4/d3c/d53/d5d/d6f/f8c 0 2024-04-26T04:42:17.831 INFO:tasks.workunit.client.0.smithi090.stdout:2/232: unlink db/d11/f26 0 2024-04-26T04:42:17.831 INFO:tasks.workunit.client.1.smithi104.stdout:5/508: write d0/d1/d4/d2a/d2b/f35 [3061696,36976] 0 2024-04-26T04:42:17.833 INFO:tasks.workunit.client.0.smithi090.stdout:5/306: symlink d4/d1a/d2a/d31/l64 0 2024-04-26T04:42:17.833 INFO:tasks.workunit.client.0.smithi090.stdout:8/217: creat dc/d1f/d49/d3b/f4a x:0 0 0 2024-04-26T04:42:17.834 INFO:tasks.workunit.client.1.smithi104.stdout:2/731: unlink d6/d1b/l64 0 2024-04-26T04:42:17.835 INFO:tasks.workunit.client.1.smithi104.stdout:0/462: truncate d0/d2/d21/d38/f54 832683 0 2024-04-26T04:42:17.837 INFO:tasks.workunit.client.0.smithi090.stdout:1/232: link d0/f2e d0/f46 0 2024-04-26T04:42:17.837 INFO:tasks.workunit.client.0.smithi090.stdout:1/233: chown d0/d4/d7/l3b 197 1 2024-04-26T04:42:17.837 INFO:tasks.workunit.client.0.smithi090.stdout:1/234: write d0/d4/d7/d17/f29 [818986,29015] 0 2024-04-26T04:42:17.838 INFO:tasks.workunit.client.0.smithi090.stdout:7/288: symlink d0/d1/d4/d3e/l68 0 2024-04-26T04:42:17.839 INFO:tasks.workunit.client.1.smithi104.stdout:5/509: link d0/d1/f3d d0/d1/d64/d26/d36/d72/faa 0 2024-04-26T04:42:17.839 INFO:tasks.workunit.client.1.smithi104.stdout:5/510: chown d0/d1/d64/d26/d36/d72/da9 20864217 1 2024-04-26T04:42:17.839 INFO:tasks.workunit.client.0.smithi090.stdout:2/233: rename db/d11/d14/d27/d28/f45 to db/d11/d14/d25/f4e 0 2024-04-26T04:42:17.842 INFO:tasks.workunit.client.1.smithi104.stdout:0/463: link d0/d2/c73 d0/c93 0 2024-04-26T04:42:17.843 INFO:tasks.workunit.client.1.smithi104.stdout:5/511: rmdir d0/d1/d4/d2a/d6b/d73 39 2024-04-26T04:42:17.844 INFO:tasks.workunit.client.1.smithi104.stdout:2/732: getdents d6/d1b/d93 0 2024-04-26T04:42:17.844 INFO:tasks.workunit.client.1.smithi104.stdout:0/464: mknod d0/d2/d10/c94 0 2024-04-26T04:42:17.844 INFO:tasks.workunit.client.1.smithi104.stdout:0/465: fdatasync d0/f7f 0 2024-04-26T04:42:17.845 INFO:tasks.workunit.client.1.smithi104.stdout:2/733: rmdir d6/d1b/d68/d81/d97 39 2024-04-26T04:42:17.845 INFO:tasks.workunit.client.1.smithi104.stdout:5/512: dread d0/d1/d3e/f51 [0,4194304] 0 2024-04-26T04:42:17.845 INFO:tasks.workunit.client.0.smithi090.stdout:1/235: truncate d0/d4/d7/d17/f18 3702329 0 2024-04-26T04:42:17.846 INFO:tasks.workunit.client.1.smithi104.stdout:0/466: mknod d0/d4/c95 0 2024-04-26T04:42:17.846 INFO:tasks.workunit.client.0.smithi090.stdout:2/234: creat db/d3f/f4f x:0 0 0 2024-04-26T04:42:17.846 INFO:tasks.workunit.client.0.smithi090.stdout:2/235: truncate db/d11/d14/d1b/f43 316052 0 2024-04-26T04:42:17.846 INFO:tasks.workunit.client.0.smithi090.stdout:2/236: dread - db/d11/d14/f4a zero size 2024-04-26T04:42:17.847 INFO:tasks.workunit.client.1.smithi104.stdout:5/513: mknod d0/d1/d64/d26/d36/d72/cab 0 2024-04-26T04:42:17.848 INFO:tasks.workunit.client.0.smithi090.stdout:1/236: unlink d0/d4/d43/c45 0 2024-04-26T04:42:17.848 INFO:tasks.workunit.client.0.smithi090.stdout:1/237: stat d0/c1f 0 2024-04-26T04:42:17.848 INFO:tasks.workunit.client.0.smithi090.stdout:6/332: dwrite d9/d1c/d4d/d58/f6c [0,4194304] 0 2024-04-26T04:42:17.848 INFO:tasks.workunit.client.1.smithi104.stdout:0/467: unlink d0/d2/d21/d27/d1e/d57/d6c/f59 0 2024-04-26T04:42:17.849 INFO:tasks.workunit.client.1.smithi104.stdout:0/468: write d0/d2/d21/d27/d1e/f65 [2022,22030] 0 2024-04-26T04:42:17.849 INFO:tasks.workunit.client.1.smithi104.stdout:5/514: symlink d0/d1/d25/lac 0 2024-04-26T04:42:17.850 INFO:tasks.workunit.client.0.smithi090.stdout:2/237: unlink db/d11/d14/d25/f2a 0 2024-04-26T04:42:17.851 INFO:tasks.workunit.client.1.smithi104.stdout:5/515: mknod d0/d1/d1e/d98/cad 0 2024-04-26T04:42:17.852 INFO:tasks.workunit.client.1.smithi104.stdout:5/516: write d0/d1/f38 [182826,15680] 0 2024-04-26T04:42:17.852 INFO:tasks.workunit.client.1.smithi104.stdout:5/517: readlink d0/d1/d64/d26/l46 0 2024-04-26T04:42:17.852 INFO:tasks.workunit.client.1.smithi104.stdout:5/518: write d0/d1/d4/d2a/d2b/d37/d48/f70 [3832968,127177] 0 2024-04-26T04:42:17.852 INFO:tasks.workunit.client.1.smithi104.stdout:5/519: creat d0/d1/d1e/d98/fae x:0 0 0 2024-04-26T04:42:17.853 INFO:tasks.workunit.client.1.smithi104.stdout:5/520: creat d0/d89/faf x:0 0 0 2024-04-26T04:42:17.853 INFO:tasks.workunit.client.1.smithi104.stdout:5/521: write d0/d89/faf [150964,6944] 0 2024-04-26T04:42:17.854 INFO:tasks.workunit.client.0.smithi090.stdout:6/333: mknod d9/d1c/d4d/d6d/c82 0 2024-04-26T04:42:17.854 INFO:tasks.workunit.client.0.smithi090.stdout:6/334: dread d9/f11 [0,4194304] 0 2024-04-26T04:42:17.856 INFO:tasks.workunit.client.1.smithi104.stdout:2/734: write d6/d8/f70 [1253665,13877] 0 2024-04-26T04:42:17.857 INFO:tasks.workunit.client.1.smithi104.stdout:2/735: write d6/d1b/d68/d81/d97/f9a [313070,77401] 0 2024-04-26T04:42:17.857 INFO:tasks.workunit.client.1.smithi104.stdout:2/736: chown d6/de/d85/ca4 78153486 1 2024-04-26T04:42:17.859 INFO:tasks.workunit.client.1.smithi104.stdout:2/737: unlink d6/de/f40 0 2024-04-26T04:42:17.859 INFO:tasks.workunit.client.1.smithi104.stdout:2/738: stat d6/de/d8e/f60 0 2024-04-26T04:42:17.859 INFO:tasks.workunit.client.0.smithi090.stdout:6/335: link d9/l39 d9/d21/d40/d4c/l83 0 2024-04-26T04:42:17.860 INFO:tasks.workunit.client.0.smithi090.stdout:5/307: dwrite d4/f46 [0,4194304] 0 2024-04-26T04:42:17.861 INFO:tasks.workunit.client.0.smithi090.stdout:9/186: dwrite db/f26 [4194304,4194304] 0 2024-04-26T04:42:17.861 INFO:tasks.workunit.client.0.smithi090.stdout:9/187: write db/dd/f1d [1068233,61633] 0 2024-04-26T04:42:17.862 INFO:tasks.workunit.client.0.smithi090.stdout:9/188: creat db/f41 x:0 0 0 2024-04-26T04:42:17.862 INFO:tasks.workunit.client.0.smithi090.stdout:1/238: write d0/d4/f15 [2331786,125440] 0 2024-04-26T04:42:17.862 INFO:tasks.workunit.client.1.smithi104.stdout:2/739: symlink d6/d1b/d51/lf5 0 2024-04-26T04:42:17.868 INFO:tasks.workunit.client.1.smithi104.stdout:0/469: dread d0/d2/d10/f71 [0,4194304] 0 2024-04-26T04:42:17.870 INFO:tasks.workunit.client.0.smithi090.stdout:8/218: truncate dc/d1f/f43 1323318 0 2024-04-26T04:42:17.874 INFO:tasks.workunit.client.0.smithi090.stdout:9/189: dread db/d1e/f27 [0,4194304] 0 2024-04-26T04:42:17.877 INFO:tasks.workunit.client.0.smithi090.stdout:1/239: creat d0/d4/d43/f47 x:0 0 0 2024-04-26T04:42:17.879 INFO:tasks.workunit.client.1.smithi104.stdout:5/522: write d0/f4e [2021355,71545] 0 2024-04-26T04:42:17.879 INFO:tasks.workunit.client.0.smithi090.stdout:9/190: creat db/dd/d18/d38/f42 x:0 0 0 2024-04-26T04:42:17.879 INFO:tasks.workunit.client.0.smithi090.stdout:9/191: creat db/dd/d18/f43 x:0 0 0 2024-04-26T04:42:17.880 INFO:tasks.workunit.client.1.smithi104.stdout:0/470: dread d0/d2/d21/d49/f7e [0,4194304] 0 2024-04-26T04:42:17.881 INFO:tasks.workunit.client.0.smithi090.stdout:1/240: creat d0/d4/f48 x:0 0 0 2024-04-26T04:42:17.881 INFO:tasks.workunit.client.0.smithi090.stdout:1/241: dread - d0/d4/d7/f33 zero size 2024-04-26T04:42:17.882 INFO:tasks.workunit.client.1.smithi104.stdout:5/523: creat d0/d1/d4/d2a/d6b/d91/fb0 x:0 0 0 2024-04-26T04:42:17.883 INFO:tasks.workunit.client.1.smithi104.stdout:0/471: rmdir d0/d2/d21/d49/d5e 39 2024-04-26T04:42:17.883 INFO:tasks.workunit.client.1.smithi104.stdout:0/472: fdatasync d0/d2/d21/d27/d1e/d57/d6c/f56 0 2024-04-26T04:42:17.883 INFO:tasks.workunit.client.1.smithi104.stdout:0/473: dread - d0/d2/d21/d27/f83 zero size 2024-04-26T04:42:17.884 INFO:tasks.workunit.client.1.smithi104.stdout:5/524: creat d0/d1/d1e/fb1 x:0 0 0 2024-04-26T04:42:17.884 INFO:tasks.workunit.client.1.smithi104.stdout:5/525: readlink d0/d1/d4/l9 0 2024-04-26T04:42:17.884 INFO:tasks.workunit.client.1.smithi104.stdout:5/526: write d0/f5 [906296,106583] 0 2024-04-26T04:42:17.885 INFO:tasks.workunit.client.0.smithi090.stdout:1/242: mkdir d0/d4/d7/d49 0 2024-04-26T04:42:17.885 INFO:tasks.workunit.client.1.smithi104.stdout:0/474: truncate d0/d2/d10/f5a 1783565 0 2024-04-26T04:42:17.887 INFO:tasks.workunit.client.1.smithi104.stdout:5/527: mkdir d0/d89/db2 0 2024-04-26T04:42:17.887 INFO:tasks.workunit.client.1.smithi104.stdout:0/475: creat d0/d2/d21/d27/d1e/f96 x:0 0 0 2024-04-26T04:42:17.888 INFO:tasks.workunit.client.0.smithi090.stdout:9/192: write db/f3d [3674704,63458] 0 2024-04-26T04:42:17.889 INFO:tasks.workunit.client.0.smithi090.stdout:9/193: dread - db/dd/d18/f43 zero size 2024-04-26T04:42:17.889 INFO:tasks.workunit.client.0.smithi090.stdout:9/194: write db/dd/d18/f2a [219746,31973] 0 2024-04-26T04:42:17.889 INFO:tasks.workunit.client.1.smithi104.stdout:5/528: mkdir d0/d1/d64/d26/d36/da7/db3 0 2024-04-26T04:42:17.890 INFO:tasks.workunit.client.0.smithi090.stdout:2/238: rmdir db/d11/d14 39 2024-04-26T04:42:17.892 INFO:tasks.workunit.client.1.smithi104.stdout:5/529: dread d0/d1/f52 [0,4194304] 0 2024-04-26T04:42:17.892 INFO:tasks.workunit.client.1.smithi104.stdout:5/530: chown d0/d1/d64/c6e 59488 1 2024-04-26T04:42:17.892 INFO:tasks.workunit.client.1.smithi104.stdout:5/531: truncate d0/d1/d1e/fb1 480434 0 2024-04-26T04:42:17.892 INFO:tasks.workunit.client.0.smithi090.stdout:6/336: dwrite d9/d21/d3b/f46 [0,4194304] 0 2024-04-26T04:42:17.892 INFO:tasks.workunit.client.0.smithi090.stdout:9/195: unlink db/d1e/c3e 0 2024-04-26T04:42:17.892 INFO:tasks.workunit.client.0.smithi090.stdout:5/308: dwrite d4/d1a/d1f/d53/d59/d5e/f5d [0,4194304] 0 2024-04-26T04:42:17.895 INFO:tasks.workunit.client.1.smithi104.stdout:0/476: rmdir d0/d2/d21/d27/d1e/d57 39 2024-04-26T04:42:17.895 INFO:tasks.workunit.client.1.smithi104.stdout:0/477: stat d0 0 2024-04-26T04:42:17.895 INFO:tasks.workunit.client.1.smithi104.stdout:0/478: write d0/d4/d3b/f77 [432324,101811] 0 2024-04-26T04:42:17.895 INFO:tasks.workunit.client.1.smithi104.stdout:0/479: stat d0/d2/d10/l37 0 2024-04-26T04:42:17.895 INFO:tasks.workunit.client.0.smithi090.stdout:2/239: link db/d11/l46 db/d3f/l50 0 2024-04-26T04:42:17.896 INFO:tasks.workunit.client.0.smithi090.stdout:9/196: rmdir db/d16 39 2024-04-26T04:42:17.896 INFO:tasks.workunit.client.0.smithi090.stdout:9/197: write db/f1b [1439158,111301] 0 2024-04-26T04:42:17.899 INFO:tasks.workunit.client.0.smithi090.stdout:5/309: rename d4/f37 to d4/d1a/d3f/f65 0 2024-04-26T04:42:17.899 INFO:tasks.workunit.client.0.smithi090.stdout:9/198: dread f3 [0,4194304] 0 2024-04-26T04:42:17.901 INFO:tasks.workunit.client.1.smithi104.stdout:0/480: write d0/d2/d10/f5a [2470297,35199] 0 2024-04-26T04:42:17.901 INFO:tasks.workunit.client.1.smithi104.stdout:0/481: chown d0/d4/d3c/c74 107678 1 2024-04-26T04:42:17.902 INFO:tasks.workunit.client.1.smithi104.stdout:0/482: rmdir d0/d4/d3c/d53/d5d 39 2024-04-26T04:42:17.904 INFO:tasks.workunit.client.1.smithi104.stdout:0/483: creat d0/d2/d21/f97 x:0 0 0 2024-04-26T04:42:17.904 INFO:tasks.workunit.client.1.smithi104.stdout:0/484: truncate d0/d81/f8a 351253 0 2024-04-26T04:42:17.905 INFO:tasks.workunit.client.0.smithi090.stdout:2/240: write db/dc/f3c [5043154,78561] 0 2024-04-26T04:42:17.908 INFO:tasks.workunit.client.0.smithi090.stdout:5/310: rename d4/d1a/d1f/d53/d59/d5e/f5d to d4/d1a/df/d36/d42/d49/f66 0 2024-04-26T04:42:17.908 INFO:tasks.workunit.client.0.smithi090.stdout:5/311: chown d4/d1a/d1f/c4c 510941 1 2024-04-26T04:42:17.910 INFO:tasks.workunit.client.0.smithi090.stdout:8/219: write dc/d1f/f43 [646661,116274] 0 2024-04-26T04:42:17.910 INFO:tasks.workunit.client.0.smithi090.stdout:8/220: chown dc/l26 6 1 2024-04-26T04:42:17.910 INFO:tasks.workunit.client.0.smithi090.stdout:8/221: fdatasync dc/d1f/d3a/f3e 0 2024-04-26T04:42:17.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/312: symlink d4/d1a/l67 0 2024-04-26T04:42:17.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/313: chown d4/d1a/l67 6742902 1 2024-04-26T04:42:17.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/314: write d4/d1a/d45/f32 [2778163,13049] 0 2024-04-26T04:42:17.911 INFO:tasks.workunit.client.0.smithi090.stdout:5/315: stat d4/d1a/df/d36 0 2024-04-26T04:42:17.914 INFO:tasks.workunit.client.0.smithi090.stdout:8/222: unlink dc/d1f/f33 0 2024-04-26T04:42:17.916 INFO:tasks.workunit.client.0.smithi090.stdout:8/223: unlink dc/d1f/d49/f2e 0 2024-04-26T04:42:17.920 INFO:tasks.workunit.client.1.smithi104.stdout:0/485: dread d0/d2/d21/d49/d5e/f78 [0,4194304] 0 2024-04-26T04:42:17.922 INFO:tasks.workunit.client.1.smithi104.stdout:0/486: symlink d0/d4/d3b/l98 0 2024-04-26T04:42:17.925 INFO:tasks.workunit.client.0.smithi090.stdout:1/243: dwrite d0/d4/d7/d17/d22/f2f [0,4194304] 0 2024-04-26T04:42:17.935 INFO:tasks.workunit.client.0.smithi090.stdout:1/244: mkdir d0/d4/d7/d4a 0 2024-04-26T04:42:17.936 INFO:tasks.workunit.client.0.smithi090.stdout:9/199: rename db/d1e to db/dd/d44 0 2024-04-26T04:42:17.937 INFO:tasks.workunit.client.0.smithi090.stdout:1/245: mkdir d0/d4/d7/d4b 0 2024-04-26T04:42:17.938 INFO:tasks.workunit.client.0.smithi090.stdout:9/200: symlink db/dd/d18/d38/d39/l45 0 2024-04-26T04:42:17.939 INFO:tasks.workunit.client.0.smithi090.stdout:1/246: creat d0/d4/d7/d4b/f4c x:0 0 0 2024-04-26T04:42:17.939 INFO:tasks.workunit.client.0.smithi090.stdout:1/247: write d0/d4/d43/f47 [99832,76464] 0 2024-04-26T04:42:17.940 INFO:tasks.workunit.client.0.smithi090.stdout:2/241: dwrite db/d11/d14/d27/d28/f3d [0,4194304] 0 2024-04-26T04:42:17.940 INFO:tasks.workunit.client.0.smithi090.stdout:2/242: getdents db/d11/d14/d25/d2d/d48 0 2024-04-26T04:42:17.940 INFO:tasks.workunit.client.0.smithi090.stdout:9/201: truncate db/f13 3814219 0 2024-04-26T04:42:17.940 INFO:tasks.workunit.client.0.smithi090.stdout:9/202: stat db/dd/d18/d38/d39/d3f 0 2024-04-26T04:42:17.942 INFO:tasks.workunit.client.0.smithi090.stdout:1/248: mkdir d0/d4/d43/d4d 0 2024-04-26T04:42:17.942 INFO:tasks.workunit.client.0.smithi090.stdout:1/249: stat d0/d4/d7/d17/d22/d31/f39 0 2024-04-26T04:42:17.943 INFO:tasks.workunit.client.1.smithi104.stdout:5/532: dwrite d0/d1/d1e/f30 [0,4194304] 0 2024-04-26T04:42:17.943 INFO:tasks.workunit.client.0.smithi090.stdout:6/337: dwrite d9/f24 [0,4194304] 0 2024-04-26T04:42:17.943 INFO:tasks.workunit.client.0.smithi090.stdout:6/338: write d9/d1c/d31/f41 [801479,8434] 0 2024-04-26T04:42:17.943 INFO:tasks.workunit.client.0.smithi090.stdout:5/316: dwrite d4/d1a/f19 [0,4194304] 0 2024-04-26T04:42:17.944 INFO:tasks.workunit.client.0.smithi090.stdout:2/243: creat db/d11/d14/d1b/f51 x:0 0 0 2024-04-26T04:42:17.945 INFO:tasks.workunit.client.1.smithi104.stdout:5/533: unlink d0/d1/d64/f23 0 2024-04-26T04:42:17.946 INFO:tasks.workunit.client.0.smithi090.stdout:9/203: creat db/d16/f46 x:0 0 0 2024-04-26T04:42:17.947 INFO:tasks.workunit.client.0.smithi090.stdout:1/250: symlink d0/d4/d7/d17/d22/d31/l4e 0 2024-04-26T04:42:17.951 INFO:tasks.workunit.client.1.smithi104.stdout:5/534: creat d0/d1/d64/d26/d36/da7/db3/fb4 x:0 0 0 2024-04-26T04:42:17.952 INFO:tasks.workunit.client.1.smithi104.stdout:5/535: mkdir d0/d1/d4/d2a/d6b/d73/da4/db5 0 2024-04-26T04:42:17.952 INFO:tasks.workunit.client.1.smithi104.stdout:5/536: creat d0/d41/d4f/fb6 x:0 0 0 2024-04-26T04:42:17.952 INFO:tasks.workunit.client.1.smithi104.stdout:5/537: chown d0/d1/f2 42 1 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:4/606: sync 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:9/419: sync 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:9/420: fdatasync d11/f27 0 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:6/679: sync 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:1/562: sync 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:9/421: write d11/f44 [659214,110848] 0 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:9/422: stat d11/d15/d32/f56 0 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:1/563: write d0/d6/d19/d21/f6d [1581624,105323] 0 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/416: sync 2024-04-26T04:42:17.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/417: chown d0/d1c/d27/f40 342413 1 2024-04-26T04:42:17.956 INFO:tasks.workunit.client.1.smithi104.stdout:8/418: chown d0/d1c/d2d/f32 124321906 1 2024-04-26T04:42:17.956 INFO:tasks.workunit.client.1.smithi104.stdout:8/419: creat d0/d33/f82 x:0 0 0 2024-04-26T04:42:17.956 INFO:tasks.workunit.client.1.smithi104.stdout:8/420: creat d0/d50/f83 x:0 0 0 2024-04-26T04:42:17.956 INFO:tasks.workunit.client.1.smithi104.stdout:8/421: creat d0/d1c/f84 x:0 0 0 2024-04-26T04:42:17.956 INFO:tasks.workunit.client.1.smithi104.stdout:3/599: sync 2024-04-26T04:42:17.956 INFO:tasks.workunit.client.1.smithi104.stdout:3/600: truncate d5/d9/dc/d29/fcc 173941 0 2024-04-26T04:42:17.957 INFO:tasks.workunit.client.1.smithi104.stdout:7/470: sync 2024-04-26T04:42:17.957 INFO:tasks.workunit.client.1.smithi104.stdout:7/471: fsync d2/f75 0 2024-04-26T04:42:17.957 INFO:tasks.workunit.client.1.smithi104.stdout:7/472: write d2/d4/d32/d35/d44/f91 [842539,68544] 0 2024-04-26T04:42:17.957 INFO:tasks.workunit.client.1.smithi104.stdout:5/538: rename d0/d1/f38 to d0/d1/d4/d2a/d2b/fb7 0 2024-04-26T04:42:17.958 INFO:tasks.workunit.client.0.smithi090.stdout:6/339: rename d9/fd to d9/d1c/d4d/d6d/f84 0 2024-04-26T04:42:17.958 INFO:tasks.workunit.client.0.smithi090.stdout:6/340: write d9/d3c/f61 [1037108,123989] 0 2024-04-26T04:42:17.960 INFO:tasks.workunit.client.1.smithi104.stdout:4/607: creat d4/d7/d1d/d2e/d74/dbd/dd3/fd7 x:0 0 0 2024-04-26T04:42:17.960 INFO:tasks.workunit.client.1.smithi104.stdout:4/608: truncate d4/d7/d1d/d1e/d24/d48/f4e 4704254 0 2024-04-26T04:42:17.960 INFO:tasks.workunit.client.1.smithi104.stdout:4/609: truncate d4/d7/d1d/d1e/d24/d4a/f82 1805334 0 2024-04-26T04:42:17.960 INFO:tasks.workunit.client.1.smithi104.stdout:4/610: write d4/d20/f9a [901355,121805] 0 2024-04-26T04:42:17.960 INFO:tasks.workunit.client.0.smithi090.stdout:2/244: getdents db/dc 0 2024-04-26T04:42:17.961 INFO:tasks.workunit.client.0.smithi090.stdout:2/245: dread - db/d11/d14/f4a zero size 2024-04-26T04:42:17.961 INFO:tasks.workunit.client.1.smithi104.stdout:9/423: dread d11/d15/d32/d4c/d54/f61 [0,4194304] 0 2024-04-26T04:42:17.961 INFO:tasks.workunit.client.1.smithi104.stdout:6/680: truncate d0/d9/f4a 431791 0 2024-04-26T04:42:17.961 INFO:tasks.workunit.client.1.smithi104.stdout:6/681: truncate d0/d9/dc6/f7c 1012500 0 2024-04-26T04:42:17.961 INFO:tasks.workunit.client.0.smithi090.stdout:1/251: mknod d0/d4/d7/d4a/c4f 0 2024-04-26T04:42:17.964 INFO:tasks.workunit.client.0.smithi090.stdout:5/317: rename d4/d1a/d2a/d31/l64 to d4/d1a/df/d36/l68 0 2024-04-26T04:42:17.966 INFO:tasks.workunit.client.0.smithi090.stdout:2/246: link db/cf db/c52 0 2024-04-26T04:42:17.967 INFO:tasks.workunit.client.1.smithi104.stdout:1/564: rmdir d0/d6/d19/d21/d59 39 2024-04-26T04:42:17.967 INFO:tasks.workunit.client.1.smithi104.stdout:1/565: chown d0/d6/d19/d21/d66/l24 95168 1 2024-04-26T04:42:17.967 INFO:tasks.workunit.client.1.smithi104.stdout:1/566: stat d0/d6/ca 0 2024-04-26T04:42:17.967 INFO:tasks.workunit.client.1.smithi104.stdout:1/567: readlink d0/d6/d19/d21/d58/lb9 0 2024-04-26T04:42:17.967 INFO:tasks.workunit.client.1.smithi104.stdout:1/568: chown d0/d6/d19/d21/d66/d20/da3/la8 18 1 2024-04-26T04:42:17.968 INFO:tasks.workunit.client.1.smithi104.stdout:1/569: truncate d0/d6/d2e/f86 1269924 0 2024-04-26T04:42:17.968 INFO:tasks.workunit.client.1.smithi104.stdout:8/422: mkdir d0/d1c/d59/d85 0 2024-04-26T04:42:17.968 INFO:tasks.workunit.client.1.smithi104.stdout:1/570: dread d0/d6/d19/d21/d59/f73 [0,4194304] 0 2024-04-26T04:42:17.969 INFO:tasks.workunit.client.0.smithi090.stdout:6/341: dread d9/d21/d40/f4b [0,4194304] 0 2024-04-26T04:42:17.969 INFO:tasks.workunit.client.1.smithi104.stdout:3/601: creat d5/d9/d86/fd2 x:0 0 0 2024-04-26T04:42:17.969 INFO:tasks.workunit.client.1.smithi104.stdout:3/602: creat d5/d9/dc/d29/d7d/d8e/fd3 x:0 0 0 2024-04-26T04:42:17.971 INFO:tasks.workunit.client.1.smithi104.stdout:5/539: creat d0/d1/d64/d26/fb8 x:0 0 0 2024-04-26T04:42:17.971 INFO:tasks.workunit.client.1.smithi104.stdout:5/540: creat d0/d89/fb9 x:0 0 0 2024-04-26T04:42:17.975 INFO:tasks.workunit.client.0.smithi090.stdout:1/252: mkdir d0/d4/d7/d17/d22/d50 0 2024-04-26T04:42:17.975 INFO:tasks.workunit.client.0.smithi090.stdout:8/224: rmdir dc 39 2024-04-26T04:42:17.975 INFO:tasks.workunit.client.0.smithi090.stdout:1/253: read - d0/d4/d7/d4b/f4c zero size 2024-04-26T04:42:17.975 INFO:tasks.workunit.client.0.smithi090.stdout:8/225: truncate dc/d34/f40 782246 0 2024-04-26T04:42:17.975 INFO:tasks.workunit.client.0.smithi090.stdout:5/318: truncate d4/d1a/f43 841095 0 2024-04-26T04:42:17.977 INFO:tasks.workunit.client.0.smithi090.stdout:6/342: creat d9/d1c/d4d/f85 x:0 0 0 2024-04-26T04:42:17.977 INFO:tasks.workunit.client.0.smithi090.stdout:6/343: chown d9/d21/d3b/d62/f26 182 1 2024-04-26T04:42:17.977 INFO:tasks.workunit.client.0.smithi090.stdout:6/344: chown d9/d21/d40/d4c/c56 386 1 2024-04-26T04:42:17.978 INFO:tasks.workunit.client.0.smithi090.stdout:8/226: symlink dc/d1f/d49/d3b/l4b 0 2024-04-26T04:42:17.980 INFO:tasks.workunit.client.0.smithi090.stdout:9/204: dwrite db/d16/f46 [0,4194304] 0 2024-04-26T04:42:17.980 INFO:tasks.workunit.client.0.smithi090.stdout:9/205: creat db/f47 x:0 0 0 2024-04-26T04:42:17.981 INFO:tasks.workunit.client.1.smithi104.stdout:4/611: rmdir d4/d7/d1d/d1e/d15/dc6 0 2024-04-26T04:42:17.981 INFO:tasks.workunit.client.1.smithi104.stdout:9/424: rename d11/d1d/d34/f5d to d11/d15/d32/d35/f80 0 2024-04-26T04:42:17.982 INFO:tasks.workunit.client.1.smithi104.stdout:6/682: creat d0/d9/da8/fea x:0 0 0 2024-04-26T04:42:17.982 INFO:tasks.workunit.client.1.smithi104.stdout:6/683: write d0/d9/d13/d41/d82/dc8/f5e [1425768,33877] 0 2024-04-26T04:42:17.982 INFO:tasks.workunit.client.1.smithi104.stdout:6/684: dread - d0/d9/dc6/fb5 zero size 2024-04-26T04:42:17.982 INFO:tasks.workunit.client.1.smithi104.stdout:6/685: creat d0/d9/d13/d41/d82/dc8/dba/daa/feb x:0 0 0 2024-04-26T04:42:17.984 INFO:tasks.workunit.client.0.smithi090.stdout:6/345: mkdir d9/d21/d3b/d86 0 2024-04-26T04:42:17.985 INFO:tasks.workunit.client.1.smithi104.stdout:8/423: truncate d0/d1c/d27/f72 6667448 0 2024-04-26T04:42:17.985 INFO:tasks.workunit.client.1.smithi104.stdout:8/424: chown d0/d1c/c63 4431949 1 2024-04-26T04:42:17.985 INFO:tasks.workunit.client.1.smithi104.stdout:8/425: dread - d0/d33/d35/f36 zero size 2024-04-26T04:42:17.985 INFO:tasks.workunit.client.1.smithi104.stdout:8/426: creat d0/d1c/d27/f86 x:0 0 0 2024-04-26T04:42:17.986 INFO:tasks.workunit.client.1.smithi104.stdout:8/427: read d0/d33/f2c [270013,128902] 0 2024-04-26T04:42:17.986 INFO:tasks.workunit.client.1.smithi104.stdout:8/428: chown d0/d1c/d2d/l54 427 1 2024-04-26T04:42:17.987 INFO:tasks.workunit.client.1.smithi104.stdout:8/429: read d0/f2e [199736,48352] 0 2024-04-26T04:42:17.987 INFO:tasks.workunit.client.1.smithi104.stdout:8/430: stat d0/d50/f68 0 2024-04-26T04:42:17.989 INFO:tasks.workunit.client.1.smithi104.stdout:3/603: unlink d5/d9/dc/l79 0 2024-04-26T04:42:17.992 INFO:tasks.workunit.client.1.smithi104.stdout:9/425: mknod d11/d15/d1b/d6d/c81 0 2024-04-26T04:42:17.994 INFO:tasks.workunit.client.0.smithi090.stdout:8/227: dread fa [0,4194304] 0 2024-04-26T04:42:17.994 INFO:tasks.workunit.client.0.smithi090.stdout:0/273: sync 2024-04-26T04:42:17.997 INFO:tasks.workunit.client.0.smithi090.stdout:0/274: rename d0/l3a to d0/d4b/l61 0 2024-04-26T04:42:17.998 INFO:tasks.workunit.client.1.smithi104.stdout:6/686: rename d0/d9/d13/d41/d96/dac/d8e to d0/d9/d13/d41/d96/dac/dec 0 2024-04-26T04:42:17.998 INFO:tasks.workunit.client.0.smithi090.stdout:0/275: getdents d0/d4b/d5d 0 2024-04-26T04:42:18.000 INFO:tasks.workunit.client.0.smithi090.stdout:0/276: rmdir d0/d4b/d5d/d48 39 2024-04-26T04:42:18.001 INFO:tasks.workunit.client.1.smithi104.stdout:8/431: unlink d0/d1c/d59/f69 0 2024-04-26T04:42:18.002 INFO:tasks.workunit.client.1.smithi104.stdout:3/604: mkdir d5/d9/d80/dbc/dd4 0 2024-04-26T04:42:18.003 INFO:tasks.workunit.client.0.smithi090.stdout:0/277: rename d0/d4b/d5d/l2b to d0/d9/d45/l62 0 2024-04-26T04:42:18.004 INFO:tasks.workunit.client.1.smithi104.stdout:6/687: mkdir d0/d9/d13/d41/d96/ded 0 2024-04-26T04:42:18.004 INFO:tasks.workunit.client.1.smithi104.stdout:6/688: write d0/d9/d13/d44/f97 [4913029,84965] 0 2024-04-26T04:42:18.004 INFO:tasks.workunit.client.1.smithi104.stdout:6/689: readlink d0/d9/d13/d44/d71/l23 0 2024-04-26T04:42:18.005 INFO:tasks.workunit.client.1.smithi104.stdout:5/541: dread d0/d1/d4/d2a/d2b/d37/f57 [0,4194304] 0 2024-04-26T04:42:18.008 INFO:tasks.workunit.client.1.smithi104.stdout:5/542: link d0/d1/d1e/f21 d0/d1/d4/d2a/d6b/d73/d95/fba 0 2024-04-26T04:42:18.008 INFO:tasks.workunit.client.1.smithi104.stdout:5/543: chown d0/d89/fb9 11198388 1 2024-04-26T04:42:18.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/278: write d0/d9/f3e [422176,47134] 0 2024-04-26T04:42:18.020 INFO:tasks.workunit.client.0.smithi090.stdout:0/279: symlink d0/df/l63 0 2024-04-26T04:42:18.020 INFO:tasks.workunit.client.0.smithi090.stdout:0/280: mkdir d0/d9/d64 0 2024-04-26T04:42:18.021 INFO:tasks.workunit.client.1.smithi104.stdout:5/544: dread d0/f6 [0,4194304] 0 2024-04-26T04:42:18.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/281: symlink d0/d9/d45/d53/l65 0 2024-04-26T04:42:18.021 INFO:tasks.workunit.client.1.smithi104.stdout:5/545: creat d0/d1/d4/fbb x:0 0 0 2024-04-26T04:42:18.024 INFO:tasks.workunit.client.1.smithi104.stdout:5/546: dread d0/d1/d64/f19 [0,4194304] 0 2024-04-26T04:42:18.065 INFO:tasks.workunit.client.1.smithi104.stdout:1/571: dwrite d0/d6/d19/d21/d66/d30/d81/d9d/f9f [0,4194304] 0 2024-04-26T04:42:18.069 INFO:tasks.workunit.client.1.smithi104.stdout:8/432: write d0/d1c/d27/d47/f38 [400920,101561] 0 2024-04-26T04:42:18.069 INFO:tasks.workunit.client.1.smithi104.stdout:8/433: write d0/d33/f7b [379117,112313] 0 2024-04-26T04:42:18.069 INFO:tasks.workunit.client.1.smithi104.stdout:8/434: fsync d0/d33/f2c 0 2024-04-26T04:42:18.069 INFO:tasks.workunit.client.1.smithi104.stdout:1/572: mkdir d0/d6/d19/d21/d66/d20/d2a/dbc 0 2024-04-26T04:42:18.070 INFO:tasks.workunit.client.1.smithi104.stdout:1/573: dread d0/d6/d19/d21/d66/d29/d3f/f7b [0,4194304] 0 2024-04-26T04:42:18.070 INFO:tasks.workunit.client.1.smithi104.stdout:1/574: stat d0/d6/d19/d21/d66/d20/d2a/d56/d8c/c8f 0 2024-04-26T04:42:18.072 INFO:tasks.workunit.client.1.smithi104.stdout:1/575: mknod d0/d6/d2e/d32/d4c/db1/cbd 0 2024-04-26T04:42:18.073 INFO:tasks.workunit.client.0.smithi090.stdout:8/228: write dc/d1f/d49/f28 [2295594,24893] 0 2024-04-26T04:42:18.073 INFO:tasks.workunit.client.0.smithi090.stdout:8/229: dread - dc/d1f/d49/d3b/f4a zero size 2024-04-26T04:42:18.074 INFO:tasks.workunit.client.0.smithi090.stdout:8/230: dread - dc/f1d zero size 2024-04-26T04:42:18.074 INFO:tasks.workunit.client.0.smithi090.stdout:8/231: stat fa 0 2024-04-26T04:42:18.074 INFO:tasks.workunit.client.0.smithi090.stdout:8/232: chown dc/d1f/d49/c47 190 1 2024-04-26T04:42:18.074 INFO:tasks.workunit.client.1.smithi104.stdout:1/576: rename d0/d6/d19/d21/d66/l3d to d0/d6/d19/lbe 0 2024-04-26T04:42:18.075 INFO:tasks.workunit.client.0.smithi090.stdout:8/233: mknod dc/d1f/c4c 0 2024-04-26T04:42:18.077 INFO:tasks.workunit.client.0.smithi090.stdout:8/234: mkdir dc/d34/d4d 0 2024-04-26T04:42:18.077 INFO:tasks.workunit.client.0.smithi090.stdout:8/235: truncate dc/d1f/d49/d3b/f4a 918714 0 2024-04-26T04:42:18.077 INFO:tasks.workunit.client.0.smithi090.stdout:8/236: read dc/d1f/f27 [3399396,72636] 0 2024-04-26T04:42:18.077 INFO:tasks.workunit.client.1.smithi104.stdout:8/435: dread d0/d1c/d2d/f32 [0,4194304] 0 2024-04-26T04:42:18.078 INFO:tasks.workunit.client.0.smithi090.stdout:8/237: creat dc/d34/d4d/f4e x:0 0 0 2024-04-26T04:42:18.078 INFO:tasks.workunit.client.0.smithi090.stdout:8/238: chown dc/d1f/l2a 322 1 2024-04-26T04:42:18.078 INFO:tasks.workunit.client.0.smithi090.stdout:8/239: write dc/d1f/f27 [2105845,96120] 0 2024-04-26T04:42:18.080 INFO:tasks.workunit.client.1.smithi104.stdout:8/436: truncate d0/d33/d51/f71 129876 0 2024-04-26T04:42:18.082 INFO:tasks.workunit.client.1.smithi104.stdout:8/437: truncate d0/f1e 1340976 0 2024-04-26T04:42:18.086 INFO:tasks.workunit.client.1.smithi104.stdout:8/438: rename d0/d50/f68 to d0/d33/d35/f87 0 2024-04-26T04:42:18.086 INFO:tasks.workunit.client.1.smithi104.stdout:8/439: readlink d0/d1c/d2d/l54 0 2024-04-26T04:42:18.086 INFO:tasks.workunit.client.1.smithi104.stdout:8/440: fdatasync d0/d1c/d2d/f48 0 2024-04-26T04:42:18.087 INFO:tasks.workunit.client.1.smithi104.stdout:8/441: read - d0/d1c/d27/f5c zero size 2024-04-26T04:42:18.087 INFO:tasks.workunit.client.1.smithi104.stdout:8/442: stat d0/l2 0 2024-04-26T04:42:18.087 INFO:tasks.workunit.client.1.smithi104.stdout:8/443: symlink d0/d33/d51/l88 0 2024-04-26T04:42:18.094 INFO:tasks.workunit.client.1.smithi104.stdout:8/444: dread d0/d1c/d2d/f32 [0,4194304] 0 2024-04-26T04:42:18.096 INFO:tasks.workunit.client.1.smithi104.stdout:8/445: unlink d0/d33/c75 0 2024-04-26T04:42:18.096 INFO:tasks.workunit.client.1.smithi104.stdout:8/446: symlink d0/d1c/d59/d85/l89 0 2024-04-26T04:42:18.097 INFO:tasks.workunit.client.1.smithi104.stdout:8/447: mknod d0/d33/d51/c8a 0 2024-04-26T04:42:18.099 INFO:tasks.workunit.client.1.smithi104.stdout:8/448: symlink d0/l8b 0 2024-04-26T04:42:18.100 INFO:tasks.workunit.client.1.smithi104.stdout:8/449: unlink d0/d33/f82 0 2024-04-26T04:42:18.100 INFO:tasks.workunit.client.1.smithi104.stdout:8/450: write d0/d33/d35/f36 [838801,20646] 0 2024-04-26T04:42:18.101 INFO:tasks.workunit.client.1.smithi104.stdout:8/451: unlink d0/d1c/d27/d46/l4a 0 2024-04-26T04:42:18.101 INFO:tasks.workunit.client.1.smithi104.stdout:8/452: chown d0/ld 0 1 2024-04-26T04:42:18.104 INFO:tasks.workunit.client.1.smithi104.stdout:8/453: rename d0/d50/f83 to d0/d1c/d2d/f8c 0 2024-04-26T04:42:18.105 INFO:tasks.workunit.client.1.smithi104.stdout:8/454: mknod d0/d50/c8d 0 2024-04-26T04:42:18.105 INFO:tasks.workunit.client.1.smithi104.stdout:8/455: chown d0/d1c/d2d/l3a 38899 1 2024-04-26T04:42:18.105 INFO:tasks.workunit.client.1.smithi104.stdout:8/456: readlink d0/d1c/d27/l4b 0 2024-04-26T04:42:18.105 INFO:tasks.workunit.client.1.smithi104.stdout:4/612: dwrite d4/d7/d1d/d1e/d8f/db6/d7d/fa1 [0,4194304] 0 2024-04-26T04:42:18.105 INFO:tasks.workunit.client.1.smithi104.stdout:8/457: mknod d0/d1c/d2d/d7a/c8e 0 2024-04-26T04:42:18.106 INFO:tasks.workunit.client.0.smithi090.stdout:5/319: dwrite d4/d1a/d1f/f2b [4194304,4194304] 0 2024-04-26T04:42:18.106 INFO:tasks.workunit.client.0.smithi090.stdout:5/320: fsync d4/d1a/d3f/f52 0 2024-04-26T04:42:18.106 INFO:tasks.workunit.client.0.smithi090.stdout:2/247: dwrite db/dc/f3c [4194304,4194304] 0 2024-04-26T04:42:18.107 INFO:tasks.workunit.client.1.smithi104.stdout:3/605: dwrite d5/db2/f1b [0,4194304] 0 2024-04-26T04:42:18.110 INFO:tasks.workunit.client.1.smithi104.stdout:4/613: mkdir d4/d7/d1d/d1e/d15/dd8 0 2024-04-26T04:42:18.110 INFO:tasks.workunit.client.1.smithi104.stdout:4/614: chown d4/d7/d1d/d2e/d74/dbd/dd3/fd7 6 1 2024-04-26T04:42:18.110 INFO:tasks.workunit.client.0.smithi090.stdout:2/248: link db/d11/d14/d25/f37 db/d3f/f53 0 2024-04-26T04:42:18.112 INFO:tasks.workunit.client.0.smithi090.stdout:2/249: rename db/d3f/f53 to db/d11/d14/d25/d41/f54 0 2024-04-26T04:42:18.113 INFO:tasks.workunit.client.0.smithi090.stdout:2/250: truncate db/d11/d14/d27/d2f/f32 4285676 0 2024-04-26T04:42:18.113 INFO:tasks.workunit.client.1.smithi104.stdout:8/458: dread d0/d33/f1f [0,4194304] 0 2024-04-26T04:42:18.114 INFO:tasks.workunit.client.1.smithi104.stdout:3/606: mkdir d5/d9/dd5 0 2024-04-26T04:42:18.115 INFO:tasks.workunit.client.0.smithi090.stdout:2/251: link db/dc/l1a db/d3f/l55 0 2024-04-26T04:42:18.115 INFO:tasks.workunit.client.0.smithi090.stdout:2/252: stat db/dc/l29 0 2024-04-26T04:42:18.116 INFO:tasks.workunit.client.0.smithi090.stdout:8/240: dwrite dc/f1d [0,4194304] 0 2024-04-26T04:42:18.116 INFO:tasks.workunit.client.0.smithi090.stdout:8/241: fsync dc/f1a 0 2024-04-26T04:42:18.116 INFO:tasks.workunit.client.0.smithi090.stdout:6/346: dwrite d9/d1c/f47 [0,4194304] 0 2024-04-26T04:42:18.116 INFO:tasks.workunit.client.0.smithi090.stdout:6/347: fsync f1 0 2024-04-26T04:42:18.116 INFO:tasks.workunit.client.0.smithi090.stdout:6/348: write d9/d3c/f64 [595855,130411] 0 2024-04-26T04:42:18.117 INFO:tasks.workunit.client.1.smithi104.stdout:9/426: dwrite d11/d15/f3d [0,4194304] 0 2024-04-26T04:42:18.117 INFO:tasks.workunit.client.1.smithi104.stdout:5/547: dwrite d0/d41/f74 [0,4194304] 0 2024-04-26T04:42:18.117 INFO:tasks.workunit.client.1.smithi104.stdout:6/690: dwrite d0/d9/d13/d41/d82/dc8/d80/f75 [4194304,4194304] 0 2024-04-26T04:42:18.117 INFO:tasks.workunit.client.1.smithi104.stdout:5/548: dread - d0/d1/d4/d2a/d2b/d5c/f8a zero size 2024-04-26T04:42:18.117 INFO:tasks.workunit.client.1.smithi104.stdout:9/427: chown d11/l1c 14074 1 2024-04-26T04:42:18.117 INFO:tasks.workunit.client.1.smithi104.stdout:5/549: creat d0/d1/d1e/d98/fbc x:0 0 0 2024-04-26T04:42:18.117 INFO:tasks.workunit.client.1.smithi104.stdout:6/691: write d0/d9/f87 [4490495,130311] 0 2024-04-26T04:42:18.118 INFO:tasks.workunit.client.0.smithi090.stdout:2/253: link db/d11/d14/d1b/f51 db/d11/d14/d27/d28/f56 0 2024-04-26T04:42:18.124 INFO:tasks.workunit.client.0.smithi090.stdout:8/242: unlink dc/d1f/l2f 0 2024-04-26T04:42:18.125 INFO:tasks.workunit.client.1.smithi104.stdout:4/615: mknod d4/d7/d1d/d8e/cd9 0 2024-04-26T04:42:18.125 INFO:tasks.workunit.client.1.smithi104.stdout:4/616: creat d4/d7/d1d/d1e/d24/d4a/fda x:0 0 0 2024-04-26T04:42:18.125 INFO:tasks.workunit.client.1.smithi104.stdout:4/617: creat d4/d7/d1d/d1e/d24/d48/fdb x:0 0 0 2024-04-26T04:42:18.128 INFO:tasks.workunit.client.1.smithi104.stdout:3/607: write d5/d9/dc/d29/d7d/d8e/fbe [2066748,60371] 0 2024-04-26T04:42:18.128 INFO:tasks.workunit.client.1.smithi104.stdout:3/608: chown d5/d9/dc/d32/f77 7 1 2024-04-26T04:42:18.128 INFO:tasks.workunit.client.1.smithi104.stdout:3/609: readlink d5/d9/dc/l11 0 2024-04-26T04:42:18.129 INFO:tasks.workunit.client.0.smithi090.stdout:0/282: dread d0/d9/f3e [0,4194304] 0 2024-04-26T04:42:18.129 INFO:tasks.workunit.client.0.smithi090.stdout:0/283: write d0/f4f [909673,71477] 0 2024-04-26T04:42:18.129 INFO:tasks.workunit.client.0.smithi090.stdout:0/284: readlink d0/l50 0 2024-04-26T04:42:18.130 INFO:tasks.workunit.client.1.smithi104.stdout:8/459: rename d0/d33/d35/f36 to d0/d50/f8f 0 2024-04-26T04:42:18.130 INFO:tasks.workunit.client.1.smithi104.stdout:8/460: creat d0/d1c/d27/f90 x:0 0 0 2024-04-26T04:42:18.130 INFO:tasks.workunit.client.1.smithi104.stdout:8/461: dread - d0/d1c/d2d/f8c zero size 2024-04-26T04:42:18.131 INFO:tasks.workunit.client.1.smithi104.stdout:8/462: truncate d0/d33/f2c 795519 0 2024-04-26T04:42:18.131 INFO:tasks.workunit.client.1.smithi104.stdout:8/463: stat d0/d33/f2b 0 2024-04-26T04:42:18.131 INFO:tasks.workunit.client.1.smithi104.stdout:8/464: creat d0/d1c/f91 x:0 0 0 2024-04-26T04:42:18.131 INFO:tasks.workunit.client.1.smithi104.stdout:8/465: write d0/d33/d35/f45 [2832856,95527] 0 2024-04-26T04:42:18.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/349: rename d9/d21/d3b/d62/c3f to d9/d21/c87 0 2024-04-26T04:42:18.131 INFO:tasks.workunit.client.0.smithi090.stdout:6/350: truncate d9/d1c/d4d/f85 89329 0 2024-04-26T04:42:18.134 INFO:tasks.workunit.client.1.smithi104.stdout:5/550: dread d0/f22 [4194304,4194304] 0 2024-04-26T04:42:18.134 INFO:tasks.workunit.client.0.smithi090.stdout:2/254: mknod db/d11/d14/d27/d2f/d42/c57 0 2024-04-26T04:42:18.135 INFO:tasks.workunit.client.0.smithi090.stdout:5/321: dwrite d4/d1a/d3f/f44 [0,4194304] 0 2024-04-26T04:42:18.136 INFO:tasks.workunit.client.0.smithi090.stdout:0/285: truncate d0/d9/f17 2387235 0 2024-04-26T04:42:18.138 INFO:tasks.workunit.client.0.smithi090.stdout:6/351: mknod d9/d21/c88 0 2024-04-26T04:42:18.138 INFO:tasks.workunit.client.0.smithi090.stdout:6/352: write d9/f68 [242141,56816] 0 2024-04-26T04:42:18.138 INFO:tasks.workunit.client.0.smithi090.stdout:6/353: write d9/d21/d40/d4c/f59 [843482,20010] 0 2024-04-26T04:42:18.141 INFO:tasks.workunit.client.0.smithi090.stdout:6/354: dread d9/d1c/d31/f3a [0,4194304] 0 2024-04-26T04:42:18.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/286: link d0/de/f39 d0/d9/d45/f66 0 2024-04-26T04:42:18.143 INFO:tasks.workunit.client.0.smithi090.stdout:6/355: link d9/d21/d3b/c81 d9/d1c/d31/d69/c89 0 2024-04-26T04:42:18.144 INFO:tasks.workunit.client.0.smithi090.stdout:0/287: chown d0/d4b/d5d/l32 857608947 1 2024-04-26T04:42:18.146 INFO:tasks.workunit.client.0.smithi090.stdout:6/356: mknod d9/d21/c8a 0 2024-04-26T04:42:18.146 INFO:tasks.workunit.client.0.smithi090.stdout:6/357: dread - d9/d21/d40/d4c/f7b zero size 2024-04-26T04:42:18.146 INFO:tasks.workunit.client.0.smithi090.stdout:6/358: readlink d9/l5e 0 2024-04-26T04:42:18.148 INFO:tasks.workunit.client.0.smithi090.stdout:2/255: write db/d11/f17 [2941973,72163] 0 2024-04-26T04:42:18.149 INFO:tasks.workunit.client.0.smithi090.stdout:0/288: readlink d0/d4b/d5d/l5f 0 2024-04-26T04:42:18.149 INFO:tasks.workunit.client.0.smithi090.stdout:0/289: fdatasync d0/df/f25 0 2024-04-26T04:42:18.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/256: write db/d11/d14/d25/f3e [498698,65302] 0 2024-04-26T04:42:18.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/257: dread - db/d11/f44 zero size 2024-04-26T04:42:18.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/258: creat db/d11/d14/d27/d4b/f58 x:0 0 0 2024-04-26T04:42:18.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/259: truncate db/d11/d14/d25/f49 982210 0 2024-04-26T04:42:18.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/260: dread - db/d11/d14/d27/d4b/f58 zero size 2024-04-26T04:42:18.150 INFO:tasks.workunit.client.0.smithi090.stdout:2/261: chown db/d11/d14 9410 1 2024-04-26T04:42:18.150 INFO:tasks.workunit.client.0.smithi090.stdout:5/322: dread d4/fe [0,4194304] 0 2024-04-26T04:42:18.152 INFO:tasks.workunit.client.0.smithi090.stdout:3/173: sync 2024-04-26T04:42:18.152 INFO:tasks.workunit.client.0.smithi090.stdout:4/297: sync 2024-04-26T04:42:18.152 INFO:tasks.workunit.client.0.smithi090.stdout:3/174: chown d1/f8 22407 1 2024-04-26T04:42:18.152 INFO:tasks.workunit.client.0.smithi090.stdout:4/298: chown d3/db/d12/d20/d24/d30/c42 19 1 2024-04-26T04:42:18.154 INFO:tasks.workunit.client.0.smithi090.stdout:3/175: symlink d1/d6/d2c/l3b 0 2024-04-26T04:42:18.157 INFO:tasks.workunit.client.1.smithi104.stdout:6/692: dwrite d0/d9/d13/d44/d71/fa3 [0,4194304] 0 2024-04-26T04:42:18.157 INFO:tasks.workunit.client.1.smithi104.stdout:5/551: chown d0/d1/d4/d2a/d2b/d37/f9c 6 1 2024-04-26T04:42:18.160 INFO:tasks.workunit.client.0.smithi090.stdout:3/176: dread d1/d17/f1f [0,4194304] 0 2024-04-26T04:42:18.163 INFO:tasks.workunit.client.1.smithi104.stdout:6/693: creat d0/d9/d13/d41/d82/dc8/d80/fee x:0 0 0 2024-04-26T04:42:18.166 INFO:tasks.workunit.client.1.smithi104.stdout:9/428: write d11/d15/f3d [4530583,115945] 0 2024-04-26T04:42:18.166 INFO:tasks.workunit.client.1.smithi104.stdout:9/429: chown d11/f16 630013 1 2024-04-26T04:42:18.167 INFO:tasks.workunit.client.1.smithi104.stdout:9/430: creat d11/d15/d32/d35/d7b/f82 x:0 0 0 2024-04-26T04:42:18.169 INFO:tasks.workunit.client.0.smithi090.stdout:8/243: dwrite fa [0,4194304] 0 2024-04-26T04:42:18.169 INFO:tasks.workunit.client.0.smithi090.stdout:2/262: fdatasync db/d11/d14/d25/f3e 0 2024-04-26T04:42:18.172 INFO:tasks.workunit.client.1.smithi104.stdout:3/610: dwrite d5/d9/dc/d29/f58 [0,4194304] 0 2024-04-26T04:42:18.172 INFO:tasks.workunit.client.1.smithi104.stdout:3/611: fdatasync d5/d9/dc/d69/f3e 0 2024-04-26T04:42:18.172 INFO:tasks.workunit.client.1.smithi104.stdout:3/612: fsync d5/d9/dc/d29/faa 0 2024-04-26T04:42:18.174 INFO:tasks.workunit.client.0.smithi090.stdout:8/244: mkdir dc/d1f/d49/d4f 0 2024-04-26T04:42:18.176 INFO:tasks.workunit.client.1.smithi104.stdout:3/613: mkdir d5/d9/dc/d29/dd6 0 2024-04-26T04:42:18.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/245: mknod dc/d1f/c50 0 2024-04-26T04:42:18.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/246: readlink dc/d1f/l37 0 2024-04-26T04:42:18.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/247: write dc/d1f/d3a/f44 [792417,111083] 0 2024-04-26T04:42:18.177 INFO:tasks.workunit.client.1.smithi104.stdout:2/740: sync 2024-04-26T04:42:18.178 INFO:tasks.workunit.client.0.smithi090.stdout:7/289: sync 2024-04-26T04:42:18.179 INFO:tasks.workunit.client.0.smithi090.stdout:2/263: dread db/d11/d14/d25/f3a [0,4194304] 0 2024-04-26T04:42:18.179 INFO:tasks.workunit.client.0.smithi090.stdout:8/248: link dc/l26 dc/d34/l51 0 2024-04-26T04:42:18.180 INFO:tasks.workunit.client.1.smithi104.stdout:8/466: dwrite d0/f2f [4194304,4194304] 0 2024-04-26T04:42:18.180 INFO:tasks.workunit.client.1.smithi104.stdout:8/467: truncate d0/d50/f8f 1273767 0 2024-04-26T04:42:18.180 INFO:tasks.workunit.client.1.smithi104.stdout:8/468: creat d0/d33/d51/f92 x:0 0 0 2024-04-26T04:42:18.181 INFO:tasks.workunit.client.0.smithi090.stdout:6/359: dwrite d9/d1c/d4d/d6d/d66/f75 [0,4194304] 0 2024-04-26T04:42:18.182 INFO:tasks.workunit.client.0.smithi090.stdout:7/290: truncate d0/d1/d4/d3e/f4b 3017003 0 2024-04-26T04:42:18.182 INFO:tasks.workunit.client.1.smithi104.stdout:8/469: dread d0/d1c/f4c [0,4194304] 0 2024-04-26T04:42:18.183 INFO:tasks.workunit.client.0.smithi090.stdout:2/264: truncate db/dc/f12 828670 0 2024-04-26T04:42:18.183 INFO:tasks.workunit.client.0.smithi090.stdout:2/265: stat l5 0 2024-04-26T04:42:18.185 INFO:tasks.workunit.client.0.smithi090.stdout:8/249: creat dc/d1f/d49/d39/f52 x:0 0 0 2024-04-26T04:42:18.187 INFO:tasks.workunit.client.0.smithi090.stdout:4/299: dwrite d3/db/d12/d20/d24/d30/f49 [0,4194304] 0 2024-04-26T04:42:18.187 INFO:tasks.workunit.client.0.smithi090.stdout:4/300: write d3/f48 [264220,52891] 0 2024-04-26T04:42:18.187 INFO:tasks.workunit.client.0.smithi090.stdout:6/360: write d9/d1c/d4d/d6d/f45 [176480,122088] 0 2024-04-26T04:42:18.187 INFO:tasks.workunit.client.0.smithi090.stdout:4/301: chown d3/db/d12/d20/d45 0 1 2024-04-26T04:42:18.187 INFO:tasks.workunit.client.0.smithi090.stdout:7/291: mknod d0/d1/c69 0 2024-04-26T04:42:18.187 INFO:tasks.workunit.client.0.smithi090.stdout:7/292: fdatasync d0/db/d53/fa 0 2024-04-26T04:42:18.188 INFO:tasks.workunit.client.0.smithi090.stdout:2/266: truncate db/f33 812848 0 2024-04-26T04:42:18.188 INFO:tasks.workunit.client.0.smithi090.stdout:2/267: chown db/dc/l1a 1990387 1 2024-04-26T04:42:18.188 INFO:tasks.workunit.client.0.smithi090.stdout:2/268: truncate db/d11/d14/d25/f3a 237824 0 2024-04-26T04:42:18.189 INFO:tasks.workunit.client.1.smithi104.stdout:5/552: read d0/d1/d4/f63 [3225391,96996] 0 2024-04-26T04:42:18.189 INFO:tasks.workunit.client.1.smithi104.stdout:5/553: truncate d0/d1/d4/d2a/d6b/d91/fb0 274851 0 2024-04-26T04:42:18.189 INFO:tasks.workunit.client.1.smithi104.stdout:8/470: link d0/d33/d35/l3d d0/d33/d35/d6d/l93 0 2024-04-26T04:42:18.189 INFO:tasks.workunit.client.1.smithi104.stdout:5/554: write d0/d1/d64/d26/d36/da7/fa8 [4276414,72061] 0 2024-04-26T04:42:18.189 INFO:tasks.workunit.client.1.smithi104.stdout:8/471: dread d0/d33/f2c [0,4194304] 0 2024-04-26T04:42:18.189 INFO:tasks.workunit.client.0.smithi090.stdout:8/250: rename dc/d1f/c42 to dc/d1f/d49/d39/c53 0 2024-04-26T04:42:18.191 INFO:tasks.workunit.client.1.smithi104.stdout:5/555: mkdir d0/d1/d4/d2a/d6b/d91/dbd 0 2024-04-26T04:42:18.192 INFO:tasks.workunit.client.0.smithi090.stdout:6/361: mknod d9/d3c/c8b 0 2024-04-26T04:42:18.196 INFO:tasks.workunit.client.0.smithi090.stdout:7/293: symlink d0/db/d15/d30/l6a 0 2024-04-26T04:42:18.196 INFO:tasks.workunit.client.0.smithi090.stdout:4/302: rename d3/db/d12/f17 to d3/db/d12/d20/d24/d30/f5a 0 2024-04-26T04:42:18.197 INFO:tasks.workunit.client.1.smithi104.stdout:2/741: write d6/d1b/d51/d6a/fc5 [2823308,81824] 0 2024-04-26T04:42:18.197 INFO:tasks.workunit.client.1.smithi104.stdout:2/742: dread - d6/d1b/d93/fe5 zero size 2024-04-26T04:42:18.199 INFO:tasks.workunit.client.0.smithi090.stdout:7/294: creat d0/d1/d4/f6b x:0 0 0 2024-04-26T04:42:18.199 INFO:tasks.workunit.client.0.smithi090.stdout:7/295: stat d0/d1/d4/d43/d4a 0 2024-04-26T04:42:18.199 INFO:tasks.workunit.client.1.smithi104.stdout:2/743: mkdir d6/d8/df6 0 2024-04-26T04:42:18.199 INFO:tasks.workunit.client.1.smithi104.stdout:2/744: fsync d6/d8/d5a/fb3 0 2024-04-26T04:42:18.200 INFO:tasks.workunit.client.0.smithi090.stdout:4/303: creat d3/db/d12/f5b x:0 0 0 2024-04-26T04:42:18.200 INFO:tasks.workunit.client.0.smithi090.stdout:5/323: dwrite d4/d1a/f17 [0,4194304] 0 2024-04-26T04:42:18.201 INFO:tasks.workunit.client.0.smithi090.stdout:8/251: creat dc/d1f/d49/d3b/d3d/f54 x:0 0 0 2024-04-26T04:42:18.201 INFO:tasks.workunit.client.1.smithi104.stdout:9/431: dwrite d11/d15/d32/d35/f5e [0,4194304] 0 2024-04-26T04:42:18.201 INFO:tasks.workunit.client.1.smithi104.stdout:9/432: creat d11/d15/d32/f83 x:0 0 0 2024-04-26T04:42:18.202 INFO:tasks.workunit.client.0.smithi090.stdout:7/296: link d0/d1/d4/d43/c49 d0/d1/d4/c6c 0 2024-04-26T04:42:18.204 INFO:tasks.workunit.client.0.smithi090.stdout:3/177: dwrite d1/d9/fb [4194304,4194304] 0 2024-04-26T04:42:18.204 INFO:tasks.workunit.client.0.smithi090.stdout:3/178: fsync d1/d17/f1f 0 2024-04-26T04:42:18.205 INFO:tasks.workunit.client.1.smithi104.stdout:9/433: mkdir d11/d53/d84 0 2024-04-26T04:42:18.205 INFO:tasks.workunit.client.1.smithi104.stdout:9/434: write d11/f12 [4211222,80367] 0 2024-04-26T04:42:18.205 INFO:tasks.workunit.client.1.smithi104.stdout:9/435: chown d11/d1d/d34/d69/f79 0 1 2024-04-26T04:42:18.207 INFO:tasks.workunit.client.0.smithi090.stdout:5/324: link d4/d1a/d3f/f52 d4/d1a/d1f/d53/d59/f69 0 2024-04-26T04:42:18.209 INFO:tasks.workunit.client.0.smithi090.stdout:8/252: mkdir dc/d1f/d49/d39/d55 0 2024-04-26T04:42:18.209 INFO:tasks.workunit.client.0.smithi090.stdout:8/253: fdatasync dc/d1f/f36 0 2024-04-26T04:42:18.210 INFO:tasks.workunit.client.0.smithi090.stdout:7/297: mknod d0/d1/d4/d43/d4a/c6d 0 2024-04-26T04:42:18.212 INFO:tasks.workunit.client.1.smithi104.stdout:2/745: write d6/d8/d3e/f49 [814303,61408] 0 2024-04-26T04:42:18.212 INFO:tasks.workunit.client.1.smithi104.stdout:2/746: getdents d6/d1b/d93 0 2024-04-26T04:42:18.216 INFO:tasks.workunit.client.0.smithi090.stdout:3/179: write d1/f11 [3755007,35610] 0 2024-04-26T04:42:18.216 INFO:tasks.workunit.client.0.smithi090.stdout:4/304: dread d3/db/d12/d20/d24/f52 [0,4194304] 0 2024-04-26T04:42:18.217 INFO:tasks.workunit.client.1.smithi104.stdout:3/614: rename d5/d9/dc/d29 to d5/d9/dc/d32/d48/dac/dd7 0 2024-04-26T04:42:18.217 INFO:tasks.workunit.client.1.smithi104.stdout:3/615: fsync d5/db2/d72/f99 0 2024-04-26T04:42:18.223 INFO:tasks.workunit.client.1.smithi104.stdout:5/556: rename d0/d89/fb9 to d0/d1/d64/d26/d36/d72/fbe 0 2024-04-26T04:42:18.226 INFO:tasks.workunit.client.1.smithi104.stdout:8/472: dwrite d0/d1c/f4c [0,4194304] 0 2024-04-26T04:42:18.229 INFO:tasks.workunit.client.1.smithi104.stdout:9/436: rename d11/d15/d32/f66 to d11/d53/f85 0 2024-04-26T04:42:18.230 INFO:tasks.workunit.client.0.smithi090.stdout:0/290: dwrite d0/f4f [0,4194304] 0 2024-04-26T04:42:18.230 INFO:tasks.workunit.client.0.smithi090.stdout:0/291: write d0/d9/f17 [1123139,38505] 0 2024-04-26T04:42:18.231 INFO:tasks.workunit.client.0.smithi090.stdout:5/325: creat d4/d1a/d1f/d53/d59/d5e/f6a x:0 0 0 2024-04-26T04:42:18.232 INFO:tasks.workunit.client.0.smithi090.stdout:5/326: fsync d4/f1e 0 2024-04-26T04:42:18.234 INFO:tasks.workunit.client.1.smithi104.stdout:2/747: rename d6/d1b/d68/d81/c96 to d6/d1b/d51/db8/cf7 0 2024-04-26T04:42:18.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/254: rename fa to dc/d1f/d49/d39/d55/f56 0 2024-04-26T04:42:18.235 INFO:tasks.workunit.client.0.smithi090.stdout:8/255: read - dc/f30 zero size 2024-04-26T04:42:18.236 INFO:tasks.workunit.client.0.smithi090.stdout:7/298: creat d0/db/d15/d29/f6e x:0 0 0 2024-04-26T04:42:18.237 INFO:tasks.workunit.client.0.smithi090.stdout:6/362: dwrite d9/d21/d3b/d62/f25 [0,4194304] 0 2024-04-26T04:42:18.238 INFO:tasks.workunit.client.1.smithi104.stdout:9/437: unlink d11/d15/d32/l5c 0 2024-04-26T04:42:18.238 INFO:tasks.workunit.client.1.smithi104.stdout:9/438: chown d11/d1d/d34/d69/f79 33690 1 2024-04-26T04:42:18.238 INFO:tasks.workunit.client.1.smithi104.stdout:9/439: truncate d11/d15/d1b/f72 610230 0 2024-04-26T04:42:18.238 INFO:tasks.workunit.client.1.smithi104.stdout:8/473: getdents d0 0 2024-04-26T04:42:18.240 INFO:tasks.workunit.client.1.smithi104.stdout:5/557: rename d0/d1/d4/d2a/d2b/d37/d48/c61 to d0/d41/d4f/cbf 0 2024-04-26T04:42:18.240 INFO:tasks.workunit.client.1.smithi104.stdout:5/558: read - d0/d1/d1e/d98/fbc zero size 2024-04-26T04:42:18.241 INFO:tasks.workunit.client.1.smithi104.stdout:5/559: dread - d0/d1/d64/d26/f44 zero size 2024-04-26T04:42:18.241 INFO:tasks.workunit.client.1.smithi104.stdout:9/440: creat d11/d53/f86 x:0 0 0 2024-04-26T04:42:18.242 INFO:tasks.workunit.client.1.smithi104.stdout:5/560: getdents d0/d1/d4/d2a/d6b/d91 0 2024-04-26T04:42:18.246 INFO:tasks.workunit.client.0.smithi090.stdout:7/299: link d0/db/d15/c2a d0/db/d15/d30/c6f 0 2024-04-26T04:42:18.246 INFO:tasks.workunit.client.0.smithi090.stdout:7/300: chown d0/db/d53/l41 14869765 1 2024-04-26T04:42:18.249 INFO:tasks.workunit.client.0.smithi090.stdout:6/363: creat d9/d1c/d4d/d58/f8c x:0 0 0 2024-04-26T04:42:18.249 INFO:tasks.workunit.client.0.smithi090.stdout:5/327: write d4/d1a/df/d36/f4d [3424684,28490] 0 2024-04-26T04:42:18.252 INFO:tasks.workunit.client.0.smithi090.stdout:2/269: dwrite db/d11/d14/d25/f37 [0,4194304] 0 2024-04-26T04:42:18.252 INFO:tasks.workunit.client.0.smithi090.stdout:6/364: rename d9/d21/d40/d4c/d6f/l73 to d9/d1c/d31/d69/l8d 0 2024-04-26T04:42:18.252 INFO:tasks.workunit.client.0.smithi090.stdout:6/365: creat d9/d1c/d4d/f8e x:0 0 0 2024-04-26T04:42:18.254 INFO:tasks.workunit.client.0.smithi090.stdout:5/328: mkdir d4/d1a/df/d36/d6b 0 2024-04-26T04:42:18.257 INFO:tasks.workunit.client.0.smithi090.stdout:2/270: mkdir db/d59 0 2024-04-26T04:42:18.257 INFO:tasks.workunit.client.0.smithi090.stdout:2/271: write db/d11/d14/d25/f4e [725388,53342] 0 2024-04-26T04:42:18.257 INFO:tasks.workunit.client.0.smithi090.stdout:2/272: write db/d11/d14/d25/f49 [1045460,71348] 0 2024-04-26T04:42:18.259 INFO:tasks.workunit.client.0.smithi090.stdout:6/366: creat d9/d1c/d4d/d6d/f8f x:0 0 0 2024-04-26T04:42:18.260 INFO:tasks.workunit.client.0.smithi090.stdout:5/329: symlink d4/d1a/df/d36/l6c 0 2024-04-26T04:42:18.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/273: creat db/f5a x:0 0 0 2024-04-26T04:42:18.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/274: chown db/d11/d14/d1b/d20/d4c 240438507 1 2024-04-26T04:42:18.263 INFO:tasks.workunit.client.0.smithi090.stdout:2/275: rmdir db/d3f 39 2024-04-26T04:42:18.263 INFO:tasks.workunit.client.0.smithi090.stdout:2/276: write db/d11/d14/d27/d2f/f32 [3910090,125837] 0 2024-04-26T04:42:18.264 INFO:tasks.workunit.client.0.smithi090.stdout:7/301: dread d0/db/d15/d29/fc [0,4194304] 0 2024-04-26T04:42:18.275 INFO:tasks.workunit.client.0.smithi090.stdout:3/180: dwrite d1/d17/f1f [0,4194304] 0 2024-04-26T04:42:18.276 INFO:tasks.workunit.client.0.smithi090.stdout:4/305: dwrite d3/db/f19 [0,4194304] 0 2024-04-26T04:42:18.277 INFO:tasks.workunit.client.0.smithi090.stdout:6/367: write d9/d21/d3b/d62/f26 [72773,96361] 0 2024-04-26T04:42:18.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/330: write d4/f1e [1676652,72859] 0 2024-04-26T04:42:18.279 INFO:tasks.workunit.client.0.smithi090.stdout:4/306: mkdir d3/d13/d57/d5c 0 2024-04-26T04:42:18.279 INFO:tasks.workunit.client.0.smithi090.stdout:6/368: truncate d9/d1c/d4d/d58/f6c 1639526 0 2024-04-26T04:42:18.281 INFO:tasks.workunit.client.0.smithi090.stdout:0/292: dwrite d0/f38 [4194304,4194304] 0 2024-04-26T04:42:18.281 INFO:tasks.workunit.client.0.smithi090.stdout:0/293: chown d0/d9 127 1 2024-04-26T04:42:18.282 INFO:tasks.workunit.client.1.smithi104.stdout:3/616: dwrite d5/d9/dc/d32/d48/dac/dd7/d49/f82 [0,4194304] 0 2024-04-26T04:42:18.282 INFO:tasks.workunit.client.1.smithi104.stdout:3/617: stat d5/d9/dc/d32/d48/dac/la6 0 2024-04-26T04:42:18.283 INFO:tasks.workunit.client.0.smithi090.stdout:5/331: creat d4/d1a/df/f6d x:0 0 0 2024-04-26T04:42:18.283 INFO:tasks.workunit.client.0.smithi090.stdout:4/307: getdents d3/db/d12/d20/d45 0 2024-04-26T04:42:18.284 INFO:tasks.workunit.client.1.smithi104.stdout:8/474: dwrite d0/d1c/f7e [0,4194304] 0 2024-04-26T04:42:18.284 INFO:tasks.workunit.client.1.smithi104.stdout:3/618: mknod d5/d9/dc/d32/d48/dac/dd7/d7d/cd8 0 2024-04-26T04:42:18.287 INFO:tasks.workunit.client.0.smithi090.stdout:0/294: getdents d0/d9/d45/d53 0 2024-04-26T04:42:18.287 INFO:tasks.workunit.client.1.smithi104.stdout:3/619: read d5/db2/f3b [939209,41832] 0 2024-04-26T04:42:18.288 INFO:tasks.workunit.client.1.smithi104.stdout:5/561: dwrite d0/d41/f74 [0,4194304] 0 2024-04-26T04:42:18.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/181: dread d1/d9/fa [0,4194304] 0 2024-04-26T04:42:18.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/182: dread - d1/d17/f1a zero size 2024-04-26T04:42:18.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/183: creat d1/f3c x:0 0 0 2024-04-26T04:42:18.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/184: rename d1/d9 to d1/d9/d3d 22 2024-04-26T04:42:18.290 INFO:tasks.workunit.client.1.smithi104.stdout:9/441: dwrite d11/d15/d32/d35/f65 [0,4194304] 0 2024-04-26T04:42:18.293 INFO:tasks.workunit.client.1.smithi104.stdout:3/620: mkdir d5/d9/dc/d32/dd9 0 2024-04-26T04:42:18.293 INFO:tasks.workunit.client.1.smithi104.stdout:3/621: chown d5/db2/d60/faf 1752428 1 2024-04-26T04:42:18.293 INFO:tasks.workunit.client.1.smithi104.stdout:3/622: write d5/db2/d46/f4d [728933,13486] 0 2024-04-26T04:42:18.293 INFO:tasks.workunit.client.1.smithi104.stdout:3/623: chown d5/d9/fe 100314939 1 2024-04-26T04:42:18.293 INFO:tasks.workunit.client.1.smithi104.stdout:3/624: write d5/d9/dc/f21 [1367894,73368] 0 2024-04-26T04:42:18.296 INFO:tasks.workunit.client.0.smithi090.stdout:3/185: creat d1/d17/f3e x:0 0 0 2024-04-26T04:42:18.296 INFO:tasks.workunit.client.0.smithi090.stdout:3/186: write d1/d9/fa [2980073,20204] 0 2024-04-26T04:42:18.296 INFO:tasks.workunit.client.0.smithi090.stdout:3/187: chown d1/cd 1550 1 2024-04-26T04:42:18.297 INFO:tasks.workunit.client.0.smithi090.stdout:6/369: write d9/f24 [365512,129852] 0 2024-04-26T04:42:18.297 INFO:tasks.workunit.client.0.smithi090.stdout:6/370: truncate d9/d1c/d31/d79/f7e 350256 0 2024-04-26T04:42:18.298 INFO:tasks.workunit.client.0.smithi090.stdout:1/254: sync 2024-04-26T04:42:18.299 INFO:tasks.workunit.client.0.smithi090.stdout:9/206: sync 2024-04-26T04:42:18.299 INFO:tasks.workunit.client.0.smithi090.stdout:9/207: chown db/dd/c15 41892938 1 2024-04-26T04:42:18.299 INFO:tasks.workunit.client.0.smithi090.stdout:9/208: write db/f47 [672870,80368] 0 2024-04-26T04:42:18.302 INFO:tasks.workunit.client.1.smithi104.stdout:8/475: read d0/f1a [8228035,16129] 0 2024-04-26T04:42:18.302 INFO:tasks.workunit.client.1.smithi104.stdout:8/476: chown d0/d1c/d27/f86 14568 1 2024-04-26T04:42:18.303 INFO:tasks.workunit.client.0.smithi090.stdout:3/188: symlink d1/l3f 0 2024-04-26T04:42:18.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/256: rmdir dc/d1f/d49/d39 39 2024-04-26T04:42:18.304 INFO:tasks.workunit.client.0.smithi090.stdout:5/332: write d4/d1a/d1f/f2b [2195542,93789] 0 2024-04-26T04:42:18.304 INFO:tasks.workunit.client.1.smithi104.stdout:9/442: mkdir d11/d1d/d87 0 2024-04-26T04:42:18.305 INFO:tasks.workunit.client.0.smithi090.stdout:1/255: rename d0/f46 to d0/d4/d7/d4a/f51 0 2024-04-26T04:42:18.308 INFO:tasks.workunit.client.0.smithi090.stdout:4/308: write d3/db/d12/d20/d2d/d2e/f35 [2203857,95649] 0 2024-04-26T04:42:18.308 INFO:tasks.workunit.client.0.smithi090.stdout:8/257: write dc/d1f/d49/d3b/f4a [1514600,94472] 0 2024-04-26T04:42:18.308 INFO:tasks.workunit.client.0.smithi090.stdout:8/258: read - dc/d1f/f35 zero size 2024-04-26T04:42:18.308 INFO:tasks.workunit.client.0.smithi090.stdout:8/259: truncate dc/d1f/d3a/f44 1474235 0 2024-04-26T04:42:18.309 INFO:tasks.workunit.client.1.smithi104.stdout:8/477: symlink d0/l94 0 2024-04-26T04:42:18.309 INFO:tasks.workunit.client.1.smithi104.stdout:8/478: creat d0/d33/d35/d6d/f95 x:0 0 0 2024-04-26T04:42:18.310 INFO:tasks.workunit.client.0.smithi090.stdout:5/333: mknod d4/d1a/df/d36/d42/c6e 0 2024-04-26T04:42:18.312 INFO:tasks.workunit.client.1.smithi104.stdout:0/487: sync 2024-04-26T04:42:18.312 INFO:tasks.workunit.client.1.smithi104.stdout:0/488: stat d0/d2/d21/d38/f34 0 2024-04-26T04:42:18.312 INFO:tasks.workunit.client.1.smithi104.stdout:7/473: sync 2024-04-26T04:42:18.313 INFO:tasks.workunit.client.1.smithi104.stdout:1/577: sync 2024-04-26T04:42:18.313 INFO:tasks.workunit.client.1.smithi104.stdout:1/578: write d0/d6/d19/d21/d66/d20/d2a/f61 [1933313,92225] 0 2024-04-26T04:42:18.313 INFO:tasks.workunit.client.1.smithi104.stdout:0/489: creat d0/d2/d21/d49/f99 x:0 0 0 2024-04-26T04:42:18.313 INFO:tasks.workunit.client.1.smithi104.stdout:0/490: chown d0/d2/d21/d27/f7b 98 1 2024-04-26T04:42:18.313 INFO:tasks.workunit.client.1.smithi104.stdout:0/491: dread - d0/d2/d21/d49/d5e/f7d zero size 2024-04-26T04:42:18.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/256: creat d0/d4/d7/d17/d22/f52 x:0 0 0 2024-04-26T04:42:18.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/257: chown d0/c2d 13201 1 2024-04-26T04:42:18.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/258: truncate d0/d4/d7/d17/f3d 8459 0 2024-04-26T04:42:18.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/259: stat d0/d4/d7/l3b 0 2024-04-26T04:42:18.315 INFO:tasks.workunit.client.0.smithi090.stdout:4/309: symlink d3/db/d12/d20/l5d 0 2024-04-26T04:42:18.316 INFO:tasks.workunit.client.0.smithi090.stdout:8/260: mkdir dc/d1f/d49/d3b/d3d/d57 0 2024-04-26T04:42:18.317 INFO:tasks.workunit.client.1.smithi104.stdout:7/474: rmdir d2/d4/d1d/d34/d7f/d8f 0 2024-04-26T04:42:18.317 INFO:tasks.workunit.client.1.smithi104.stdout:0/492: mkdir d0/d2/d21/d27/d1e/d57/d6c/d85/d9a 0 2024-04-26T04:42:18.318 INFO:tasks.workunit.client.1.smithi104.stdout:0/493: chown d0/d2/d10/d3d/f72 349 1 2024-04-26T04:42:18.318 INFO:tasks.workunit.client.1.smithi104.stdout:0/494: read - d0/d2/d10/d84/f90 zero size 2024-04-26T04:42:18.318 INFO:tasks.workunit.client.1.smithi104.stdout:0/495: write d0/d2/d21/d27/d1e/d57/d6c/f56 [5175921,59253] 0 2024-04-26T04:42:18.319 INFO:tasks.workunit.client.0.smithi090.stdout:5/334: creat d4/d1a/df/d36/d42/d56/f6f x:0 0 0 2024-04-26T04:42:18.322 INFO:tasks.workunit.client.1.smithi104.stdout:7/475: symlink d2/d4/d32/d3d/d41/d7e/l9b 0 2024-04-26T04:42:18.322 INFO:tasks.workunit.client.1.smithi104.stdout:7/476: chown d2/d4/d32/d3d 47097058 1 2024-04-26T04:42:18.322 INFO:tasks.workunit.client.1.smithi104.stdout:8/479: write d0/d1c/f24 [1272349,46567] 0 2024-04-26T04:42:18.322 INFO:tasks.workunit.client.1.smithi104.stdout:8/480: write d0/d1c/f84 [742592,108558] 0 2024-04-26T04:42:18.324 INFO:tasks.workunit.client.0.smithi090.stdout:1/260: mkdir d0/d4/d7/d4b/d53 0 2024-04-26T04:42:18.324 INFO:tasks.workunit.client.0.smithi090.stdout:1/261: write d0/f2e [1287188,73336] 0 2024-04-26T04:42:18.324 INFO:tasks.workunit.client.0.smithi090.stdout:6/371: dread d9/d21/d3b/d62/f28 [0,4194304] 0 2024-04-26T04:42:18.324 INFO:tasks.workunit.client.0.smithi090.stdout:7/302: dwrite d0/db/d53/f16 [0,4194304] 0 2024-04-26T04:42:18.324 INFO:tasks.workunit.client.0.smithi090.stdout:7/303: getdents d0/db/d53/d5e 0 2024-04-26T04:42:18.324 INFO:tasks.workunit.client.0.smithi090.stdout:7/304: chown d0/db 129579 1 2024-04-26T04:42:18.325 INFO:tasks.workunit.client.0.smithi090.stdout:7/305: stat d0/d1/d4/d3e/l68 0 2024-04-26T04:42:18.325 INFO:tasks.workunit.client.0.smithi090.stdout:7/306: dread - d0/db/d53/f56 zero size 2024-04-26T04:42:18.325 INFO:tasks.workunit.client.0.smithi090.stdout:6/372: creat d9/d21/d40/d4c/f90 x:0 0 0 2024-04-26T04:42:18.325 INFO:tasks.workunit.client.0.smithi090.stdout:6/373: chown d9/c1b 29 1 2024-04-26T04:42:18.325 INFO:tasks.workunit.client.0.smithi090.stdout:4/310: symlink d3/db/d12/d20/d2d/l5e 0 2024-04-26T04:42:18.325 INFO:tasks.workunit.client.0.smithi090.stdout:5/335: symlink d4/d1a/df/d36/d42/d49/l70 0 2024-04-26T04:42:18.327 INFO:tasks.workunit.client.1.smithi104.stdout:8/481: mknod d0/d1c/d27/d47/c96 0 2024-04-26T04:42:18.327 INFO:tasks.workunit.client.1.smithi104.stdout:5/562: dwrite d0/d89/fa2 [0,4194304] 0 2024-04-26T04:42:18.327 INFO:tasks.workunit.client.1.smithi104.stdout:2/748: dwrite d6/d1b/d51/d6a/fc5 [0,4194304] 0 2024-04-26T04:42:18.328 INFO:tasks.workunit.client.1.smithi104.stdout:2/749: chown d6/de/le3 1 1 2024-04-26T04:42:18.329 INFO:tasks.workunit.client.0.smithi090.stdout:0/295: dwrite d0/de/f39 [0,4194304] 0 2024-04-26T04:42:18.329 INFO:tasks.workunit.client.0.smithi090.stdout:8/261: dread dc/f1a [0,4194304] 0 2024-04-26T04:42:18.329 INFO:tasks.workunit.client.0.smithi090.stdout:7/307: link d0/d1/f52 d0/db/df/f70 0 2024-04-26T04:42:18.329 INFO:tasks.workunit.client.0.smithi090.stdout:6/374: mknod d9/d77/c91 0 2024-04-26T04:42:18.330 INFO:tasks.workunit.client.0.smithi090.stdout:4/311: mknod d3/c5f 0 2024-04-26T04:42:18.330 INFO:tasks.workunit.client.0.smithi090.stdout:4/312: stat d3/db/d12/d20/f41 0 2024-04-26T04:42:18.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/308: dread d0/db/d53/f17 [0,4194304] 0 2024-04-26T04:42:18.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/309: write d0/f45 [779695,95841] 0 2024-04-26T04:42:18.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/310: creat d0/d1/f71 x:0 0 0 2024-04-26T04:42:18.331 INFO:tasks.workunit.client.0.smithi090.stdout:7/311: dread - d0/db/df/d1c/d23/f60 zero size 2024-04-26T04:42:18.332 INFO:tasks.workunit.client.1.smithi104.stdout:2/750: creat d6/d1b/d68/d81/d97/ff8 x:0 0 0 2024-04-26T04:42:18.336 INFO:tasks.workunit.client.0.smithi090.stdout:0/296: creat d0/df/f67 x:0 0 0 2024-04-26T04:42:18.336 INFO:tasks.workunit.client.0.smithi090.stdout:0/297: readlink d0/df/l1a 0 2024-04-26T04:42:18.336 INFO:tasks.workunit.client.0.smithi090.stdout:0/298: chown d0/d4b/d5d 2042802 1 2024-04-26T04:42:18.336 INFO:tasks.workunit.client.0.smithi090.stdout:6/375: rename d9/d21/d3b/d62 to d9/d1c/d4d/d6d/d92 0 2024-04-26T04:42:18.336 INFO:tasks.workunit.client.0.smithi090.stdout:6/376: write d9/d21/d40/f74 [20514,24351] 0 2024-04-26T04:42:18.337 INFO:tasks.workunit.client.0.smithi090.stdout:1/262: dread d0/d4/d7/d17/d22/f2f [0,4194304] 0 2024-04-26T04:42:18.338 INFO:tasks.workunit.client.0.smithi090.stdout:6/377: truncate d9/d21/d3b/f51 3118591 0 2024-04-26T04:42:18.340 INFO:tasks.workunit.client.0.smithi090.stdout:5/336: dread d4/d1a/df/f22 [0,4194304] 0 2024-04-26T04:42:18.343 INFO:tasks.workunit.client.0.smithi090.stdout:5/337: unlink d4/l2d 0 2024-04-26T04:42:18.343 INFO:tasks.workunit.client.0.smithi090.stdout:5/338: chown d4/d1a/d3f/f44 1 1 2024-04-26T04:42:18.344 INFO:tasks.workunit.client.1.smithi104.stdout:5/563: creat d0/d1/d4/d2a/d2b/d37/d99/fc0 x:0 0 0 2024-04-26T04:42:18.344 INFO:tasks.workunit.client.0.smithi090.stdout:5/339: mkdir d4/d1a/df/d36/d42/d56/d71 0 2024-04-26T04:42:18.344 INFO:tasks.workunit.client.0.smithi090.stdout:5/340: write d4/d1a/df/d4a/f51 [1823572,53545] 0 2024-04-26T04:42:18.345 INFO:tasks.workunit.client.1.smithi104.stdout:2/751: mknod d6/de/d1d/dcf/cf9 0 2024-04-26T04:42:18.346 INFO:tasks.workunit.client.1.smithi104.stdout:5/564: mkdir d0/d1/d4/d2a/d2b/d37/d48/dc1 0 2024-04-26T04:42:18.347 INFO:tasks.workunit.client.0.smithi090.stdout:8/262: write dc/ff [2947707,109816] 0 2024-04-26T04:42:18.356 INFO:tasks.workunit.client.0.smithi090.stdout:6/378: dread f1 [0,4194304] 0 2024-04-26T04:42:18.357 INFO:tasks.workunit.client.0.smithi090.stdout:6/379: truncate d9/d21/f5f 738115 0 2024-04-26T04:42:18.357 INFO:tasks.workunit.client.0.smithi090.stdout:6/380: chown d9/d1c/d31/d69/c89 76240727 1 2024-04-26T04:42:18.357 INFO:tasks.workunit.client.0.smithi090.stdout:6/381: creat d9/d1c/d4d/d6d/f93 x:0 0 0 2024-04-26T04:42:18.357 INFO:tasks.workunit.client.0.smithi090.stdout:6/382: chown d9/d1c/d4d 12629 1 2024-04-26T04:42:18.357 INFO:tasks.workunit.client.1.smithi104.stdout:2/752: dread d6/de/d8e/f4c [0,4194304] 0 2024-04-26T04:42:18.357 INFO:tasks.workunit.client.1.smithi104.stdout:2/753: chown d6/d1b/d68/d81/fa2 4902 1 2024-04-26T04:42:18.357 INFO:tasks.workunit.client.1.smithi104.stdout:2/754: chown d6/d1b/d93/fc4 0 1 2024-04-26T04:42:18.380 INFO:tasks.workunit.client.1.smithi104.stdout:2/755: write d6/d1b/d93/faa [2104785,72139] 0 2024-04-26T04:42:18.380 INFO:tasks.workunit.client.1.smithi104.stdout:2/756: readlink d6/de/le3 0 2024-04-26T04:42:18.380 INFO:tasks.workunit.client.1.smithi104.stdout:2/757: write d6/d1b/f31 [1026797,33057] 0 2024-04-26T04:42:18.382 INFO:tasks.workunit.client.1.smithi104.stdout:8/482: rmdir d0/d1c 39 2024-04-26T04:42:18.382 INFO:tasks.workunit.client.1.smithi104.stdout:8/483: write d0/d1c/d2d/f8c [431498,31101] 0 2024-04-26T04:42:18.383 INFO:tasks.workunit.client.1.smithi104.stdout:8/484: mknod d0/d1c/d27/d47/d67/c97 0 2024-04-26T04:42:18.383 INFO:tasks.workunit.client.1.smithi104.stdout:8/485: creat d0/d33/d35/d6d/f98 x:0 0 0 2024-04-26T04:42:18.383 INFO:tasks.workunit.client.1.smithi104.stdout:8/486: chown d0/l4 359 1 2024-04-26T04:42:18.383 INFO:tasks.workunit.client.1.smithi104.stdout:9/443: dwrite d11/d15/d32/d4c/d54/f77 [0,4194304] 0 2024-04-26T04:42:18.383 INFO:tasks.workunit.client.1.smithi104.stdout:0/496: dwrite d0/f4d [0,4194304] 0 2024-04-26T04:42:18.384 INFO:tasks.workunit.client.1.smithi104.stdout:0/497: chown d0/f91 776949976 1 2024-04-26T04:42:18.389 INFO:tasks.workunit.client.1.smithi104.stdout:9/444: mknod d11/d15/d32/c88 0 2024-04-26T04:42:18.393 INFO:tasks.workunit.client.1.smithi104.stdout:0/498: rename d0/d2/d21/d27/d1e/l69 to d0/d2/d21/d38/l9b 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:1/263: dwrite d0/d4/d7/d17/d22/f2f [0,4194304] 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:1/264: creat d0/d4/d43/f54 x:0 0 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:1/265: readlink d0/l6 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/312: dwrite d0/db/d15/d29/f44 [0,4194304] 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:5/341: dwrite d4/d1a/f48 [0,4194304] 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/313: write d0/db/df/d1c/d23/f60 [709453,95049] 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/314: fdatasync d0/f4e 0 2024-04-26T04:42:18.395 INFO:tasks.workunit.client.0.smithi090.stdout:0/299: dwrite d0/de/f36 [0,4194304] 0 2024-04-26T04:42:18.396 INFO:tasks.workunit.client.0.smithi090.stdout:0/300: truncate d0/d4b/f52 854862 0 2024-04-26T04:42:18.396 INFO:tasks.workunit.client.1.smithi104.stdout:0/499: rmdir d0/d2/d21/d27/d1e/d57 39 2024-04-26T04:42:18.397 INFO:tasks.workunit.client.1.smithi104.stdout:0/500: rename d0/d4/d3c to d0/d2/d21/d49/d5e/d9c 0 2024-04-26T04:42:18.398 INFO:tasks.workunit.client.0.smithi090.stdout:5/342: dread d4/d1a/df/f22 [0,4194304] 0 2024-04-26T04:42:18.399 INFO:tasks.workunit.client.0.smithi090.stdout:8/263: dwrite dc/d1f/d49/f20 [0,4194304] 0 2024-04-26T04:42:18.399 INFO:tasks.workunit.client.0.smithi090.stdout:8/264: readlink dc/d1f/d49/l16 0 2024-04-26T04:42:18.399 INFO:tasks.workunit.client.0.smithi090.stdout:8/265: chown dc/d1f/d49/f28 202 1 2024-04-26T04:42:18.400 INFO:tasks.workunit.client.1.smithi104.stdout:5/565: dwrite d0/d1/d64/f92 [0,4194304] 0 2024-04-26T04:42:18.400 INFO:tasks.workunit.client.1.smithi104.stdout:5/566: write d0/d41/f4d [2439669,89641] 0 2024-04-26T04:42:18.400 INFO:tasks.workunit.client.1.smithi104.stdout:5/567: chown d0/d1/d64/d26/d36/d72/cab 0 1 2024-04-26T04:42:18.401 INFO:tasks.workunit.client.0.smithi090.stdout:6/383: dwrite d9/f12 [4194304,4194304] 0 2024-04-26T04:42:18.401 INFO:tasks.workunit.client.0.smithi090.stdout:6/384: write d9/f14 [2736923,17999] 0 2024-04-26T04:42:18.401 INFO:tasks.workunit.client.0.smithi090.stdout:6/385: read d9/f24 [1363789,55872] 0 2024-04-26T04:42:18.403 INFO:tasks.workunit.client.1.smithi104.stdout:5/568: rename d0/d1/d4/d2a/l49 to d0/d41/d4f/lc2 0 2024-04-26T04:42:18.404 INFO:tasks.workunit.client.1.smithi104.stdout:5/569: write d0/d1/d64/d26/d36/da7/fa8 [4214528,34891] 0 2024-04-26T04:42:18.404 INFO:tasks.workunit.client.1.smithi104.stdout:5/570: chown d0/d1/d4/lc 4300 1 2024-04-26T04:42:18.408 INFO:tasks.workunit.client.0.smithi090.stdout:0/301: read d0/de/f39 [3384625,98533] 0 2024-04-26T04:42:18.410 INFO:tasks.workunit.client.0.smithi090.stdout:5/343: dread d4/d1a/f48 [0,4194304] 0 2024-04-26T04:42:18.410 INFO:tasks.workunit.client.0.smithi090.stdout:5/344: fsync d4/d1a/df/d36/f54 0 2024-04-26T04:42:18.418 INFO:tasks.workunit.client.1.smithi104.stdout:5/571: dread d0/d1/d64/f6c [0,4194304] 0 2024-04-26T04:42:18.419 INFO:tasks.workunit.client.1.smithi104.stdout:5/572: rename d0/d1/c15 to d0/d1/d4/d2a/cc3 0 2024-04-26T04:42:18.421 INFO:tasks.workunit.client.1.smithi104.stdout:5/573: link d0/d1/d64/f92 d0/d1/d4/d2a/d6b/d91/fc4 0 2024-04-26T04:42:18.421 INFO:tasks.workunit.client.1.smithi104.stdout:5/574: fdatasync d0/d1/d3e/f79 0 2024-04-26T04:42:18.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/266: mkdir d0/d55 0 2024-04-26T04:42:18.422 INFO:tasks.workunit.client.1.smithi104.stdout:5/575: truncate d0/d1/d4/d2a/d6b/d73/d95/fba 1771430 0 2024-04-26T04:42:18.430 INFO:tasks.workunit.client.1.smithi104.stdout:8/487: dwrite d0/d1c/d27/d46/f78 [0,4194304] 0 2024-04-26T04:42:18.439 INFO:tasks.workunit.client.1.smithi104.stdout:0/501: dwrite d0/d2/d21/d49/d5e/f7d [0,4194304] 0 2024-04-26T04:42:18.440 INFO:tasks.workunit.client.1.smithi104.stdout:9/445: dwrite d11/d15/d32/d35/f80 [0,4194304] 0 2024-04-26T04:42:18.440 INFO:tasks.workunit.client.1.smithi104.stdout:9/446: creat d11/f89 x:0 0 0 2024-04-26T04:42:18.440 INFO:tasks.workunit.client.1.smithi104.stdout:8/488: write d0/d33/d35/f87 [3805492,41374] 0 2024-04-26T04:42:18.442 INFO:tasks.workunit.client.0.smithi090.stdout:6/386: dwrite d9/d1c/d4d/f8e [0,4194304] 0 2024-04-26T04:42:18.442 INFO:tasks.workunit.client.0.smithi090.stdout:6/387: chown d9/d1c/d4d/d58 10664 1 2024-04-26T04:42:18.442 INFO:tasks.workunit.client.0.smithi090.stdout:5/345: dwrite d4/f1e [4194304,4194304] 0 2024-04-26T04:42:18.442 INFO:tasks.workunit.client.0.smithi090.stdout:1/267: dwrite d0/d4/d7/d17/f2a [0,4194304] 0 2024-04-26T04:42:18.448 INFO:tasks.workunit.client.1.smithi104.stdout:9/447: mkdir d11/d15/d32/d4c/d8a 0 2024-04-26T04:42:18.451 INFO:tasks.workunit.client.1.smithi104.stdout:8/489: mknod d0/d1c/d27/c99 0 2024-04-26T04:42:18.451 INFO:tasks.workunit.client.1.smithi104.stdout:8/490: read d0/d33/f2c [551317,85582] 0 2024-04-26T04:42:18.454 INFO:tasks.workunit.client.1.smithi104.stdout:5/576: read d0/d1/d1e/f21 [1667017,51716] 0 2024-04-26T04:42:18.454 INFO:tasks.workunit.client.0.smithi090.stdout:6/388: read d9/f14 [8355657,41292] 0 2024-04-26T04:42:18.456 INFO:tasks.workunit.client.1.smithi104.stdout:8/491: read d0/f1e [527585,118122] 0 2024-04-26T04:42:18.456 INFO:tasks.workunit.client.1.smithi104.stdout:5/577: truncate d0/d1/d1e/f56 414383 0 2024-04-26T04:42:18.456 INFO:tasks.workunit.client.1.smithi104.stdout:5/578: write d0/d1/d4/d2a/d6b/d73/d95/fa5 [943975,81538] 0 2024-04-26T04:42:18.458 INFO:tasks.workunit.client.1.smithi104.stdout:8/492: rmdir d0/d1c/d59 39 2024-04-26T04:42:18.460 INFO:tasks.workunit.client.1.smithi104.stdout:5/579: creat d0/d1/d4/d2a/d6b/d91/dbd/fc5 x:0 0 0 2024-04-26T04:42:18.461 INFO:tasks.workunit.client.1.smithi104.stdout:8/493: mkdir d0/d9a 0 2024-04-26T04:42:18.461 INFO:tasks.workunit.client.1.smithi104.stdout:8/494: write d0/d33/d35/d6d/f74 [997233,1153] 0 2024-04-26T04:42:18.461 INFO:tasks.workunit.client.1.smithi104.stdout:8/495: creat d0/d33/d35/d6d/f9b x:0 0 0 2024-04-26T04:42:18.461 INFO:tasks.workunit.client.1.smithi104.stdout:8/496: creat d0/d33/d35/d6d/f9c x:0 0 0 2024-04-26T04:42:18.461 INFO:tasks.workunit.client.1.smithi104.stdout:5/580: mkdir d0/d1/d4/d2a/d2b/d37/d48/dc6 0 2024-04-26T04:42:18.463 INFO:tasks.workunit.client.0.smithi090.stdout:5/346: dwrite d4/d1a/df/d36/d42/d56/f6f [0,4194304] 0 2024-04-26T04:42:18.465 INFO:tasks.workunit.client.1.smithi104.stdout:8/497: mknod d0/d33/c9d 0 2024-04-26T04:42:18.465 INFO:tasks.workunit.client.1.smithi104.stdout:8/498: dread - d0/d1c/d27/f86 zero size 2024-04-26T04:42:18.465 INFO:tasks.workunit.client.1.smithi104.stdout:8/499: link d0/d1c/d2d/l54 d0/d9a/l9e 0 2024-04-26T04:42:18.467 INFO:tasks.workunit.client.1.smithi104.stdout:8/500: rmdir d0/d1c/d27/d47/d6b 0 2024-04-26T04:42:18.467 INFO:tasks.workunit.client.1.smithi104.stdout:8/501: stat d0/f2f 0 2024-04-26T04:42:18.469 INFO:tasks.workunit.client.1.smithi104.stdout:0/502: dwrite d0/f2f [0,4194304] 0 2024-04-26T04:42:18.472 INFO:tasks.workunit.client.1.smithi104.stdout:9/448: dwrite d11/d15/f18 [4194304,4194304] 0 2024-04-26T04:42:18.476 INFO:tasks.workunit.client.1.smithi104.stdout:0/503: creat d0/d2/d21/d49/d5e/d9c/d53/f9d x:0 0 0 2024-04-26T04:42:18.479 INFO:tasks.workunit.client.1.smithi104.stdout:9/449: truncate f10 605084 0 2024-04-26T04:42:18.480 INFO:tasks.workunit.client.1.smithi104.stdout:0/504: link d0/d2/d21/d27/d1e/f96 d0/d2/d10/d3d/f9e 0 2024-04-26T04:42:18.487 INFO:tasks.workunit.client.1.smithi104.stdout:5/581: dwrite d0/d1/d64/d26/d36/d72/faa [0,4194304] 0 2024-04-26T04:42:18.488 INFO:tasks.workunit.client.0.smithi090.stdout:8/266: mkdir dc/d1f/d49/d4f/d58 0 2024-04-26T04:42:18.489 INFO:tasks.workunit.client.1.smithi104.stdout:8/502: dwrite d0/d33/f7b [0,4194304] 0 2024-04-26T04:42:18.491 INFO:tasks.workunit.client.1.smithi104.stdout:5/582: mkdir d0/d1/d4/d2a/d6b/d91/dc7 0 2024-04-26T04:42:18.492 INFO:tasks.workunit.client.1.smithi104.stdout:5/583: stat d0/d41/f4d 0 2024-04-26T04:42:18.498 INFO:tasks.workunit.client.0.smithi090.stdout:1/268: unlink d0/c35 0 2024-04-26T04:42:18.498 INFO:tasks.workunit.client.0.smithi090.stdout:8/267: dread dc/d1f/d49/f17 [0,4194304] 0 2024-04-26T04:42:18.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/347: mkdir d4/d1a/df/d72 0 2024-04-26T04:42:18.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/348: write d4/fe [2989140,43679] 0 2024-04-26T04:42:18.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/349: chown d4/d1a/df/d36/d42/d49 5217239 1 2024-04-26T04:42:18.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/350: truncate d4/d1a/df/d36/f4d 4280753 0 2024-04-26T04:42:18.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/315: getdents d0/db/d15/d30 0 2024-04-26T04:42:18.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/316: stat d0/d1/f38 0 2024-04-26T04:42:18.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/317: write d0/db/d53/f2d [419551,19592] 0 2024-04-26T04:42:18.504 INFO:tasks.workunit.client.0.smithi090.stdout:7/318: dread - d0/d1/d4/f6b zero size 2024-04-26T04:42:18.506 INFO:tasks.workunit.client.0.smithi090.stdout:0/302: link d0/d9/d45/d53/l54 d0/de/l68 0 2024-04-26T04:42:18.507 INFO:tasks.workunit.client.0.smithi090.stdout:1/269: rename d0/d4/d7/d17/f38 to d0/d4/d7/d4a/f56 0 2024-04-26T04:42:18.509 INFO:tasks.workunit.client.0.smithi090.stdout:8/268: symlink dc/d1f/d3a/l59 0 2024-04-26T04:42:18.509 INFO:tasks.workunit.client.0.smithi090.stdout:8/269: write dc/f1a [2141905,117246] 0 2024-04-26T04:42:18.509 INFO:tasks.workunit.client.1.smithi104.stdout:0/505: dwrite d0/d2/d21/d27/f82 [0,4194304] 0 2024-04-26T04:42:18.510 INFO:tasks.workunit.client.0.smithi090.stdout:5/351: truncate d4/d1a/df/d36/d42/d49/f66 11084 0 2024-04-26T04:42:18.510 INFO:tasks.workunit.client.0.smithi090.stdout:5/352: rename d4/d1a to d4/d1a/d1f/d73 22 2024-04-26T04:42:18.511 INFO:tasks.workunit.client.0.smithi090.stdout:7/319: symlink d0/db/d15/d30/l72 0 2024-04-26T04:42:18.512 INFO:tasks.workunit.client.0.smithi090.stdout:7/320: write d0/f3c [752047,117697] 0 2024-04-26T04:42:18.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/270: mkdir dc/d5a 0 2024-04-26T04:42:18.513 INFO:tasks.workunit.client.0.smithi090.stdout:8/271: write dc/d1f/f43 [1571198,3078] 0 2024-04-26T04:42:18.513 INFO:tasks.workunit.client.1.smithi104.stdout:0/506: unlink d0/d2/f23 0 2024-04-26T04:42:18.514 INFO:tasks.workunit.client.1.smithi104.stdout:4/618: sync 2024-04-26T04:42:18.515 INFO:tasks.workunit.client.1.smithi104.stdout:6/694: sync 2024-04-26T04:42:18.515 INFO:tasks.workunit.client.0.smithi090.stdout:5/353: rename d4/d1a/d1f/d53/d59/f69 to d4/d1a/d45/f74 0 2024-04-26T04:42:18.515 INFO:tasks.workunit.client.0.smithi090.stdout:7/321: stat d0/db/d15/d30/l54 0 2024-04-26T04:42:18.515 INFO:tasks.workunit.client.0.smithi090.stdout:7/322: chown d0/db/d15/d29/l48 387380765 1 2024-04-26T04:42:18.516 INFO:tasks.workunit.client.1.smithi104.stdout:0/507: rename d0/c4a to d0/d2/d21/d27/d1e/d57/d6c/d85/d9a/c9f 0 2024-04-26T04:42:18.516 INFO:tasks.workunit.client.0.smithi090.stdout:8/272: getdents dc/d1f/d49 0 2024-04-26T04:42:18.519 INFO:tasks.workunit.client.1.smithi104.stdout:6/695: rename d0/d9/d13/d41/d82/dc8/c17 to d0/d9/d13/d41/d82/dc8/dba/daa/dda/cef 0 2024-04-26T04:42:18.520 INFO:tasks.workunit.client.1.smithi104.stdout:0/508: mkdir d0/d2/da0 0 2024-04-26T04:42:18.520 INFO:tasks.workunit.client.1.smithi104.stdout:0/509: chown d0/d2/d21/d49/f55 0 1 2024-04-26T04:42:18.521 INFO:tasks.workunit.client.0.smithi090.stdout:0/303: write d0/de/f36 [432776,32878] 0 2024-04-26T04:42:18.522 INFO:tasks.workunit.client.1.smithi104.stdout:6/696: unlink d0/l8a 0 2024-04-26T04:42:18.524 INFO:tasks.workunit.client.1.smithi104.stdout:0/510: unlink d0/f7f 0 2024-04-26T04:42:18.525 INFO:tasks.workunit.client.0.smithi090.stdout:0/304: link d0/d4b/l61 d0/d9/d45/l69 0 2024-04-26T04:42:18.525 INFO:tasks.workunit.client.0.smithi090.stdout:8/273: dread dc/d1f/f36 [0,4194304] 0 2024-04-26T04:42:18.526 INFO:tasks.workunit.client.1.smithi104.stdout:9/450: dwrite f4 [4194304,4194304] 0 2024-04-26T04:42:18.526 INFO:tasks.workunit.client.1.smithi104.stdout:6/697: mknod d0/d9/d13/dcd/dd9/cf0 0 2024-04-26T04:42:18.528 INFO:tasks.workunit.client.1.smithi104.stdout:5/584: dwrite d0/d1/d1e/d98/fae [0,4194304] 0 2024-04-26T04:42:18.528 INFO:tasks.workunit.client.1.smithi104.stdout:5/585: truncate d0/d1/d64/f19 740094 0 2024-04-26T04:42:18.529 INFO:tasks.workunit.client.0.smithi090.stdout:0/305: creat d0/d4b/d5d/f6a x:0 0 0 2024-04-26T04:42:18.533 INFO:tasks.workunit.client.0.smithi090.stdout:7/323: write d0/f8 [4839539,46617] 0 2024-04-26T04:42:18.534 INFO:tasks.workunit.client.1.smithi104.stdout:5/586: mkdir d0/d1/d4/d2a/d6b/d91/dc8 0 2024-04-26T04:42:18.536 INFO:tasks.workunit.client.0.smithi090.stdout:5/354: dwrite d4/d1a/d3f/f29 [0,4194304] 0 2024-04-26T04:42:18.536 INFO:tasks.workunit.client.0.smithi090.stdout:1/270: dwrite d0/d4/f48 [0,4194304] 0 2024-04-26T04:42:18.537 INFO:tasks.workunit.client.1.smithi104.stdout:8/503: dwrite d0/d1c/d2d/f8c [0,4194304] 0 2024-04-26T04:42:18.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/511: rename d0/d81 to d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1 0 2024-04-26T04:42:18.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/512: write d0/d2/d21/d49/d5e/d9c/f86 [4818591,74554] 0 2024-04-26T04:42:18.546 INFO:tasks.workunit.client.0.smithi090.stdout:0/306: write d0/f38 [5546582,68492] 0 2024-04-26T04:42:18.546 INFO:tasks.workunit.client.1.smithi104.stdout:8/504: unlink d0/d50/l52 0 2024-04-26T04:42:18.550 INFO:tasks.workunit.client.1.smithi104.stdout:0/513: truncate d0/d2/d21/d49/d5e/d9c/f4b 2233483 0 2024-04-26T04:42:18.550 INFO:tasks.workunit.client.1.smithi104.stdout:0/514: write d0/d2/d21/f45 [1180480,58478] 0 2024-04-26T04:42:18.550 INFO:tasks.workunit.client.1.smithi104.stdout:0/515: write d0/d2/d21/d49/d5e/d9c/d53/d5d/f88 [744131,111972] 0 2024-04-26T04:42:18.550 INFO:tasks.workunit.client.1.smithi104.stdout:0/516: readlink d0/d2/d10/d3d/l87 0 2024-04-26T04:42:18.550 INFO:tasks.workunit.client.1.smithi104.stdout:0/517: readlink d0/d4/d3b/l98 0 2024-04-26T04:42:18.551 INFO:tasks.workunit.client.0.smithi090.stdout:8/274: dwrite dc/d1f/f27 [0,4194304] 0 2024-04-26T04:42:18.551 INFO:tasks.workunit.client.1.smithi104.stdout:6/698: dread d0/d9/fbc [0,4194304] 0 2024-04-26T04:42:18.551 INFO:tasks.workunit.client.1.smithi104.stdout:8/505: rename d0/d33/c22 to d0/d33/d35/d6d/c9f 0 2024-04-26T04:42:18.552 INFO:tasks.workunit.client.1.smithi104.stdout:8/506: fsync d0/d1c/d27/d46/f6f 0 2024-04-26T04:42:18.552 INFO:tasks.workunit.client.1.smithi104.stdout:8/507: chown d0/d1c/d27/d47/c96 33 1 2024-04-26T04:42:18.552 INFO:tasks.workunit.client.1.smithi104.stdout:8/508: fsync d0/d1c/d27/f5b 0 2024-04-26T04:42:18.552 INFO:tasks.workunit.client.1.smithi104.stdout:0/518: creat d0/d2/d21/d27/d1e/d57/d6c/d85/fa2 x:0 0 0 2024-04-26T04:42:18.554 INFO:tasks.workunit.client.1.smithi104.stdout:8/509: symlink d0/d1c/d27/d47/d67/la0 0 2024-04-26T04:42:18.555 INFO:tasks.workunit.client.0.smithi090.stdout:1/271: symlink d0/d55/l57 0 2024-04-26T04:42:18.556 INFO:tasks.workunit.client.0.smithi090.stdout:0/307: creat d0/df/f6b x:0 0 0 2024-04-26T04:42:18.556 INFO:tasks.workunit.client.0.smithi090.stdout:0/308: chown d0/d9/d45/c60 399 1 2024-04-26T04:42:18.557 INFO:tasks.workunit.client.1.smithi104.stdout:0/519: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3 0 2024-04-26T04:42:18.559 INFO:tasks.workunit.client.1.smithi104.stdout:8/510: mknod d0/d9a/ca1 0 2024-04-26T04:42:18.561 INFO:tasks.workunit.client.0.smithi090.stdout:1/272: mkdir d0/d4/d43/d4d/d58 0 2024-04-26T04:42:18.562 INFO:tasks.workunit.client.0.smithi090.stdout:0/309: creat d0/df/f6c x:0 0 0 2024-04-26T04:42:18.562 INFO:tasks.workunit.client.1.smithi104.stdout:8/511: truncate d0/d1c/d27/d47/f25 1113457 0 2024-04-26T04:42:18.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/275: symlink dc/d1f/d49/d3b/l5b 0 2024-04-26T04:42:18.563 INFO:tasks.workunit.client.0.smithi090.stdout:0/310: write d0/df/f3c [425256,20637] 0 2024-04-26T04:42:18.563 INFO:tasks.workunit.client.0.smithi090.stdout:0/311: write d0/de/f39 [2647704,130964] 0 2024-04-26T04:42:18.564 INFO:tasks.workunit.client.0.smithi090.stdout:1/273: symlink d0/d4/d43/d4d/l59 0 2024-04-26T04:42:18.564 INFO:tasks.workunit.client.0.smithi090.stdout:1/274: creat d0/f5a x:0 0 0 2024-04-26T04:42:18.565 INFO:tasks.workunit.client.0.smithi090.stdout:1/275: creat d0/d4/d7/d4b/f5b x:0 0 0 2024-04-26T04:42:18.565 INFO:tasks.workunit.client.0.smithi090.stdout:1/276: stat d0/d4/d7/d49 0 2024-04-26T04:42:18.565 INFO:tasks.workunit.client.0.smithi090.stdout:8/276: getdents dc/d34 0 2024-04-26T04:42:18.565 INFO:tasks.workunit.client.0.smithi090.stdout:8/277: stat dc/d1f/d49/d3b/d3d/d57 0 2024-04-26T04:42:18.565 INFO:tasks.workunit.client.1.smithi104.stdout:8/512: rename d0/d1c/d27/f5b to d0/d33/d51/fa2 0 2024-04-26T04:42:18.566 INFO:tasks.workunit.client.0.smithi090.stdout:0/312: mkdir d0/d4b/d6d 0 2024-04-26T04:42:18.567 INFO:tasks.workunit.client.1.smithi104.stdout:8/513: mknod d0/d1c/d59/ca3 0 2024-04-26T04:42:18.567 INFO:tasks.workunit.client.0.smithi090.stdout:1/277: truncate d0/d4/f15 3072990 0 2024-04-26T04:42:18.568 INFO:tasks.workunit.client.1.smithi104.stdout:8/514: mkdir d0/d33/d35/d6d/da4 0 2024-04-26T04:42:18.569 INFO:tasks.workunit.client.1.smithi104.stdout:8/515: fdatasync d0/f4e 0 2024-04-26T04:42:18.569 INFO:tasks.workunit.client.0.smithi090.stdout:8/278: link dc/d1f/d49/d3b/d3d/f54 dc/f5c 0 2024-04-26T04:42:18.570 INFO:tasks.workunit.client.1.smithi104.stdout:8/516: symlink d0/d33/d51/la5 0 2024-04-26T04:42:18.571 INFO:tasks.workunit.client.0.smithi090.stdout:8/279: mkdir dc/d5a/d5d 0 2024-04-26T04:42:18.572 INFO:tasks.workunit.client.1.smithi104.stdout:8/517: mkdir d0/d33/da6 0 2024-04-26T04:42:18.572 INFO:tasks.workunit.client.1.smithi104.stdout:8/518: chown d0/l6 3 1 2024-04-26T04:42:18.574 INFO:tasks.workunit.client.1.smithi104.stdout:8/519: mknod d0/d1c/d27/d47/d67/d76/ca7 0 2024-04-26T04:42:18.581 INFO:tasks.workunit.client.1.smithi104.stdout:9/451: dwrite d11/d15/f18 [4194304,4194304] 0 2024-04-26T04:42:18.581 INFO:tasks.workunit.client.1.smithi104.stdout:9/452: chown d11/f44 64805906 1 2024-04-26T04:42:18.581 INFO:tasks.workunit.client.1.smithi104.stdout:9/453: truncate d11/d15/f59 351382 0 2024-04-26T04:42:18.581 INFO:tasks.workunit.client.1.smithi104.stdout:9/454: dread - d11/d15/d32/d4c/d54/f7a zero size 2024-04-26T04:42:18.581 INFO:tasks.workunit.client.1.smithi104.stdout:5/587: dwrite d0/d1/d1e/f30 [0,4194304] 0 2024-04-26T04:42:18.583 INFO:tasks.workunit.client.1.smithi104.stdout:9/455: symlink d11/d15/d32/d35/l8b 0 2024-04-26T04:42:18.586 INFO:tasks.workunit.client.1.smithi104.stdout:5/588: mknod d0/d1/d4/d90/cc9 0 2024-04-26T04:42:18.586 INFO:tasks.workunit.client.0.smithi090.stdout:7/324: dwrite d0/db/d53/f2d [0,4194304] 0 2024-04-26T04:42:18.587 INFO:tasks.workunit.client.1.smithi104.stdout:3/625: sync 2024-04-26T04:42:18.588 INFO:tasks.workunit.client.1.smithi104.stdout:5/589: link d0/d1/d4/l5a d0/d1/d64/d26/lca 0 2024-04-26T04:42:18.590 INFO:tasks.workunit.client.0.smithi090.stdout:7/325: rename d0/db/d15/d30/l39 to d0/db/d15/d30/l73 0 2024-04-26T04:42:18.590 INFO:tasks.workunit.client.0.smithi090.stdout:7/326: truncate d0/db/df/d4d/f63 790602 0 2024-04-26T04:42:18.591 INFO:tasks.workunit.client.1.smithi104.stdout:8/520: getdents d0/d50 0 2024-04-26T04:42:18.592 INFO:tasks.workunit.client.0.smithi090.stdout:7/327: mknod d0/db/c74 0 2024-04-26T04:42:18.593 INFO:tasks.workunit.client.1.smithi104.stdout:5/590: symlink d0/d1/d1e/lcb 0 2024-04-26T04:42:18.594 INFO:tasks.workunit.client.0.smithi090.stdout:7/328: symlink d0/d1/d4/d43/d4a/l75 0 2024-04-26T04:42:18.596 INFO:tasks.workunit.client.1.smithi104.stdout:8/521: creat d0/d33/da6/fa8 x:0 0 0 2024-04-26T04:42:18.596 INFO:tasks.workunit.client.1.smithi104.stdout:8/522: stat d0/d1c/d27/d47/d67/d76/c77 0 2024-04-26T04:42:18.597 INFO:tasks.workunit.client.0.smithi090.stdout:7/329: creat d0/db/d15/d30/d4f/f76 x:0 0 0 2024-04-26T04:42:18.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/278: dwrite d0/d4/d7/d17/f2a [0,4194304] 0 2024-04-26T04:42:18.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/279: write d0/d4/d7/d17/d22/f52 [962880,25640] 0 2024-04-26T04:42:18.598 INFO:tasks.workunit.client.0.smithi090.stdout:8/280: dwrite dc/d1f/d3a/f44 [0,4194304] 0 2024-04-26T04:42:18.598 INFO:tasks.workunit.client.0.smithi090.stdout:8/281: truncate dc/d1f/d3a/f44 4582975 0 2024-04-26T04:42:18.598 INFO:tasks.workunit.client.0.smithi090.stdout:8/282: creat dc/d1f/d49/d3b/d3d/f5e x:0 0 0 2024-04-26T04:42:18.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/330: symlink d0/d1/d4/d43/l77 0 2024-04-26T04:42:18.598 INFO:tasks.workunit.client.0.smithi090.stdout:5/355: dwrite d4/d1a/d45/f3c [0,4194304] 0 2024-04-26T04:42:18.599 INFO:tasks.workunit.client.1.smithi104.stdout:5/591: link d0/f1d d0/d1/d64/d26/d36/fcc 0 2024-04-26T04:42:18.599 INFO:tasks.workunit.client.1.smithi104.stdout:5/592: write d0/d1/d3e/f59 [595442,66968] 0 2024-04-26T04:42:18.599 INFO:tasks.workunit.client.1.smithi104.stdout:3/626: read d5/d9/dc/d32/d48/dac/dd7/f56 [2886364,40352] 0 2024-04-26T04:42:18.600 INFO:tasks.workunit.client.1.smithi104.stdout:6/699: dwrite d0/d9/d13/d41/d82/dc8/f94 [0,4194304] 0 2024-04-26T04:42:18.600 INFO:tasks.workunit.client.1.smithi104.stdout:8/523: symlink d0/la9 0 2024-04-26T04:42:18.600 INFO:tasks.workunit.client.1.smithi104.stdout:8/524: stat d0/d1c/d2d/l3a 0 2024-04-26T04:42:18.602 INFO:tasks.workunit.client.1.smithi104.stdout:5/593: creat d0/d1/d4/d2a/d6b/d91/dc7/fcd x:0 0 0 2024-04-26T04:42:18.602 INFO:tasks.workunit.client.1.smithi104.stdout:5/594: chown d0/d1/d4/d2a/d6b/d73/da4 3284 1 2024-04-26T04:42:18.605 INFO:tasks.workunit.client.1.smithi104.stdout:6/700: mkdir d0/d9/d13/d41/d96/dac/df1 0 2024-04-26T04:42:18.606 INFO:tasks.workunit.client.0.smithi090.stdout:8/283: mknod dc/d1f/d3a/c5f 0 2024-04-26T04:42:18.607 INFO:tasks.workunit.client.1.smithi104.stdout:5/595: creat d0/d1/d64/d26/d36/d72/da9/fce x:0 0 0 2024-04-26T04:42:18.607 INFO:tasks.workunit.client.1.smithi104.stdout:5/596: write d0/f93 [358234,1241] 0 2024-04-26T04:42:18.611 INFO:tasks.workunit.client.1.smithi104.stdout:6/701: creat d0/d9/d13/d41/d96/dac/dc4/ff2 x:0 0 0 2024-04-26T04:42:18.614 INFO:tasks.workunit.client.0.smithi090.stdout:5/356: write d4/d1a/d45/f4f [2407036,41051] 0 2024-04-26T04:42:18.614 INFO:tasks.workunit.client.0.smithi090.stdout:0/313: rmdir d0/d4b 39 2024-04-26T04:42:18.615 INFO:tasks.workunit.client.0.smithi090.stdout:1/280: write d0/d4/f15 [2385360,28696] 0 2024-04-26T04:42:18.618 INFO:tasks.workunit.client.1.smithi104.stdout:6/702: link d0/d9/d25/f85 d0/ff3 0 2024-04-26T04:42:18.618 INFO:tasks.workunit.client.1.smithi104.stdout:6/703: fdatasync d0/d9/d13/d41/d96/dac/dc4/fcf 0 2024-04-26T04:42:18.618 INFO:tasks.workunit.client.1.smithi104.stdout:6/704: fdatasync d0/d9/d13/d41/d96/fb0 0 2024-04-26T04:42:18.619 INFO:tasks.workunit.client.1.smithi104.stdout:6/705: truncate d0/dd8/fe9 1470540 0 2024-04-26T04:42:18.620 INFO:tasks.workunit.client.1.smithi104.stdout:5/597: dread d0/d1/f3d [0,4194304] 0 2024-04-26T04:42:18.620 INFO:tasks.workunit.client.1.smithi104.stdout:5/598: dread - d0/d1/f65 zero size 2024-04-26T04:42:18.620 INFO:tasks.workunit.client.1.smithi104.stdout:5/599: write d0/f66 [723274,15606] 0 2024-04-26T04:42:18.621 INFO:tasks.workunit.client.0.smithi090.stdout:5/357: truncate d4/d1a/d3f/f44 3683231 0 2024-04-26T04:42:18.622 INFO:tasks.workunit.client.0.smithi090.stdout:0/314: creat d0/de/f6e x:0 0 0 2024-04-26T04:42:18.623 INFO:tasks.workunit.client.1.smithi104.stdout:5/600: symlink d0/d1/d4/d2a/lcf 0 2024-04-26T04:42:18.623 INFO:tasks.workunit.client.1.smithi104.stdout:5/601: dread - d0/d1/d4/d2a/d2b/d37/d99/fc0 zero size 2024-04-26T04:42:18.623 INFO:tasks.workunit.client.1.smithi104.stdout:5/602: fdatasync d0/d1/d4/d2a/d6b/d91/dbd/fc5 0 2024-04-26T04:42:18.624 INFO:tasks.workunit.client.1.smithi104.stdout:9/456: dwrite d11/d15/f30 [0,4194304] 0 2024-04-26T04:42:18.624 INFO:tasks.workunit.client.0.smithi090.stdout:1/281: creat d0/d4/d7/d17/d22/f5c x:0 0 0 2024-04-26T04:42:18.625 INFO:tasks.workunit.client.0.smithi090.stdout:0/315: symlink d0/d9/d45/d53/l6f 0 2024-04-26T04:42:18.627 INFO:tasks.workunit.client.1.smithi104.stdout:9/457: mknod d11/c8c 0 2024-04-26T04:42:18.628 INFO:tasks.workunit.client.1.smithi104.stdout:9/458: fdatasync d11/d15/d32/d4c/d54/f75 0 2024-04-26T04:42:18.629 INFO:tasks.workunit.client.1.smithi104.stdout:9/459: rename d11/d1d/d34/d69/l7d to d11/d15/d32/d35/d7b/l8d 0 2024-04-26T04:42:18.630 INFO:tasks.workunit.client.1.smithi104.stdout:9/460: symlink d11/d15/d1b/d6d/l8e 0 2024-04-26T04:42:18.630 INFO:tasks.workunit.client.1.smithi104.stdout:9/461: fsync d11/d15/d32/d35/f57 0 2024-04-26T04:42:18.630 INFO:tasks.workunit.client.1.smithi104.stdout:9/462: fsync d11/f44 0 2024-04-26T04:42:18.631 INFO:tasks.workunit.client.1.smithi104.stdout:9/463: mknod d11/d1d/d71/c8f 0 2024-04-26T04:42:18.631 INFO:tasks.workunit.client.1.smithi104.stdout:9/464: chown d11/d15/d32/d35/d7b 25120 1 2024-04-26T04:42:18.631 INFO:tasks.workunit.client.1.smithi104.stdout:9/465: fdatasync d11/d15/d32/d35/f65 0 2024-04-26T04:42:18.631 INFO:tasks.workunit.client.1.smithi104.stdout:6/706: dread d0/d9/d13/d41/d82/dc8/d80/f75 [4194304,4194304] 0 2024-04-26T04:42:18.632 INFO:tasks.workunit.client.1.smithi104.stdout:9/466: mkdir d11/d90 0 2024-04-26T04:42:18.633 INFO:tasks.workunit.client.1.smithi104.stdout:9/467: mknod d11/d15/d32/d4c/d7c/c91 0 2024-04-26T04:42:18.633 INFO:tasks.workunit.client.1.smithi104.stdout:9/468: dread - d11/d53/f85 zero size 2024-04-26T04:42:18.634 INFO:tasks.workunit.client.1.smithi104.stdout:9/469: mkdir d11/d15/d32/d35/d7b/d92 0 2024-04-26T04:42:18.634 INFO:tasks.workunit.client.1.smithi104.stdout:9/470: chown d11/d15/d1b/f37 77 1 2024-04-26T04:42:18.635 INFO:tasks.workunit.client.0.smithi090.stdout:5/358: write d4/d1a/d3f/f52 [4915218,63363] 0 2024-04-26T04:42:18.635 INFO:tasks.workunit.client.0.smithi090.stdout:5/359: write d4/d1a/df/d4a/f51 [2182555,111492] 0 2024-04-26T04:42:18.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/471: rename d11/d15/f67 to d11/d15/d32/f93 0 2024-04-26T04:42:18.636 INFO:tasks.workunit.client.1.smithi104.stdout:9/472: readlink d11/d15/l2b 0 2024-04-26T04:42:18.636 INFO:tasks.workunit.client.1.smithi104.stdout:9/473: fsync d11/d1d/d34/f6c 0 2024-04-26T04:42:18.636 INFO:tasks.workunit.client.1.smithi104.stdout:9/474: mkdir d11/d1d/d71/d94 0 2024-04-26T04:42:18.638 INFO:tasks.workunit.client.0.smithi090.stdout:5/360: chown d4/f13 59583006 1 2024-04-26T04:42:18.638 INFO:tasks.workunit.client.0.smithi090.stdout:5/361: creat d4/d1a/df/d36/d42/d56/f75 x:0 0 0 2024-04-26T04:42:18.641 INFO:tasks.workunit.client.1.smithi104.stdout:6/707: dread d0/d9/d13/d41/d82/dc8/fb6 [0,4194304] 0 2024-04-26T04:42:18.648 INFO:tasks.workunit.client.1.smithi104.stdout:0/520: dwrite d0/d2/d21/d27/f6d [4194304,4194304] 0 2024-04-26T04:42:18.649 INFO:tasks.workunit.client.1.smithi104.stdout:8/525: dwrite d0/fe [0,4194304] 0 2024-04-26T04:42:18.650 INFO:tasks.workunit.client.0.smithi090.stdout:8/284: dwrite dc/d1f/f36 [0,4194304] 0 2024-04-26T04:42:18.651 INFO:tasks.workunit.client.1.smithi104.stdout:3/627: dwrite d5/d9/fe [0,4194304] 0 2024-04-26T04:42:18.652 INFO:tasks.workunit.client.0.smithi090.stdout:8/285: rename dc/d1f/f43 to dc/d1f/d49/d3b/d3d/d57/f60 0 2024-04-26T04:42:18.655 INFO:tasks.workunit.client.0.smithi090.stdout:8/286: mknod dc/d1f/d49/d39/c61 0 2024-04-26T04:42:18.656 INFO:tasks.workunit.client.0.smithi090.stdout:8/287: write dc/d1f/f35 [755072,100021] 0 2024-04-26T04:42:18.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/316: dread d0/de/f39 [0,4194304] 0 2024-04-26T04:42:18.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/317: write d0/d4b/f52 [1157419,99087] 0 2024-04-26T04:42:18.659 INFO:tasks.workunit.client.0.smithi090.stdout:0/318: fsync d0/d9/f17 0 2024-04-26T04:42:18.661 INFO:tasks.workunit.client.1.smithi104.stdout:0/521: mknod d0/d2/d21/ca4 0 2024-04-26T04:42:18.662 INFO:tasks.workunit.client.1.smithi104.stdout:0/522: fsync d0/d2/d21/d49/d5e/d9c/f4b 0 2024-04-26T04:42:18.667 INFO:tasks.workunit.client.1.smithi104.stdout:3/628: mknod d5/db2/d46/d95/cda 0 2024-04-26T04:42:18.668 INFO:tasks.workunit.client.0.smithi090.stdout:0/319: symlink d0/d9/d45/d53/l70 0 2024-04-26T04:42:18.670 INFO:tasks.workunit.client.0.smithi090.stdout:0/320: truncate d0/de/f2c 1267075 0 2024-04-26T04:42:18.672 INFO:tasks.workunit.client.0.smithi090.stdout:7/331: truncate d0/db/df/d4d/f63 561308 0 2024-04-26T04:42:18.673 INFO:tasks.workunit.client.1.smithi104.stdout:0/523: rmdir d0/d2/d21/d27 39 2024-04-26T04:42:18.673 INFO:tasks.workunit.client.0.smithi090.stdout:0/321: rename d0/de/f47 to d0/de/f71 0 2024-04-26T04:42:18.674 INFO:tasks.workunit.client.1.smithi104.stdout:0/524: dread d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/f8a [0,4194304] 0 2024-04-26T04:42:18.675 INFO:tasks.workunit.client.0.smithi090.stdout:7/332: symlink d0/d1/l78 0 2024-04-26T04:42:18.676 INFO:tasks.workunit.client.1.smithi104.stdout:0/525: symlink d0/d2/d10/la5 0 2024-04-26T04:42:18.677 INFO:tasks.workunit.client.0.smithi090.stdout:0/322: unlink d0/l4a 0 2024-04-26T04:42:18.677 INFO:tasks.workunit.client.1.smithi104.stdout:3/629: rename d5/d9/dc to d5/db2/da3/dd0/ddb 0 2024-04-26T04:42:18.678 INFO:tasks.workunit.client.1.smithi104.stdout:3/630: mknod d5/d9/d86/cdc 0 2024-04-26T04:42:18.678 INFO:tasks.workunit.client.1.smithi104.stdout:3/631: stat d5/db2/da3 0 2024-04-26T04:42:18.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/632: rename d5/db2/da3/dd0/ddb/d32/d47/lce to d5/d9/d80/dbc/ldd 0 2024-04-26T04:42:18.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/633: dread - d5/db2/da3/dd0/ddb/f36 zero size 2024-04-26T04:42:18.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/634: chown d5/db2/da3/dd0/ddb/d32/d48 251 1 2024-04-26T04:42:18.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/635: chown f3 126756 1 2024-04-26T04:42:18.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/636: creat d5/d9/d80/dbc/fde x:0 0 0 2024-04-26T04:42:18.680 INFO:tasks.workunit.client.1.smithi104.stdout:3/637: chown d5/db2/da3/fc4 1538683 1 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.0.smithi090.stdout:7/333: rename d0/d1/d4/d43/l77 to d0/db/df/d4d/l79 0 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.0.smithi090.stdout:7/334: chown d0/d1/d4/d43 676285 1 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.0.smithi090.stdout:7/335: chown d0/d1 166 1 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.1.smithi104.stdout:3/638: read d5/db2/da3/dd0/ddb/d69/fab [872694,12833] 0 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.1.smithi104.stdout:0/526: dread d0/d2/d10/f64 [4194304,4194304] 0 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.1.smithi104.stdout:0/527: dread - d0/d2/d10/d3d/f9e zero size 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.1.smithi104.stdout:0/528: creat d0/d2/d21/d49/d5e/d9c/d53/fa6 x:0 0 0 2024-04-26T04:42:18.681 INFO:tasks.workunit.client.1.smithi104.stdout:5/603: dwrite d0/f93 [0,4194304] 0 2024-04-26T04:42:18.682 INFO:tasks.workunit.client.0.smithi090.stdout:0/323: unlink d0/f38 0 2024-04-26T04:42:18.683 INFO:tasks.workunit.client.0.smithi090.stdout:7/336: creat d0/db/df/d1c/f7a x:0 0 0 2024-04-26T04:42:18.683 INFO:tasks.workunit.client.0.smithi090.stdout:7/337: read - d0/db/d15/f57 zero size 2024-04-26T04:42:18.685 INFO:tasks.workunit.client.0.smithi090.stdout:7/338: truncate d0/d1/d4/d3e/f4b 1575708 0 2024-04-26T04:42:18.685 INFO:tasks.workunit.client.1.smithi104.stdout:0/529: unlink d0/d2/d21/d27/d1e/d57/l7a 0 2024-04-26T04:42:18.685 INFO:tasks.workunit.client.1.smithi104.stdout:0/530: fdatasync d0/d2/d21/d49/d5e/d9c/d53/f9d 0 2024-04-26T04:42:18.685 INFO:tasks.workunit.client.1.smithi104.stdout:5/604: mknod d0/d1/d4/d2a/d6b/d91/dc7/cd0 0 2024-04-26T04:42:18.685 INFO:tasks.workunit.client.1.smithi104.stdout:3/639: dread d5/db2/da3/dd0/ddb/f16 [0,4194304] 0 2024-04-26T04:42:18.686 INFO:tasks.workunit.client.1.smithi104.stdout:0/531: dread d0/d2/d21/d27/d1e/f65 [0,4194304] 0 2024-04-26T04:42:18.686 INFO:tasks.workunit.client.1.smithi104.stdout:5/605: link d0/d1/c87 d0/d1/d4/d2a/cd1 0 2024-04-26T04:42:18.688 INFO:tasks.workunit.client.1.smithi104.stdout:3/640: creat d5/db2/da3/dd0/ddb/d32/d48/fdf x:0 0 0 2024-04-26T04:42:18.688 INFO:tasks.workunit.client.1.smithi104.stdout:3/641: read - d5/db2/da3/dd0/ddb/d69/fb6 zero size 2024-04-26T04:42:18.688 INFO:tasks.workunit.client.1.smithi104.stdout:3/642: write d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/fc6 [164064,39364] 0 2024-04-26T04:42:18.688 INFO:tasks.workunit.client.0.smithi090.stdout:5/362: dwrite d4/d1a/d45/f4f [0,4194304] 0 2024-04-26T04:42:18.689 INFO:tasks.workunit.client.0.smithi090.stdout:8/288: dwrite dc/f5c [0,4194304] 0 2024-04-26T04:42:18.690 INFO:tasks.workunit.client.1.smithi104.stdout:5/606: creat d0/d1/d4/d2a/fd2 x:0 0 0 2024-04-26T04:42:18.690 INFO:tasks.workunit.client.1.smithi104.stdout:5/607: stat d0/d1/d4/d2a/d2b/d37/d48/dc1 0 2024-04-26T04:42:18.690 INFO:tasks.workunit.client.1.smithi104.stdout:5/608: stat d0/d1/d4/d2a/d2b/d37/d48/dc1 0 2024-04-26T04:42:18.690 INFO:tasks.workunit.client.1.smithi104.stdout:0/532: dread d0/d2/d21/d49/d5e/f78 [0,4194304] 0 2024-04-26T04:42:18.690 INFO:tasks.workunit.client.1.smithi104.stdout:5/609: symlink d0/d1/d4/d2a/d6b/d91/dc7/ld3 0 2024-04-26T04:42:18.691 INFO:tasks.workunit.client.0.smithi090.stdout:5/363: unlink d4/d1a/l12 0 2024-04-26T04:42:18.692 INFO:tasks.workunit.client.1.smithi104.stdout:8/526: dwrite d0/f1a [4194304,4194304] 0 2024-04-26T04:42:18.692 INFO:tasks.workunit.client.1.smithi104.stdout:8/527: write d0/d1c/d27/f72 [6674290,12485] 0 2024-04-26T04:42:18.692 INFO:tasks.workunit.client.1.smithi104.stdout:8/528: write d0/d33/d35/d6d/f9c [266396,35700] 0 2024-04-26T04:42:18.692 INFO:tasks.workunit.client.1.smithi104.stdout:8/529: fsync d0/f9 0 2024-04-26T04:42:18.693 INFO:tasks.workunit.client.0.smithi090.stdout:7/339: dread d0/db/d15/d29/f18 [0,4194304] 0 2024-04-26T04:42:18.693 INFO:tasks.workunit.client.1.smithi104.stdout:3/643: dread d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/f74 [0,4194304] 0 2024-04-26T04:42:18.693 INFO:tasks.workunit.client.1.smithi104.stdout:5/610: rename d0/d1/d4/d2a/d6b/d91/dc8 to d0/d1/d1e/d98/dd4 0 2024-04-26T04:42:18.693 INFO:tasks.workunit.client.1.smithi104.stdout:3/644: fsync d5/db2/da3/dd0/ddb/f28 0 2024-04-26T04:42:18.696 INFO:tasks.workunit.client.1.smithi104.stdout:3/645: write d5/db2/f3b [83386,50909] 0 2024-04-26T04:42:18.698 INFO:tasks.workunit.client.0.smithi090.stdout:0/324: dread d0/d9/f13 [0,4194304] 0 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.0.smithi090.stdout:0/325: dread - d0/d4b/d5d/f6a zero size 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.0.smithi090.stdout:8/289: creat dc/d5a/f62 x:0 0 0 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:8/530: dread d0/d1c/d27/f40 [0,4194304] 0 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:8/531: write d0/d1c/d27/d47/f38 [489313,8773] 0 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:7/477: sync 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/579: sync 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:2/758: sync 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:7/478: readlink d2/l83 0 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/580: write d0/d6/d19/f3a [2017352,49294] 0 2024-04-26T04:42:18.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/581: chown d0/d6/d19/d1b/da5 230856 1 2024-04-26T04:42:18.700 INFO:tasks.workunit.client.0.smithi090.stdout:7/340: mkdir d0/db/df/d4d/d7b 0 2024-04-26T04:42:18.701 INFO:tasks.workunit.client.1.smithi104.stdout:7/479: mkdir d2/d4/d32/d5e/d9c 0 2024-04-26T04:42:18.701 INFO:tasks.workunit.client.1.smithi104.stdout:7/480: readlink d2/d4/l25 0 2024-04-26T04:42:18.701 INFO:tasks.workunit.client.1.smithi104.stdout:7/481: chown d2/d4/d32/d3d/l92 6156635 1 2024-04-26T04:42:18.704 INFO:tasks.workunit.client.1.smithi104.stdout:2/759: mknod d6/d1b/d68/cfa 0 2024-04-26T04:42:18.704 INFO:tasks.workunit.client.0.smithi090.stdout:8/290: truncate dc/d1f/d49/f28 2754165 0 2024-04-26T04:42:18.705 INFO:tasks.workunit.client.0.smithi090.stdout:4/313: sync 2024-04-26T04:42:18.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/189: sync 2024-04-26T04:42:18.706 INFO:tasks.workunit.client.0.smithi090.stdout:6/389: sync 2024-04-26T04:42:18.706 INFO:tasks.workunit.client.0.smithi090.stdout:2/277: sync 2024-04-26T04:42:18.706 INFO:tasks.workunit.client.0.smithi090.stdout:9/209: sync 2024-04-26T04:42:18.707 INFO:tasks.workunit.client.1.smithi104.stdout:1/582: creat d0/fbf x:0 0 0 2024-04-26T04:42:18.707 INFO:tasks.workunit.client.0.smithi090.stdout:7/341: dread d0/db/df/d4d/f63 [0,4194304] 0 2024-04-26T04:42:18.707 INFO:tasks.workunit.client.0.smithi090.stdout:7/342: chown d0/f4e 441452 1 2024-04-26T04:42:18.707 INFO:tasks.workunit.client.0.smithi090.stdout:7/343: chown d0/db 149252713 1 2024-04-26T04:42:18.708 INFO:tasks.workunit.client.0.smithi090.stdout:5/364: dread d4/d1a/df/d4a/f51 [0,4194304] 0 2024-04-26T04:42:18.708 INFO:tasks.workunit.client.0.smithi090.stdout:5/365: stat d4/d1a/d1f/f3a 0 2024-04-26T04:42:18.709 INFO:tasks.workunit.client.0.smithi090.stdout:0/326: rename d0/d9/l10 to d0/d4b/d5d/l72 0 2024-04-26T04:42:18.710 INFO:tasks.workunit.client.0.smithi090.stdout:2/278: mkdir db/d11/d14/d25/d2d/d48/d5b 0 2024-04-26T04:42:18.712 INFO:tasks.workunit.client.1.smithi104.stdout:2/760: link d6/d8/d3e/fd6 d6/d1b/ffb 0 2024-04-26T04:42:18.714 INFO:tasks.workunit.client.0.smithi090.stdout:5/366: unlink d4/d1a/df/d36/l68 0 2024-04-26T04:42:18.714 INFO:tasks.workunit.client.0.smithi090.stdout:5/367: write d4/d1a/df/d36/f5c [183218,51741] 0 2024-04-26T04:42:18.716 INFO:tasks.workunit.client.1.smithi104.stdout:1/583: creat d0/d6/fc0 x:0 0 0 2024-04-26T04:42:18.717 INFO:tasks.workunit.client.0.smithi090.stdout:4/314: rename d3/d13/d57 to d3/db/d60 0 2024-04-26T04:42:18.719 INFO:tasks.workunit.client.1.smithi104.stdout:1/584: link d0/d6/d2e/d32/d6a/l96 d0/d6/d9e/lc1 0 2024-04-26T04:42:18.720 INFO:tasks.workunit.client.0.smithi090.stdout:3/190: rename d1/f11 to d1/d17/d36/f40 0 2024-04-26T04:42:18.720 INFO:tasks.workunit.client.0.smithi090.stdout:3/191: write d1/d6/f14 [3543964,78286] 0 2024-04-26T04:42:18.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:18 smithi090 ceph-mon[26648]: pgmap v251: 65 pgs: 65 active+clean; 1.1 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 111 MiB/s rd, 141 MiB/s wr, 572 op/s 2024-04-26T04:42:18.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:18.722 INFO:tasks.workunit.client.1.smithi104.stdout:1/585: symlink d0/d6/d19/d21/d66/d30/d7e/lc2 0 2024-04-26T04:42:18.722 INFO:tasks.workunit.client.0.smithi090.stdout:4/315: link d3/db/d12/f4e d3/d13/f61 0 2024-04-26T04:42:18.724 INFO:tasks.workunit.client.1.smithi104.stdout:1/586: getdents d0/d6/d2e/d32/d6a 0 2024-04-26T04:42:18.724 INFO:tasks.workunit.client.1.smithi104.stdout:1/587: dread d0/d6/d19/d21/d66/d20/d2a/d56/d8c/f99 [0,4194304] 0 2024-04-26T04:42:18.725 INFO:tasks.workunit.client.0.smithi090.stdout:5/368: write d4/d1a/df/f22 [1206093,111432] 0 2024-04-26T04:42:18.726 INFO:tasks.workunit.client.0.smithi090.stdout:3/192: symlink d1/d17/d36/d33/l41 0 2024-04-26T04:42:18.726 INFO:tasks.workunit.client.1.smithi104.stdout:1/588: mkdir d0/d6/d2e/d32/d4c/db1/dc3 0 2024-04-26T04:42:18.727 INFO:tasks.workunit.client.0.smithi090.stdout:0/327: rename d0/d4b/d5d/l3f to d0/d4b/l73 0 2024-04-26T04:42:18.728 INFO:tasks.workunit.client.1.smithi104.stdout:1/589: creat d0/d6/d19/d1b/d35/fc4 x:0 0 0 2024-04-26T04:42:18.730 INFO:tasks.workunit.client.0.smithi090.stdout:3/193: mkdir d1/d17/d36/d33/d42 0 2024-04-26T04:42:18.730 INFO:tasks.workunit.client.0.smithi090.stdout:3/194: write d1/f29 [487697,24147] 0 2024-04-26T04:42:18.731 INFO:tasks.workunit.client.1.smithi104.stdout:2/761: dread d6/de/d1d/f83 [0,4194304] 0 2024-04-26T04:42:18.731 INFO:tasks.workunit.client.1.smithi104.stdout:2/762: mkdir d6/d1b/d68/d81/dfc 0 2024-04-26T04:42:18.734 INFO:tasks.workunit.client.0.smithi090.stdout:2/279: rename db/d3f to db/d11/d14/d27/d2f/d42/d5c 0 2024-04-26T04:42:18.737 INFO:tasks.workunit.client.0.smithi090.stdout:3/195: mkdir d1/d17/d36/d33/d43 0 2024-04-26T04:42:18.737 INFO:tasks.workunit.client.0.smithi090.stdout:3/196: write d1/d9/f38 [506199,23420] 0 2024-04-26T04:42:18.739 INFO:tasks.workunit.client.0.smithi090.stdout:0/328: link d0/d9/d45/l5a d0/df/l74 0 2024-04-26T04:42:18.739 INFO:tasks.workunit.client.0.smithi090.stdout:0/329: readlink d0/d4b/l73 0 2024-04-26T04:42:18.740 INFO:tasks.workunit.client.0.smithi090.stdout:0/330: chown d0/de/f71 212 1 2024-04-26T04:42:18.741 INFO:tasks.workunit.client.1.smithi104.stdout:7/482: dwrite d2/d4/d32/d35/f7a [0,4194304] 0 2024-04-26T04:42:18.742 INFO:tasks.workunit.client.1.smithi104.stdout:8/532: dwrite d0/d1c/d27/f66 [0,4194304] 0 2024-04-26T04:42:18.742 INFO:tasks.workunit.client.1.smithi104.stdout:8/533: write d0/d1c/d27/d47/f38 [1635960,127631] 0 2024-04-26T04:42:18.742 INFO:tasks.workunit.client.0.smithi090.stdout:9/210: dwrite db/f26 [0,4194304] 0 2024-04-26T04:42:18.742 INFO:tasks.workunit.client.0.smithi090.stdout:9/211: write db/dd/d18/f36 [1283975,59239] 0 2024-04-26T04:42:18.743 INFO:tasks.workunit.client.0.smithi090.stdout:6/390: dwrite d9/d1c/d4d/d6d/f84 [0,4194304] 0 2024-04-26T04:42:18.743 INFO:tasks.workunit.client.0.smithi090.stdout:3/197: mknod d1/d17/d36/c44 0 2024-04-26T04:42:18.744 INFO:tasks.workunit.client.0.smithi090.stdout:5/369: read d4/d1a/d45/f74 [3037364,86405] 0 2024-04-26T04:42:18.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/331: rename d0/cd to d0/d9/d64/c75 0 2024-04-26T04:42:18.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/332: write d0/d9/f41 [1438257,35585] 0 2024-04-26T04:42:18.744 INFO:tasks.workunit.client.0.smithi090.stdout:0/333: truncate d0/df/f67 312222 0 2024-04-26T04:42:18.745 INFO:tasks.workunit.client.1.smithi104.stdout:7/483: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d 0 2024-04-26T04:42:18.747 INFO:tasks.workunit.client.0.smithi090.stdout:9/212: mknod db/c48 0 2024-04-26T04:42:18.749 INFO:tasks.workunit.client.1.smithi104.stdout:8/534: link d0/c41 d0/d9a/caa 0 2024-04-26T04:42:18.750 INFO:tasks.workunit.client.1.smithi104.stdout:7/484: mkdir d2/d4/d32/d3d/d9e 0 2024-04-26T04:42:18.751 INFO:tasks.workunit.client.1.smithi104.stdout:7/485: write d2/d4/f33 [4371434,61802] 0 2024-04-26T04:42:18.751 INFO:tasks.workunit.client.0.smithi090.stdout:2/280: dread db/d11/d14/d25/d41/f54 [0,4194304] 0 2024-04-26T04:42:18.752 INFO:tasks.workunit.client.0.smithi090.stdout:6/391: mkdir d9/d1c/d4d/d58/d94 0 2024-04-26T04:42:18.753 INFO:tasks.workunit.client.0.smithi090.stdout:3/198: truncate d1/f8 6235919 0 2024-04-26T04:42:18.759 INFO:tasks.workunit.client.0.smithi090.stdout:9/213: mknod db/d16/c49 0 2024-04-26T04:42:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:18 smithi104 ceph-mon[32369]: pgmap v251: 65 pgs: 65 active+clean; 1.1 GiB data, 3.4 GiB used, 533 GiB / 536 GiB avail; 111 MiB/s rd, 141 MiB/s wr, 572 op/s 2024-04-26T04:42:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:18.762 INFO:tasks.workunit.client.0.smithi090.stdout:3/199: symlink d1/d17/d36/d2a/d3a/l45 0 2024-04-26T04:42:18.762 INFO:tasks.workunit.client.0.smithi090.stdout:3/200: fdatasync d1/d17/d36/f24 0 2024-04-26T04:42:18.762 INFO:tasks.workunit.client.0.smithi090.stdout:3/201: dread - d1/d17/d36/f32 zero size 2024-04-26T04:42:18.765 INFO:tasks.workunit.client.1.smithi104.stdout:7/486: dread d2/d4/d32/d3d/d47/f61 [0,4194304] 0 2024-04-26T04:42:18.768 INFO:tasks.workunit.client.0.smithi090.stdout:0/334: dread d0/d4b/d5d/f24 [0,4194304] 0 2024-04-26T04:42:18.768 INFO:tasks.workunit.client.0.smithi090.stdout:0/335: fsync d0/de/f2f 0 2024-04-26T04:42:18.768 INFO:tasks.workunit.client.0.smithi090.stdout:0/336: fsync d0/d9/f13 0 2024-04-26T04:42:18.770 INFO:tasks.workunit.client.0.smithi090.stdout:0/337: unlink d0/d4b/d5d/f24 0 2024-04-26T04:42:18.770 INFO:tasks.workunit.client.0.smithi090.stdout:0/338: write d0/d9/d45/f55 [1004547,109146] 0 2024-04-26T04:42:18.771 INFO:tasks.workunit.client.0.smithi090.stdout:0/339: dread - d0/d4b/d5d/d48/f56 zero size 2024-04-26T04:42:18.772 INFO:tasks.workunit.client.0.smithi090.stdout:0/340: truncate d0/d9/f3e 2354633 0 2024-04-26T04:42:18.773 INFO:tasks.workunit.client.1.smithi104.stdout:1/590: dwrite d0/d6/d19/d21/d66/f83 [0,4194304] 0 2024-04-26T04:42:18.774 INFO:tasks.workunit.client.0.smithi090.stdout:0/341: mknod d0/c76 0 2024-04-26T04:42:18.776 INFO:tasks.workunit.client.1.smithi104.stdout:1/591: link d0/d6/d19/d1b/d35/c65 d0/d6/d19/d21/d66/d20/d2a/d56/cc5 0 2024-04-26T04:42:18.784 INFO:tasks.workunit.client.1.smithi104.stdout:2/763: dwrite d6/d8/d5a/f54 [0,4194304] 0 2024-04-26T04:42:18.820 INFO:tasks.workunit.client.1.smithi104.stdout:7/487: dwrite d2/d4/d32/d35/d37/f3a [0,4194304] 0 2024-04-26T04:42:18.820 INFO:tasks.workunit.client.1.smithi104.stdout:8/535: fdatasync d0/d1c/d27/d47/f38 0 2024-04-26T04:42:18.820 INFO:tasks.workunit.client.1.smithi104.stdout:8/536: unlink d0/d1c/d27/f5c 0 2024-04-26T04:42:18.820 INFO:tasks.workunit.client.1.smithi104.stdout:1/592: dwrite d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:18.820 INFO:tasks.workunit.client.1.smithi104.stdout:1/593: truncate d0/d6/d19/d21/d66/d20/d2a/d56/d8c/f99 287651 0 2024-04-26T04:42:18.821 INFO:tasks.workunit.client.1.smithi104.stdout:1/594: fsync d0/d6/d19/d21/d59/f6e 0 2024-04-26T04:42:18.821 INFO:tasks.workunit.client.1.smithi104.stdout:1/595: mkdir d0/d6/d19/d21/d66/d30/d7e/dc6 0 2024-04-26T04:42:18.821 INFO:tasks.workunit.client.1.smithi104.stdout:1/596: stat d0/d6/d19/d21/d66/d20/d2a/f6f 0 2024-04-26T04:42:18.821 INFO:tasks.workunit.client.1.smithi104.stdout:1/597: truncate d0/d6/d2e/f7c 1214553 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:6/392: dwrite d9/d1c/d4d/d6d/f93 [0,4194304] 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/370: dwrite d4/d1a/df/f28 [0,4194304] 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/281: dwrite db/f5a [0,4194304] 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:0/342: dread d0/f3b [0,4194304] 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:0/343: fdatasync d0/d9/d45/f55 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/282: mkdir db/d11/d14/d1b/d20/d4c/d5d 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/283: fsync db/dc/f12 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/284: write db/d11/d14/d25/f4e [1388587,18826] 0 2024-04-26T04:42:18.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/285: chown db/d11/d14/d25/d41/f54 1671347265 1 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:0/344: creat d0/df/f77 x:0 0 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/286: creat db/d11/d14/d25/f5e x:0 0 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:0/345: unlink d0/f3b 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/287: symlink db/d11/d14/d27/d2f/d42/l5f 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/288: getdents db/d11/d14 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/289: dread db/d11/d14/d25/f3a [0,4194304] 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:0/346: dread d0/d9/f13 [0,4194304] 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:0/347: write d0/d9/f3e [1620593,40702] 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/290: symlink db/d11/d14/d1b/d20/l60 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/291: fdatasync db/d11/d14/d25/d41/f54 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/292: truncate db/d11/d14/d1b/d20/f24 4145903 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:0/348: link d0/d9/d45/f59 d0/de/f78 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/293: creat db/d11/d14/d27/d28/f61 x:0 0 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/294: rename db/f5a to db/d59/f62 0 2024-04-26T04:42:18.823 INFO:tasks.workunit.client.0.smithi090.stdout:6/393: dwrite d9/d1c/d4d/d58/f8c [0,4194304] 0 2024-04-26T04:42:18.824 INFO:tasks.workunit.client.0.smithi090.stdout:2/295: write db/d11/d14/d25/d2d/f36 [33239,112447] 0 2024-04-26T04:42:18.824 INFO:tasks.workunit.client.0.smithi090.stdout:2/296: mkdir db/d11/d14/d63 0 2024-04-26T04:42:18.824 INFO:tasks.workunit.client.1.smithi104.stdout:1/598: truncate d0/d6/d19/d21/d66/d20/d2a/f6f 2099042 0 2024-04-26T04:42:18.824 INFO:tasks.workunit.client.1.smithi104.stdout:1/599: creat d0/d6/d2e/d32/fc7 x:0 0 0 2024-04-26T04:42:18.825 INFO:tasks.workunit.client.0.smithi090.stdout:2/297: symlink db/d11/d14/d25/d2d/l64 0 2024-04-26T04:42:18.825 INFO:tasks.workunit.client.0.smithi090.stdout:2/298: write db/d11/d14/d25/f3e [546451,99834] 0 2024-04-26T04:42:18.828 INFO:tasks.workunit.client.0.smithi090.stdout:2/299: mkdir db/d11/d14/d25/d65 0 2024-04-26T04:42:18.829 INFO:tasks.workunit.client.0.smithi090.stdout:2/300: mkdir db/d59/d66 0 2024-04-26T04:42:18.829 INFO:tasks.workunit.client.0.smithi090.stdout:2/301: getdents db/d11/d14/d1b/d20/d4c/d5d 0 2024-04-26T04:42:18.832 INFO:tasks.workunit.client.0.smithi090.stdout:6/394: dread d9/d1c/d4d/d6d/d92/f25 [4194304,4194304] 0 2024-04-26T04:42:18.832 INFO:tasks.workunit.client.0.smithi090.stdout:6/395: creat d9/d1c/d31/f95 x:0 0 0 2024-04-26T04:42:18.832 INFO:tasks.workunit.client.0.smithi090.stdout:6/396: write d9/f5c [114678,63248] 0 2024-04-26T04:42:18.833 INFO:tasks.workunit.client.0.smithi090.stdout:6/397: symlink d9/d21/d40/d4c/l96 0 2024-04-26T04:42:18.833 INFO:tasks.workunit.client.0.smithi090.stdout:2/302: dread db/d11/d14/d25/f49 [0,4194304] 0 2024-04-26T04:42:18.836 INFO:tasks.workunit.client.0.smithi090.stdout:2/303: mknod db/d11/d14/d27/d28/c67 0 2024-04-26T04:42:18.836 INFO:tasks.workunit.client.0.smithi090.stdout:2/304: readlink db/d11/l2e 0 2024-04-26T04:42:18.836 INFO:tasks.workunit.client.0.smithi090.stdout:2/305: dread - db/dc/f1f zero size 2024-04-26T04:42:18.836 INFO:tasks.workunit.client.0.smithi090.stdout:2/306: symlink db/d11/d14/d27/d28/l68 0 2024-04-26T04:42:18.837 INFO:tasks.workunit.client.0.smithi090.stdout:2/307: mkdir db/d59/d66/d69 0 2024-04-26T04:42:18.837 INFO:tasks.workunit.client.0.smithi090.stdout:2/308: chown db/d11/d14/d25/f37 42781 1 2024-04-26T04:42:18.837 INFO:tasks.workunit.client.0.smithi090.stdout:2/309: readlink db/d11/l46 0 2024-04-26T04:42:18.845 INFO:tasks.workunit.client.1.smithi104.stdout:7/488: dwrite d2/f75 [0,4194304] 0 2024-04-26T04:42:18.845 INFO:tasks.workunit.client.1.smithi104.stdout:2/764: dwrite d6/d1b/d68/d81/fa2 [0,4194304] 0 2024-04-26T04:42:18.845 INFO:tasks.workunit.client.1.smithi104.stdout:1/600: dwrite d0/d6/d19/d1b/da5/fa6 [0,4194304] 0 2024-04-26T04:42:18.846 INFO:tasks.workunit.client.0.smithi090.stdout:5/371: dwrite d4/f46 [0,4194304] 0 2024-04-26T04:42:18.846 INFO:tasks.workunit.client.1.smithi104.stdout:8/537: dwrite d0/f4e [0,4194304] 0 2024-04-26T04:42:18.848 INFO:tasks.workunit.client.1.smithi104.stdout:2/765: truncate d6/f62 1619264 0 2024-04-26T04:42:18.854 INFO:tasks.workunit.client.1.smithi104.stdout:1/601: creat d0/d6/d19/d21/d59/fc8 x:0 0 0 2024-04-26T04:42:18.854 INFO:tasks.workunit.client.1.smithi104.stdout:1/602: dread - d0/fbf zero size 2024-04-26T04:42:18.855 INFO:tasks.workunit.client.1.smithi104.stdout:1/603: chown d0/d6/c34 6830 1 2024-04-26T04:42:18.859 INFO:tasks.workunit.client.1.smithi104.stdout:8/538: rename d0/d33/f2b to d0/d33/fab 0 2024-04-26T04:42:18.859 INFO:tasks.workunit.client.1.smithi104.stdout:8/539: chown d0/d33/d35/d6d/f9b 232 1 2024-04-26T04:42:18.859 INFO:tasks.workunit.client.1.smithi104.stdout:8/540: stat d0/d1c/d2d/c58 0 2024-04-26T04:42:18.860 INFO:tasks.workunit.client.0.smithi090.stdout:5/372: read d4/d1a/d3f/f44 [1963883,10293] 0 2024-04-26T04:42:18.862 INFO:tasks.workunit.client.1.smithi104.stdout:8/541: creat d0/d1c/d27/d47/d67/d76/d7f/fac x:0 0 0 2024-04-26T04:42:18.863 INFO:tasks.workunit.client.0.smithi090.stdout:5/373: link d4/f1e d4/d1a/d1f/d53/f76 0 2024-04-26T04:42:18.863 INFO:tasks.workunit.client.1.smithi104.stdout:8/542: rename d0/d1c/l5d to d0/d1c/d2d/d7a/lad 0 2024-04-26T04:42:18.867 INFO:tasks.workunit.client.0.smithi090.stdout:5/374: truncate d4/d1a/df/d4a/f55 471566 0 2024-04-26T04:42:18.867 INFO:tasks.workunit.client.0.smithi090.stdout:0/349: rmdir d0/de 39 2024-04-26T04:42:18.867 INFO:tasks.workunit.client.0.smithi090.stdout:5/375: creat d4/d1a/df/d72/f77 x:0 0 0 2024-04-26T04:42:18.868 INFO:tasks.workunit.client.0.smithi090.stdout:0/350: symlink d0/d4b/d6d/l79 0 2024-04-26T04:42:18.868 INFO:tasks.workunit.client.0.smithi090.stdout:0/351: chown d0/d9/d45/d53/l5b 56816184 1 2024-04-26T04:42:18.870 INFO:tasks.workunit.client.0.smithi090.stdout:5/376: creat d4/d1a/f78 x:0 0 0 2024-04-26T04:42:18.870 INFO:tasks.workunit.client.1.smithi104.stdout:1/604: write d0/d6/d49/f4a [2342124,96444] 0 2024-04-26T04:42:18.873 INFO:tasks.workunit.client.1.smithi104.stdout:7/489: dwrite d2/d4/d5a/f7b [0,4194304] 0 2024-04-26T04:42:18.873 INFO:tasks.workunit.client.0.smithi090.stdout:2/310: dwrite db/d11/d14/d27/d2f/f32 [0,4194304] 0 2024-04-26T04:42:18.873 INFO:tasks.workunit.client.0.smithi090.stdout:2/311: stat db/c21 0 2024-04-26T04:42:18.875 INFO:tasks.workunit.client.1.smithi104.stdout:2/766: dwrite d6/d1b/d68/feb [0,4194304] 0 2024-04-26T04:42:18.876 INFO:tasks.workunit.client.0.smithi090.stdout:0/352: link d0/df/f6c d0/f7a 0 2024-04-26T04:42:18.878 INFO:tasks.workunit.client.0.smithi090.stdout:0/353: mknod d0/d4b/d5d/c7b 0 2024-04-26T04:42:18.879 INFO:tasks.workunit.client.0.smithi090.stdout:0/354: write d0/df/f6b [388014,105890] 0 2024-04-26T04:42:18.883 INFO:tasks.workunit.client.0.smithi090.stdout:0/355: mkdir d0/d9/d7c 0 2024-04-26T04:42:18.895 INFO:tasks.workunit.client.1.smithi104.stdout:2/767: dread d6/d8/d3e/f2a [4194304,4194304] 0 2024-04-26T04:42:18.895 INFO:tasks.workunit.client.1.smithi104.stdout:2/768: write d6/d1b/d93/faa [2687227,16804] 0 2024-04-26T04:42:18.895 INFO:tasks.workunit.client.1.smithi104.stdout:2/769: chown d6/d8/d3e 0 1 2024-04-26T04:42:18.895 INFO:tasks.workunit.client.1.smithi104.stdout:2/770: dread - d6/d1b/d93/fc4 zero size 2024-04-26T04:42:18.895 INFO:tasks.workunit.client.1.smithi104.stdout:2/771: fsync d6/d1b/d68/f95 0 2024-04-26T04:42:18.896 INFO:tasks.workunit.client.1.smithi104.stdout:7/490: dread d2/d4/d5a/f68 [0,4194304] 0 2024-04-26T04:42:18.897 INFO:tasks.workunit.client.1.smithi104.stdout:7/491: mkdir d2/d4/d32/d5e/d9f 0 2024-04-26T04:42:18.898 INFO:tasks.workunit.client.1.smithi104.stdout:2/772: dread d6/de/d1d/f4a [0,4194304] 0 2024-04-26T04:42:18.898 INFO:tasks.workunit.client.1.smithi104.stdout:2/773: symlink d6/d8/d3e/lfd 0 2024-04-26T04:42:18.906 INFO:tasks.workunit.client.1.smithi104.stdout:4/619: sync 2024-04-26T04:42:18.908 INFO:tasks.workunit.client.1.smithi104.stdout:4/620: mknod d4/d7/d1d/d1e/d24/d48/cdc 0 2024-04-26T04:42:18.910 INFO:tasks.workunit.client.1.smithi104.stdout:1/605: dwrite d0/d6/d49/f87 [0,4194304] 0 2024-04-26T04:42:18.911 INFO:tasks.workunit.client.0.smithi090.stdout:0/356: dwrite d0/de/f2f [0,4194304] 0 2024-04-26T04:42:18.912 INFO:tasks.workunit.client.1.smithi104.stdout:7/492: dread d2/d4/d32/d35/f7a [0,4194304] 0 2024-04-26T04:42:18.912 INFO:tasks.workunit.client.1.smithi104.stdout:7/493: write d2/d4/f6e [374516,123881] 0 2024-04-26T04:42:18.912 INFO:tasks.workunit.client.1.smithi104.stdout:8/543: dwrite d0/d1c/d27/d47/d67/d76/d7f/fac [0,4194304] 0 2024-04-26T04:42:18.913 INFO:tasks.workunit.client.0.smithi090.stdout:2/312: dwrite db/d11/d14/d25/d41/f54 [0,4194304] 0 2024-04-26T04:42:18.913 INFO:tasks.workunit.client.0.smithi090.stdout:2/313: truncate db/dc/f1f 769265 0 2024-04-26T04:42:18.914 INFO:tasks.workunit.client.1.smithi104.stdout:1/606: rename d0/d6/d19/d21/d66/d29/f39 to d0/d6/d19/d1b/d35/fc9 0 2024-04-26T04:42:18.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/357: mkdir d0/d4b/d5d/d48/d7d 0 2024-04-26T04:42:18.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/358: chown d0/df/f6c 119 1 2024-04-26T04:42:18.916 INFO:tasks.workunit.client.1.smithi104.stdout:7/494: mkdir d2/d4/d5a/da0 0 2024-04-26T04:42:18.916 INFO:tasks.workunit.client.1.smithi104.stdout:7/495: readlink d2/d4/d1d/d34/l77 0 2024-04-26T04:42:18.916 INFO:tasks.workunit.client.1.smithi104.stdout:7/496: write d2/d4/d5a/f99 [215564,25182] 0 2024-04-26T04:42:18.920 INFO:tasks.workunit.client.1.smithi104.stdout:7/497: stat d2/d4/l25 0 2024-04-26T04:42:18.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/314: unlink db/dc/f1e 0 2024-04-26T04:42:18.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/315: dread - db/d11/d14/d27/d2f/d42/d5c/f4f zero size 2024-04-26T04:42:18.923 INFO:tasks.workunit.client.1.smithi104.stdout:7/498: rename c1 to d2/d4/d32/d3d/d9e/ca1 0 2024-04-26T04:42:18.923 INFO:tasks.workunit.client.1.smithi104.stdout:4/621: dwrite d4/d7/d1d/f43 [0,4194304] 0 2024-04-26T04:42:18.923 INFO:tasks.workunit.client.1.smithi104.stdout:4/622: chown d4/d7/d1d/d1e/d15/l25 494080881 1 2024-04-26T04:42:18.924 INFO:tasks.workunit.client.0.smithi090.stdout:2/316: creat db/d11/d14/d25/d2d/f6a x:0 0 0 2024-04-26T04:42:18.924 INFO:tasks.workunit.client.1.smithi104.stdout:7/499: rename d2/d4/d32/d35/f7a to d2/d4/d32/d35/d44/fa2 0 2024-04-26T04:42:18.924 INFO:tasks.workunit.client.0.smithi090.stdout:0/359: rmdir d0/df 39 2024-04-26T04:42:18.925 INFO:tasks.workunit.client.1.smithi104.stdout:1/607: dread d0/d6/d19/d21/d66/d20/d2a/f31 [0,4194304] 0 2024-04-26T04:42:18.926 INFO:tasks.workunit.client.0.smithi090.stdout:2/317: mknod db/d11/d14/d63/c6b 0 2024-04-26T04:42:18.927 INFO:tasks.workunit.client.1.smithi104.stdout:1/608: symlink d0/d6/d19/d21/d66/d20/d2a/lca 0 2024-04-26T04:42:18.927 INFO:tasks.workunit.client.0.smithi090.stdout:0/360: rmdir d0/d9/d45/d53 39 2024-04-26T04:42:18.928 INFO:tasks.workunit.client.1.smithi104.stdout:1/609: dread d0/d6/d19/d21/d59/f73 [0,4194304] 0 2024-04-26T04:42:18.928 INFO:tasks.workunit.client.1.smithi104.stdout:1/610: read - d0/d6/fc0 zero size 2024-04-26T04:42:18.928 INFO:tasks.workunit.client.1.smithi104.stdout:4/623: write d4/d7/f8 [743847,113464] 0 2024-04-26T04:42:18.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/361: mkdir d0/d9/d7c/d7e 0 2024-04-26T04:42:18.931 INFO:tasks.workunit.client.1.smithi104.stdout:1/611: link d0/d6/d19/d21/l71 d0/d6/d19/d21/d66/d29/d3f/lcb 0 2024-04-26T04:42:18.931 INFO:tasks.workunit.client.1.smithi104.stdout:1/612: fsync d0/d6/d2e/f86 0 2024-04-26T04:42:18.931 INFO:tasks.workunit.client.1.smithi104.stdout:1/613: write d0/d6/d19/d21/d66/f41 [2727247,106050] 0 2024-04-26T04:42:18.931 INFO:tasks.workunit.client.1.smithi104.stdout:1/614: dread - d0/d6/d2e/d32/fac zero size 2024-04-26T04:42:18.932 INFO:tasks.workunit.client.1.smithi104.stdout:1/615: readlink d0/d6/d9e/lc1 0 2024-04-26T04:42:18.934 INFO:tasks.workunit.client.0.smithi090.stdout:0/362: rmdir d0/d9/d7c 39 2024-04-26T04:42:18.934 INFO:tasks.workunit.client.0.smithi090.stdout:0/363: chown d0/de/f36 8460795 1 2024-04-26T04:42:18.939 INFO:tasks.workunit.client.0.smithi090.stdout:2/318: rmdir db 39 2024-04-26T04:42:18.940 INFO:tasks.workunit.client.1.smithi104.stdout:7/500: dwrite d2/d4/d1d/d34/f40 [0,4194304] 0 2024-04-26T04:42:18.945 INFO:tasks.workunit.client.1.smithi104.stdout:1/616: write d0/d6/d49/f68 [770634,56026] 0 2024-04-26T04:42:18.945 INFO:tasks.workunit.client.1.smithi104.stdout:1/617: stat d0/d6/d2e/d32/d4d/da1 0 2024-04-26T04:42:18.948 INFO:tasks.workunit.client.1.smithi104.stdout:1/618: creat d0/d6/d19/d21/d66/d20/d2a/fcc x:0 0 0 2024-04-26T04:42:18.948 INFO:tasks.workunit.client.1.smithi104.stdout:1/619: fsync d0/d6/d19/d21/d66/f83 0 2024-04-26T04:42:18.950 INFO:tasks.workunit.client.1.smithi104.stdout:1/620: creat d0/d6/d19/d21/d66/d30/fcd x:0 0 0 2024-04-26T04:42:18.950 INFO:tasks.workunit.client.1.smithi104.stdout:7/501: dread d2/d4/d32/d35/d37/f6f [0,4194304] 0 2024-04-26T04:42:18.950 INFO:tasks.workunit.client.1.smithi104.stdout:1/621: creat d0/d6/d19/d21/d66/d30/d7e/dc6/fce x:0 0 0 2024-04-26T04:42:18.951 INFO:tasks.workunit.client.1.smithi104.stdout:7/502: getdents d2/d4/d32/d3d/d41/d5f/d6c 0 2024-04-26T04:42:18.952 INFO:tasks.workunit.client.1.smithi104.stdout:1/622: getdents d0/d6/d19/d21/d66/d29/d3f 0 2024-04-26T04:42:18.953 INFO:tasks.workunit.client.1.smithi104.stdout:7/503: creat d2/d4/d32/d3d/d41/d5f/d6d/fa3 x:0 0 0 2024-04-26T04:42:18.954 INFO:tasks.workunit.client.1.smithi104.stdout:7/504: creat d2/d4/d32/d35/d44/fa4 x:0 0 0 2024-04-26T04:42:18.954 INFO:tasks.workunit.client.1.smithi104.stdout:1/623: mknod d0/d6/d2e/d32/d4c/db1/dc3/ccf 0 2024-04-26T04:42:18.955 INFO:tasks.workunit.client.1.smithi104.stdout:7/505: symlink d2/d4/d32/d3d/d41/d5f/la5 0 2024-04-26T04:42:18.955 INFO:tasks.workunit.client.1.smithi104.stdout:7/506: fdatasync d2/d4/d32/d60/f8c 0 2024-04-26T04:42:18.958 INFO:tasks.workunit.client.1.smithi104.stdout:7/507: symlink d2/d4/d32/d5e/la6 0 2024-04-26T04:42:18.958 INFO:tasks.workunit.client.1.smithi104.stdout:7/508: chown d2/d4/l22 36404 1 2024-04-26T04:42:18.958 INFO:tasks.workunit.client.1.smithi104.stdout:7/509: chown d2/d4/d32/d3d/d41/d5f/d6c/d89 41791840 1 2024-04-26T04:42:18.958 INFO:tasks.workunit.client.1.smithi104.stdout:1/624: rmdir d0/d6/d19/d21/d66/d20/d2a/d56/d8c 39 2024-04-26T04:42:18.958 INFO:tasks.workunit.client.1.smithi104.stdout:1/625: stat d0/d6/d2e/d32/d4c 0 2024-04-26T04:42:18.958 INFO:tasks.workunit.client.1.smithi104.stdout:1/626: chown d0/d6/d2e/d32/d4d/da1 51085871 1 2024-04-26T04:42:18.959 INFO:tasks.workunit.client.1.smithi104.stdout:7/510: creat d2/d4/d1d/fa7 x:0 0 0 2024-04-26T04:42:18.960 INFO:tasks.workunit.client.1.smithi104.stdout:7/511: unlink d2/d4/d32/d35/c43 0 2024-04-26T04:42:18.960 INFO:tasks.workunit.client.1.smithi104.stdout:4/624: dwrite d4/d7/d1d/d1e/d24/d59/fbc [0,4194304] 0 2024-04-26T04:42:18.961 INFO:tasks.workunit.client.1.smithi104.stdout:4/625: creat d4/d20/d55/d56/da3/fdd x:0 0 0 2024-04-26T04:42:18.961 INFO:tasks.workunit.client.1.smithi104.stdout:4/626: dread - d4/d7/d1d/d1e/d24/d4a/f50 zero size 2024-04-26T04:42:18.961 INFO:tasks.workunit.client.1.smithi104.stdout:4/627: chown d4/db/f12 331773 1 2024-04-26T04:42:18.963 INFO:tasks.workunit.client.0.smithi090.stdout:2/319: dwrite db/d11/d14/d25/f49 [0,4194304] 0 2024-04-26T04:42:18.963 INFO:tasks.workunit.client.0.smithi090.stdout:0/364: dwrite d0/de/f36 [0,4194304] 0 2024-04-26T04:42:18.965 INFO:tasks.workunit.client.1.smithi104.stdout:4/628: symlink d4/d7/d1d/d1e/d8f/lde 0 2024-04-26T04:42:18.965 INFO:tasks.workunit.client.1.smithi104.stdout:4/629: fsync d4/d7/d1d/d1e/d24/f68 0 2024-04-26T04:42:18.965 INFO:tasks.workunit.client.1.smithi104.stdout:4/630: creat d4/d7/fdf x:0 0 0 2024-04-26T04:42:18.965 INFO:tasks.workunit.client.0.smithi090.stdout:2/320: truncate db/d11/d14/d25/f3e 305323 0 2024-04-26T04:42:18.968 INFO:tasks.workunit.client.0.smithi090.stdout:0/365: symlink d0/d9/d64/l7f 0 2024-04-26T04:42:18.968 INFO:tasks.workunit.client.0.smithi090.stdout:0/366: chown d0/d4b/f52 870 1 2024-04-26T04:42:18.968 INFO:tasks.workunit.client.0.smithi090.stdout:0/367: write d0/fc [962160,95756] 0 2024-04-26T04:42:18.968 INFO:tasks.workunit.client.0.smithi090.stdout:0/368: write d0/de/f2c [1293763,66191] 0 2024-04-26T04:42:18.968 INFO:tasks.workunit.client.0.smithi090.stdout:0/369: write d0/df/f67 [1079063,25644] 0 2024-04-26T04:42:18.968 INFO:tasks.workunit.client.0.smithi090.stdout:2/321: rmdir db/d11/d14/d1b/d20/d4c 39 2024-04-26T04:42:18.969 INFO:tasks.workunit.client.1.smithi104.stdout:7/512: getdents d2/d4/d32/d5e 0 2024-04-26T04:42:18.969 INFO:tasks.workunit.client.1.smithi104.stdout:7/513: creat d2/d4/d1d/fa8 x:0 0 0 2024-04-26T04:42:18.970 INFO:tasks.workunit.client.0.smithi090.stdout:0/370: creat d0/d4b/d5d/d48/f80 x:0 0 0 2024-04-26T04:42:18.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/322: creat db/d11/d14/d1b/f6c x:0 0 0 2024-04-26T04:42:18.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/323: read - db/d11/d14/d1b/f6c zero size 2024-04-26T04:42:18.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/324: creat db/d11/d14/d1b/f6d x:0 0 0 2024-04-26T04:42:18.972 INFO:tasks.workunit.client.1.smithi104.stdout:4/631: mknod d4/d7/d1d/d1e/d15/dd8/ce0 0 2024-04-26T04:42:18.972 INFO:tasks.workunit.client.1.smithi104.stdout:7/514: mknod d2/d4/d32/d3d/d41/d7e/ca9 0 2024-04-26T04:42:18.972 INFO:tasks.workunit.client.1.smithi104.stdout:7/515: creat d2/d4/d32/d3d/d41/d5f/d6d/faa x:0 0 0 2024-04-26T04:42:18.973 INFO:tasks.workunit.client.1.smithi104.stdout:1/627: dwrite d0/d6/d19/f3a [0,4194304] 0 2024-04-26T04:42:18.975 INFO:tasks.workunit.client.0.smithi090.stdout:2/325: link db/d11/d14/d27/d28/f56 db/d11/d14/d27/d28/f6e 0 2024-04-26T04:42:18.977 INFO:tasks.workunit.client.1.smithi104.stdout:4/632: getdents d4/d7/d1d/d1e/d24/d59 0 2024-04-26T04:42:18.977 INFO:tasks.workunit.client.0.smithi090.stdout:0/371: symlink d0/de/l81 0 2024-04-26T04:42:18.978 INFO:tasks.workunit.client.1.smithi104.stdout:7/516: link d2/d4/d32/d3d/d41/d5f/d6d/faa d2/d4/d5a/da0/fab 0 2024-04-26T04:42:18.978 INFO:tasks.workunit.client.1.smithi104.stdout:7/517: dread - d2/d4/d1d/fa8 zero size 2024-04-26T04:42:18.979 INFO:tasks.workunit.client.0.smithi090.stdout:2/326: mkdir db/dc/d6f 0 2024-04-26T04:42:18.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/372: chown d0/de/l22 59 1 2024-04-26T04:42:18.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/373: dread - d0/f7a zero size 2024-04-26T04:42:18.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/374: readlink d0/df/l1a 0 2024-04-26T04:42:18.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/375: chown d0/d9/d45/l62 31314 1 2024-04-26T04:42:18.981 INFO:tasks.workunit.client.0.smithi090.stdout:2/327: mkdir db/d11/d14/d27/d2f/d70 0 2024-04-26T04:42:18.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/376: write d0/d9/d45/f55 [467955,45906] 0 2024-04-26T04:42:18.981 INFO:tasks.workunit.client.0.smithi090.stdout:0/377: readlink d0/d4b/d5d/l32 0 2024-04-26T04:42:18.982 INFO:tasks.workunit.client.0.smithi090.stdout:2/328: mkdir db/d11/d14/d27/d2f/d70/d71 0 2024-04-26T04:42:18.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/329: read - db/d11/d14/f4a zero size 2024-04-26T04:42:18.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/330: readlink db/dc/l2c 0 2024-04-26T04:42:18.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/331: fsync db/dc/f3c 0 2024-04-26T04:42:18.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/332: chown db/d11/d14/d25/f49 192 1 2024-04-26T04:42:18.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/333: dread - db/d11/d14/d27/d28/f61 zero size 2024-04-26T04:42:18.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/334: dread - db/d11/d14/d27/d28/f6e zero size 2024-04-26T04:42:18.985 INFO:tasks.workunit.client.0.smithi090.stdout:0/378: symlink d0/df/l82 0 2024-04-26T04:42:18.985 INFO:tasks.workunit.client.0.smithi090.stdout:0/379: fdatasync d0/d9/f43 0 2024-04-26T04:42:18.986 INFO:tasks.workunit.client.1.smithi104.stdout:1/628: dread d0/d6/d19/d21/d66/d30/d81/d9d/f9f [0,4194304] 0 2024-04-26T04:42:18.987 INFO:tasks.workunit.client.0.smithi090.stdout:0/380: write d0/d4b/f52 [131932,38204] 0 2024-04-26T04:42:18.987 INFO:tasks.workunit.client.1.smithi104.stdout:1/629: stat d0/d6/d2e/d32/l97 0 2024-04-26T04:42:18.989 INFO:tasks.workunit.client.1.smithi104.stdout:1/630: mkdir d0/d6/d2e/d32/d4c/dd0 0 2024-04-26T04:42:18.989 INFO:tasks.workunit.client.1.smithi104.stdout:1/631: readlink d0/d6/d2e/lad 0 2024-04-26T04:42:18.990 INFO:tasks.workunit.client.1.smithi104.stdout:4/633: dread d4/d7/d1d/d1e/d24/f76 [0,4194304] 0 2024-04-26T04:42:18.990 INFO:tasks.workunit.client.1.smithi104.stdout:4/634: dread - d4/f69 zero size 2024-04-26T04:42:18.990 INFO:tasks.workunit.client.1.smithi104.stdout:1/632: rmdir d0/d6/d9e 39 2024-04-26T04:42:18.990 INFO:tasks.workunit.client.1.smithi104.stdout:1/633: stat d0/d6/d19/d1b/d35/c70 0 2024-04-26T04:42:18.991 INFO:tasks.workunit.client.0.smithi090.stdout:0/381: rename d0/d9/d45/l69 to d0/d9/l83 0 2024-04-26T04:42:18.992 INFO:tasks.workunit.client.0.smithi090.stdout:0/382: write d0/d9/f41 [1508079,85230] 0 2024-04-26T04:42:18.992 INFO:tasks.workunit.client.1.smithi104.stdout:4/635: rmdir d4/d7/d1d/d1e/d24/d59/dc1 39 2024-04-26T04:42:18.992 INFO:tasks.workunit.client.1.smithi104.stdout:7/518: read d2/d4/d32/d35/d37/f3a [1565633,97097] 0 2024-04-26T04:42:18.992 INFO:tasks.workunit.client.1.smithi104.stdout:1/634: dread d0/d6/d19/d21/d66/d20/d2a/d56/d8c/f99 [0,4194304] 0 2024-04-26T04:42:18.992 INFO:tasks.workunit.client.1.smithi104.stdout:1/635: chown d0/d6/d19/d21/d66/d30/d7e/dc6/fce 1568825 1 2024-04-26T04:42:18.993 INFO:tasks.workunit.client.1.smithi104.stdout:4/636: symlink d4/d20/le1 0 2024-04-26T04:42:18.993 INFO:tasks.workunit.client.1.smithi104.stdout:4/637: write d4/db/d6b/fa2 [471052,81849] 0 2024-04-26T04:42:18.993 INFO:tasks.workunit.client.1.smithi104.stdout:7/519: dread d2/d4/d32/d35/d44/f91 [0,4194304] 0 2024-04-26T04:42:18.996 INFO:tasks.workunit.client.0.smithi090.stdout:0/383: fsync d0/d4b/f52 0 2024-04-26T04:42:18.998 INFO:tasks.workunit.client.1.smithi104.stdout:9/475: sync 2024-04-26T04:42:18.998 INFO:tasks.workunit.client.1.smithi104.stdout:9/476: creat d11/f95 x:0 0 0 2024-04-26T04:42:18.998 INFO:tasks.workunit.client.1.smithi104.stdout:1/636: read d0/f14 [8295471,32816] 0 2024-04-26T04:42:18.999 INFO:tasks.workunit.client.1.smithi104.stdout:1/637: creat d0/d6/d19/d1b/d35/fd1 x:0 0 0 2024-04-26T04:42:19.000 INFO:tasks.workunit.client.1.smithi104.stdout:4/638: read d4/f98 [3868607,57280] 0 2024-04-26T04:42:19.000 INFO:tasks.workunit.client.1.smithi104.stdout:9/477: link d11/d15/f18 d11/d15/d1b/d6d/f96 0 2024-04-26T04:42:19.001 INFO:tasks.workunit.client.1.smithi104.stdout:4/639: mknod d4/db/ce2 0 2024-04-26T04:42:19.006 INFO:tasks.workunit.client.1.smithi104.stdout:7/520: dread d2/d4/d32/d60/f63 [0,4194304] 0 2024-04-26T04:42:19.011 INFO:tasks.workunit.client.1.smithi104.stdout:7/521: truncate d2/d4/f6e 159308 0 2024-04-26T04:42:19.014 INFO:tasks.workunit.client.1.smithi104.stdout:7/522: symlink d2/d4/d32/d3d/d9e/lac 0 2024-04-26T04:42:19.014 INFO:tasks.workunit.client.1.smithi104.stdout:7/523: chown d2/d4/d32/d35/d37/l65 571388437 1 2024-04-26T04:42:19.015 INFO:tasks.workunit.client.1.smithi104.stdout:7/524: dread - d2/d4/d32/d35/d44/d5b/f7c zero size 2024-04-26T04:42:19.030 INFO:tasks.workunit.client.1.smithi104.stdout:7/525: dread d2/d4/f11 [4194304,4194304] 0 2024-04-26T04:42:19.032 INFO:tasks.workunit.client.1.smithi104.stdout:7/526: rmdir d2/d4/d32/d5e/d9c 0 2024-04-26T04:42:19.032 INFO:tasks.workunit.client.1.smithi104.stdout:7/527: truncate d2/d4/d32/d35/f87 508193 0 2024-04-26T04:42:19.032 INFO:tasks.workunit.client.1.smithi104.stdout:7/528: readlink d2/d4/d1d/d34/l77 0 2024-04-26T04:42:19.032 INFO:tasks.workunit.client.1.smithi104.stdout:7/529: chown d2/d4/f74 1 1 2024-04-26T04:42:19.041 INFO:tasks.workunit.client.1.smithi104.stdout:1/638: dwrite d0/d6/d19/d1b/d35/fc9 [0,4194304] 0 2024-04-26T04:42:19.041 INFO:tasks.workunit.client.1.smithi104.stdout:1/639: truncate d0/d6/d2e/f86 381052 0 2024-04-26T04:42:19.042 INFO:tasks.workunit.client.1.smithi104.stdout:4/640: dwrite d4/d7/d1d/d1e/d24/d4a/fac [0,4194304] 0 2024-04-26T04:42:19.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/282: sync 2024-04-26T04:42:19.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/283: write d0/d4/d7/d17/d22/f23 [291126,19298] 0 2024-04-26T04:42:19.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/284: chown d0/d4/f9 121578454 1 2024-04-26T04:42:19.044 INFO:tasks.workunit.client.0.smithi090.stdout:1/285: mknod d0/d4/d7/d49/c5d 0 2024-04-26T04:42:19.045 INFO:tasks.workunit.client.0.smithi090.stdout:1/286: fsync d0/d4/d7/d17/d22/f52 0 2024-04-26T04:42:19.045 INFO:tasks.workunit.client.0.smithi090.stdout:1/287: write d0/d4/d7/d17/d22/f52 [1169837,47178] 0 2024-04-26T04:42:19.046 INFO:tasks.workunit.client.0.smithi090.stdout:1/288: creat d0/d4/d7/d49/f5e x:0 0 0 2024-04-26T04:42:19.047 INFO:tasks.workunit.client.0.smithi090.stdout:1/289: fdatasync d0/d4/f14 0 2024-04-26T04:42:19.054 INFO:tasks.workunit.client.0.smithi090.stdout:1/290: dread d0/d4/f48 [0,4194304] 0 2024-04-26T04:42:19.055 INFO:tasks.workunit.client.0.smithi090.stdout:1/291: truncate d0/d4/f15 964485 0 2024-04-26T04:42:19.055 INFO:tasks.workunit.client.0.smithi090.stdout:1/292: chown d0/d4/d43/f54 3937828 1 2024-04-26T04:42:19.066 INFO:tasks.workunit.client.0.smithi090.stdout:1/293: write d0/d4/d7/d17/d22/f2f [284956,2224] 0 2024-04-26T04:42:19.067 INFO:tasks.workunit.client.0.smithi090.stdout:1/294: getdents d0/d4/d7/d4a 0 2024-04-26T04:42:19.070 INFO:tasks.workunit.client.1.smithi104.stdout:0/533: sync 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.1.smithi104.stdout:6/708: sync 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.1.smithi104.stdout:3/646: sync 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.1.smithi104.stdout:5/611: sync 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.1.smithi104.stdout:7/530: dwrite d2/d4/d32/f69 [0,4194304] 0 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.1.smithi104.stdout:4/641: dwrite d4/d7/d1d/d1e/d24/d59/fae [0,4194304] 0 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.0.smithi090.stdout:0/384: dwrite d0/de/f39 [0,4194304] 0 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.0.smithi090.stdout:0/385: write d0/d9/f17 [1386970,94992] 0 2024-04-26T04:42:19.071 INFO:tasks.workunit.client.0.smithi090.stdout:1/295: dread d0/d4/f27 [0,4194304] 0 2024-04-26T04:42:19.072 INFO:tasks.workunit.client.0.smithi090.stdout:0/386: mkdir d0/d9/d45/d84 0 2024-04-26T04:42:19.074 INFO:tasks.workunit.client.1.smithi104.stdout:0/534: rename d0/d2/d21/d49/d5e/d9c/d53/d5d/f75 to d0/d2/d21/d49/d5e/fa7 0 2024-04-26T04:42:19.074 INFO:tasks.workunit.client.1.smithi104.stdout:0/535: fdatasync d0/d2/d21/d49/f7e 0 2024-04-26T04:42:19.074 INFO:tasks.workunit.client.1.smithi104.stdout:0/536: chown d0/d2/d21/d27 0 1 2024-04-26T04:42:19.076 INFO:tasks.workunit.client.1.smithi104.stdout:6/709: mknod d0/d9/d13/d41/d96/cf4 0 2024-04-26T04:42:19.082 INFO:tasks.workunit.client.1.smithi104.stdout:3/647: creat d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/fe0 x:0 0 0 2024-04-26T04:42:19.082 INFO:tasks.workunit.client.1.smithi104.stdout:4/642: creat d4/d7/d1d/d1e/d24/d59/db8/fe3 x:0 0 0 2024-04-26T04:42:19.083 INFO:tasks.workunit.client.1.smithi104.stdout:4/643: creat d4/d7/d1d/d1e/d24/fe4 x:0 0 0 2024-04-26T04:42:19.083 INFO:tasks.workunit.client.1.smithi104.stdout:7/531: mkdir d2/dad 0 2024-04-26T04:42:19.083 INFO:tasks.workunit.client.1.smithi104.stdout:7/532: fsync d2/d4/d32/d60/f63 0 2024-04-26T04:42:19.083 INFO:tasks.workunit.client.1.smithi104.stdout:7/533: chown d2/d4/d32/d3d/d41/d5f/d6d/faa 5748905 1 2024-04-26T04:42:19.083 INFO:tasks.workunit.client.1.smithi104.stdout:0/537: creat d0/d2/d21/d27/d1e/d57/d6c/d85/fa8 x:0 0 0 2024-04-26T04:42:19.083 INFO:tasks.workunit.client.1.smithi104.stdout:3/648: symlink d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/dd6/le1 0 2024-04-26T04:42:19.083 INFO:tasks.workunit.client.1.smithi104.stdout:3/649: fsync d5/db2/d60/fa7 0 2024-04-26T04:42:19.084 INFO:tasks.workunit.client.1.smithi104.stdout:3/650: write d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/fe0 [170523,31908] 0 2024-04-26T04:42:19.084 INFO:tasks.workunit.client.1.smithi104.stdout:3/651: read - d5/db2/da3/dd0/ddb/f36 zero size 2024-04-26T04:42:19.088 INFO:tasks.workunit.client.1.smithi104.stdout:4/644: rename d4/d7/d1d/f7a to d4/d7/d1d/d2e/d74/fe5 0 2024-04-26T04:42:19.089 INFO:tasks.workunit.client.1.smithi104.stdout:7/534: mknod d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/cae 0 2024-04-26T04:42:19.089 INFO:tasks.workunit.client.1.smithi104.stdout:7/535: fsync d2/d4/d32/d60/f82 0 2024-04-26T04:42:19.090 INFO:tasks.workunit.client.1.smithi104.stdout:0/538: truncate d0/d2/d10/f5a 1947877 0 2024-04-26T04:42:19.093 INFO:tasks.workunit.client.1.smithi104.stdout:7/536: link d2/d4/d32/d3d/d47/f61 d2/d4/d1d/d34/d7f/faf 0 2024-04-26T04:42:19.094 INFO:tasks.workunit.client.1.smithi104.stdout:0/539: write d0/fa [2153510,33844] 0 2024-04-26T04:42:19.095 INFO:tasks.workunit.client.1.smithi104.stdout:0/540: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fa9 x:0 0 0 2024-04-26T04:42:19.097 INFO:tasks.workunit.client.1.smithi104.stdout:3/652: dread d5/db2/da3/fa4 [0,4194304] 0 2024-04-26T04:42:19.101 INFO:tasks.workunit.client.1.smithi104.stdout:4/645: write d4/d7/ff [6846689,32788] 0 2024-04-26T04:42:19.102 INFO:tasks.workunit.client.0.smithi090.stdout:0/387: dwrite d0/df/f6b [0,4194304] 0 2024-04-26T04:42:19.105 INFO:tasks.workunit.client.1.smithi104.stdout:4/646: creat d4/d7/d1d/d1e/d24/d59/db8/fe6 x:0 0 0 2024-04-26T04:42:19.106 INFO:tasks.workunit.client.1.smithi104.stdout:4/647: readlink d4/d20/d55/d56/l34 0 2024-04-26T04:42:19.106 INFO:tasks.workunit.client.1.smithi104.stdout:4/648: stat d4/d20/d55/d56/da3/fc4 0 2024-04-26T04:42:19.107 INFO:tasks.workunit.client.1.smithi104.stdout:3/653: dread d5/d9/fe [0,4194304] 0 2024-04-26T04:42:19.108 INFO:tasks.workunit.client.1.smithi104.stdout:4/649: unlink d4/d7/d1d/d8e/f97 0 2024-04-26T04:42:19.110 INFO:tasks.workunit.client.1.smithi104.stdout:3/654: dread d5/db2/f91 [0,4194304] 0 2024-04-26T04:42:19.111 INFO:tasks.workunit.client.1.smithi104.stdout:4/650: unlink d4/d7/d1d/d1e/d24/d59/fb7 0 2024-04-26T04:42:19.112 INFO:tasks.workunit.client.1.smithi104.stdout:3/655: truncate d5/db2/da3/dd0/ddb/d32/d48/f78 353815 0 2024-04-26T04:42:19.112 INFO:tasks.workunit.client.1.smithi104.stdout:3/656: read - d5/db2/da3/fc4 zero size 2024-04-26T04:42:19.114 INFO:tasks.workunit.client.1.smithi104.stdout:4/651: rename d4/c35 to d4/d7/d1d/d1e/d24/d48/ce7 0 2024-04-26T04:42:19.114 INFO:tasks.workunit.client.1.smithi104.stdout:3/657: truncate d5/f54 2129993 0 2024-04-26T04:42:19.116 INFO:tasks.workunit.client.1.smithi104.stdout:3/658: rename d5/db2/d46/d95/fa5 to d5/db2/fe2 0 2024-04-26T04:42:19.116 INFO:tasks.workunit.client.1.smithi104.stdout:3/659: creat d5/d9/d80/da2/fe3 x:0 0 0 2024-04-26T04:42:19.116 INFO:tasks.workunit.client.1.smithi104.stdout:3/660: chown d5/db2/da3/dd0/ddb/d69 3764095 1 2024-04-26T04:42:19.117 INFO:tasks.workunit.client.1.smithi104.stdout:3/661: mkdir d5/d9/de4 0 2024-04-26T04:42:19.126 INFO:tasks.workunit.client.1.smithi104.stdout:5/612: dwrite d0/d1/d1e/f8f [0,4194304] 0 2024-04-26T04:42:19.126 INFO:tasks.workunit.client.1.smithi104.stdout:5/613: read - d0/d1/d4/fbb zero size 2024-04-26T04:42:19.126 INFO:tasks.workunit.client.1.smithi104.stdout:5/614: write d0/d41/f4d [671000,127922] 0 2024-04-26T04:42:19.127 INFO:tasks.workunit.client.1.smithi104.stdout:5/615: chown d0/d89 159 1 2024-04-26T04:42:19.133 INFO:tasks.workunit.client.1.smithi104.stdout:4/652: dread d4/d7/d1d/d1e/d24/d4a/fac [0,4194304] 0 2024-04-26T04:42:19.133 INFO:tasks.workunit.client.0.smithi090.stdout:1/296: dwrite d0/d4/f48 [4194304,4194304] 0 2024-04-26T04:42:19.134 INFO:tasks.workunit.client.1.smithi104.stdout:4/653: dread d4/db/fe [0,4194304] 0 2024-04-26T04:42:19.134 INFO:tasks.workunit.client.1.smithi104.stdout:4/654: creat d4/d7/fe8 x:0 0 0 2024-04-26T04:42:19.136 INFO:tasks.workunit.client.0.smithi090.stdout:1/297: rmdir d0/d4/d7/d17/d22/d50 0 2024-04-26T04:42:19.136 INFO:tasks.workunit.client.0.smithi090.stdout:1/298: stat d0/d4/f15 0 2024-04-26T04:42:19.137 INFO:tasks.workunit.client.1.smithi104.stdout:7/537: dwrite d2/f42 [0,4194304] 0 2024-04-26T04:42:19.137 INFO:tasks.workunit.client.1.smithi104.stdout:6/710: dwrite d0/d9/d13/d44/d71/fa3 [0,4194304] 0 2024-04-26T04:42:19.137 INFO:tasks.workunit.client.1.smithi104.stdout:6/711: truncate d0/d9/f4a 1118096 0 2024-04-26T04:42:19.137 INFO:tasks.workunit.client.1.smithi104.stdout:0/541: dwrite d0/d2/d21/d49/d5e/d9c/d53/fa6 [0,4194304] 0 2024-04-26T04:42:19.139 INFO:tasks.workunit.client.1.smithi104.stdout:3/662: dwrite d5/d9/f26 [4194304,4194304] 0 2024-04-26T04:42:19.139 INFO:tasks.workunit.client.0.smithi090.stdout:0/388: dwrite d0/df/f6b [0,4194304] 0 2024-04-26T04:42:19.139 INFO:tasks.workunit.client.1.smithi104.stdout:4/655: link d4/db/f4b d4/d7/d1d/d2e/d74/dbd/dd3/fe9 0 2024-04-26T04:42:19.145 INFO:tasks.workunit.client.0.smithi090.stdout:0/389: rename d0/d9/d45/c60 to d0/d9/d45/c85 0 2024-04-26T04:42:19.146 INFO:tasks.workunit.client.1.smithi104.stdout:5/616: dwrite d0/d1/f3d [0,4194304] 0 2024-04-26T04:42:19.146 INFO:tasks.workunit.client.1.smithi104.stdout:5/617: write d0/d1/d64/f85 [264348,63725] 0 2024-04-26T04:42:19.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/390: mknod d0/d4b/d5d/d48/c86 0 2024-04-26T04:42:19.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/391: dread - d0/d4b/d5d/d48/f80 zero size 2024-04-26T04:42:19.147 INFO:tasks.workunit.client.1.smithi104.stdout:6/712: mkdir d0/d9/d13/dcd/dd9/df5 0 2024-04-26T04:42:19.147 INFO:tasks.workunit.client.1.smithi104.stdout:3/663: dread d5/db2/da3/dd0/ddb/d32/f5a [0,4194304] 0 2024-04-26T04:42:19.149 INFO:tasks.workunit.client.0.smithi090.stdout:0/392: creat d0/d9/d45/d53/f87 x:0 0 0 2024-04-26T04:42:19.149 INFO:tasks.workunit.client.0.smithi090.stdout:0/393: chown d0/d4b 57103072 1 2024-04-26T04:42:19.149 INFO:tasks.workunit.client.0.smithi090.stdout:0/394: getdents d0/d9/d7c/d7e 0 2024-04-26T04:42:19.150 INFO:tasks.workunit.client.0.smithi090.stdout:0/395: rmdir d0/d9 39 2024-04-26T04:42:19.151 INFO:tasks.workunit.client.1.smithi104.stdout:7/538: read d2/d4/d32/d3d/d47/f8a [3889749,22346] 0 2024-04-26T04:42:19.153 INFO:tasks.workunit.client.0.smithi090.stdout:0/396: creat d0/d9/d64/f88 x:0 0 0 2024-04-26T04:42:19.157 INFO:tasks.workunit.client.0.smithi090.stdout:1/299: dwrite d0/d4/d7/d17/d22/d31/f34 [0,4194304] 0 2024-04-26T04:42:19.160 INFO:tasks.workunit.client.1.smithi104.stdout:4/656: dread d4/d7/d1d/d1e/d24/d4a/fac [0,4194304] 0 2024-04-26T04:42:19.160 INFO:tasks.workunit.client.1.smithi104.stdout:4/657: readlink d4/d7/d1d/d1e/d24/d59/l62 0 2024-04-26T04:42:19.160 INFO:tasks.workunit.client.1.smithi104.stdout:4/658: truncate d4/d7/d1d/d1e/d24/d48/fdb 449673 0 2024-04-26T04:42:19.164 INFO:tasks.workunit.client.0.smithi090.stdout:1/300: unlink d0/c2d 0 2024-04-26T04:42:19.165 INFO:tasks.workunit.client.0.smithi090.stdout:0/397: dread d0/d4b/d5d/f27 [0,4194304] 0 2024-04-26T04:42:19.166 INFO:tasks.workunit.client.0.smithi090.stdout:0/398: truncate d0/d9/f17 944539 0 2024-04-26T04:42:19.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/344: sync 2024-04-26T04:42:19.168 INFO:tasks.workunit.client.0.smithi090.stdout:8/291: sync 2024-04-26T04:42:19.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/345: readlink d0/db/l21 0 2024-04-26T04:42:19.168 INFO:tasks.workunit.client.0.smithi090.stdout:8/292: write dc/d1f/d3a/f3e [686549,30768] 0 2024-04-26T04:42:19.168 INFO:tasks.workunit.client.0.smithi090.stdout:8/293: chown dc/d1f/l31 30 1 2024-04-26T04:42:19.168 INFO:tasks.workunit.client.0.smithi090.stdout:0/399: link d0/d9/d64/f88 d0/de/f89 0 2024-04-26T04:42:19.170 INFO:tasks.workunit.client.0.smithi090.stdout:8/294: mknod dc/d34/c63 0 2024-04-26T04:42:19.171 INFO:tasks.workunit.client.0.smithi090.stdout:0/400: truncate d0/d9/f3e 878388 0 2024-04-26T04:42:19.173 INFO:tasks.workunit.client.0.smithi090.stdout:1/301: dread d0/d4/d7/d17/d22/f23 [0,4194304] 0 2024-04-26T04:42:19.173 INFO:tasks.workunit.client.0.smithi090.stdout:8/295: rename dc/d1f/d3a to dc/d1f/d49/d39/d55/d64 0 2024-04-26T04:42:19.176 INFO:tasks.workunit.client.0.smithi090.stdout:8/296: rename dc/f1d to dc/d1f/d49/d3b/d3d/f65 0 2024-04-26T04:42:19.178 INFO:tasks.workunit.client.0.smithi090.stdout:8/297: creat dc/d5a/d5d/f66 x:0 0 0 2024-04-26T04:42:19.180 INFO:tasks.workunit.client.0.smithi090.stdout:8/298: getdents dc/d1f/d49/d4f 0 2024-04-26T04:42:19.182 INFO:tasks.workunit.client.0.smithi090.stdout:8/299: creat dc/d1f/f67 x:0 0 0 2024-04-26T04:42:19.183 INFO:tasks.workunit.client.1.smithi104.stdout:2/774: sync 2024-04-26T04:42:19.183 INFO:tasks.workunit.client.1.smithi104.stdout:8/544: sync 2024-04-26T04:42:19.183 INFO:tasks.workunit.client.1.smithi104.stdout:0/542: dwrite d0/d2/d21/d27/d1e/d57/d6c/f56 [0,4194304] 0 2024-04-26T04:42:19.185 INFO:tasks.workunit.client.1.smithi104.stdout:4/659: dread d4/d7/d1d/d1e/d24/d59/fbc [0,4194304] 0 2024-04-26T04:42:19.185 INFO:tasks.workunit.client.1.smithi104.stdout:2/775: rmdir d6/de/d1d/dcf/de7 0 2024-04-26T04:42:19.186 INFO:tasks.workunit.client.1.smithi104.stdout:8/545: symlink d0/d33/d35/d6d/da4/lae 0 2024-04-26T04:42:19.186 INFO:tasks.workunit.client.1.smithi104.stdout:8/546: readlink d0/d1c/d27/l5a 0 2024-04-26T04:42:19.188 INFO:tasks.workunit.client.1.smithi104.stdout:0/543: creat d0/d2/d21/d27/faa x:0 0 0 2024-04-26T04:42:19.189 INFO:tasks.workunit.client.1.smithi104.stdout:4/660: mkdir d4/d20/d55/d56/da3/dea 0 2024-04-26T04:42:19.189 INFO:tasks.workunit.client.1.smithi104.stdout:4/661: chown d4/d7/d1d/d1e/d15/c92 1 1 2024-04-26T04:42:19.190 INFO:tasks.workunit.client.1.smithi104.stdout:2/776: mkdir d6/d8/df6/dfe 0 2024-04-26T04:42:19.190 INFO:tasks.workunit.client.1.smithi104.stdout:2/777: write d6/d1b/d51/d6a/dd8/fdc [969785,22548] 0 2024-04-26T04:42:19.191 INFO:tasks.workunit.client.1.smithi104.stdout:8/547: mkdir d0/d1c/d27/d47/d67/d76/daf 0 2024-04-26T04:42:19.191 INFO:tasks.workunit.client.1.smithi104.stdout:8/548: chown d0/f1a 28 1 2024-04-26T04:42:19.192 INFO:tasks.workunit.client.1.smithi104.stdout:4/662: dread d4/d20/f9a [0,4194304] 0 2024-04-26T04:42:19.192 INFO:tasks.workunit.client.1.smithi104.stdout:4/663: creat d4/d20/d55/d56/feb x:0 0 0 2024-04-26T04:42:19.193 INFO:tasks.workunit.client.1.smithi104.stdout:3/664: dwrite d5/f23 [0,4194304] 0 2024-04-26T04:42:19.193 INFO:tasks.workunit.client.1.smithi104.stdout:8/549: truncate d0/d1c/f84 174849 0 2024-04-26T04:42:19.195 INFO:tasks.workunit.client.0.smithi090.stdout:7/346: dwrite d0/db/d53/fa [4194304,4194304] 0 2024-04-26T04:42:19.195 INFO:tasks.workunit.client.1.smithi104.stdout:0/544: link d0/d2/d21/d27/c70 d0/d4/d3b/cab 0 2024-04-26T04:42:19.197 INFO:tasks.workunit.client.1.smithi104.stdout:8/550: rename d0/d1c/d2d/c58 to d0/d1c/d59/cb0 0 2024-04-26T04:42:19.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/347: creat d0/db/df/d4d/d64/d66/f7c x:0 0 0 2024-04-26T04:42:19.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/348: write d0/db/df/d4d/d64/d66/f7c [271379,72609] 0 2024-04-26T04:42:19.199 INFO:tasks.workunit.client.0.smithi090.stdout:8/300: read dc/d1f/d49/f20 [1195844,101092] 0 2024-04-26T04:42:19.199 INFO:tasks.workunit.client.0.smithi090.stdout:8/301: write dc/f14 [4819066,14945] 0 2024-04-26T04:42:19.199 INFO:tasks.workunit.client.1.smithi104.stdout:7/539: dwrite d2/d4/d32/d35/f87 [0,4194304] 0 2024-04-26T04:42:19.200 INFO:tasks.workunit.client.1.smithi104.stdout:7/540: read d2/d4/d1d/f90 [1938687,5190] 0 2024-04-26T04:42:19.200 INFO:tasks.workunit.client.1.smithi104.stdout:6/713: dwrite d0/d9/d25/f85 [0,4194304] 0 2024-04-26T04:42:19.201 INFO:tasks.workunit.client.0.smithi090.stdout:4/316: sync 2024-04-26T04:42:19.201 INFO:tasks.workunit.client.1.smithi104.stdout:5/618: dwrite d0/d1/d4/d2a/f50 [0,4194304] 0 2024-04-26T04:42:19.202 INFO:tasks.workunit.client.1.smithi104.stdout:0/545: symlink d0/d2/d21/lac 0 2024-04-26T04:42:19.202 INFO:tasks.workunit.client.1.smithi104.stdout:0/546: readlink d0/d2/d21/d27/d1e/d57/d6c/l25 0 2024-04-26T04:42:19.204 INFO:tasks.workunit.client.1.smithi104.stdout:8/551: mkdir d0/d1c/d27/d47/db1 0 2024-04-26T04:42:19.207 INFO:tasks.workunit.client.1.smithi104.stdout:3/665: write d5/db2/da3/dd0/ddb/f28 [2789772,125406] 0 2024-04-26T04:42:19.207 INFO:tasks.workunit.client.1.smithi104.stdout:3/666: truncate d5/db2/da3/dd0/ddb/d69/f6e 944975 0 2024-04-26T04:42:19.208 INFO:tasks.workunit.client.1.smithi104.stdout:4/664: dread d4/d7/f2a [0,4194304] 0 2024-04-26T04:42:19.211 INFO:tasks.workunit.client.0.smithi090.stdout:7/349: write d0/db/d53/fa [1806108,31437] 0 2024-04-26T04:42:19.213 INFO:tasks.workunit.client.0.smithi090.stdout:7/350: rename d0/db/df/l1a to d0/db/df/d4d/l7d 0 2024-04-26T04:42:19.213 INFO:tasks.workunit.client.0.smithi090.stdout:7/351: write d0/d1/f52 [1866138,26681] 0 2024-04-26T04:42:19.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/352: chown d0/db/df/l35 28151 1 2024-04-26T04:42:19.214 INFO:tasks.workunit.client.1.smithi104.stdout:7/541: dread d2/d4/d1d/d34/f40 [0,4194304] 0 2024-04-26T04:42:19.214 INFO:tasks.workunit.client.1.smithi104.stdout:7/542: read d2/f75 [4643703,117325] 0 2024-04-26T04:42:19.215 INFO:tasks.workunit.client.0.smithi090.stdout:7/353: creat d0/db/d15/f7e x:0 0 0 2024-04-26T04:42:19.215 INFO:tasks.workunit.client.1.smithi104.stdout:6/714: symlink d0/d9/d13/d41/d96/lf6 0 2024-04-26T04:42:19.227 INFO:tasks.workunit.client.1.smithi104.stdout:5/619: creat d0/d1/d4/d2a/d6b/d73/da4/fd5 x:0 0 0 2024-04-26T04:42:19.229 INFO:tasks.workunit.client.1.smithi104.stdout:0/547: link d0/d2/d21/d27/l48 d0/d2/d21/d38/lad 0 2024-04-26T04:42:19.229 INFO:tasks.workunit.client.1.smithi104.stdout:0/548: fdatasync d0/d2/d21/d27/d1e/f65 0 2024-04-26T04:42:19.230 INFO:tasks.workunit.client.0.smithi090.stdout:0/401: dread d0/d9/f17 [0,4194304] 0 2024-04-26T04:42:19.230 INFO:tasks.workunit.client.1.smithi104.stdout:3/667: link d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/fb0 d5/db2/d46/fe5 0 2024-04-26T04:42:19.232 INFO:tasks.workunit.client.1.smithi104.stdout:0/549: dread d0/d4/d3b/f77 [0,4194304] 0 2024-04-26T04:42:19.232 INFO:tasks.workunit.client.1.smithi104.stdout:0/550: write d0/d2/d21/d27/f7b [2679009,112774] 0 2024-04-26T04:42:19.233 INFO:tasks.workunit.client.1.smithi104.stdout:7/543: getdents d2/d4/d32/d3d 0 2024-04-26T04:42:19.233 INFO:tasks.workunit.client.1.smithi104.stdout:2/778: dwrite d6/de/d85/fcc [0,4194304] 0 2024-04-26T04:42:19.234 INFO:tasks.workunit.client.1.smithi104.stdout:6/715: mkdir d0/d9/d13/d41/d82/df7 0 2024-04-26T04:42:19.235 INFO:tasks.workunit.client.0.smithi090.stdout:0/402: rename d0/d9 to d0/d4b/d5d/d5c/d8a 0 2024-04-26T04:42:19.235 INFO:tasks.workunit.client.0.smithi090.stdout:0/403: fdatasync d0/de/f2c 0 2024-04-26T04:42:19.236 INFO:tasks.workunit.client.0.smithi090.stdout:8/302: dwrite dc/f5c [0,4194304] 0 2024-04-26T04:42:19.236 INFO:tasks.workunit.client.1.smithi104.stdout:4/665: dwrite d4/db/d6b/fa2 [0,4194304] 0 2024-04-26T04:42:19.236 INFO:tasks.workunit.client.1.smithi104.stdout:8/552: dwrite d0/d1c/d27/d46/f78 [4194304,4194304] 0 2024-04-26T04:42:19.237 INFO:tasks.workunit.client.0.smithi090.stdout:4/317: dwrite d3/db/d12/d20/d2d/d2e/f3a [0,4194304] 0 2024-04-26T04:42:19.237 INFO:tasks.workunit.client.0.smithi090.stdout:4/318: readlink d3/l5 0 2024-04-26T04:42:19.239 INFO:tasks.workunit.client.1.smithi104.stdout:3/668: unlink d5/db2/da3/dd0/ddb/d3a/l4f 0 2024-04-26T04:42:19.240 INFO:tasks.workunit.client.0.smithi090.stdout:8/303: rmdir dc/d1f/d49/d4f 39 2024-04-26T04:42:19.241 INFO:tasks.workunit.client.1.smithi104.stdout:7/544: stat d2/d4/f13 0 2024-04-26T04:42:19.241 INFO:tasks.workunit.client.1.smithi104.stdout:7/545: fsync d2/d4/d32/d3d/d41/d5f/d6d/fa3 0 2024-04-26T04:42:19.243 INFO:tasks.workunit.client.0.smithi090.stdout:0/404: dread d0/df/f3c [0,4194304] 0 2024-04-26T04:42:19.244 INFO:tasks.workunit.client.0.smithi090.stdout:8/304: dread dc/d1f/f24 [0,4194304] 0 2024-04-26T04:42:19.244 INFO:tasks.workunit.client.1.smithi104.stdout:2/779: truncate d6/d8/f39 2232949 0 2024-04-26T04:42:19.244 INFO:tasks.workunit.client.1.smithi104.stdout:2/780: chown d6/de/d1d/f6d 26416 1 2024-04-26T04:42:19.244 INFO:tasks.workunit.client.1.smithi104.stdout:2/781: stat d6/d1b/d68/d81/d97/d99/lf4 0 2024-04-26T04:42:19.246 INFO:tasks.workunit.client.1.smithi104.stdout:6/716: creat d0/d9/d13/ff8 x:0 0 0 2024-04-26T04:42:19.246 INFO:tasks.workunit.client.1.smithi104.stdout:6/717: fdatasync d0/d9/d13/d44/d71/da5/fb7 0 2024-04-26T04:42:19.247 INFO:tasks.workunit.client.1.smithi104.stdout:6/718: stat d0/d9/d25/c83 0 2024-04-26T04:42:19.247 INFO:tasks.workunit.client.0.smithi090.stdout:7/354: getdents d0/db/df/d4d/d64/d66 0 2024-04-26T04:42:19.252 INFO:tasks.workunit.client.0.smithi090.stdout:4/319: symlink d3/d13/d1d/l62 0 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:2/335: sync 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:9/214: sync 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:3/202: sync 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:6/398: sync 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:5/377: sync 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:9/215: creat db/dd/d18/d31/f4a x:0 0 0 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:5/378: stat d4/d1a/d45 0 2024-04-26T04:42:19.257 INFO:tasks.workunit.client.0.smithi090.stdout:0/405: mknod d0/d4b/d5d/d48/d7d/c8b 0 2024-04-26T04:42:19.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/203: mknod d1/d17/d36/d33/c46 0 2024-04-26T04:42:19.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/204: write d1/f3c [164927,50030] 0 2024-04-26T04:42:19.259 INFO:tasks.workunit.client.1.smithi104.stdout:5/620: rmdir d0/d41/d4f 39 2024-04-26T04:42:19.260 INFO:tasks.workunit.client.0.smithi090.stdout:2/336: symlink db/d11/d14/d25/d2d/l72 0 2024-04-26T04:42:19.260 INFO:tasks.workunit.client.0.smithi090.stdout:2/337: fsync db/d11/f17 0 2024-04-26T04:42:19.261 INFO:tasks.workunit.client.1.smithi104.stdout:7/546: stat d2/d4/c4a 0 2024-04-26T04:42:19.261 INFO:tasks.workunit.client.0.smithi090.stdout:5/379: mkdir d4/d1a/d2a/d79 0 2024-04-26T04:42:19.263 INFO:tasks.workunit.client.0.smithi090.stdout:0/406: creat d0/d4b/f8c x:0 0 0 2024-04-26T04:42:19.264 INFO:tasks.workunit.client.0.smithi090.stdout:3/205: truncate d1/d9/fa 2928101 0 2024-04-26T04:42:19.265 INFO:tasks.workunit.client.1.smithi104.stdout:6/719: symlink d0/d9/d13/d44/d71/da5/lf9 0 2024-04-26T04:42:19.265 INFO:tasks.workunit.client.1.smithi104.stdout:6/720: truncate d0/d9/d13/d41/d82/dc8/fd6 728314 0 2024-04-26T04:42:19.265 INFO:tasks.workunit.client.1.smithi104.stdout:6/721: write d0/d9/d25/f39 [5149265,42279] 0 2024-04-26T04:42:19.266 INFO:tasks.workunit.client.1.smithi104.stdout:6/722: write d0/d9/d13/d41/d82/dc8/dba/daa/fdf [828382,109327] 0 2024-04-26T04:42:19.266 INFO:tasks.workunit.client.1.smithi104.stdout:6/723: truncate d0/d9/d13/d44/d71/f74 1062745 0 2024-04-26T04:42:19.266 INFO:tasks.workunit.client.1.smithi104.stdout:6/724: fdatasync d0/d9/d13/d41/f7f 0 2024-04-26T04:42:19.266 INFO:tasks.workunit.client.0.smithi090.stdout:0/407: symlink d0/d4b/d5d/d5c/d8a/d7c/l8d 0 2024-04-26T04:42:19.269 INFO:tasks.workunit.client.1.smithi104.stdout:4/666: rename d4/d7/d1d/f43 to d4/d7/d1d/d1e/d24/d4a/fec 0 2024-04-26T04:42:19.272 INFO:tasks.workunit.client.0.smithi090.stdout:9/216: write db/f13 [1650842,26465] 0 2024-04-26T04:42:19.273 INFO:tasks.workunit.client.1.smithi104.stdout:7/547: dread d2/d4/d32/d35/d44/f91 [0,4194304] 0 2024-04-26T04:42:19.277 INFO:tasks.workunit.client.1.smithi104.stdout:6/725: truncate d0/d9/d13/d41/d96/dac/f67 470116 0 2024-04-26T04:42:19.277 INFO:tasks.workunit.client.1.smithi104.stdout:6/726: write d0/d9/d13/d44/d47/f4c [1545722,105646] 0 2024-04-26T04:42:19.277 INFO:tasks.workunit.client.1.smithi104.stdout:6/727: fsync d0/ff3 0 2024-04-26T04:42:19.278 INFO:tasks.workunit.client.1.smithi104.stdout:6/728: write d0/d9/d13/d44/d71/f4b [897471,122023] 0 2024-04-26T04:42:19.278 INFO:tasks.workunit.client.1.smithi104.stdout:6/729: chown d0/d9/d13/d41/d96/dac/db3/cc3 28408 1 2024-04-26T04:42:19.279 INFO:tasks.workunit.client.0.smithi090.stdout:9/217: dread db/dd/d18/d38/d39/f3b [0,4194304] 0 2024-04-26T04:42:19.279 INFO:tasks.workunit.client.1.smithi104.stdout:5/621: getdents d0/d1/d4/d2a/d6b/d91/dbd 0 2024-04-26T04:42:19.282 INFO:tasks.workunit.client.1.smithi104.stdout:7/548: truncate d2/d4/d32/f76 2028346 0 2024-04-26T04:42:19.282 INFO:tasks.workunit.client.1.smithi104.stdout:7/549: truncate d2/d4/d32/d35/d44/d5b/f7c 581929 0 2024-04-26T04:42:19.284 INFO:tasks.workunit.client.1.smithi104.stdout:4/667: mknod d4/d7/d1d/d1e/d24/d4a/ced 0 2024-04-26T04:42:19.284 INFO:tasks.workunit.client.1.smithi104.stdout:4/668: write d4/d7/d1d/d1e/d24/d48/fce [607206,109825] 0 2024-04-26T04:42:19.285 INFO:tasks.workunit.client.1.smithi104.stdout:5/622: creat d0/d1/d1e/fd6 x:0 0 0 2024-04-26T04:42:19.289 INFO:tasks.workunit.client.1.smithi104.stdout:7/550: rename d2/d4/d32/d3d/d9e to d2/d4/d1d/d34/db0 0 2024-04-26T04:42:19.289 INFO:tasks.workunit.client.1.smithi104.stdout:7/551: truncate d2/d4/d32/d3d/f50 936179 0 2024-04-26T04:42:19.289 INFO:tasks.workunit.client.1.smithi104.stdout:7/552: fdatasync d2/d4/d32/d35/d44/fa4 0 2024-04-26T04:42:19.289 INFO:tasks.workunit.client.1.smithi104.stdout:7/553: fdatasync d2/d4/f11 0 2024-04-26T04:42:19.289 INFO:tasks.workunit.client.1.smithi104.stdout:5/623: mkdir d0/d1/d64/d26/dd7 0 2024-04-26T04:42:19.290 INFO:tasks.workunit.client.1.smithi104.stdout:5/624: chown d0/d1/d4/d2a/d6b/d91/dbd 124618960 1 2024-04-26T04:42:19.290 INFO:tasks.workunit.client.0.smithi090.stdout:5/380: write d4/d1a/df/d36/d42/d56/f6f [2510557,80301] 0 2024-04-26T04:42:19.290 INFO:tasks.workunit.client.1.smithi104.stdout:9/478: sync 2024-04-26T04:42:19.290 INFO:tasks.workunit.client.1.smithi104.stdout:1/640: sync 2024-04-26T04:42:19.291 INFO:tasks.workunit.client.1.smithi104.stdout:5/625: dread d0/d1/d4/d2a/d2b/d37/d48/f80 [0,4194304] 0 2024-04-26T04:42:19.292 INFO:tasks.workunit.client.0.smithi090.stdout:5/381: creat d4/d1a/df/d4a/f7a x:0 0 0 2024-04-26T04:42:19.293 INFO:tasks.workunit.client.1.smithi104.stdout:9/479: rmdir d11/d15/d32/d35/d7b/d92 0 2024-04-26T04:42:19.295 INFO:tasks.workunit.client.1.smithi104.stdout:1/641: rename d0/d6/d19/d21/d66/d20/d2a/d56/c5c to d0/d6/d19/d21/d66/d30/d7e/dc6/cd2 0 2024-04-26T04:42:19.297 INFO:tasks.workunit.client.1.smithi104.stdout:9/480: mknod d11/d90/c97 0 2024-04-26T04:42:19.297 INFO:tasks.workunit.client.1.smithi104.stdout:9/481: fdatasync d11/d15/d32/d35/f5e 0 2024-04-26T04:42:19.298 INFO:tasks.workunit.client.1.smithi104.stdout:7/554: dread d2/d4/d1d/f90 [0,4194304] 0 2024-04-26T04:42:19.299 INFO:tasks.workunit.client.1.smithi104.stdout:1/642: mknod d0/d6/d19/d21/d66/d30/d7e/dc6/cd3 0 2024-04-26T04:42:19.300 INFO:tasks.workunit.client.1.smithi104.stdout:5/626: getdents d0/d1/d4/d2a/d2b/d37/d99 0 2024-04-26T04:42:19.303 INFO:tasks.workunit.client.1.smithi104.stdout:7/555: rmdir d2/dad 0 2024-04-26T04:42:19.305 INFO:tasks.workunit.client.0.smithi090.stdout:0/408: write d0/de/f2f [3119001,33121] 0 2024-04-26T04:42:19.305 INFO:tasks.workunit.client.1.smithi104.stdout:5/627: creat d0/d1/d4/d2a/d2b/d37/d48/dc1/fd8 x:0 0 0 2024-04-26T04:42:19.307 INFO:tasks.workunit.client.0.smithi090.stdout:5/382: dread d4/d1a/d3f/f29 [0,4194304] 0 2024-04-26T04:42:19.307 INFO:tasks.workunit.client.1.smithi104.stdout:9/482: mkdir d11/d15/d98 0 2024-04-26T04:42:19.307 INFO:tasks.workunit.client.1.smithi104.stdout:9/483: write d11/f89 [704500,104467] 0 2024-04-26T04:42:19.308 INFO:tasks.workunit.client.1.smithi104.stdout:9/484: chown d11/f68 22 1 2024-04-26T04:42:19.308 INFO:tasks.workunit.client.1.smithi104.stdout:9/485: stat d11/d15/d1b/f37 0 2024-04-26T04:42:19.308 INFO:tasks.workunit.client.1.smithi104.stdout:9/486: chown d11/d15/d32/d35 5511680 1 2024-04-26T04:42:19.308 INFO:tasks.workunit.client.0.smithi090.stdout:5/383: symlink d4/d1a/df/d36/d42/d56/d71/l7b 0 2024-04-26T04:42:19.308 INFO:tasks.workunit.client.1.smithi104.stdout:7/556: chown d2/d4/l25 98689 1 2024-04-26T04:42:19.314 INFO:tasks.workunit.client.1.smithi104.stdout:7/557: getdents d2/d4/d32/d3d/d41/d5f 0 2024-04-26T04:42:19.314 INFO:tasks.workunit.client.1.smithi104.stdout:7/558: truncate d2/d4/d32/d35/d44/d5b/f7c 1373358 0 2024-04-26T04:42:19.316 INFO:tasks.workunit.client.1.smithi104.stdout:7/559: unlink d2/d4/d32/d60/f63 0 2024-04-26T04:42:19.318 INFO:tasks.workunit.client.0.smithi090.stdout:5/384: dread d4/d1a/d3f/f29 [0,4194304] 0 2024-04-26T04:42:19.318 INFO:tasks.workunit.client.0.smithi090.stdout:5/385: write d4/d1a/df/d36/f5c [1158806,95018] 0 2024-04-26T04:42:19.320 INFO:tasks.workunit.client.0.smithi090.stdout:5/386: creat d4/d1a/d2a/f7c x:0 0 0 2024-04-26T04:42:19.321 INFO:tasks.workunit.client.0.smithi090.stdout:5/387: creat d4/d1a/df/d36/d42/d49/f7d x:0 0 0 2024-04-26T04:42:19.322 INFO:tasks.workunit.client.0.smithi090.stdout:5/388: fdatasync d4/d1a/f78 0 2024-04-26T04:42:19.322 INFO:tasks.workunit.client.0.smithi090.stdout:5/389: write d4/d1a/d3f/f65 [405588,29067] 0 2024-04-26T04:42:19.325 INFO:tasks.workunit.client.1.smithi104.stdout:8/553: dwrite d0/f4d [0,4194304] 0 2024-04-26T04:42:19.325 INFO:tasks.workunit.client.1.smithi104.stdout:8/554: write d0/f5 [1563084,34990] 0 2024-04-26T04:42:19.325 INFO:tasks.workunit.client.1.smithi104.stdout:8/555: write d0/f49 [1624707,15847] 0 2024-04-26T04:42:19.334 INFO:tasks.workunit.client.1.smithi104.stdout:8/556: dread d0/fe [0,4194304] 0 2024-04-26T04:42:19.334 INFO:tasks.workunit.client.1.smithi104.stdout:8/557: write d0/d1c/d2d/f48 [1052064,30673] 0 2024-04-26T04:42:19.334 INFO:tasks.workunit.client.1.smithi104.stdout:4/669: dread d4/d20/fab [0,4194304] 0 2024-04-26T04:42:19.336 INFO:tasks.workunit.client.1.smithi104.stdout:1/643: dread d0/d6/d2e/f7c [0,4194304] 0 2024-04-26T04:42:19.351 INFO:tasks.workunit.client.0.smithi090.stdout:8/305: dwrite dc/ff [0,4194304] 0 2024-04-26T04:42:19.352 INFO:tasks.workunit.client.0.smithi090.stdout:7/355: dwrite d0/db/d53/f56 [0,4194304] 0 2024-04-26T04:42:19.354 INFO:tasks.workunit.client.0.smithi090.stdout:8/306: link dc/d1f/d49/d3b/l5b dc/d1f/d49/l68 0 2024-04-26T04:42:19.354 INFO:tasks.workunit.client.0.smithi090.stdout:8/307: creat dc/d5a/d5d/f69 x:0 0 0 2024-04-26T04:42:19.354 INFO:tasks.workunit.client.0.smithi090.stdout:8/308: write dc/d1f/f35 [956674,42919] 0 2024-04-26T04:42:19.354 INFO:tasks.workunit.client.0.smithi090.stdout:8/309: stat dc/f14 0 2024-04-26T04:42:19.354 INFO:tasks.workunit.client.0.smithi090.stdout:8/310: creat dc/d5a/f6a x:0 0 0 2024-04-26T04:42:19.355 INFO:tasks.workunit.client.0.smithi090.stdout:8/311: truncate dc/d1f/d49/d39/d55/f56 4297542 0 2024-04-26T04:42:19.355 INFO:tasks.workunit.client.1.smithi104.stdout:0/551: dwrite d0/d2/d10/d84/f90 [0,4194304] 0 2024-04-26T04:42:19.355 INFO:tasks.workunit.client.1.smithi104.stdout:2/782: dwrite d6/d8/f5b [0,4194304] 0 2024-04-26T04:42:19.356 INFO:tasks.workunit.client.1.smithi104.stdout:9/487: dwrite d11/d15/d32/d35/f3a [0,4194304] 0 2024-04-26T04:42:19.356 INFO:tasks.workunit.client.1.smithi104.stdout:9/488: getdents d11/d1d/d71/d94 0 2024-04-26T04:42:19.356 INFO:tasks.workunit.client.1.smithi104.stdout:6/730: dwrite d0/dd8/fe9 [0,4194304] 0 2024-04-26T04:42:19.356 INFO:tasks.workunit.client.1.smithi104.stdout:6/731: chown d0/d9/d13/d44/d71/d3f 1465936003 1 2024-04-26T04:42:19.356 INFO:tasks.workunit.client.1.smithi104.stdout:6/732: write d0/d9/d25/f3e [929648,46321] 0 2024-04-26T04:42:19.357 INFO:tasks.workunit.client.1.smithi104.stdout:3/669: dwrite d5/db2/f3b [0,4194304] 0 2024-04-26T04:42:19.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/320: dwrite f1 [4194304,4194304] 0 2024-04-26T04:42:19.358 INFO:tasks.workunit.client.0.smithi090.stdout:2/338: dwrite db/d11/d14/d25/d2d/f6a [0,4194304] 0 2024-04-26T04:42:19.358 INFO:tasks.workunit.client.0.smithi090.stdout:5/390: dwrite d4/d1a/d45/f4f [0,4194304] 0 2024-04-26T04:42:19.358 INFO:tasks.workunit.client.0.smithi090.stdout:9/218: dwrite db/f1b [0,4194304] 0 2024-04-26T04:42:19.358 INFO:tasks.workunit.client.0.smithi090.stdout:9/219: dread - db/dd/d18/d31/f4a zero size 2024-04-26T04:42:19.358 INFO:tasks.workunit.client.0.smithi090.stdout:9/220: truncate db/dd/f1d 3396412 0 2024-04-26T04:42:19.358 INFO:tasks.workunit.client.0.smithi090.stdout:7/356: rename d0/db/d15/d30/l6a to d0/db/df/d4d/l7f 0 2024-04-26T04:42:19.361 INFO:tasks.workunit.client.1.smithi104.stdout:8/558: dwrite d0/d1c/f26 [4194304,4194304] 0 2024-04-26T04:42:19.363 INFO:tasks.workunit.client.0.smithi090.stdout:5/391: symlink d4/d1a/df/d36/l7e 0 2024-04-26T04:42:19.367 INFO:tasks.workunit.client.0.smithi090.stdout:9/221: symlink db/dd/d18/l4b 0 2024-04-26T04:42:19.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/222: read f3 [1623806,43425] 0 2024-04-26T04:42:19.372 INFO:tasks.workunit.client.1.smithi104.stdout:6/733: dread d0/dd8/fe9 [0,4194304] 0 2024-04-26T04:42:19.373 INFO:tasks.workunit.client.0.smithi090.stdout:2/339: rename db/d11/c15 to db/d11/d14/d27/d2f/d42/d5c/c73 0 2024-04-26T04:42:19.375 INFO:tasks.workunit.client.1.smithi104.stdout:8/559: dread d0/f4d [0,4194304] 0 2024-04-26T04:42:19.376 INFO:tasks.workunit.client.0.smithi090.stdout:9/223: truncate db/f35 3943092 0 2024-04-26T04:42:19.376 INFO:tasks.workunit.client.0.smithi090.stdout:5/392: dread d4/d1a/df/d36/d42/d49/f66 [0,4194304] 0 2024-04-26T04:42:19.377 INFO:tasks.workunit.client.0.smithi090.stdout:5/393: fsync d4/d1a/d3f/f29 0 2024-04-26T04:42:19.377 INFO:tasks.workunit.client.0.smithi090.stdout:5/394: getdents d4/d1a/d2a/d79 0 2024-04-26T04:42:19.377 INFO:tasks.workunit.client.0.smithi090.stdout:5/395: read - d4/d1a/df/d72/f77 zero size 2024-04-26T04:42:19.378 INFO:tasks.workunit.client.0.smithi090.stdout:7/357: symlink d0/db/d15/d30/l80 0 2024-04-26T04:42:19.381 INFO:tasks.workunit.client.0.smithi090.stdout:5/396: getdents d4/d1a/d45 0 2024-04-26T04:42:19.381 INFO:tasks.workunit.client.0.smithi090.stdout:5/397: chown d4/d1a/df/d36/d42/d56/d71/l7b 24701216 1 2024-04-26T04:42:19.382 INFO:tasks.workunit.client.0.smithi090.stdout:5/398: dread d4/d1a/df/d36/d42/d49/f66 [0,4194304] 0 2024-04-26T04:42:19.382 INFO:tasks.workunit.client.0.smithi090.stdout:2/340: dread db/dc/f19 [0,4194304] 0 2024-04-26T04:42:19.382 INFO:tasks.workunit.client.0.smithi090.stdout:2/341: truncate db/d11/d14/d27/f38 915688 0 2024-04-26T04:42:19.382 INFO:tasks.workunit.client.1.smithi104.stdout:7/560: mkdir d2/d4/d32/d3d/d47/db1 0 2024-04-26T04:42:19.383 INFO:tasks.workunit.client.0.smithi090.stdout:8/312: dwrite dc/d1f/d49/d39/d55/d64/f3e [0,4194304] 0 2024-04-26T04:42:19.384 INFO:tasks.workunit.client.0.smithi090.stdout:8/313: truncate dc/d1f/d49/d39/d55/d64/f3e 4963582 0 2024-04-26T04:42:19.384 INFO:tasks.workunit.client.0.smithi090.stdout:8/314: truncate dc/d1f/d49/d3b/d3d/f5e 655004 0 2024-04-26T04:42:19.384 INFO:tasks.workunit.client.0.smithi090.stdout:4/321: dwrite d3/f2c [0,4194304] 0 2024-04-26T04:42:19.385 INFO:tasks.workunit.client.0.smithi090.stdout:5/399: symlink d4/d1a/df/l7f 0 2024-04-26T04:42:19.385 INFO:tasks.workunit.client.0.smithi090.stdout:2/342: symlink db/d11/d14/d1b/l74 0 2024-04-26T04:42:19.385 INFO:tasks.workunit.client.0.smithi090.stdout:2/343: chown db/d11/d14/d27/d28/f6e 436 1 2024-04-26T04:42:19.388 INFO:tasks.workunit.client.1.smithi104.stdout:4/670: mkdir d4/d7/d1d/d2e/d74/dbd/dee 0 2024-04-26T04:42:19.389 INFO:tasks.workunit.client.1.smithi104.stdout:1/644: creat d0/d6/d19/d21/d66/d20/d2a/dbc/fd4 x:0 0 0 2024-04-26T04:42:19.390 INFO:tasks.workunit.client.0.smithi090.stdout:8/315: unlink dc/d1f/f24 0 2024-04-26T04:42:19.390 INFO:tasks.workunit.client.0.smithi090.stdout:8/316: read - dc/d1f/d49/d39/f52 zero size 2024-04-26T04:42:19.391 INFO:tasks.workunit.client.1.smithi104.stdout:0/552: getdents d0/d2/d21/d49/d5e 0 2024-04-26T04:42:19.392 INFO:tasks.workunit.client.1.smithi104.stdout:0/553: write d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/f8a [559623,43014] 0 2024-04-26T04:42:19.392 INFO:tasks.workunit.client.1.smithi104.stdout:8/560: dwrite d0/d50/f55 [0,4194304] 0 2024-04-26T04:42:19.395 INFO:tasks.workunit.client.0.smithi090.stdout:4/322: link d3/db/d12/d20/d24/f52 d3/db/d12/d20/d2d/d58/f63 0 2024-04-26T04:42:19.396 INFO:tasks.workunit.client.0.smithi090.stdout:4/323: truncate d3/db/d12/d20/d2d/d2e/f3a 4435082 0 2024-04-26T04:42:19.396 INFO:tasks.workunit.client.1.smithi104.stdout:2/783: link d6/d8/d3e/c84 d6/d1b/d68/cff 0 2024-04-26T04:42:19.397 INFO:tasks.workunit.client.0.smithi090.stdout:5/400: mkdir d4/d1a/df/d36/d42/d56/d80 0 2024-04-26T04:42:19.397 INFO:tasks.workunit.client.0.smithi090.stdout:5/401: fdatasync d4/d1a/df/f22 0 2024-04-26T04:42:19.398 INFO:tasks.workunit.client.1.smithi104.stdout:9/489: creat d11/d90/f99 x:0 0 0 2024-04-26T04:42:19.398 INFO:tasks.workunit.client.1.smithi104.stdout:9/490: fdatasync d11/d15/d32/f83 0 2024-04-26T04:42:19.398 INFO:tasks.workunit.client.1.smithi104.stdout:9/491: truncate d11/d1d/d34/d69/f78 354858 0 2024-04-26T04:42:19.399 INFO:tasks.workunit.client.1.smithi104.stdout:9/492: creat d11/d15/d32/f9a x:0 0 0 2024-04-26T04:42:19.400 INFO:tasks.workunit.client.0.smithi090.stdout:2/344: symlink db/d11/d14/d1b/d20/d4c/l75 0 2024-04-26T04:42:19.401 INFO:tasks.workunit.client.0.smithi090.stdout:8/317: rmdir dc/d1f/d49/d39/d55 39 2024-04-26T04:42:19.403 INFO:tasks.workunit.client.0.smithi090.stdout:5/402: mknod d4/d1a/d45/c81 0 2024-04-26T04:42:19.403 INFO:tasks.workunit.client.0.smithi090.stdout:5/403: dread - d4/d1a/d2a/f7c zero size 2024-04-26T04:42:19.405 INFO:tasks.workunit.client.0.smithi090.stdout:8/318: getdents dc/d1f/d49/d3b/d3d 0 2024-04-26T04:42:19.407 INFO:tasks.workunit.client.1.smithi104.stdout:0/554: read d0/d2/d21/d49/d5e/d9c/f86 [4049034,63724] 0 2024-04-26T04:42:19.407 INFO:tasks.workunit.client.1.smithi104.stdout:3/670: mkdir d5/db2/d46/db7/de6 0 2024-04-26T04:42:19.407 INFO:tasks.workunit.client.1.smithi104.stdout:9/493: read d11/f52 [275423,104349] 0 2024-04-26T04:42:19.410 INFO:tasks.workunit.client.1.smithi104.stdout:7/561: mkdir d2/d4/d32/d35/d44/d5b/db2 0 2024-04-26T04:42:19.410 INFO:tasks.workunit.client.1.smithi104.stdout:7/562: creat d2/d4/d32/d35/d44/fb3 x:0 0 0 2024-04-26T04:42:19.413 INFO:tasks.workunit.client.1.smithi104.stdout:4/671: creat d4/d7/d1d/d1e/d15/dd8/fef x:0 0 0 2024-04-26T04:42:19.417 INFO:tasks.workunit.client.1.smithi104.stdout:1/645: creat d0/d6/d19/d21/d66/d20/d2a/d56/fd5 x:0 0 0 2024-04-26T04:42:19.417 INFO:tasks.workunit.client.0.smithi090.stdout:5/404: dread d4/d1a/f19 [0,4194304] 0 2024-04-26T04:42:19.418 INFO:tasks.workunit.client.0.smithi090.stdout:5/405: write d4/d1a/d2a/d31/f34 [8583465,36641] 0 2024-04-26T04:42:19.421 INFO:tasks.workunit.client.0.smithi090.stdout:5/406: link d4/d1a/df/d36/d42/d56/f6f d4/d1a/df/d36/d42/d49/f82 0 2024-04-26T04:42:19.422 INFO:tasks.workunit.client.0.smithi090.stdout:1/302: sync 2024-04-26T04:42:19.423 INFO:tasks.workunit.client.0.smithi090.stdout:1/303: creat d0/d4/d7/d49/f5f x:0 0 0 2024-04-26T04:42:19.423 INFO:tasks.workunit.client.0.smithi090.stdout:1/304: readlink d0/d4/l41 0 2024-04-26T04:42:19.424 INFO:tasks.workunit.client.0.smithi090.stdout:8/319: dread dc/d1f/d49/d3b/d3d/f54 [0,4194304] 0 2024-04-26T04:42:19.424 INFO:tasks.workunit.client.0.smithi090.stdout:8/320: readlink dc/l11 0 2024-04-26T04:42:19.424 INFO:tasks.workunit.client.1.smithi104.stdout:0/555: write d0/d2/d21/d49/d5e/f78 [709038,112667] 0 2024-04-26T04:42:19.424 INFO:tasks.workunit.client.1.smithi104.stdout:0/556: dread - d0/d2/d21/d27/f83 zero size 2024-04-26T04:42:19.425 INFO:tasks.workunit.client.1.smithi104.stdout:8/561: rename d0/d33/d35/f45 to d0/d1c/d27/d47/d67/d76/daf/fb2 0 2024-04-26T04:42:19.425 INFO:tasks.workunit.client.0.smithi090.stdout:2/345: dwrite db/dc/f1f [0,4194304] 0 2024-04-26T04:42:19.427 INFO:tasks.workunit.client.1.smithi104.stdout:2/784: creat d6/d1b/d75/f100 x:0 0 0 2024-04-26T04:42:19.430 INFO:tasks.workunit.client.1.smithi104.stdout:6/734: dread d0/d9/d13/d41/d96/dac/f67 [0,4194304] 0 2024-04-26T04:42:19.431 INFO:tasks.workunit.client.1.smithi104.stdout:9/494: symlink d11/d1d/d71/l9b 0 2024-04-26T04:42:19.433 INFO:tasks.workunit.client.0.smithi090.stdout:1/305: write d0/d4/d7/d17/f2a [3774962,118252] 0 2024-04-26T04:42:19.433 INFO:tasks.workunit.client.0.smithi090.stdout:1/306: fsync d0/d4/d7/d4a/f56 0 2024-04-26T04:42:19.433 INFO:tasks.workunit.client.1.smithi104.stdout:5/628: rmdir d0/d1/d4/d2a 39 2024-04-26T04:42:19.435 INFO:tasks.workunit.client.1.smithi104.stdout:7/563: fdatasync d2/d4/d5a/f68 0 2024-04-26T04:42:19.438 INFO:tasks.workunit.client.0.smithi090.stdout:5/407: dread d4/d1a/d1f/f2b [4194304,4194304] 0 2024-04-26T04:42:19.438 INFO:tasks.workunit.client.0.smithi090.stdout:5/408: creat d4/d1a/d2a/f83 x:0 0 0 2024-04-26T04:42:19.439 INFO:tasks.workunit.client.0.smithi090.stdout:8/321: dread dc/d1f/f21 [0,4194304] 0 2024-04-26T04:42:19.441 INFO:tasks.workunit.client.1.smithi104.stdout:0/557: dwrite d0/d2/d21/d38/f34 [0,4194304] 0 2024-04-26T04:42:19.444 INFO:tasks.workunit.client.1.smithi104.stdout:5/629: dread d0/d41/f74 [0,4194304] 0 2024-04-26T04:42:19.445 INFO:tasks.workunit.client.1.smithi104.stdout:5/630: write d0/d1/d1e/fb1 [271849,122297] 0 2024-04-26T04:42:19.454 INFO:tasks.workunit.client.0.smithi090.stdout:5/409: dread d4/fe [0,4194304] 0 2024-04-26T04:42:19.454 INFO:tasks.workunit.client.0.smithi090.stdout:5/410: write d4/d1a/df/d36/d42/d56/f75 [188456,24904] 0 2024-04-26T04:42:19.454 INFO:tasks.workunit.client.0.smithi090.stdout:5/411: creat d4/d1a/df/d36/f84 x:0 0 0 2024-04-26T04:42:19.461 INFO:tasks.workunit.client.0.smithi090.stdout:8/322: creat dc/f6b x:0 0 0 2024-04-26T04:42:19.461 INFO:tasks.workunit.client.0.smithi090.stdout:5/412: write d4/f1e [7603976,35682] 0 2024-04-26T04:42:19.466 INFO:tasks.workunit.client.0.smithi090.stdout:8/323: rename dc/d1f/d49/d39/d55/f56 to dc/d1f/d49/d39/d55/f6c 0 2024-04-26T04:42:19.466 INFO:tasks.workunit.client.0.smithi090.stdout:8/324: chown dc/d5a/d5d 21 1 2024-04-26T04:42:19.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/413: creat d4/d1a/d2a/d79/f85 x:0 0 0 2024-04-26T04:42:19.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/414: stat d4/d1a/d1f/d53/d59 0 2024-04-26T04:42:19.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/415: fdatasync d4/d1a/d2a/f41 0 2024-04-26T04:42:19.468 INFO:tasks.workunit.client.0.smithi090.stdout:8/325: write dc/d1f/d49/d3b/d3d/f54 [1366477,104428] 0 2024-04-26T04:42:19.470 INFO:tasks.workunit.client.0.smithi090.stdout:8/326: creat dc/d1f/d49/d3b/f6d x:0 0 0 2024-04-26T04:42:19.470 INFO:tasks.workunit.client.0.smithi090.stdout:8/327: write dc/d1f/d49/f20 [506195,109128] 0 2024-04-26T04:42:19.470 INFO:tasks.workunit.client.0.smithi090.stdout:5/416: dread d4/d1a/df/d36/d42/d56/f6f [0,4194304] 0 2024-04-26T04:42:19.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/417: write d4/d1a/df/f22 [442392,69485] 0 2024-04-26T04:42:19.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/418: write d4/d1a/df/f6d [615226,1147] 0 2024-04-26T04:42:19.471 INFO:tasks.workunit.client.0.smithi090.stdout:5/419: chown d4/d1a/d45/f4f 2 1 2024-04-26T04:42:19.484 INFO:tasks.workunit.client.0.smithi090.stdout:5/420: write d4/f46 [512151,75501] 0 2024-04-26T04:42:19.494 INFO:tasks.workunit.client.0.smithi090.stdout:8/328: dwrite dc/d1f/d49/d39/d55/d64/f3e [4194304,4194304] 0 2024-04-26T04:42:19.494 INFO:tasks.workunit.client.0.smithi090.stdout:8/329: truncate dc/d34/d4d/f4e 713802 0 2024-04-26T04:42:19.496 INFO:tasks.workunit.client.0.smithi090.stdout:8/330: mkdir dc/d5a/d5d/d6e 0 2024-04-26T04:42:19.497 INFO:tasks.workunit.client.0.smithi090.stdout:8/331: creat dc/d1f/d49/d4f/d58/f6f x:0 0 0 2024-04-26T04:42:19.498 INFO:tasks.workunit.client.0.smithi090.stdout:8/332: creat dc/d1f/d49/d39/d55/f70 x:0 0 0 2024-04-26T04:42:19.499 INFO:tasks.workunit.client.1.smithi104.stdout:3/671: rename d5/f90 to d5/db2/da3/dd0/ddb/d69/fe7 0 2024-04-26T04:42:19.499 INFO:tasks.workunit.client.1.smithi104.stdout:8/562: getdents d0/d1c 0 2024-04-26T04:42:19.500 INFO:tasks.workunit.client.0.smithi090.stdout:8/333: rename dc/d5a/d5d/d6e to dc/d71 0 2024-04-26T04:42:19.501 INFO:tasks.workunit.client.1.smithi104.stdout:2/785: symlink d6/d1b/d68/l101 0 2024-04-26T04:42:19.501 INFO:tasks.workunit.client.0.smithi090.stdout:8/334: truncate dc/f1e 1775591 0 2024-04-26T04:42:19.501 INFO:tasks.workunit.client.0.smithi090.stdout:8/335: dread - dc/d5a/d5d/f69 zero size 2024-04-26T04:42:19.501 INFO:tasks.workunit.client.0.smithi090.stdout:8/336: stat dc/d1f/d49/d3b/d3d/f65 0 2024-04-26T04:42:19.502 INFO:tasks.workunit.client.0.smithi090.stdout:8/337: mknod dc/d5a/d5d/c72 0 2024-04-26T04:42:19.502 INFO:tasks.workunit.client.0.smithi090.stdout:5/421: dwrite d4/d1a/d2a/f41 [0,4194304] 0 2024-04-26T04:42:19.503 INFO:tasks.workunit.client.0.smithi090.stdout:8/338: mkdir dc/d5a/d5d/d73 0 2024-04-26T04:42:19.503 INFO:tasks.workunit.client.0.smithi090.stdout:8/339: stat dc/d1f/d49/c23 0 2024-04-26T04:42:19.503 INFO:tasks.workunit.client.0.smithi090.stdout:5/422: read d4/d1a/df/d4a/f55 [205306,77859] 0 2024-04-26T04:42:19.503 INFO:tasks.workunit.client.0.smithi090.stdout:5/423: stat d4/d1a/d2a/f83 0 2024-04-26T04:42:19.504 INFO:tasks.workunit.client.1.smithi104.stdout:6/735: mkdir d0/d9/d13/d41/d96/dac/dc4/dfa 0 2024-04-26T04:42:19.505 INFO:tasks.workunit.client.1.smithi104.stdout:9/495: mkdir d11/d15/d1b/d6d/d9c 0 2024-04-26T04:42:19.505 INFO:tasks.workunit.client.1.smithi104.stdout:9/496: stat d11/d15/d1b/l2a 0 2024-04-26T04:42:19.506 INFO:tasks.workunit.client.1.smithi104.stdout:7/564: creat d2/d4/d32/d3d/fb4 x:0 0 0 2024-04-26T04:42:19.506 INFO:tasks.workunit.client.0.smithi090.stdout:5/424: write d4/d1a/f19 [3348570,66734] 0 2024-04-26T04:42:19.507 INFO:tasks.workunit.client.0.smithi090.stdout:8/340: rename dc/f14 to dc/f74 0 2024-04-26T04:42:19.507 INFO:tasks.workunit.client.0.smithi090.stdout:8/341: fsync dc/d1f/d49/f28 0 2024-04-26T04:42:19.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/425: mkdir d4/d1a/df/d36/d6b/d86 0 2024-04-26T04:42:19.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/426: fdatasync d4/d1a/d3f/f52 0 2024-04-26T04:42:19.510 INFO:tasks.workunit.client.0.smithi090.stdout:8/342: creat dc/d1f/d49/f75 x:0 0 0 2024-04-26T04:42:19.511 INFO:tasks.workunit.client.1.smithi104.stdout:3/672: mknod d5/db2/da3/ce8 0 2024-04-26T04:42:19.511 INFO:tasks.workunit.client.0.smithi090.stdout:5/427: creat d4/d1a/df/d36/d61/f87 x:0 0 0 2024-04-26T04:42:19.512 INFO:tasks.workunit.client.1.smithi104.stdout:8/563: link d0/d1c/c29 d0/d1c/d2d/d7a/cb3 0 2024-04-26T04:42:19.512 INFO:tasks.workunit.client.1.smithi104.stdout:8/564: write d0/f49 [220490,73861] 0 2024-04-26T04:42:19.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/343: symlink dc/d5a/l76 0 2024-04-26T04:42:19.513 INFO:tasks.workunit.client.1.smithi104.stdout:2/786: symlink d6/d1b/d68/d81/dfc/l102 0 2024-04-26T04:42:19.513 INFO:tasks.workunit.client.1.smithi104.stdout:6/736: truncate d0/d9/d13/d44/f97 4871433 0 2024-04-26T04:42:19.515 INFO:tasks.workunit.client.0.smithi090.stdout:8/344: mknod dc/d1f/d49/d3b/d3d/d57/c77 0 2024-04-26T04:42:19.515 INFO:tasks.workunit.client.0.smithi090.stdout:6/399: sync 2024-04-26T04:42:19.515 INFO:tasks.workunit.client.0.smithi090.stdout:6/400: dread - d9/d21/d40/d4c/f7b zero size 2024-04-26T04:42:19.516 INFO:tasks.workunit.client.1.smithi104.stdout:0/558: getdents d0/d4/d3b 0 2024-04-26T04:42:19.516 INFO:tasks.workunit.client.0.smithi090.stdout:5/428: link d4/d1a/df/f22 d4/f88 0 2024-04-26T04:42:19.516 INFO:tasks.workunit.client.0.smithi090.stdout:5/429: fsync d4/d1a/df/d36/d42/d49/f82 0 2024-04-26T04:42:19.517 INFO:tasks.workunit.client.0.smithi090.stdout:8/345: symlink dc/l78 0 2024-04-26T04:42:19.518 INFO:tasks.workunit.client.0.smithi090.stdout:6/401: symlink d9/d1c/d31/d69/l97 0 2024-04-26T04:42:19.518 INFO:tasks.workunit.client.0.smithi090.stdout:6/402: creat d9/d1c/d4d/d6d/f98 x:0 0 0 2024-04-26T04:42:19.520 INFO:tasks.workunit.client.1.smithi104.stdout:3/673: mknod d5/db2/da3/dd0/ddb/d69/ce9 0 2024-04-26T04:42:19.520 INFO:tasks.workunit.client.0.smithi090.stdout:8/346: link dc/d1f/d49/d39/d55/d64/l59 dc/d1f/d49/d3b/d3d/d57/l79 0 2024-04-26T04:42:19.521 INFO:tasks.workunit.client.1.smithi104.stdout:2/787: creat d6/d1b/d75/f103 x:0 0 0 2024-04-26T04:42:19.522 INFO:tasks.workunit.client.1.smithi104.stdout:9/497: mknod d11/d1d/c9d 0 2024-04-26T04:42:19.523 INFO:tasks.workunit.client.1.smithi104.stdout:0/559: symlink d0/d2/d21/d27/d1e/d57/lae 0 2024-04-26T04:42:19.523 INFO:tasks.workunit.client.0.smithi090.stdout:6/403: truncate d9/d1c/d4d/d58/f8c 3274770 0 2024-04-26T04:42:19.523 INFO:tasks.workunit.client.0.smithi090.stdout:6/404: write d9/d1c/d31/f3a [396084,2000] 0 2024-04-26T04:42:19.524 INFO:tasks.workunit.client.1.smithi104.stdout:1/646: rename d0/d6/d19/d21/d66/d30 to d0/d6/d19/d21/d58/db4/dd6 0 2024-04-26T04:42:19.526 INFO:tasks.workunit.client.0.smithi090.stdout:8/347: creat dc/d1f/d49/d39/d55/d64/f7a x:0 0 0 2024-04-26T04:42:19.526 INFO:tasks.workunit.client.0.smithi090.stdout:8/348: chown dc/d1f/f21 83354 1 2024-04-26T04:42:19.526 INFO:tasks.workunit.client.1.smithi104.stdout:3/674: creat d5/db2/da3/dd0/ddb/fea x:0 0 0 2024-04-26T04:42:19.527 INFO:tasks.workunit.client.0.smithi090.stdout:8/349: truncate dc/f74 2874472 0 2024-04-26T04:42:19.527 INFO:tasks.workunit.client.0.smithi090.stdout:8/350: fsync dc/f6b 0 2024-04-26T04:42:19.528 INFO:tasks.workunit.client.1.smithi104.stdout:2/788: unlink d6/d8/l66 0 2024-04-26T04:42:19.529 INFO:tasks.workunit.client.0.smithi090.stdout:8/351: mknod dc/d5a/d5d/c7b 0 2024-04-26T04:42:19.529 INFO:tasks.workunit.client.0.smithi090.stdout:8/352: dread - dc/d1f/d49/d4f/d58/f6f zero size 2024-04-26T04:42:19.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/353: write dc/d1f/d49/d39/f52 [864830,106844] 0 2024-04-26T04:42:19.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/354: write dc/d1f/d49/d3b/d3d/f54 [3156892,100162] 0 2024-04-26T04:42:19.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/355: getdents dc/d1f/d49/d3b/d3d 0 2024-04-26T04:42:19.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/356: readlink dc/l11 0 2024-04-26T04:42:19.530 INFO:tasks.workunit.client.0.smithi090.stdout:8/357: read - dc/d1f/d49/d39/d55/d64/f7a zero size 2024-04-26T04:42:19.530 INFO:tasks.workunit.client.1.smithi104.stdout:9/498: mkdir d11/d15/d32/d4c/d9e 0 2024-04-26T04:42:19.531 INFO:tasks.workunit.client.1.smithi104.stdout:7/565: rename d2/d4/d32/d35/d44/l53 to d2/d4/d32/d35/d44/d5b/lb5 0 2024-04-26T04:42:19.532 INFO:tasks.workunit.client.0.smithi090.stdout:8/358: rename dc/d1f/d49/f75 to dc/d1f/d49/d39/f7c 0 2024-04-26T04:42:19.532 INFO:tasks.workunit.client.0.smithi090.stdout:8/359: fsync dc/d1f/d49/d3b/d3d/f65 0 2024-04-26T04:42:19.533 INFO:tasks.workunit.client.1.smithi104.stdout:1/647: creat d0/d6/d19/d1b/fd7 x:0 0 0 2024-04-26T04:42:19.533 INFO:tasks.workunit.client.1.smithi104.stdout:1/648: creat d0/d6/d19/d1b/da5/fd8 x:0 0 0 2024-04-26T04:42:19.533 INFO:tasks.workunit.client.1.smithi104.stdout:1/649: read d0/d6/d2e/d32/f62 [805455,3550] 0 2024-04-26T04:42:19.534 INFO:tasks.workunit.client.1.smithi104.stdout:3/675: symlink d5/d9/dd1/leb 0 2024-04-26T04:42:19.534 INFO:tasks.workunit.client.1.smithi104.stdout:3/676: write d5/db2/da3/dd0/ddb/d32/d48/f78 [854972,23010] 0 2024-04-26T04:42:19.535 INFO:tasks.workunit.client.1.smithi104.stdout:3/677: creat d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/fec x:0 0 0 2024-04-26T04:42:19.535 INFO:tasks.workunit.client.1.smithi104.stdout:9/499: dread d11/d15/d1b/f37 [0,4194304] 0 2024-04-26T04:42:19.535 INFO:tasks.workunit.client.1.smithi104.stdout:7/566: mkdir d2/d4/d32/d3d/d47/db6 0 2024-04-26T04:42:19.535 INFO:tasks.workunit.client.1.smithi104.stdout:7/567: write d2/d4/d32/d35/d37/f86 [403381,114412] 0 2024-04-26T04:42:19.538 INFO:tasks.workunit.client.1.smithi104.stdout:1/650: creat d0/d6/d9e/fd9 x:0 0 0 2024-04-26T04:42:19.542 INFO:tasks.workunit.client.1.smithi104.stdout:3/678: creat d5/db2/fed x:0 0 0 2024-04-26T04:42:19.542 INFO:tasks.workunit.client.1.smithi104.stdout:3/679: dread - d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/fb0 zero size 2024-04-26T04:42:19.543 INFO:tasks.workunit.client.1.smithi104.stdout:1/651: symlink d0/d6/d19/d21/d66/d20/d2a/d56/d8c/lda 0 2024-04-26T04:42:19.545 INFO:tasks.workunit.client.1.smithi104.stdout:7/568: write d2/d4/d32/d3d/d41/d5f/d6d/f8b [1954948,7850] 0 2024-04-26T04:42:19.546 INFO:tasks.workunit.client.1.smithi104.stdout:3/680: link d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/f8b d5/db2/da3/dd0/ddb/d69/fee 0 2024-04-26T04:42:19.546 INFO:tasks.workunit.client.1.smithi104.stdout:3/681: dread - d5/db2/da3/dd0/ddb/d32/d48/fdf zero size 2024-04-26T04:42:19.547 INFO:tasks.workunit.client.1.smithi104.stdout:1/652: link d0/d6/d19/d21/l37 d0/d6/d2e/d32/d4c/db1/dc3/ldb 0 2024-04-26T04:42:19.547 INFO:tasks.workunit.client.1.smithi104.stdout:1/653: stat d0/d6/d19/d1b/f7f 0 2024-04-26T04:42:19.547 INFO:tasks.workunit.client.1.smithi104.stdout:1/654: truncate d0/d6/d19/d21/d58/db4/dd6/d81/f8b 1472082 0 2024-04-26T04:42:19.547 INFO:tasks.workunit.client.1.smithi104.stdout:0/560: dwrite d0/d2/d10/f64 [0,4194304] 0 2024-04-26T04:42:19.548 INFO:tasks.workunit.client.1.smithi104.stdout:3/682: mknod d5/cef 0 2024-04-26T04:42:19.549 INFO:tasks.workunit.client.1.smithi104.stdout:1/655: dread d0/d6/d19/d21/d59/f6e [0,4194304] 0 2024-04-26T04:42:19.550 INFO:tasks.workunit.client.1.smithi104.stdout:1/656: rmdir d0/d6/d19/d21/d66/d20/d2a/d56/d67 39 2024-04-26T04:42:19.552 INFO:tasks.workunit.client.1.smithi104.stdout:1/657: dread d0/d6/d19/d21/d58/db4/dd6/d81/f8b [0,4194304] 0 2024-04-26T04:42:19.554 INFO:tasks.workunit.client.1.smithi104.stdout:1/658: creat d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/fdc x:0 0 0 2024-04-26T04:42:19.555 INFO:tasks.workunit.client.1.smithi104.stdout:1/659: unlink d0/d6/d19/d21/d66/l55 0 2024-04-26T04:42:19.555 INFO:tasks.workunit.client.1.smithi104.stdout:1/660: fdatasync d0/d6/d19/d21/d66/d20/d2a/d56/fd5 0 2024-04-26T04:42:19.555 INFO:tasks.workunit.client.1.smithi104.stdout:1/661: read - d0/d6/d19/d21/d66/d20/d2a/d56/d67/f76 zero size 2024-04-26T04:42:19.556 INFO:tasks.workunit.client.1.smithi104.stdout:1/662: creat d0/d6/d19/d1b/da7/db8/fdd x:0 0 0 2024-04-26T04:42:19.558 INFO:tasks.workunit.client.1.smithi104.stdout:3/683: write d5/db2/fe2 [358462,26939] 0 2024-04-26T04:42:19.563 INFO:tasks.workunit.client.1.smithi104.stdout:3/684: symlink d5/d9/d80/dbc/lf0 0 2024-04-26T04:42:19.563 INFO:tasks.workunit.client.1.smithi104.stdout:3/685: write d5/db2/da3/dd0/ddb/d69/f55 [4969792,102622] 0 2024-04-26T04:42:19.566 INFO:tasks.workunit.client.1.smithi104.stdout:3/686: rename d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/fcc to d5/d9/dd5/ff1 0 2024-04-26T04:42:19.569 INFO:tasks.workunit.client.1.smithi104.stdout:7/569: dwrite d2/d4/d32/d3d/d41/d5f/f8d [0,4194304] 0 2024-04-26T04:42:19.569 INFO:tasks.workunit.client.1.smithi104.stdout:0/561: dwrite d0/d2/d21/d27/d1e/d57/d6c/f66 [0,4194304] 0 2024-04-26T04:42:19.572 INFO:tasks.workunit.client.1.smithi104.stdout:3/687: dread d5/db2/da3/dd0/ddb/d32/f77 [0,4194304] 0 2024-04-26T04:42:19.597 INFO:tasks.workunit.client.1.smithi104.stdout:0/562: dwrite d0/d2/d10/f5a [0,4194304] 0 2024-04-26T04:42:19.597 INFO:tasks.workunit.client.1.smithi104.stdout:7/570: dwrite d2/d4/d32/d3d/d47/f8a [0,4194304] 0 2024-04-26T04:42:19.597 INFO:tasks.workunit.client.1.smithi104.stdout:0/563: stat d0/l2e 0 2024-04-26T04:42:19.598 INFO:tasks.workunit.client.1.smithi104.stdout:0/564: chown d0/d2/d21/d38/f34 3336 1 2024-04-26T04:42:19.601 INFO:tasks.workunit.client.1.smithi104.stdout:7/571: dread d2/d4/d32/f76 [0,4194304] 0 2024-04-26T04:42:19.607 INFO:tasks.workunit.client.1.smithi104.stdout:0/565: dread d0/d2/d21/d38/f6b [0,4194304] 0 2024-04-26T04:42:19.623 INFO:tasks.workunit.client.1.smithi104.stdout:7/572: dwrite d2/d4/d32/d3d/d47/f61 [0,4194304] 0 2024-04-26T04:42:19.626 INFO:tasks.workunit.client.1.smithi104.stdout:7/573: mknod d2/d4/d32/d3d/d47/db6/cb7 0 2024-04-26T04:42:19.628 INFO:tasks.workunit.client.1.smithi104.stdout:7/574: creat d2/fb8 x:0 0 0 2024-04-26T04:42:19.628 INFO:tasks.workunit.client.1.smithi104.stdout:7/575: dread - d2/d4/d32/d3d/fb4 zero size 2024-04-26T04:42:19.631 INFO:tasks.workunit.client.1.smithi104.stdout:7/576: dread d2/d4/d32/d3d/d41/d5f/d6d/f8b [0,4194304] 0 2024-04-26T04:42:19.633 INFO:tasks.workunit.client.1.smithi104.stdout:7/577: creat d2/d4/d32/d3d/d41/d5f/d6c/fb9 x:0 0 0 2024-04-26T04:42:19.633 INFO:tasks.workunit.client.1.smithi104.stdout:7/578: chown d2/d4/d32/d5e/la6 158 1 2024-04-26T04:42:19.634 INFO:tasks.workunit.client.1.smithi104.stdout:7/579: symlink d2/d4/d32/d5e/lba 0 2024-04-26T04:42:19.636 INFO:tasks.workunit.client.1.smithi104.stdout:7/580: getdents d2/d4/d32/d35 0 2024-04-26T04:42:19.637 INFO:tasks.workunit.client.1.smithi104.stdout:9/500: write d11/d15/d32/d4c/d54/f62 [525592,46275] 0 2024-04-26T04:42:19.637 INFO:tasks.workunit.client.1.smithi104.stdout:7/581: rename d2/c23 to d2/d4/d32/d35/cbb 0 2024-04-26T04:42:19.638 INFO:tasks.workunit.client.1.smithi104.stdout:7/582: creat d2/d4/d32/d3d/d47/fbc x:0 0 0 2024-04-26T04:42:19.638 INFO:tasks.workunit.client.1.smithi104.stdout:9/501: symlink d11/d1d/d71/l9f 0 2024-04-26T04:42:19.640 INFO:tasks.workunit.client.1.smithi104.stdout:7/583: mkdir d2/d4/d32/d3d/d41/dbd 0 2024-04-26T04:42:19.640 INFO:tasks.workunit.client.1.smithi104.stdout:7/584: chown d2/d4/d32/d3d/d41/d5f/d6d/fa3 0 1 2024-04-26T04:42:19.640 INFO:tasks.workunit.client.1.smithi104.stdout:7/585: fsync d2/d4/d5a/f99 0 2024-04-26T04:42:19.641 INFO:tasks.workunit.client.1.smithi104.stdout:9/502: mknod d11/d1d/d71/d94/ca0 0 2024-04-26T04:42:19.641 INFO:tasks.workunit.client.1.smithi104.stdout:9/503: fdatasync d11/d53/f86 0 2024-04-26T04:42:19.641 INFO:tasks.workunit.client.1.smithi104.stdout:7/586: getdents d2/d4/d5a/da0 0 2024-04-26T04:42:19.642 INFO:tasks.workunit.client.1.smithi104.stdout:9/504: symlink d11/d1d/d71/d94/la1 0 2024-04-26T04:42:19.643 INFO:tasks.workunit.client.1.smithi104.stdout:7/587: mkdir d2/d4/d32/d35/dbe 0 2024-04-26T04:42:19.644 INFO:tasks.workunit.client.1.smithi104.stdout:9/505: mknod d11/d1d/ca2 0 2024-04-26T04:42:19.644 INFO:tasks.workunit.client.1.smithi104.stdout:9/506: chown d11/d90 583 1 2024-04-26T04:42:19.644 INFO:tasks.workunit.client.1.smithi104.stdout:9/507: write d11/d15/d32/d4c/d54/f75 [344604,46320] 0 2024-04-26T04:42:19.650 INFO:tasks.workunit.client.0.smithi090.stdout:5/430: dwrite d4/d1a/df/d36/f84 [0,4194304] 0 2024-04-26T04:42:19.652 INFO:tasks.workunit.client.1.smithi104.stdout:6/737: dwrite d0/d9/d13/d41/d82/dc8/f94 [0,4194304] 0 2024-04-26T04:42:19.653 INFO:tasks.workunit.client.0.smithi090.stdout:5/431: unlink d4/d1a/f48 0 2024-04-26T04:42:19.653 INFO:tasks.workunit.client.0.smithi090.stdout:5/432: fsync d4/d1a/d3f/f52 0 2024-04-26T04:42:19.653 INFO:tasks.workunit.client.0.smithi090.stdout:5/433: creat d4/d1a/df/d4a/f89 x:0 0 0 2024-04-26T04:42:19.653 INFO:tasks.workunit.client.1.smithi104.stdout:1/663: dwrite d0/d6/d19/d21/d66/d29/d3f/fa0 [0,4194304] 0 2024-04-26T04:42:19.654 INFO:tasks.workunit.client.1.smithi104.stdout:1/664: read d0/d6/ff [2229552,88124] 0 2024-04-26T04:42:19.654 INFO:tasks.workunit.client.1.smithi104.stdout:1/665: read d0/f14 [7355302,86360] 0 2024-04-26T04:42:19.654 INFO:tasks.workunit.client.1.smithi104.stdout:1/666: fdatasync d0/d6/d19/d21/d66/d29/d3f/f5b 0 2024-04-26T04:42:19.654 INFO:tasks.workunit.client.1.smithi104.stdout:1/667: write d0/d6/d19/d21/d28/f85 [4992034,59980] 0 2024-04-26T04:42:19.654 INFO:tasks.workunit.client.1.smithi104.stdout:8/565: dwrite d0/f9 [0,4194304] 0 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/434: creat d4/d1a/d1f/d53/d59/f8a x:0 0 0 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/435: readlink d4/d1a/df/d36/d42/d49/l70 0 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/436: chown d4/d1a/d1f/d53/d59/d5e/f6a 7800 1 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/437: fdatasync d4/d1a/df/d36/d42/d56/f6f 0 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/438: fsync d4/d1a/d1f/d53/d59/d5e/f6a 0 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.0.smithi090.stdout:5/439: chown d4/d1a/c6 1868495 1 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.1.smithi104.stdout:3/688: dwrite d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/fd3 [0,4194304] 0 2024-04-26T04:42:19.657 INFO:tasks.workunit.client.1.smithi104.stdout:3/689: creat d5/db2/da3/ff2 x:0 0 0 2024-04-26T04:42:19.658 INFO:tasks.workunit.client.1.smithi104.stdout:6/738: getdents d0/d9/d13/d41/d82/dc8/d80/dca 0 2024-04-26T04:42:19.662 INFO:tasks.workunit.client.1.smithi104.stdout:1/668: mkdir d0/d6/d19/d21/d66/d20/da3/dde 0 2024-04-26T04:42:19.662 INFO:tasks.workunit.client.1.smithi104.stdout:1/669: chown d0/d6/d19/d21/d66/d29/d3f/fa0 57691 1 2024-04-26T04:42:19.662 INFO:tasks.workunit.client.1.smithi104.stdout:1/670: truncate d0/d6/d2e/f7c 2256354 0 2024-04-26T04:42:19.665 INFO:tasks.workunit.client.1.smithi104.stdout:8/566: unlink d0/d1c/d27/d46/f6f 0 2024-04-26T04:42:19.672 INFO:tasks.workunit.client.1.smithi104.stdout:6/739: dread d0/d9/d13/d41/d82/dc8/d80/f6c [0,4194304] 0 2024-04-26T04:42:19.672 INFO:tasks.workunit.client.1.smithi104.stdout:0/566: dwrite d0/d2/d21/d38/f6b [4194304,4194304] 0 2024-04-26T04:42:19.673 INFO:tasks.workunit.client.1.smithi104.stdout:0/567: creat d0/d2/d21/d27/d1e/d57/d6c/d85/faf x:0 0 0 2024-04-26T04:42:19.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/440: write d4/d1a/d1f/f3a [430618,59907] 0 2024-04-26T04:42:19.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/441: creat d4/d1a/df/d36/d42/d56/f8b x:0 0 0 2024-04-26T04:42:19.676 INFO:tasks.workunit.client.1.smithi104.stdout:1/671: link d0/d6/d19/d21/d66/l10 d0/d6/d19/d21/d66/d20/d2a/d98/ldf 0 2024-04-26T04:42:19.676 INFO:tasks.workunit.client.1.smithi104.stdout:8/567: truncate d0/f1a 7615149 0 2024-04-26T04:42:19.676 INFO:tasks.workunit.client.1.smithi104.stdout:8/568: dread - d0/d33/da6/fa8 zero size 2024-04-26T04:42:19.678 INFO:tasks.workunit.client.1.smithi104.stdout:8/569: truncate d0/d1c/d2d/f48 3323590 0 2024-04-26T04:42:19.679 INFO:tasks.workunit.client.0.smithi090.stdout:5/442: dread d4/d1a/f43 [0,4194304] 0 2024-04-26T04:42:19.679 INFO:tasks.workunit.client.0.smithi090.stdout:5/443: readlink d4/d1a/d3f/l2e 0 2024-04-26T04:42:19.679 INFO:tasks.workunit.client.0.smithi090.stdout:5/444: chown d4/d1a/df/f6d 34 1 2024-04-26T04:42:19.679 INFO:tasks.workunit.client.1.smithi104.stdout:0/568: getdents d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3 0 2024-04-26T04:42:19.680 INFO:tasks.workunit.client.1.smithi104.stdout:1/672: write d0/d6/d19/d1b/d35/fc9 [4283253,55973] 0 2024-04-26T04:42:19.680 INFO:tasks.workunit.client.0.smithi090.stdout:5/445: truncate d4/d1a/d45/f74 5286206 0 2024-04-26T04:42:19.681 INFO:tasks.workunit.client.1.smithi104.stdout:8/570: unlink d0/d1c/c29 0 2024-04-26T04:42:19.681 INFO:tasks.workunit.client.1.smithi104.stdout:8/571: write d0/d1c/d2d/f6a [1010571,118545] 0 2024-04-26T04:42:19.682 INFO:tasks.workunit.client.1.smithi104.stdout:1/673: truncate d0/d6/d2e/d32/f62 243951 0 2024-04-26T04:42:19.682 INFO:tasks.workunit.client.1.smithi104.stdout:1/674: write d0/d6/ff [5405886,106349] 0 2024-04-26T04:42:19.682 INFO:tasks.workunit.client.0.smithi090.stdout:5/446: unlink d4/d1a/df/d36/d42/c50 0 2024-04-26T04:42:19.684 INFO:tasks.workunit.client.1.smithi104.stdout:7/588: dwrite d2/d4/f6e [0,4194304] 0 2024-04-26T04:42:19.684 INFO:tasks.workunit.client.1.smithi104.stdout:1/675: rename d0/d6/d19/d21/d66/l24 to d0/d6/d49/le0 0 2024-04-26T04:42:19.685 INFO:tasks.workunit.client.0.smithi090.stdout:5/447: symlink d4/d1a/l8c 0 2024-04-26T04:42:19.685 INFO:tasks.workunit.client.0.smithi090.stdout:5/448: truncate d4/d1a/d2a/f83 56643 0 2024-04-26T04:42:19.685 INFO:tasks.workunit.client.0.smithi090.stdout:5/449: dread - d4/d1a/d1f/d53/d59/f8a zero size 2024-04-26T04:42:19.687 INFO:tasks.workunit.client.0.smithi090.stdout:8/360: dwrite dc/d1f/d49/d39/f52 [0,4194304] 0 2024-04-26T04:42:19.687 INFO:tasks.workunit.client.0.smithi090.stdout:8/361: write dc/f30 [29500,51516] 0 2024-04-26T04:42:19.687 INFO:tasks.workunit.client.1.smithi104.stdout:9/508: dwrite d11/f12 [4194304,4194304] 0 2024-04-26T04:42:19.688 INFO:tasks.workunit.client.1.smithi104.stdout:1/676: fdatasync d0/f14 0 2024-04-26T04:42:19.688 INFO:tasks.workunit.client.1.smithi104.stdout:3/690: dwrite d5/db2/da3/fa4 [4194304,4194304] 0 2024-04-26T04:42:19.689 INFO:tasks.workunit.client.1.smithi104.stdout:7/589: link d2/d4/d32/d35/d44/d5b/lb5 d2/d4/d1d/d34/db0/lbf 0 2024-04-26T04:42:19.689 INFO:tasks.workunit.client.1.smithi104.stdout:7/590: dread - d2/d4/d32/d3d/d41/d5f/d6d/faa zero size 2024-04-26T04:42:19.689 INFO:tasks.workunit.client.1.smithi104.stdout:7/591: getdents d2/d4/d32/d3d/d41/d5f 0 2024-04-26T04:42:19.690 INFO:tasks.workunit.client.1.smithi104.stdout:7/592: truncate d2/d4/d32/d3d/d41/d5f/d6c/fb9 897070 0 2024-04-26T04:42:19.690 INFO:tasks.workunit.client.1.smithi104.stdout:7/593: fdatasync d2/f56 0 2024-04-26T04:42:19.696 INFO:tasks.workunit.client.1.smithi104.stdout:1/677: creat d0/d6/d19/d21/d66/d20/d2a/d56/d8c/fe1 x:0 0 0 2024-04-26T04:42:19.696 INFO:tasks.workunit.client.1.smithi104.stdout:1/678: write d0/d6/d2e/d32/d4c/fb0 [526270,110825] 0 2024-04-26T04:42:19.697 INFO:tasks.workunit.client.1.smithi104.stdout:1/679: dread d0/d6/d19/d21/d59/f6e [0,4194304] 0 2024-04-26T04:42:19.700 INFO:tasks.workunit.client.0.smithi090.stdout:5/450: rmdir d4/d1a 39 2024-04-26T04:42:19.701 INFO:tasks.workunit.client.0.smithi090.stdout:8/362: mknod dc/d5a/d5d/c7d 0 2024-04-26T04:42:19.702 INFO:tasks.workunit.client.1.smithi104.stdout:3/691: mkdir d5/db2/da3/df3 0 2024-04-26T04:42:19.703 INFO:tasks.workunit.client.0.smithi090.stdout:8/363: symlink dc/d1f/d49/d39/l7e 0 2024-04-26T04:42:19.704 INFO:tasks.workunit.client.1.smithi104.stdout:9/509: dwrite d11/d15/f59 [0,4194304] 0 2024-04-26T04:42:19.704 INFO:tasks.workunit.client.1.smithi104.stdout:8/572: dwrite d0/d1c/f7e [0,4194304] 0 2024-04-26T04:42:19.705 INFO:tasks.workunit.client.1.smithi104.stdout:1/680: rename d0/d6/d2e/d32/d4c/db1/cbd to d0/d6/d2e/d32/d4c/db1/dc3/ce2 0 2024-04-26T04:42:19.705 INFO:tasks.workunit.client.1.smithi104.stdout:1/681: truncate d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/fdc 485139 0 2024-04-26T04:42:19.705 INFO:tasks.workunit.client.1.smithi104.stdout:1/682: write d0/d6/d19/d21/d58/db4/dd6/d7e/dc6/fce [559832,116061] 0 2024-04-26T04:42:19.705 INFO:tasks.workunit.client.0.smithi090.stdout:8/364: rename dc/d1f/d49/f46 to dc/d1f/d49/d4f/f7f 0 2024-04-26T04:42:19.712 INFO:tasks.workunit.client.1.smithi104.stdout:8/573: creat d0/d50/fb4 x:0 0 0 2024-04-26T04:42:19.713 INFO:tasks.workunit.client.1.smithi104.stdout:3/692: getdents d5/d9/d80/dbc 0 2024-04-26T04:42:19.713 INFO:tasks.workunit.client.1.smithi104.stdout:3/693: chown d5/d9/c96 1754 1 2024-04-26T04:42:19.713 INFO:tasks.workunit.client.1.smithi104.stdout:3/694: fdatasync d5/db2/da3/dd0/ddb/d32/f77 0 2024-04-26T04:42:19.713 INFO:tasks.workunit.client.1.smithi104.stdout:3/695: fdatasync d5/db2/da3/dd0/ddb/d69/fbf 0 2024-04-26T04:42:19.713 INFO:tasks.workunit.client.1.smithi104.stdout:3/696: chown d5/f24 4 1 2024-04-26T04:42:19.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/365: dread dc/d1f/f36 [0,4194304] 0 2024-04-26T04:42:19.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/366: readlink dc/d1f/d49/d39/l7e 0 2024-04-26T04:42:19.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/367: dread - dc/d1f/d49/d3b/f6d zero size 2024-04-26T04:42:19.714 INFO:tasks.workunit.client.0.smithi090.stdout:8/368: readlink dc/l78 0 2024-04-26T04:42:19.715 INFO:tasks.workunit.client.1.smithi104.stdout:9/510: mkdir d11/d15/da3 0 2024-04-26T04:42:19.715 INFO:tasks.workunit.client.1.smithi104.stdout:9/511: dread - d11/d15/d32/f83 zero size 2024-04-26T04:42:19.715 INFO:tasks.workunit.client.1.smithi104.stdout:9/512: creat d11/d15/d1b/fa4 x:0 0 0 2024-04-26T04:42:19.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/369: rename dc/d1f/c4c to dc/c80 0 2024-04-26T04:42:19.716 INFO:tasks.workunit.client.0.smithi090.stdout:8/370: creat dc/d1f/d49/d39/f81 x:0 0 0 2024-04-26T04:42:19.717 INFO:tasks.workunit.client.1.smithi104.stdout:8/574: symlink d0/d1c/d27/d47/d67/d76/d7f/lb5 0 2024-04-26T04:42:19.718 INFO:tasks.workunit.client.1.smithi104.stdout:3/697: mknod d5/db2/da3/df3/cf4 0 2024-04-26T04:42:19.720 INFO:tasks.workunit.client.0.smithi090.stdout:8/371: dread dc/d1f/d49/d3b/f4a [0,4194304] 0 2024-04-26T04:42:19.720 INFO:tasks.workunit.client.0.smithi090.stdout:8/372: write dc/d5a/f62 [98716,104475] 0 2024-04-26T04:42:19.721 INFO:tasks.workunit.client.0.smithi090.stdout:8/373: mknod dc/d71/c82 0 2024-04-26T04:42:19.722 INFO:tasks.workunit.client.1.smithi104.stdout:8/575: truncate d0/d33/d35/d6d/f74 128714 0 2024-04-26T04:42:19.722 INFO:tasks.workunit.client.1.smithi104.stdout:8/576: chown d0/d33/d51/f92 75289 1 2024-04-26T04:42:19.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/374: creat dc/d1f/d49/d3b/d3d/d57/f83 x:0 0 0 2024-04-26T04:42:19.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/375: readlink dc/d1f/d49/l68 0 2024-04-26T04:42:19.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/376: chown dc/d1f/d49/d39/f7c 12 1 2024-04-26T04:42:19.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/377: chown dc/d1f/d49/d3b/f6d 27629742 1 2024-04-26T04:42:19.723 INFO:tasks.workunit.client.1.smithi104.stdout:7/594: truncate d2/d4/d32/d35/d44/d5b/f7c 129652 0 2024-04-26T04:42:19.723 INFO:tasks.workunit.client.1.smithi104.stdout:7/595: write d2/d4/d32/d35/d44/fa4 [955685,48695] 0 2024-04-26T04:42:19.723 INFO:tasks.workunit.client.1.smithi104.stdout:7/596: chown d2/d4/d32/d3d/d47 956 1 2024-04-26T04:42:19.723 INFO:tasks.workunit.client.1.smithi104.stdout:7/597: write d2/d4/d5a/f68 [2292021,128342] 0 2024-04-26T04:42:19.724 INFO:tasks.workunit.client.0.smithi090.stdout:8/378: creat dc/d34/f84 x:0 0 0 2024-04-26T04:42:19.725 INFO:tasks.workunit.client.0.smithi090.stdout:8/379: stat dc/c18 0 2024-04-26T04:42:19.725 INFO:tasks.workunit.client.1.smithi104.stdout:8/577: unlink d0/d9a/ca1 0 2024-04-26T04:42:19.725 INFO:tasks.workunit.client.1.smithi104.stdout:8/578: write d0/d1c/d27/d47/d67/d76/daf/fb2 [2053216,35469] 0 2024-04-26T04:42:19.725 INFO:tasks.workunit.client.1.smithi104.stdout:8/579: creat d0/d33/da6/fb6 x:0 0 0 2024-04-26T04:42:19.725 INFO:tasks.workunit.client.1.smithi104.stdout:8/580: write d0/d33/d35/d6d/f9b [72535,58560] 0 2024-04-26T04:42:19.726 INFO:tasks.workunit.client.1.smithi104.stdout:7/598: mkdir d2/d4/d32/d3d/d41/dc0 0 2024-04-26T04:42:19.726 INFO:tasks.workunit.client.1.smithi104.stdout:9/513: write d11/f16 [1728812,127986] 0 2024-04-26T04:42:19.726 INFO:tasks.workunit.client.1.smithi104.stdout:9/514: write d11/f44 [1032253,39398] 0 2024-04-26T04:42:19.727 INFO:tasks.workunit.client.1.smithi104.stdout:8/581: symlink d0/d1c/d2d/d7a/lb7 0 2024-04-26T04:42:19.728 INFO:tasks.workunit.client.1.smithi104.stdout:7/599: getdents d2/d4/d32/d35/d44/d5b 0 2024-04-26T04:42:19.729 INFO:tasks.workunit.client.0.smithi090.stdout:3/206: sync 2024-04-26T04:42:19.729 INFO:tasks.workunit.client.0.smithi090.stdout:3/207: readlink d1/d17/d36/d2a/d3a/l45 0 2024-04-26T04:42:19.730 INFO:tasks.workunit.client.1.smithi104.stdout:9/515: mknod d11/d15/da3/ca5 0 2024-04-26T04:42:19.730 INFO:tasks.workunit.client.1.smithi104.stdout:9/516: chown d11/d1d/d34/f3c 6 1 2024-04-26T04:42:19.730 INFO:tasks.workunit.client.1.smithi104.stdout:8/582: dread d0/f1e [0,4194304] 0 2024-04-26T04:42:19.731 INFO:tasks.workunit.client.0.smithi090.stdout:3/208: getdents d1/d17/d36/d2a/d3a 0 2024-04-26T04:42:19.731 INFO:tasks.workunit.client.0.smithi090.stdout:5/451: dwrite d4/d1a/df/d4a/f51 [4194304,4194304] 0 2024-04-26T04:42:19.733 INFO:tasks.workunit.client.0.smithi090.stdout:8/380: write dc/f74 [1660928,92425] 0 2024-04-26T04:42:19.734 INFO:tasks.workunit.client.1.smithi104.stdout:1/683: dwrite d0/d6/d19/d21/d59/f6e [0,4194304] 0 2024-04-26T04:42:19.734 INFO:tasks.workunit.client.1.smithi104.stdout:1/684: stat d0/d6/d2e/d32/d4c/dd0 0 2024-04-26T04:42:19.735 INFO:tasks.workunit.client.1.smithi104.stdout:8/583: creat d0/d1c/d27/d47/d67/d76/daf/fb8 x:0 0 0 2024-04-26T04:42:19.747 INFO:tasks.workunit.client.1.smithi104.stdout:1/685: rmdir d0/d6/d19/d1b/da7/db8 39 2024-04-26T04:42:19.747 INFO:tasks.workunit.client.1.smithi104.stdout:1/686: chown d0/d6/d19/d21/d66/d20/d2a/d56/d67 919 1 2024-04-26T04:42:19.750 INFO:tasks.workunit.client.1.smithi104.stdout:1/687: rename d0/d6/d19/d1b/d35/fc9 to d0/d6/d19/d21/d58/db4/dd6/fe3 0 2024-04-26T04:42:19.752 INFO:tasks.workunit.client.0.smithi090.stdout:9/224: sync 2024-04-26T04:42:19.753 INFO:tasks.workunit.client.0.smithi090.stdout:0/409: sync 2024-04-26T04:42:19.754 INFO:tasks.workunit.client.0.smithi090.stdout:9/225: mkdir db/dd/d18/d31/d4c 0 2024-04-26T04:42:19.755 INFO:tasks.workunit.client.0.smithi090.stdout:9/226: getdents db/dd/d44 0 2024-04-26T04:42:19.759 INFO:tasks.workunit.client.1.smithi104.stdout:9/517: getdents d11/d15/d1b 0 2024-04-26T04:42:19.762 INFO:tasks.workunit.client.1.smithi104.stdout:9/518: unlink d11/d15/d32/d35/f5e 0 2024-04-26T04:42:19.762 INFO:tasks.workunit.client.1.smithi104.stdout:9/519: fdatasync d11/d15/d32/d4c/d54/f62 0 2024-04-26T04:42:19.763 INFO:tasks.workunit.client.1.smithi104.stdout:9/520: chown f10 2276525 1 2024-04-26T04:42:19.764 INFO:tasks.workunit.client.1.smithi104.stdout:9/521: unlink d11/d1d/d71/d94/la1 0 2024-04-26T04:42:19.766 INFO:tasks.workunit.client.1.smithi104.stdout:4/672: sync 2024-04-26T04:42:19.766 INFO:tasks.workunit.client.1.smithi104.stdout:4/673: write d4/d7/fe8 [137031,21245] 0 2024-04-26T04:42:19.768 INFO:tasks.workunit.client.1.smithi104.stdout:4/674: mkdir d4/db/d6b/df0 0 2024-04-26T04:42:19.777 INFO:tasks.workunit.client.1.smithi104.stdout:2/789: sync 2024-04-26T04:42:19.777 INFO:tasks.workunit.client.1.smithi104.stdout:5/631: sync 2024-04-26T04:42:19.777 INFO:tasks.workunit.client.1.smithi104.stdout:2/790: chown d6/d1b/d51/c9e 885 1 2024-04-26T04:42:19.777 INFO:tasks.workunit.client.1.smithi104.stdout:5/632: chown d0/d1/d1e/f56 49963925 1 2024-04-26T04:42:19.778 INFO:tasks.workunit.client.0.smithi090.stdout:3/209: dwrite d1/f27 [0,4194304] 0 2024-04-26T04:42:19.778 INFO:tasks.workunit.client.1.smithi104.stdout:5/633: rmdir d0/d1/d64/d26/d36/da7/db3 39 2024-04-26T04:42:19.779 INFO:tasks.workunit.client.0.smithi090.stdout:8/381: dwrite dc/d1f/d49/d3b/f4a [0,4194304] 0 2024-04-26T04:42:19.779 INFO:tasks.workunit.client.0.smithi090.stdout:8/382: chown dc/f1e 37 1 2024-04-26T04:42:19.779 INFO:tasks.workunit.client.0.smithi090.stdout:8/383: stat dc/d5a/d5d/f66 0 2024-04-26T04:42:19.781 INFO:tasks.workunit.client.1.smithi104.stdout:7/600: dwrite d2/d4/f11 [0,4194304] 0 2024-04-26T04:42:19.782 INFO:tasks.workunit.client.0.smithi090.stdout:3/210: link d1/d17/c23 d1/d9/c47 0 2024-04-26T04:42:19.783 INFO:tasks.workunit.client.0.smithi090.stdout:8/384: creat dc/d1f/d49/d3b/f85 x:0 0 0 2024-04-26T04:42:19.783 INFO:tasks.workunit.client.0.smithi090.stdout:8/385: readlink dc/d1f/l48 0 2024-04-26T04:42:19.784 INFO:tasks.workunit.client.1.smithi104.stdout:7/601: stat d2/d4/d32/d35/cbb 0 2024-04-26T04:42:19.784 INFO:tasks.workunit.client.1.smithi104.stdout:4/675: dread d4/d7/d1d/f45 [0,4194304] 0 2024-04-26T04:42:19.784 INFO:tasks.workunit.client.1.smithi104.stdout:4/676: chown d4/d7/d1d/d1e/d15/c16 6732252 1 2024-04-26T04:42:19.784 INFO:tasks.workunit.client.0.smithi090.stdout:3/211: mknod d1/d17/d36/c48 0 2024-04-26T04:42:19.786 INFO:tasks.workunit.client.1.smithi104.stdout:4/677: symlink d4/d7/d1d/d1e/d8f/db6/lf1 0 2024-04-26T04:42:19.786 INFO:tasks.workunit.client.0.smithi090.stdout:3/212: creat d1/d6/d2c/f49 x:0 0 0 2024-04-26T04:42:19.787 INFO:tasks.workunit.client.1.smithi104.stdout:4/678: creat d4/d7/d1d/d1e/d24/d59/ff2 x:0 0 0 2024-04-26T04:42:19.788 INFO:tasks.workunit.client.1.smithi104.stdout:4/679: chown d4/d7/d1d/d1e/d8f/db6/f9c 38805 1 2024-04-26T04:42:19.789 INFO:tasks.workunit.client.1.smithi104.stdout:4/680: truncate d4/d7/f8 1474956 0 2024-04-26T04:42:19.789 INFO:tasks.workunit.client.1.smithi104.stdout:4/681: truncate d4/d7/fdf 914084 0 2024-04-26T04:42:19.790 INFO:tasks.workunit.client.0.smithi090.stdout:0/410: dwrite d0/d4b/d5d/d5c/f5e [0,4194304] 0 2024-04-26T04:42:19.794 INFO:tasks.workunit.client.1.smithi104.stdout:7/602: dread d2/d4/f26 [0,4194304] 0 2024-04-26T04:42:19.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/411: rename d0/d4b/d5d/d5c/d8a/d7c/d7e to d0/d4b/d5d/d5c/d8a/d45/d53/d8e 0 2024-04-26T04:42:19.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/412: readlink d0/l50 0 2024-04-26T04:42:19.795 INFO:tasks.workunit.client.0.smithi090.stdout:0/413: chown d0 701 1 2024-04-26T04:42:19.796 INFO:tasks.workunit.client.1.smithi104.stdout:7/603: mknod d2/d4/cc1 0 2024-04-26T04:42:19.796 INFO:tasks.workunit.client.0.smithi090.stdout:0/414: creat d0/d4b/d5d/d48/d7d/f8f x:0 0 0 2024-04-26T04:42:19.797 INFO:tasks.workunit.client.1.smithi104.stdout:7/604: creat d2/d4/d32/d35/dbe/fc2 x:0 0 0 2024-04-26T04:42:19.802 INFO:tasks.workunit.client.1.smithi104.stdout:8/584: dwrite d0/d1c/f6e [0,4194304] 0 2024-04-26T04:42:19.803 INFO:tasks.workunit.client.1.smithi104.stdout:9/522: dwrite d11/d15/f30 [0,4194304] 0 2024-04-26T04:42:19.804 INFO:tasks.workunit.client.0.smithi090.stdout:0/415: read d0/de/f2c [1111845,9114] 0 2024-04-26T04:42:19.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/688: rename d0/d6/d19/d21/d58/db4/dd6 to d0/d6/d19/d21/d66/de4 0 2024-04-26T04:42:19.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/689: chown d0/d6/d19/d1b/da7/db8 472889 1 2024-04-26T04:42:19.807 INFO:tasks.workunit.client.1.smithi104.stdout:2/791: dwrite d6/d1b/d51/d6a/f7d [0,4194304] 0 2024-04-26T04:42:19.809 INFO:tasks.workunit.client.1.smithi104.stdout:4/682: dwrite d4/d20/f9a [0,4194304] 0 2024-04-26T04:42:19.812 INFO:tasks.workunit.client.1.smithi104.stdout:5/634: write d0/d1/d4/d2a/d2b/d37/f57 [528413,65466] 0 2024-04-26T04:42:19.817 INFO:tasks.workunit.client.1.smithi104.stdout:1/690: mkdir d0/d6/d19/d21/d66/d20/d2a/d56/de5 0 2024-04-26T04:42:19.820 INFO:tasks.workunit.client.1.smithi104.stdout:4/683: mkdir d4/d7/d1d/d1e/d15/df3 0 2024-04-26T04:42:19.821 INFO:tasks.workunit.client.1.smithi104.stdout:4/684: fdatasync d4/d7/d1d/d1e/d8f/db6/f87 0 2024-04-26T04:42:19.821 INFO:tasks.workunit.client.1.smithi104.stdout:4/685: write d4/db/fe [925185,111461] 0 2024-04-26T04:42:19.822 INFO:tasks.workunit.client.1.smithi104.stdout:8/585: dwrite d0/d1c/f6e [0,4194304] 0 2024-04-26T04:42:19.822 INFO:tasks.workunit.client.1.smithi104.stdout:8/586: readlink d0/d1c/d27/d47/l14 0 2024-04-26T04:42:19.822 INFO:tasks.workunit.client.1.smithi104.stdout:9/523: getdents d11/d1d 0 2024-04-26T04:42:19.824 INFO:tasks.workunit.client.1.smithi104.stdout:1/691: link d0/d6/d2e/d32/fc7 d0/d6/d19/d1b/da7/db8/fe6 0 2024-04-26T04:42:19.827 INFO:tasks.workunit.client.1.smithi104.stdout:9/524: link d11/f12 d11/d15/d32/d35/d7b/fa6 0 2024-04-26T04:42:19.827 INFO:tasks.workunit.client.1.smithi104.stdout:9/525: creat d11/d1d/d34/d69/fa7 x:0 0 0 2024-04-26T04:42:19.827 INFO:tasks.workunit.client.1.smithi104.stdout:9/526: dread - d11/d1d/f6b zero size 2024-04-26T04:42:19.827 INFO:tasks.workunit.client.1.smithi104.stdout:1/692: rmdir d0/d6/d19/d1b/da7/db8 39 2024-04-26T04:42:19.830 INFO:tasks.workunit.client.1.smithi104.stdout:9/527: symlink d11/d15/d1b/d6d/d9c/la8 0 2024-04-26T04:42:19.830 INFO:tasks.workunit.client.1.smithi104.stdout:9/528: fdatasync d11/d15/d1b/f60 0 2024-04-26T04:42:19.830 INFO:tasks.workunit.client.1.smithi104.stdout:9/529: read d11/f17 [2025937,45528] 0 2024-04-26T04:42:19.832 INFO:tasks.workunit.client.1.smithi104.stdout:9/530: mknod d11/d15/d32/d4c/ca9 0 2024-04-26T04:42:19.832 INFO:tasks.workunit.client.1.smithi104.stdout:9/531: stat d11/d15/d32/d4c/d8a 0 2024-04-26T04:42:19.833 INFO:tasks.workunit.client.1.smithi104.stdout:8/587: dread d0/d1c/d27/d47/f25 [0,4194304] 0 2024-04-26T04:42:19.834 INFO:tasks.workunit.client.1.smithi104.stdout:9/532: creat d11/d53/d84/faa x:0 0 0 2024-04-26T04:42:19.835 INFO:tasks.workunit.client.1.smithi104.stdout:8/588: truncate d0/f1a 7730917 0 2024-04-26T04:42:19.837 INFO:tasks.workunit.client.1.smithi104.stdout:9/533: mknod d11/d15/d98/cab 0 2024-04-26T04:42:19.847 INFO:tasks.workunit.client.1.smithi104.stdout:2/792: dwrite d6/d8/d3e/f49 [4194304,4194304] 0 2024-04-26T04:42:19.849 INFO:tasks.workunit.client.1.smithi104.stdout:2/793: mknod d6/d8/c104 0 2024-04-26T04:42:19.849 INFO:tasks.workunit.client.1.smithi104.stdout:2/794: fsync d6/d1b/d93/faa 0 2024-04-26T04:42:19.852 INFO:tasks.workunit.client.0.smithi090.stdout:0/416: dwrite d0/d4b/d5d/d5c/d8a/d45/f59 [0,4194304] 0 2024-04-26T04:42:19.855 INFO:tasks.workunit.client.0.smithi090.stdout:0/417: creat d0/de/f90 x:0 0 0 2024-04-26T04:42:19.855 INFO:tasks.workunit.client.0.smithi090.stdout:0/418: truncate d0/df/f25 1287207 0 2024-04-26T04:42:19.859 INFO:tasks.workunit.client.1.smithi104.stdout:4/686: dwrite d4/f98 [4194304,4194304] 0 2024-04-26T04:42:19.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/419: dread d0/d4b/d5d/d5c/d8a/f41 [0,4194304] 0 2024-04-26T04:42:19.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/420: truncate d0/d4b/d5d/d5c/d8a/d45/f55 1408160 0 2024-04-26T04:42:19.866 INFO:tasks.workunit.client.1.smithi104.stdout:5/635: dwrite d0/d1/d4/d2a/d6b/d91/dbd/fc5 [0,4194304] 0 2024-04-26T04:42:19.870 INFO:tasks.workunit.client.1.smithi104.stdout:5/636: creat d0/d1/d4/d2a/d6b/d91/dbd/fd9 x:0 0 0 2024-04-26T04:42:19.871 INFO:tasks.workunit.client.1.smithi104.stdout:5/637: stat d0/d1/d64/d26/c6d 0 2024-04-26T04:42:19.871 INFO:tasks.workunit.client.1.smithi104.stdout:5/638: write d0/d1/d1e/f27 [923248,110562] 0 2024-04-26T04:42:19.872 INFO:tasks.workunit.client.1.smithi104.stdout:4/687: dread d4/d7/ff [0,4194304] 0 2024-04-26T04:42:19.874 INFO:tasks.workunit.client.1.smithi104.stdout:5/639: rename d0/d1/d4/la to d0/d1/d64/d26/d36/lda 0 2024-04-26T04:42:19.874 INFO:tasks.workunit.client.1.smithi104.stdout:5/640: read d0/d1/d64/f31 [3680627,114040] 0 2024-04-26T04:42:19.878 INFO:tasks.workunit.client.1.smithi104.stdout:4/688: dread d4/d7/d1d/d1e/d24/d4a/f82 [0,4194304] 0 2024-04-26T04:42:19.880 INFO:tasks.workunit.client.1.smithi104.stdout:8/589: dwrite d0/d1c/d2d/f32 [0,4194304] 0 2024-04-26T04:42:19.880 INFO:tasks.workunit.client.1.smithi104.stdout:4/689: rename d4/d7/d1d/d1e/c95 to d4/d7/d1d/d1e/d8f/cf4 0 2024-04-26T04:42:19.881 INFO:tasks.workunit.client.1.smithi104.stdout:8/590: readlink d0/d33/d35/l61 0 2024-04-26T04:42:19.883 INFO:tasks.workunit.client.0.smithi090.stdout:2/346: sync 2024-04-26T04:42:19.883 INFO:tasks.workunit.client.0.smithi090.stdout:7/358: sync 2024-04-26T04:42:19.883 INFO:tasks.workunit.client.0.smithi090.stdout:4/324: sync 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:1/307: sync 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:6/405: sync 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:2/347: chown db 1924075 1 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:1/308: dread - d0/d4/d7/f33 zero size 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:1/309: chown d0/l10 166031641 1 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:1/310: getdents d0/d4/d7/d4a 0 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:7/359: write d0/db/d15/d29/f6e [950990,110342] 0 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:6/406: write d9/d3c/f61 [1799220,82641] 0 2024-04-26T04:42:19.884 INFO:tasks.workunit.client.0.smithi090.stdout:7/360: fsync d0/db/d15/f57 0 2024-04-26T04:42:19.885 INFO:tasks.workunit.client.1.smithi104.stdout:8/591: link d0/d33/d51/c8a d0/d1c/d27/d47/d67/d76/daf/cb9 0 2024-04-26T04:42:19.885 INFO:tasks.workunit.client.1.smithi104.stdout:4/690: getdents d4/d7/d1d/d1e/d24/d48 0 2024-04-26T04:42:19.885 INFO:tasks.workunit.client.1.smithi104.stdout:4/691: fsync d4/db/f12 0 2024-04-26T04:42:19.885 INFO:tasks.workunit.client.1.smithi104.stdout:4/692: creat d4/d20/d55/d56/ff5 x:0 0 0 2024-04-26T04:42:19.885 INFO:tasks.workunit.client.1.smithi104.stdout:9/534: dwrite d11/d15/d32/d35/d7b/fa6 [0,4194304] 0 2024-04-26T04:42:19.886 INFO:tasks.workunit.client.1.smithi104.stdout:1/693: dwrite d0/d6/d49/f4a [4194304,4194304] 0 2024-04-26T04:42:19.886 INFO:tasks.workunit.client.1.smithi104.stdout:9/535: stat d11/d1d/f31 0 2024-04-26T04:42:19.886 INFO:tasks.workunit.client.1.smithi104.stdout:1/694: fsync d0/d6/d49/f4a 0 2024-04-26T04:42:19.886 INFO:tasks.workunit.client.1.smithi104.stdout:2/795: dwrite d6/d1b/d51/d6a/f6f [0,4194304] 0 2024-04-26T04:42:19.886 INFO:tasks.workunit.client.0.smithi090.stdout:4/325: truncate d3/d13/f1f 508642 0 2024-04-26T04:42:19.886 INFO:tasks.workunit.client.0.smithi090.stdout:2/348: creat db/f76 x:0 0 0 2024-04-26T04:42:19.887 INFO:tasks.workunit.client.1.smithi104.stdout:4/693: mknod d4/d7/d1d/d2e/d74/dbd/dd3/cf6 0 2024-04-26T04:42:19.888 INFO:tasks.workunit.client.0.smithi090.stdout:6/407: symlink d9/d21/d3b/l99 0 2024-04-26T04:42:19.889 INFO:tasks.workunit.client.1.smithi104.stdout:1/695: write d0/d6/d19/d21/d66/de4/d81/d9d/f9f [2222877,107426] 0 2024-04-26T04:42:19.889 INFO:tasks.workunit.client.1.smithi104.stdout:1/696: write d0/d6/d2e/d32/d4c/fb0 [676582,594] 0 2024-04-26T04:42:19.889 INFO:tasks.workunit.client.1.smithi104.stdout:1/697: chown d0/d6/c7a 24155099 1 2024-04-26T04:42:19.890 INFO:tasks.workunit.client.0.smithi090.stdout:4/326: truncate d3/d13/f61 1890577 0 2024-04-26T04:42:19.890 INFO:tasks.workunit.client.0.smithi090.stdout:0/421: dwrite d0/df/f3c [0,4194304] 0 2024-04-26T04:42:19.891 INFO:tasks.workunit.client.1.smithi104.stdout:8/592: symlink d0/d50/lba 0 2024-04-26T04:42:19.892 INFO:tasks.workunit.client.1.smithi104.stdout:5/641: dwrite d0/d1/d4/d2a/d6b/d91/dc7/fcd [0,4194304] 0 2024-04-26T04:42:19.892 INFO:tasks.workunit.client.1.smithi104.stdout:5/642: chown d0/d1/d4/d2a/d6b/c83 0 1 2024-04-26T04:42:19.892 INFO:tasks.workunit.client.1.smithi104.stdout:5/643: fsync d0/d1/d64/f19 0 2024-04-26T04:42:19.897 INFO:tasks.workunit.client.0.smithi090.stdout:0/422: creat d0/d4b/d5d/d5c/f91 x:0 0 0 2024-04-26T04:42:19.898 INFO:tasks.workunit.client.0.smithi090.stdout:2/349: dread db/d11/d14/d25/f49 [0,4194304] 0 2024-04-26T04:42:19.898 INFO:tasks.workunit.client.0.smithi090.stdout:2/350: chown db/d11/d14/d27/d2f/d42/d5c/c73 2 1 2024-04-26T04:42:19.898 INFO:tasks.workunit.client.1.smithi104.stdout:9/536: mknod d11/d1d/d71/cac 0 2024-04-26T04:42:19.899 INFO:tasks.workunit.client.0.smithi090.stdout:0/423: mknod d0/d4b/d5d/d5c/d8a/c92 0 2024-04-26T04:42:19.902 INFO:tasks.workunit.client.0.smithi090.stdout:2/351: rename db/d11/d14/d1b/d20/d4c/d5d to db/d59/d66/d77 0 2024-04-26T04:42:19.902 INFO:tasks.workunit.client.0.smithi090.stdout:2/352: readlink db/d11/d14/d25/l31 0 2024-04-26T04:42:19.902 INFO:tasks.workunit.client.0.smithi090.stdout:0/424: dread d0/d4b/d5d/d5c/d8a/f17 [0,4194304] 0 2024-04-26T04:42:19.903 INFO:tasks.workunit.client.1.smithi104.stdout:2/796: mkdir d6/d1b/d68/d105 0 2024-04-26T04:42:19.904 INFO:tasks.workunit.client.0.smithi090.stdout:2/353: unlink db/d11/d14/d25/f5e 0 2024-04-26T04:42:19.905 INFO:tasks.workunit.client.1.smithi104.stdout:4/694: rename d4/d7/d1d/d1e/d24/f76 to d4/d7/d1d/d8e/ff7 0 2024-04-26T04:42:19.905 INFO:tasks.workunit.client.1.smithi104.stdout:4/695: write d4/d7/f1f [1704915,93117] 0 2024-04-26T04:42:19.906 INFO:tasks.workunit.client.0.smithi090.stdout:0/425: mkdir d0/d4b/d5d/d5c/d8a/d45/d93 0 2024-04-26T04:42:19.907 INFO:tasks.workunit.client.1.smithi104.stdout:8/593: truncate d0/d1c/d27/f72 420302 0 2024-04-26T04:42:19.915 INFO:tasks.workunit.client.1.smithi104.stdout:9/537: write d11/d15/f18 [5215386,32748] 0 2024-04-26T04:42:19.915 INFO:tasks.workunit.client.1.smithi104.stdout:9/538: chown d11/d15/d1b/d6d 2588 1 2024-04-26T04:42:19.915 INFO:tasks.workunit.client.1.smithi104.stdout:5/644: dread - d0/d1/d4/d2a/d2b/d37/f8e zero size 2024-04-26T04:42:19.915 INFO:tasks.workunit.client.1.smithi104.stdout:5/645: truncate d0/d1/d1e/f56 899532 0 2024-04-26T04:42:19.916 INFO:tasks.workunit.client.1.smithi104.stdout:5/646: chown d0/d1/d64/f19 464 1 2024-04-26T04:42:19.916 INFO:tasks.workunit.client.1.smithi104.stdout:5/647: readlink d0/d1/d64/d26/d36/lda 0 2024-04-26T04:42:19.916 INFO:tasks.workunit.client.1.smithi104.stdout:2/797: symlink d6/de/d1d/dcf/l106 0 2024-04-26T04:42:19.916 INFO:tasks.workunit.client.1.smithi104.stdout:2/798: write d6/de/d8e/f69 [1874274,92133] 0 2024-04-26T04:42:19.916 INFO:tasks.workunit.client.1.smithi104.stdout:2/799: chown d6/d1b/d51/d6a/cee 19 1 2024-04-26T04:42:19.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/354: write db/d11/d14/d27/d28/f3d [3681910,104723] 0 2024-04-26T04:42:19.918 INFO:tasks.workunit.client.1.smithi104.stdout:8/594: mknod d0/d1c/d2d/d7a/cbb 0 2024-04-26T04:42:19.920 INFO:tasks.workunit.client.1.smithi104.stdout:9/539: symlink d11/d15/da3/lad 0 2024-04-26T04:42:19.920 INFO:tasks.workunit.client.1.smithi104.stdout:9/540: chown d11/d15/d98 7793736 1 2024-04-26T04:42:19.920 INFO:tasks.workunit.client.1.smithi104.stdout:9/541: chown f10 5 1 2024-04-26T04:42:19.920 INFO:tasks.workunit.client.1.smithi104.stdout:9/542: readlink d11/d1d/l4d 0 2024-04-26T04:42:19.920 INFO:tasks.workunit.client.0.smithi090.stdout:2/355: link db/d11/d14/d27/d2f/f32 db/d59/f78 0 2024-04-26T04:42:19.921 INFO:tasks.workunit.client.1.smithi104.stdout:5/648: link d0/d1/d4/c7e d0/d1/d64/d26/cdb 0 2024-04-26T04:42:19.921 INFO:tasks.workunit.client.1.smithi104.stdout:5/649: chown d0/d1/d4/d2a/f50 438 1 2024-04-26T04:42:19.921 INFO:tasks.workunit.client.1.smithi104.stdout:5/650: write d0/d1/d4/d2a/d2b/d37/d48/f70 [3979410,91075] 0 2024-04-26T04:42:19.921 INFO:tasks.workunit.client.1.smithi104.stdout:5/651: chown d0/d1/d64/d26/f44 4 1 2024-04-26T04:42:19.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/356: unlink db/d11/d14/d27/d4b/f58 0 2024-04-26T04:42:19.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/357: write db/d11/d14/d1b/f6c [966202,35523] 0 2024-04-26T04:42:19.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/358: stat db/d11 0 2024-04-26T04:42:19.922 INFO:tasks.workunit.client.1.smithi104.stdout:8/595: write d0/f2f [9215872,7329] 0 2024-04-26T04:42:19.923 INFO:tasks.workunit.client.0.smithi090.stdout:2/359: truncate db/d11/f17 3209670 0 2024-04-26T04:42:19.923 INFO:tasks.workunit.client.1.smithi104.stdout:9/543: rename d11/d53/f85 to d11/d15/d32/d35/d7b/fae 0 2024-04-26T04:42:19.924 INFO:tasks.workunit.client.1.smithi104.stdout:9/544: chown d11/f44 15363 1 2024-04-26T04:42:19.924 INFO:tasks.workunit.client.0.smithi090.stdout:1/311: dwrite d0/f2 [0,4194304] 0 2024-04-26T04:42:19.924 INFO:tasks.workunit.client.0.smithi090.stdout:2/360: creat db/d59/d66/d77/f79 x:0 0 0 2024-04-26T04:42:19.925 INFO:tasks.workunit.client.1.smithi104.stdout:8/596: mknod d0/d1c/d2d/d7a/cbc 0 2024-04-26T04:42:19.926 INFO:tasks.workunit.client.1.smithi104.stdout:9/545: symlink d11/d53/d84/laf 0 2024-04-26T04:42:19.926 INFO:tasks.workunit.client.0.smithi090.stdout:1/312: symlink d0/d4/d7/d4b/d53/l60 0 2024-04-26T04:42:19.927 INFO:tasks.workunit.client.0.smithi090.stdout:2/361: symlink db/dc/d6f/l7a 0 2024-04-26T04:42:19.927 INFO:tasks.workunit.client.0.smithi090.stdout:2/362: chown db/d11/d14/d27/d2f 1001 1 2024-04-26T04:42:19.928 INFO:tasks.workunit.client.1.smithi104.stdout:8/597: mknod d0/d1c/d27/d47/db1/cbd 0 2024-04-26T04:42:19.928 INFO:tasks.workunit.client.0.smithi090.stdout:1/313: mknod d0/d4/d7/d4a/c61 0 2024-04-26T04:42:19.928 INFO:tasks.workunit.client.0.smithi090.stdout:1/314: truncate d0/d4/d7/d17/d22/f5c 626207 0 2024-04-26T04:42:19.928 INFO:tasks.workunit.client.0.smithi090.stdout:1/315: stat d0/d4/d7/d17/d22/d31/f39 0 2024-04-26T04:42:19.929 INFO:tasks.workunit.client.0.smithi090.stdout:1/316: dread d0/d4/d7/d17/d22/f5c [0,4194304] 0 2024-04-26T04:42:19.929 INFO:tasks.workunit.client.0.smithi090.stdout:1/317: dread - d0/d4/d7/d49/f5e zero size 2024-04-26T04:42:19.929 INFO:tasks.workunit.client.0.smithi090.stdout:1/318: readlink d0/d4/d7/l37 0 2024-04-26T04:42:19.929 INFO:tasks.workunit.client.0.smithi090.stdout:1/319: fsync d0/d4/d43/f54 0 2024-04-26T04:42:19.930 INFO:tasks.workunit.client.1.smithi104.stdout:9/546: rmdir d11/d15/d32/d4c/d9e 0 2024-04-26T04:42:19.935 INFO:tasks.workunit.client.1.smithi104.stdout:8/598: rename d0/d1c/d2d/d7a/cb3 to d0/d33/d35/d6d/da4/cbe 0 2024-04-26T04:42:19.935 INFO:tasks.workunit.client.1.smithi104.stdout:8/599: stat d0/d33/d51/fa2 0 2024-04-26T04:42:19.936 INFO:tasks.workunit.client.1.smithi104.stdout:9/547: symlink d11/lb0 0 2024-04-26T04:42:19.936 INFO:tasks.workunit.client.1.smithi104.stdout:9/548: write d11/d15/d32/f9a [781031,115484] 0 2024-04-26T04:42:19.936 INFO:tasks.workunit.client.1.smithi104.stdout:9/549: fsync d11/f20 0 2024-04-26T04:42:19.939 INFO:tasks.workunit.client.1.smithi104.stdout:9/550: truncate d11/d15/f6a 113825 0 2024-04-26T04:42:19.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/361: dwrite d0/db/d53/f56 [0,4194304] 0 2024-04-26T04:42:19.946 INFO:tasks.workunit.client.1.smithi104.stdout:1/698: dwrite d0/d6/d19/d21/d59/f73 [0,4194304] 0 2024-04-26T04:42:19.948 INFO:tasks.workunit.client.0.smithi090.stdout:2/363: dread db/dc/f1f [0,4194304] 0 2024-04-26T04:42:19.948 INFO:tasks.workunit.client.0.smithi090.stdout:7/362: mknod d0/db/d53/d5e/c81 0 2024-04-26T04:42:19.949 INFO:tasks.workunit.client.0.smithi090.stdout:2/364: creat db/d11/d14/d27/f7b x:0 0 0 2024-04-26T04:42:19.951 INFO:tasks.workunit.client.0.smithi090.stdout:7/363: rename d0/f4e to d0/db/df/d1c/d23/f82 0 2024-04-26T04:42:19.952 INFO:tasks.workunit.client.0.smithi090.stdout:7/364: fdatasync d0/db/df/f70 0 2024-04-26T04:42:19.952 INFO:tasks.workunit.client.0.smithi090.stdout:7/365: fsync d0/db/df/f42 0 2024-04-26T04:42:19.952 INFO:tasks.workunit.client.0.smithi090.stdout:6/408: dwrite d9/d21/d3b/f51 [0,4194304] 0 2024-04-26T04:42:19.952 INFO:tasks.workunit.client.0.smithi090.stdout:0/426: dwrite d0/d4b/d5d/d5c/f5e [4194304,4194304] 0 2024-04-26T04:42:19.952 INFO:tasks.workunit.client.1.smithi104.stdout:4/696: dwrite d4/db/fe [0,4194304] 0 2024-04-26T04:42:19.953 INFO:tasks.workunit.client.1.smithi104.stdout:4/697: chown d4/d7/d1d/d1e/fad 60 1 2024-04-26T04:42:19.953 INFO:tasks.workunit.client.1.smithi104.stdout:4/698: truncate d4/d7/d1d/d1e/d15/dd8/fef 71490 0 2024-04-26T04:42:19.953 INFO:tasks.workunit.client.1.smithi104.stdout:5/652: dwrite d0/d1/d64/f85 [0,4194304] 0 2024-04-26T04:42:19.953 INFO:tasks.workunit.client.1.smithi104.stdout:5/653: readlink d0/l16 0 2024-04-26T04:42:19.954 INFO:tasks.workunit.client.0.smithi090.stdout:2/365: unlink db/d11/d14/d1b/d20/c22 0 2024-04-26T04:42:19.956 INFO:tasks.workunit.client.0.smithi090.stdout:7/366: getdents d0/d1/d4 0 2024-04-26T04:42:19.957 INFO:tasks.workunit.client.0.smithi090.stdout:4/327: read d3/db/d12/f4e [1483536,87734] 0 2024-04-26T04:42:19.957 INFO:tasks.workunit.client.0.smithi090.stdout:4/328: stat f0 0 2024-04-26T04:42:19.958 INFO:tasks.workunit.client.0.smithi090.stdout:2/366: symlink db/d11/d14/d27/d2f/d42/l7c 0 2024-04-26T04:42:19.958 INFO:tasks.workunit.client.0.smithi090.stdout:2/367: fsync db/d11/d14/d25/f4e 0 2024-04-26T04:42:19.959 INFO:tasks.workunit.client.1.smithi104.stdout:1/699: creat d0/d6/d19/d21/d66/d29/d3f/fe7 x:0 0 0 2024-04-26T04:42:19.959 INFO:tasks.workunit.client.1.smithi104.stdout:1/700: chown d0/d6/d2e/d32/d4c/db1/dc3/ccf 299546688 1 2024-04-26T04:42:19.962 INFO:tasks.workunit.client.0.smithi090.stdout:7/367: truncate d0/db/df/d4d/f63 360595 0 2024-04-26T04:42:19.963 INFO:tasks.workunit.client.1.smithi104.stdout:2/800: getdents d6/de/d1d/dcf 0 2024-04-26T04:42:19.963 INFO:tasks.workunit.client.0.smithi090.stdout:2/368: rmdir db/d11/d14 39 2024-04-26T04:42:19.964 INFO:tasks.workunit.client.1.smithi104.stdout:0/569: sync 2024-04-26T04:42:19.964 INFO:tasks.workunit.client.1.smithi104.stdout:6/740: sync 2024-04-26T04:42:19.965 INFO:tasks.workunit.client.1.smithi104.stdout:3/698: sync 2024-04-26T04:42:19.969 INFO:tasks.workunit.client.1.smithi104.stdout:2/801: symlink d6/de/d1d/dcf/l107 0 2024-04-26T04:42:19.969 INFO:tasks.workunit.client.1.smithi104.stdout:2/802: truncate d6/d8/d3e/fa3 753465 0 2024-04-26T04:42:19.969 INFO:tasks.workunit.client.1.smithi104.stdout:2/803: fdatasync d6/d8/d5a/f2e 0 2024-04-26T04:42:19.970 INFO:tasks.workunit.client.0.smithi090.stdout:2/369: rmdir db/d11/d14/d25/d2d 39 2024-04-26T04:42:19.970 INFO:tasks.workunit.client.1.smithi104.stdout:0/570: readlink d0/l5f 0 2024-04-26T04:42:19.970 INFO:tasks.workunit.client.1.smithi104.stdout:0/571: creat d0/d2/d10/d84/fb0 x:0 0 0 2024-04-26T04:42:19.970 INFO:tasks.workunit.client.1.smithi104.stdout:0/572: stat d0/l5f 0 2024-04-26T04:42:19.970 INFO:tasks.workunit.client.1.smithi104.stdout:0/573: creat d0/d2/d21/fb1 x:0 0 0 2024-04-26T04:42:19.971 INFO:tasks.workunit.client.0.smithi090.stdout:1/320: dwrite d0/d4/d7/d17/d22/d31/f39 [0,4194304] 0 2024-04-26T04:42:19.971 INFO:tasks.workunit.client.0.smithi090.stdout:7/368: symlink d0/db/d15/d30/l83 0 2024-04-26T04:42:19.972 INFO:tasks.workunit.client.1.smithi104.stdout:9/551: dwrite d11/d15/d32/d35/f80 [0,4194304] 0 2024-04-26T04:42:19.972 INFO:tasks.workunit.client.1.smithi104.stdout:2/804: symlink d6/d1b/d68/d81/dce/l108 0 2024-04-26T04:42:19.973 INFO:tasks.workunit.client.1.smithi104.stdout:2/805: read d6/d8/d3e/f49 [2944941,71913] 0 2024-04-26T04:42:19.973 INFO:tasks.workunit.client.1.smithi104.stdout:2/806: write d6/d8/d5a/fb3 [105107,35110] 0 2024-04-26T04:42:19.973 INFO:tasks.workunit.client.1.smithi104.stdout:2/807: chown d6/de/ccb 10 1 2024-04-26T04:42:19.974 INFO:tasks.workunit.client.1.smithi104.stdout:2/808: write d6/d1b/d93/fc4 [300552,104761] 0 2024-04-26T04:42:19.974 INFO:tasks.workunit.client.0.smithi090.stdout:2/370: creat db/d11/d14/d25/d2d/d48/d5b/f7d x:0 0 0 2024-04-26T04:42:19.974 INFO:tasks.workunit.client.0.smithi090.stdout:0/427: write d0/df/f3c [440705,93172] 0 2024-04-26T04:42:19.976 INFO:tasks.workunit.client.0.smithi090.stdout:1/321: dread d0/d4/d7/d17/d22/f52 [0,4194304] 0 2024-04-26T04:42:19.976 INFO:tasks.workunit.client.0.smithi090.stdout:7/369: mknod d0/db/df/d4d/d64/d66/c84 0 2024-04-26T04:42:19.976 INFO:tasks.workunit.client.0.smithi090.stdout:4/329: dread d3/db/d12/d20/d24/d30/f5a [0,4194304] 0 2024-04-26T04:42:19.976 INFO:tasks.workunit.client.0.smithi090.stdout:4/330: write d3/db/d12/d20/d24/f32 [1298817,22622] 0 2024-04-26T04:42:19.977 INFO:tasks.workunit.client.1.smithi104.stdout:6/741: dread d0/d9/d13/d41/f7f [0,4194304] 0 2024-04-26T04:42:19.979 INFO:tasks.workunit.client.1.smithi104.stdout:0/574: truncate d0/d2/d10/d3d/f8d 3794893 0 2024-04-26T04:42:19.980 INFO:tasks.workunit.client.1.smithi104.stdout:0/575: stat d0/d2/d21/d27/f30 0 2024-04-26T04:42:19.980 INFO:tasks.workunit.client.0.smithi090.stdout:0/428: creat d0/d4b/d5d/d5c/d8a/f94 x:0 0 0 2024-04-26T04:42:19.983 INFO:tasks.workunit.client.1.smithi104.stdout:9/552: mknod d11/d53/cb1 0 2024-04-26T04:42:19.983 INFO:tasks.workunit.client.1.smithi104.stdout:3/699: dread d5/db2/da3/dd0/ddb/d69/fee [0,4194304] 0 2024-04-26T04:42:19.985 INFO:tasks.workunit.client.0.smithi090.stdout:0/429: creat d0/d4b/d5d/d5c/d8a/d7c/f95 x:0 0 0 2024-04-26T04:42:19.987 INFO:tasks.workunit.client.1.smithi104.stdout:0/576: mkdir d0/d2/d21/d49/d5e/d9c/db2 0 2024-04-26T04:42:19.989 INFO:tasks.workunit.client.1.smithi104.stdout:3/700: mkdir d5/db2/da3/dd0/ddb/d3a/df5 0 2024-04-26T04:42:19.991 INFO:tasks.workunit.client.0.smithi090.stdout:0/430: rename d0/d4b/d5d/d5c/d8a/f41 to d0/d4b/d5d/d5c/d8a/d45/f96 0 2024-04-26T04:42:19.991 INFO:tasks.workunit.client.0.smithi090.stdout:0/431: write d0/d4b/d5d/d5c/d8a/f3e [1471199,48154] 0 2024-04-26T04:42:19.993 INFO:tasks.workunit.client.1.smithi104.stdout:3/701: symlink d5/d9/d86/lf6 0 2024-04-26T04:42:19.994 INFO:tasks.workunit.client.1.smithi104.stdout:3/702: rmdir d5/db2/d72 39 2024-04-26T04:42:19.996 INFO:tasks.workunit.client.1.smithi104.stdout:4/699: dwrite d4/d7/d1d/d1e/d24/d4a/fec [4194304,4194304] 0 2024-04-26T04:42:19.996 INFO:tasks.workunit.client.0.smithi090.stdout:0/432: link d0/l50 d0/d4b/d5d/d5c/d8a/d45/d53/d8e/l97 0 2024-04-26T04:42:19.996 INFO:tasks.workunit.client.1.smithi104.stdout:4/700: chown d4/d7/d1d/d1e/d15 1368 1 2024-04-26T04:42:19.998 INFO:tasks.workunit.client.0.smithi090.stdout:1/322: write d0/d4/d7/d17/f18 [2073991,85017] 0 2024-04-26T04:42:19.998 INFO:tasks.workunit.client.0.smithi090.stdout:1/323: write d0/d4/f15 [1887049,79573] 0 2024-04-26T04:42:19.999 INFO:tasks.workunit.client.1.smithi104.stdout:7/605: sync 2024-04-26T04:42:19.999 INFO:tasks.workunit.client.1.smithi104.stdout:7/606: fsync d2/d4/d5a/da0/fab 0 2024-04-26T04:42:19.999 INFO:tasks.workunit.client.0.smithi090.stdout:0/433: link d0/d4b/d5d/d48/d7d/c8b d0/d4b/d6d/c98 0 2024-04-26T04:42:19.999 INFO:tasks.workunit.client.0.smithi090.stdout:2/371: read db/d11/d14/d25/f37 [526313,10497] 0 2024-04-26T04:42:20.000 INFO:tasks.workunit.client.0.smithi090.stdout:1/324: mkdir d0/d4/d62 0 2024-04-26T04:42:20.001 INFO:tasks.workunit.client.1.smithi104.stdout:7/607: link d2/d4/d32/d3d/d47/fbc d2/d4/d32/d3d/d47/db1/fc3 0 2024-04-26T04:42:20.002 INFO:tasks.workunit.client.1.smithi104.stdout:7/608: truncate d2/d4/d5a/f68 2214817 0 2024-04-26T04:42:20.004 INFO:tasks.workunit.client.1.smithi104.stdout:7/609: unlink d2/d4/d32/d35/d37/f3a 0 2024-04-26T04:42:20.005 INFO:tasks.workunit.client.1.smithi104.stdout:7/610: mknod d2/d4/d32/d3d/d41/d5f/d6c/d94/cc4 0 2024-04-26T04:42:20.005 INFO:tasks.workunit.client.1.smithi104.stdout:7/611: write d2/d4/d1d/fa8 [95806,36612] 0 2024-04-26T04:42:20.006 INFO:tasks.workunit.client.1.smithi104.stdout:7/612: mknod d2/cc5 0 2024-04-26T04:42:20.007 INFO:tasks.workunit.client.1.smithi104.stdout:7/613: creat d2/d4/d1d/d34/fc6 x:0 0 0 2024-04-26T04:42:20.007 INFO:tasks.workunit.client.1.smithi104.stdout:7/614: read - d2/d4/d32/d3d/d47/db1/fc3 zero size 2024-04-26T04:42:20.008 INFO:tasks.workunit.client.1.smithi104.stdout:7/615: symlink d2/d4/lc7 0 2024-04-26T04:42:20.010 INFO:tasks.workunit.client.1.smithi104.stdout:3/703: dread d5/db2/f34 [0,4194304] 0 2024-04-26T04:42:20.010 INFO:tasks.workunit.client.1.smithi104.stdout:7/616: unlink d2/f42 0 2024-04-26T04:42:20.010 INFO:tasks.workunit.client.1.smithi104.stdout:7/617: readlink d2/d4/d32/d3d/d41/d7e/l9b 0 2024-04-26T04:42:20.011 INFO:tasks.workunit.client.1.smithi104.stdout:7/618: fsync d2/d4/d32/d3d/d47/f93 0 2024-04-26T04:42:20.011 INFO:tasks.workunit.client.1.smithi104.stdout:3/704: symlink d5/d9/d80/lf7 0 2024-04-26T04:42:20.011 INFO:tasks.workunit.client.1.smithi104.stdout:7/619: symlink d2/d4/d32/d3d/d47/lc8 0 2024-04-26T04:42:20.012 INFO:tasks.workunit.client.1.smithi104.stdout:1/701: dwrite d0/d6/d19/d21/d66/de4/fe3 [4194304,4194304] 0 2024-04-26T04:42:20.012 INFO:tasks.workunit.client.1.smithi104.stdout:1/702: truncate d0/d6/d19/d21/d66/d20/d2a/dbc/fd4 601903 0 2024-04-26T04:42:20.012 INFO:tasks.workunit.client.1.smithi104.stdout:7/620: mknod d2/d4/d32/d3d/d41/dbd/cc9 0 2024-04-26T04:42:20.012 INFO:tasks.workunit.client.1.smithi104.stdout:7/621: write d2/d4/d32/d35/d44/fb3 [165324,41557] 0 2024-04-26T04:42:20.012 INFO:tasks.workunit.client.0.smithi090.stdout:6/409: dwrite d9/d1c/d31/d79/f7e [0,4194304] 0 2024-04-26T04:42:20.013 INFO:tasks.workunit.client.1.smithi104.stdout:3/705: symlink d5/db2/da3/dd0/ddb/lf8 0 2024-04-26T04:42:20.014 INFO:tasks.workunit.client.1.smithi104.stdout:7/622: symlink d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/lca 0 2024-04-26T04:42:20.015 INFO:tasks.workunit.client.1.smithi104.stdout:3/706: link d5/db2/da3/dd0/ddb/d69/fb1 d5/db2/da3/dd0/ddb/d32/d47/ff9 0 2024-04-26T04:42:20.016 INFO:tasks.workunit.client.1.smithi104.stdout:7/623: mknod d2/d4/d32/d3d/d41/d5f/d6c/d94/ccb 0 2024-04-26T04:42:20.017 INFO:tasks.workunit.client.1.smithi104.stdout:9/553: dwrite d11/d53/d84/faa [0,4194304] 0 2024-04-26T04:42:20.017 INFO:tasks.workunit.client.1.smithi104.stdout:9/554: chown d11/d15/d1b/d6d/l8e 6 1 2024-04-26T04:42:20.018 INFO:tasks.workunit.client.1.smithi104.stdout:6/742: dwrite d0/d9/d13/d44/fbe [4194304,4194304] 0 2024-04-26T04:42:20.018 INFO:tasks.workunit.client.1.smithi104.stdout:6/743: dread - d0/d9/dc6/fb5 zero size 2024-04-26T04:42:20.018 INFO:tasks.workunit.client.1.smithi104.stdout:5/654: dwrite d0/f22 [4194304,4194304] 0 2024-04-26T04:42:20.018 INFO:tasks.workunit.client.1.smithi104.stdout:5/655: chown d0/d1/d4/d2a/d2b/fb7 1 1 2024-04-26T04:42:20.018 INFO:tasks.workunit.client.1.smithi104.stdout:5/656: chown d0/d1/d3e/f51 0 1 2024-04-26T04:42:20.018 INFO:tasks.workunit.client.1.smithi104.stdout:5/657: write d0/d1/d4/d2a/d6b/d73/d95/fba [2155386,16473] 0 2024-04-26T04:42:20.018 INFO:tasks.workunit.client.1.smithi104.stdout:3/707: rename d5/db2/da3/dd0/ddb/d69/c8a to d5/db2/da3/df3/cfa 0 2024-04-26T04:42:20.022 INFO:tasks.workunit.client.1.smithi104.stdout:9/555: mkdir d11/d90/db2 0 2024-04-26T04:42:20.022 INFO:tasks.workunit.client.1.smithi104.stdout:2/809: dwrite d6/d8/d3e/fa3 [0,4194304] 0 2024-04-26T04:42:20.022 INFO:tasks.workunit.client.1.smithi104.stdout:2/810: readlink d6/d1b/d51/laf 0 2024-04-26T04:42:20.030 INFO:tasks.workunit.client.1.smithi104.stdout:9/556: mkdir d11/d1d/d34/db3 0 2024-04-26T04:42:20.030 INFO:tasks.workunit.client.1.smithi104.stdout:9/557: write d11/d15/d32/d35/d7b/f82 [277849,16699] 0 2024-04-26T04:42:20.032 INFO:tasks.workunit.client.1.smithi104.stdout:6/744: dread d0/d9/d13/d41/d82/dc8/f94 [0,4194304] 0 2024-04-26T04:42:20.032 INFO:tasks.workunit.client.1.smithi104.stdout:6/745: readlink d0/d9/d13/d41/d82/dc8/d80/dca/ld0 0 2024-04-26T04:42:20.032 INFO:tasks.workunit.client.1.smithi104.stdout:6/746: dread - d0/d9/da8/fea zero size 2024-04-26T04:42:20.032 INFO:tasks.workunit.client.1.smithi104.stdout:6/747: fsync d0/d9/f87 0 2024-04-26T04:42:20.032 INFO:tasks.workunit.client.1.smithi104.stdout:6/748: fsync d0/d9/d13/d41/d82/dc8/d80/f61 0 2024-04-26T04:42:20.038 INFO:tasks.workunit.client.1.smithi104.stdout:9/558: creat d11/d15/d32/d4c/d7c/fb4 x:0 0 0 2024-04-26T04:42:20.039 INFO:tasks.workunit.client.1.smithi104.stdout:6/749: mkdir d0/d9/d13/d41/d82/df7/dfb 0 2024-04-26T04:42:20.041 INFO:tasks.workunit.client.1.smithi104.stdout:2/811: rmdir d6/d1b/d51/d6a 39 2024-04-26T04:42:20.041 INFO:tasks.workunit.client.1.smithi104.stdout:2/812: write d6/d8/d5a/fb3 [1032779,99102] 0 2024-04-26T04:42:20.041 INFO:tasks.workunit.client.1.smithi104.stdout:3/708: getdents d5/db2/da3/dd0/ddb/d3a 0 2024-04-26T04:42:20.041 INFO:tasks.workunit.client.1.smithi104.stdout:3/709: dread - d5/db2/da3/dd0/ddb/d32/d48/fc7 zero size 2024-04-26T04:42:20.043 INFO:tasks.workunit.client.1.smithi104.stdout:9/559: unlink d11/d15/d1b/d6d/c81 0 2024-04-26T04:42:20.044 INFO:tasks.workunit.client.1.smithi104.stdout:2/813: link d6/d1b/d68/l101 d6/d1b/d51/db8/l109 0 2024-04-26T04:42:20.045 INFO:tasks.workunit.client.1.smithi104.stdout:2/814: creat d6/d1b/d75/f10a x:0 0 0 2024-04-26T04:42:20.048 INFO:tasks.workunit.client.1.smithi104.stdout:2/815: creat d6/d8/df6/f10b x:0 0 0 2024-04-26T04:42:20.048 INFO:tasks.workunit.client.1.smithi104.stdout:2/816: truncate d6/d1b/d68/d81/d97/f9a 564770 0 2024-04-26T04:42:20.053 INFO:tasks.workunit.client.1.smithi104.stdout:9/560: dread d11/f16 [0,4194304] 0 2024-04-26T04:42:20.054 INFO:tasks.workunit.client.1.smithi104.stdout:6/750: read d0/d9/d13/d44/d71/fa3 [2189688,54201] 0 2024-04-26T04:42:20.054 INFO:tasks.workunit.client.1.smithi104.stdout:6/751: write d0/d9/d13/d41/d82/dc8/f5e [1755961,85903] 0 2024-04-26T04:42:20.054 INFO:tasks.workunit.client.1.smithi104.stdout:9/561: creat d11/d15/d1b/fb5 x:0 0 0 2024-04-26T04:42:20.054 INFO:tasks.workunit.client.1.smithi104.stdout:9/562: write d11/d15/d1b/f72 [1529133,94082] 0 2024-04-26T04:42:20.055 INFO:tasks.workunit.client.1.smithi104.stdout:9/563: stat d11/d15/d32/d4c/d54/f62 0 2024-04-26T04:42:20.055 INFO:tasks.workunit.client.1.smithi104.stdout:9/564: creat d11/d53/fb6 x:0 0 0 2024-04-26T04:42:20.055 INFO:tasks.workunit.client.1.smithi104.stdout:6/752: unlink d0/d9/c1e 0 2024-04-26T04:42:20.058 INFO:tasks.workunit.client.1.smithi104.stdout:9/565: mknod d11/d1d/d71/d94/cb7 0 2024-04-26T04:42:20.061 INFO:tasks.workunit.client.1.smithi104.stdout:2/817: read d6/f10 [4480693,23214] 0 2024-04-26T04:42:20.061 INFO:tasks.workunit.client.1.smithi104.stdout:2/818: truncate d6/d1b/d51/f87 554114 0 2024-04-26T04:42:20.063 INFO:tasks.workunit.client.1.smithi104.stdout:6/753: mkdir d0/d9/d13/d44/d47/da4/dfc 0 2024-04-26T04:42:20.063 INFO:tasks.workunit.client.0.smithi090.stdout:0/434: rename d0/d4b/l61 to d0/d4b/d5d/d5c/d8a/d7c/l99 0 2024-04-26T04:42:20.063 INFO:tasks.workunit.client.0.smithi090.stdout:0/435: chown d0/d4b/d5d/d5c/d8a/d45 999362 1 2024-04-26T04:42:20.063 INFO:tasks.workunit.client.0.smithi090.stdout:0/436: dread - d0/d4b/d5d/d5c/d8a/f44 zero size 2024-04-26T04:42:20.063 INFO:tasks.workunit.client.0.smithi090.stdout:0/437: dread - d0/de/f90 zero size 2024-04-26T04:42:20.064 INFO:tasks.workunit.client.0.smithi090.stdout:2/372: symlink db/d11/d14/d63/l7e 0 2024-04-26T04:42:20.064 INFO:tasks.workunit.client.0.smithi090.stdout:2/373: truncate db/d11/d14/d27/d28/f3d 4587212 0 2024-04-26T04:42:20.065 INFO:tasks.workunit.client.1.smithi104.stdout:6/754: mknod d0/d9/d13/d41/d96/ded/cfd 0 2024-04-26T04:42:20.065 INFO:tasks.workunit.client.1.smithi104.stdout:6/755: getdents d0/d9/d13/d41/d96/dac/dec/dbd 0 2024-04-26T04:42:20.065 INFO:tasks.workunit.client.1.smithi104.stdout:6/756: truncate d0/d9/d13/d41/d82/dc8/d80/fa7 623492 0 2024-04-26T04:42:20.065 INFO:tasks.workunit.client.0.smithi090.stdout:1/325: creat d0/d55/f63 x:0 0 0 2024-04-26T04:42:20.065 INFO:tasks.workunit.client.1.smithi104.stdout:6/757: link d0/d9/d13/d41/d82/dc8/fb6 d0/d9/d13/d41/d96/ded/ffe 0 2024-04-26T04:42:20.066 INFO:tasks.workunit.client.1.smithi104.stdout:6/758: read - d0/d9/d25/fd7 zero size 2024-04-26T04:42:20.066 INFO:tasks.workunit.client.1.smithi104.stdout:6/759: stat d0/d9/d13/d44/d47/fb1 0 2024-04-26T04:42:20.066 INFO:tasks.workunit.client.1.smithi104.stdout:2/819: dread d6/d8/f70 [0,4194304] 0 2024-04-26T04:42:20.067 INFO:tasks.workunit.client.1.smithi104.stdout:6/760: truncate d0/d9/d13/d41/d82/dc8/d80/fc2 1586450 0 2024-04-26T04:42:20.067 INFO:tasks.workunit.client.1.smithi104.stdout:6/761: chown d0/d9/d25/f68 1668506 1 2024-04-26T04:42:20.067 INFO:tasks.workunit.client.1.smithi104.stdout:2/820: chown d6/f8b 344114713 1 2024-04-26T04:42:20.067 INFO:tasks.workunit.client.1.smithi104.stdout:6/762: write d0/d9/d13/d44/d71/d3f/f5d [915556,12803] 0 2024-04-26T04:42:20.067 INFO:tasks.workunit.client.1.smithi104.stdout:6/763: stat d0/d9/d13/d41/f7f 0 2024-04-26T04:42:20.068 INFO:tasks.workunit.client.1.smithi104.stdout:2/821: link d6/d8/d3e/fc0 d6/d8/df6/dfe/f10c 0 2024-04-26T04:42:20.069 INFO:tasks.workunit.client.0.smithi090.stdout:7/370: dwrite d0/db/d15/d29/f6e [0,4194304] 0 2024-04-26T04:42:20.069 INFO:tasks.workunit.client.1.smithi104.stdout:6/764: rename d0/d9/d13/d44/d71/d3f/c7d to d0/d9/d13/d41/d96/cff 0 2024-04-26T04:42:20.075 INFO:tasks.workunit.client.1.smithi104.stdout:9/566: read d11/d15/d32/d35/f65 [1459816,7760] 0 2024-04-26T04:42:20.076 INFO:tasks.workunit.client.1.smithi104.stdout:7/624: dwrite d2/d4/d1d/d34/f40 [0,4194304] 0 2024-04-26T04:42:20.076 INFO:tasks.workunit.client.1.smithi104.stdout:1/703: dwrite d0/d6/d19/d21/d59/f6e [0,4194304] 0 2024-04-26T04:42:20.078 INFO:tasks.workunit.client.1.smithi104.stdout:5/658: dwrite d0/d1/d4/d2a/d2b/f35 [0,4194304] 0 2024-04-26T04:42:20.080 INFO:tasks.workunit.client.1.smithi104.stdout:3/710: dwrite d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/faa [8388608,4194304] 0 2024-04-26T04:42:20.080 INFO:tasks.workunit.client.0.smithi090.stdout:4/331: dwrite d3/db/d12/d20/d24/f32 [0,4194304] 0 2024-04-26T04:42:20.080 INFO:tasks.workunit.client.1.smithi104.stdout:9/567: mkdir d11/d1d/d71/d94/db8 0 2024-04-26T04:42:20.082 INFO:tasks.workunit.client.0.smithi090.stdout:6/410: mkdir d9/d77/d9a 0 2024-04-26T04:42:20.084 INFO:tasks.workunit.client.1.smithi104.stdout:7/625: creat d2/d4/d32/fcc x:0 0 0 2024-04-26T04:42:20.087 INFO:tasks.workunit.client.0.smithi090.stdout:0/438: creat d0/d4b/d6d/f9a x:0 0 0 2024-04-26T04:42:20.087 INFO:tasks.workunit.client.0.smithi090.stdout:0/439: dread - d0/d4b/d5d/d5c/d8a/d7c/f95 zero size 2024-04-26T04:42:20.087 INFO:tasks.workunit.client.0.smithi090.stdout:0/440: truncate d0/d4b/f52 1662225 0 2024-04-26T04:42:20.088 INFO:tasks.workunit.client.0.smithi090.stdout:0/441: fdatasync d0/de/f39 0 2024-04-26T04:42:20.090 INFO:tasks.workunit.client.1.smithi104.stdout:1/704: dread d0/d6/d19/d21/d66/de4/fe3 [4194304,4194304] 0 2024-04-26T04:42:20.090 INFO:tasks.workunit.client.1.smithi104.stdout:1/705: dread - d0/d6/d2e/d32/fc7 zero size 2024-04-26T04:42:20.090 INFO:tasks.workunit.client.1.smithi104.stdout:1/706: fdatasync d0/d6/d19/d21/d66/d29/d3f/fe7 0 2024-04-26T04:42:20.090 INFO:tasks.workunit.client.1.smithi104.stdout:0/577: write d0/d4/f11 [1779422,65932] 0 2024-04-26T04:42:20.090 INFO:tasks.workunit.client.1.smithi104.stdout:0/578: write d0/d2/d21/d49/f99 [705541,112572] 0 2024-04-26T04:42:20.090 INFO:tasks.workunit.client.1.smithi104.stdout:2/822: dwrite d6/de/d85/fcc [0,4194304] 0 2024-04-26T04:42:20.091 INFO:tasks.workunit.client.0.smithi090.stdout:7/371: mknod d0/d1/d4/d43/d65/c85 0 2024-04-26T04:42:20.091 INFO:tasks.workunit.client.1.smithi104.stdout:3/711: unlink d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/dc2/cc8 0 2024-04-26T04:42:20.092 INFO:tasks.workunit.client.0.smithi090.stdout:4/332: mkdir d3/d13/d64 0 2024-04-26T04:42:20.093 INFO:tasks.workunit.client.1.smithi104.stdout:9/568: link d11/d15/d98/cab d11/d15/d32/d4c/d54/cb9 0 2024-04-26T04:42:20.095 INFO:tasks.workunit.client.0.smithi090.stdout:0/442: creat d0/d4b/d5d/d5c/d8a/d45/f9b x:0 0 0 2024-04-26T04:42:20.095 INFO:tasks.workunit.client.0.smithi090.stdout:0/443: creat d0/d4b/d5d/d5c/f9c x:0 0 0 2024-04-26T04:42:20.095 INFO:tasks.workunit.client.0.smithi090.stdout:1/326: symlink d0/d4/d7/d17/d22/l64 0 2024-04-26T04:42:20.096 INFO:tasks.workunit.client.0.smithi090.stdout:4/333: creat d3/db/d12/d20/d45/d53/f65 x:0 0 0 2024-04-26T04:42:20.096 INFO:tasks.workunit.client.0.smithi090.stdout:4/334: readlink d3/l33 0 2024-04-26T04:42:20.097 INFO:tasks.workunit.client.0.smithi090.stdout:4/335: chown d3/d13/d1d/l62 1 1 2024-04-26T04:42:20.097 INFO:tasks.workunit.client.1.smithi104.stdout:0/579: creat d0/d2/d21/d49/d5e/fb3 x:0 0 0 2024-04-26T04:42:20.099 INFO:tasks.workunit.client.0.smithi090.stdout:7/372: dread d0/db/df/f70 [0,4194304] 0 2024-04-26T04:42:20.099 INFO:tasks.workunit.client.0.smithi090.stdout:7/373: readlink d0/db/d15/d30/d4f/l5c 0 2024-04-26T04:42:20.099 INFO:tasks.workunit.client.0.smithi090.stdout:7/374: write d0/d1/d4/f6b [884734,71924] 0 2024-04-26T04:42:20.101 INFO:tasks.workunit.client.0.smithi090.stdout:2/374: rename db/d11/d14/d27/d4b to db/d11/d7f 0 2024-04-26T04:42:20.101 INFO:tasks.workunit.client.0.smithi090.stdout:2/375: write db/d11/d14/d25/f3a [360581,77173] 0 2024-04-26T04:42:20.101 INFO:tasks.workunit.client.0.smithi090.stdout:2/376: read db/d11/d14/d25/d41/f54 [2254062,39348] 0 2024-04-26T04:42:20.102 INFO:tasks.workunit.client.1.smithi104.stdout:5/659: dwrite d0/d1/d4/d2a/d6b/d73/da4/fd5 [0,4194304] 0 2024-04-26T04:42:20.102 INFO:tasks.workunit.client.1.smithi104.stdout:5/660: dread - d0/d1/d64/d26/d36/d72/fbe zero size 2024-04-26T04:42:20.102 INFO:tasks.workunit.client.1.smithi104.stdout:2/823: creat d6/d8/d3e/f10d x:0 0 0 2024-04-26T04:42:20.103 INFO:tasks.workunit.client.1.smithi104.stdout:2/824: chown d6/d8/df6/dfe 26120874 1 2024-04-26T04:42:20.103 INFO:tasks.workunit.client.1.smithi104.stdout:2/825: dread - d6/d8/d3e/f10d zero size 2024-04-26T04:42:20.103 INFO:tasks.workunit.client.1.smithi104.stdout:5/661: dread d0/d1/d3e/f51 [0,4194304] 0 2024-04-26T04:42:20.103 INFO:tasks.workunit.client.1.smithi104.stdout:5/662: stat d0/f66 0 2024-04-26T04:42:20.103 INFO:tasks.workunit.client.1.smithi104.stdout:5/663: write d0/d1/f2 [2307864,118031] 0 2024-04-26T04:42:20.103 INFO:tasks.workunit.client.1.smithi104.stdout:3/712: creat d5/db2/da3/dd0/ddb/d32/dd9/ffb x:0 0 0 2024-04-26T04:42:20.104 INFO:tasks.workunit.client.1.smithi104.stdout:3/713: truncate d5/db2/da3/dd0/ddb/f21 2275425 0 2024-04-26T04:42:20.105 INFO:tasks.workunit.client.1.smithi104.stdout:9/569: rename d11/d1d/d71/l9b to d11/d15/d32/d35/d7b/lba 0 2024-04-26T04:42:20.105 INFO:tasks.workunit.client.1.smithi104.stdout:9/570: write d11/d1d/f6b [1028829,117062] 0 2024-04-26T04:42:20.107 INFO:tasks.workunit.client.0.smithi090.stdout:0/444: symlink d0/l9d 0 2024-04-26T04:42:20.107 INFO:tasks.workunit.client.0.smithi090.stdout:7/375: symlink d0/db/d53/l86 0 2024-04-26T04:42:20.107 INFO:tasks.workunit.client.0.smithi090.stdout:9/227: sync 2024-04-26T04:42:20.107 INFO:tasks.workunit.client.0.smithi090.stdout:3/213: sync 2024-04-26T04:42:20.107 INFO:tasks.workunit.client.0.smithi090.stdout:5/452: sync 2024-04-26T04:42:20.107 INFO:tasks.workunit.client.0.smithi090.stdout:8/386: sync 2024-04-26T04:42:20.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/411: rename d9/d1c/d4d/d6d/f8f to d9/d21/f9b 0 2024-04-26T04:42:20.108 INFO:tasks.workunit.client.0.smithi090.stdout:2/377: creat db/d11/d14/d27/d2f/d42/d5c/f80 x:0 0 0 2024-04-26T04:42:20.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/412: write d9/d1c/d31/f3a [142769,29477] 0 2024-04-26T04:42:20.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/413: fsync d9/d1c/d4d/d6d/d66/f75 0 2024-04-26T04:42:20.108 INFO:tasks.workunit.client.0.smithi090.stdout:5/453: link d4/d1a/d45/f74 d4/d1a/df/f8d 0 2024-04-26T04:42:20.109 INFO:tasks.workunit.client.0.smithi090.stdout:3/214: symlink d1/d17/d36/d33/d42/l4a 0 2024-04-26T04:42:20.109 INFO:tasks.workunit.client.0.smithi090.stdout:3/215: readlink d1/d6/d2c/l3b 0 2024-04-26T04:42:20.109 INFO:tasks.workunit.client.0.smithi090.stdout:2/378: mkdir db/d11/d14/d25/d2d/d48/d81 0 2024-04-26T04:42:20.112 INFO:tasks.workunit.client.0.smithi090.stdout:9/228: dread db/dd/d18/d38/d39/f3b [0,4194304] 0 2024-04-26T04:42:20.113 INFO:tasks.workunit.client.1.smithi104.stdout:5/664: mkdir d0/d1/d64/d26/ddc 0 2024-04-26T04:42:20.114 INFO:tasks.workunit.client.1.smithi104.stdout:2/826: getdents d6/d8/d5a 0 2024-04-26T04:42:20.114 INFO:tasks.workunit.client.1.smithi104.stdout:2/827: write d6/d1b/d68/f95 [572226,115061] 0 2024-04-26T04:42:20.114 INFO:tasks.workunit.client.0.smithi090.stdout:6/414: rename d9/d1c/d31/d69/l97 to d9/d1c/d4d/l9c 0 2024-04-26T04:42:20.116 INFO:tasks.workunit.client.0.smithi090.stdout:5/454: mkdir d4/d1a/df/d36/d8e 0 2024-04-26T04:42:20.117 INFO:tasks.workunit.client.1.smithi104.stdout:3/714: creat d5/d9/d80/ffc x:0 0 0 2024-04-26T04:42:20.118 INFO:tasks.workunit.client.0.smithi090.stdout:3/216: link d1/d6/d1b/f35 d1/d17/d36/d2a/d3a/f4b 0 2024-04-26T04:42:20.119 INFO:tasks.workunit.client.1.smithi104.stdout:1/707: read d0/d6/d2e/d32/f92 [4248918,58045] 0 2024-04-26T04:42:20.120 INFO:tasks.workunit.client.1.smithi104.stdout:9/571: rmdir d11/d90/db2 0 2024-04-26T04:42:20.120 INFO:tasks.workunit.client.1.smithi104.stdout:9/572: truncate d11/d1d/d34/f6c 1019989 0 2024-04-26T04:42:20.121 INFO:tasks.workunit.client.0.smithi090.stdout:2/379: link db/dc/d6f/l7a db/dc/l82 0 2024-04-26T04:42:20.121 INFO:tasks.workunit.client.0.smithi090.stdout:2/380: dread - db/f76 zero size 2024-04-26T04:42:20.121 INFO:tasks.workunit.client.0.smithi090.stdout:0/445: dread d0/f4f [0,4194304] 0 2024-04-26T04:42:20.121 INFO:tasks.workunit.client.0.smithi090.stdout:5/455: dread d4/d1a/f43 [0,4194304] 0 2024-04-26T04:42:20.121 INFO:tasks.workunit.client.0.smithi090.stdout:5/456: stat d4/d1a/d1f/f3a 0 2024-04-26T04:42:20.122 INFO:tasks.workunit.client.1.smithi104.stdout:5/665: symlink d0/d1/d4/d2a/d2b/d37/d48/dc6/ldd 0 2024-04-26T04:42:20.122 INFO:tasks.workunit.client.1.smithi104.stdout:5/666: chown d0/d1/d4/d2a/d2b/d37/d48/f80 1118 1 2024-04-26T04:42:20.123 INFO:tasks.workunit.client.0.smithi090.stdout:2/381: creat db/d11/d14/d27/d2f/d70/d71/f83 x:0 0 0 2024-04-26T04:42:20.123 INFO:tasks.workunit.client.1.smithi104.stdout:3/715: dread d5/db2/da3/dd0/ddb/d69/fe7 [0,4194304] 0 2024-04-26T04:42:20.123 INFO:tasks.workunit.client.1.smithi104.stdout:3/716: creat d5/d9/d80/ffd x:0 0 0 2024-04-26T04:42:20.124 INFO:tasks.workunit.client.1.smithi104.stdout:1/708: mknod d0/d6/d19/d21/d28/ce8 0 2024-04-26T04:42:20.124 INFO:tasks.workunit.client.0.smithi090.stdout:0/446: symlink d0/d4b/d5d/d5c/d8a/d45/d53/l9e 0 2024-04-26T04:42:20.124 INFO:tasks.workunit.client.0.smithi090.stdout:0/447: dread - d0/d4b/d5d/f6a zero size 2024-04-26T04:42:20.127 INFO:tasks.workunit.client.0.smithi090.stdout:5/457: creat d4/d1a/d1f/d53/d59/f8f x:0 0 0 2024-04-26T04:42:20.127 INFO:tasks.workunit.client.0.smithi090.stdout:2/382: rmdir db/d11/d14/d63 39 2024-04-26T04:42:20.128 INFO:tasks.workunit.client.0.smithi090.stdout:2/383: creat db/f84 x:0 0 0 2024-04-26T04:42:20.128 INFO:tasks.workunit.client.1.smithi104.stdout:9/573: symlink d11/d15/d32/d4c/d8a/lbb 0 2024-04-26T04:42:20.129 INFO:tasks.workunit.client.0.smithi090.stdout:0/448: mknod d0/d4b/d5d/d5c/d8a/d45/d53/d8e/c9f 0 2024-04-26T04:42:20.131 INFO:tasks.workunit.client.0.smithi090.stdout:2/384: symlink db/d59/d66/l85 0 2024-04-26T04:42:20.131 INFO:tasks.workunit.client.1.smithi104.stdout:5/667: creat d0/d1/d64/fde x:0 0 0 2024-04-26T04:42:20.132 INFO:tasks.workunit.client.0.smithi090.stdout:0/449: mknod d0/d4b/d5d/d5c/d8a/d45/d93/ca0 0 2024-04-26T04:42:20.133 INFO:tasks.workunit.client.1.smithi104.stdout:1/709: dread d0/d6/d49/f4a [4194304,4194304] 0 2024-04-26T04:42:20.135 INFO:tasks.workunit.client.1.smithi104.stdout:3/717: mknod d5/db2/da3/dd0/cfe 0 2024-04-26T04:42:20.135 INFO:tasks.workunit.client.1.smithi104.stdout:7/626: getdents d2/d4/d32 0 2024-04-26T04:42:20.135 INFO:tasks.workunit.client.1.smithi104.stdout:7/627: readlink d2/d4/l25 0 2024-04-26T04:42:20.135 INFO:tasks.workunit.client.0.smithi090.stdout:0/450: creat d0/d4b/d5d/d5c/d8a/d45/d93/fa1 x:0 0 0 2024-04-26T04:42:20.135 INFO:tasks.workunit.client.0.smithi090.stdout:2/385: getdents db/d11/d14/d1b/d20 0 2024-04-26T04:42:20.136 INFO:tasks.workunit.client.0.smithi090.stdout:2/386: read db/d11/d14/d27/d28/f3d [3219294,67085] 0 2024-04-26T04:42:20.136 INFO:tasks.workunit.client.0.smithi090.stdout:9/229: dread db/d16/f33 [0,4194304] 0 2024-04-26T04:42:20.137 INFO:tasks.workunit.client.0.smithi090.stdout:0/451: symlink d0/d4b/d5d/d5c/d8a/d45/d93/la2 0 2024-04-26T04:42:20.137 INFO:tasks.workunit.client.0.smithi090.stdout:9/230: creat db/dd/d18/d31/d4c/f4d x:0 0 0 2024-04-26T04:42:20.138 INFO:tasks.workunit.client.1.smithi104.stdout:9/574: unlink d11/d1d/c2f 0 2024-04-26T04:42:20.138 INFO:tasks.workunit.client.1.smithi104.stdout:5/668: getdents d0/d7b 0 2024-04-26T04:42:20.139 INFO:tasks.workunit.client.0.smithi090.stdout:0/452: unlink d0/d4b/d6d/l79 0 2024-04-26T04:42:20.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/453: creat d0/d4b/d5d/d5c/d8a/d45/d53/fa3 x:0 0 0 2024-04-26T04:42:20.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/454: dread - d0/d4b/d5d/d5c/d8a/f94 zero size 2024-04-26T04:42:20.142 INFO:tasks.workunit.client.0.smithi090.stdout:0/455: dread - d0/d4b/d5d/d5c/d8a/d45/d53/fa3 zero size 2024-04-26T04:42:20.143 INFO:tasks.workunit.client.0.smithi090.stdout:0/456: write d0/d4b/d5d/d5c/d8a/d45/f9b [745554,110039] 0 2024-04-26T04:42:20.143 INFO:tasks.workunit.client.1.smithi104.stdout:3/718: symlink d5/d9/d80/lff 0 2024-04-26T04:42:20.143 INFO:tasks.workunit.client.1.smithi104.stdout:3/719: dread - d5/d9/d80/ffd zero size 2024-04-26T04:42:20.143 INFO:tasks.workunit.client.1.smithi104.stdout:3/720: creat d5/d9/d80/f100 x:0 0 0 2024-04-26T04:42:20.144 INFO:tasks.workunit.client.1.smithi104.stdout:3/721: write d5/db2/da3/dd0/ddb/d69/fe7 [4921856,72941] 0 2024-04-26T04:42:20.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/458: write d4/d1a/d1f/f2b [4376105,53780] 0 2024-04-26T04:42:20.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/457: mknod d0/d4b/d5d/d5c/d8a/d64/ca4 0 2024-04-26T04:42:20.151 INFO:tasks.workunit.client.0.smithi090.stdout:6/415: write d9/d21/d3b/f51 [3909269,109318] 0 2024-04-26T04:42:20.151 INFO:tasks.workunit.client.0.smithi090.stdout:6/416: creat d9/d77/f9d x:0 0 0 2024-04-26T04:42:20.151 INFO:tasks.workunit.client.0.smithi090.stdout:6/417: fdatasync d9/d1c/d4d/d6d/d92/f25 0 2024-04-26T04:42:20.155 INFO:tasks.workunit.client.0.smithi090.stdout:9/231: dread db/dd/ff [0,4194304] 0 2024-04-26T04:42:20.155 INFO:tasks.workunit.client.0.smithi090.stdout:9/232: creat db/dd/d18/d31/f4e x:0 0 0 2024-04-26T04:42:20.155 INFO:tasks.workunit.client.1.smithi104.stdout:0/580: dwrite d0/d2/d10/d3d/f9e [0,4194304] 0 2024-04-26T04:42:20.156 INFO:tasks.workunit.client.0.smithi090.stdout:6/418: unlink d9/d21/d40/c72 0 2024-04-26T04:42:20.160 INFO:tasks.workunit.client.0.smithi090.stdout:9/233: link db/f41 db/dd/d18/d31/d4c/f4f 0 2024-04-26T04:42:20.161 INFO:tasks.workunit.client.1.smithi104.stdout:2/828: dwrite d6/d1b/d75/f103 [0,4194304] 0 2024-04-26T04:42:20.161 INFO:tasks.workunit.client.0.smithi090.stdout:3/217: dwrite d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:20.161 INFO:tasks.workunit.client.0.smithi090.stdout:4/336: dwrite f0 [0,4194304] 0 2024-04-26T04:42:20.163 INFO:tasks.workunit.client.0.smithi090.stdout:3/218: mknod d1/d6/c4c 0 2024-04-26T04:42:20.163 INFO:tasks.workunit.client.0.smithi090.stdout:1/327: dwrite d0/d4/d43/f54 [0,4194304] 0 2024-04-26T04:42:20.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/387: dread db/f10 [0,4194304] 0 2024-04-26T04:42:20.163 INFO:tasks.workunit.client.0.smithi090.stdout:1/328: stat d0/d4/d7/d17/f29 0 2024-04-26T04:42:20.164 INFO:tasks.workunit.client.0.smithi090.stdout:1/329: write d0/d4/d7/f33 [809115,119093] 0 2024-04-26T04:42:20.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/376: dwrite d0/d1/f52 [0,4194304] 0 2024-04-26T04:42:20.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/337: getdents d3/db/d12/d20/d2d/d2e 0 2024-04-26T04:42:20.165 INFO:tasks.workunit.client.1.smithi104.stdout:0/581: mknod d0/d2/d21/d49/d5e/d9c/d53/d5d/d6f/cb4 0 2024-04-26T04:42:20.172 INFO:tasks.workunit.client.0.smithi090.stdout:2/388: unlink db/d11/d14/d25/d2d/l72 0 2024-04-26T04:42:20.175 INFO:tasks.workunit.client.1.smithi104.stdout:2/829: mknod d6/d8/d5a/c10e 0 2024-04-26T04:42:20.175 INFO:tasks.workunit.client.1.smithi104.stdout:2/830: fsync d6/d8/fea 0 2024-04-26T04:42:20.176 INFO:tasks.workunit.client.0.smithi090.stdout:9/234: dread db/d16/f33 [0,4194304] 0 2024-04-26T04:42:20.176 INFO:tasks.workunit.client.0.smithi090.stdout:9/235: write db/dd/d18/f43 [754042,106766] 0 2024-04-26T04:42:20.177 INFO:tasks.workunit.client.1.smithi104.stdout:2/831: creat d6/d8/df6/dfe/f10f x:0 0 0 2024-04-26T04:42:20.177 INFO:tasks.workunit.client.1.smithi104.stdout:2/832: chown d6/d8/d5a 1570 1 2024-04-26T04:42:20.178 INFO:tasks.workunit.client.0.smithi090.stdout:1/330: symlink d0/d4/d7/d4b/d53/l65 0 2024-04-26T04:42:20.181 INFO:tasks.workunit.client.0.smithi090.stdout:4/338: dread d3/db/f19 [0,4194304] 0 2024-04-26T04:42:20.181 INFO:tasks.workunit.client.0.smithi090.stdout:9/236: rename db/dd/d44/c2f to db/c50 0 2024-04-26T04:42:20.183 INFO:tasks.workunit.client.0.smithi090.stdout:1/331: truncate d0/d4/f26 418108 0 2024-04-26T04:42:20.187 INFO:tasks.workunit.client.0.smithi090.stdout:1/332: creat d0/d4/d7/d17/d22/f66 x:0 0 0 2024-04-26T04:42:20.188 INFO:tasks.workunit.client.0.smithi090.stdout:9/237: getdents db/d16 0 2024-04-26T04:42:20.189 INFO:tasks.workunit.client.1.smithi104.stdout:7/628: dwrite d2/d4/d1d/f90 [0,4194304] 0 2024-04-26T04:42:20.190 INFO:tasks.workunit.client.1.smithi104.stdout:7/629: creat d2/d4/d32/d35/d44/d5b/fcd x:0 0 0 2024-04-26T04:42:20.193 INFO:tasks.workunit.client.0.smithi090.stdout:9/238: link db/dd/d18/c2e db/dd/d18/c51 0 2024-04-26T04:42:20.201 INFO:tasks.workunit.client.0.smithi090.stdout:4/339: write d3/db/d12/d20/d2d/d58/f63 [3772958,19849] 0 2024-04-26T04:42:20.202 INFO:tasks.workunit.client.0.smithi090.stdout:4/340: dread d3/d13/f21 [0,4194304] 0 2024-04-26T04:42:20.203 INFO:tasks.workunit.client.0.smithi090.stdout:4/341: getdents d3/d13/d1d/d50 0 2024-04-26T04:42:20.205 INFO:tasks.workunit.client.0.smithi090.stdout:3/219: dread d1/d6/f14 [0,4194304] 0 2024-04-26T04:42:20.208 INFO:tasks.workunit.client.0.smithi090.stdout:3/220: rename d1/d6/d2c/l3b to d1/d6/d1b/l4d 0 2024-04-26T04:42:20.208 INFO:tasks.workunit.client.0.smithi090.stdout:3/221: chown d1/f8 13944 1 2024-04-26T04:42:20.208 INFO:tasks.workunit.client.0.smithi090.stdout:3/222: write d1/d17/f3e [86268,124259] 0 2024-04-26T04:42:20.208 INFO:tasks.workunit.client.0.smithi090.stdout:3/223: fdatasync d1/d6/f14 0 2024-04-26T04:42:20.208 INFO:tasks.workunit.client.0.smithi090.stdout:3/224: fsync d1/d17/d36/f32 0 2024-04-26T04:42:20.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/225: symlink d1/d17/d36/d2a/l4e 0 2024-04-26T04:42:20.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/226: creat d1/d17/f4f x:0 0 0 2024-04-26T04:42:20.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/227: readlink d1/d6/d1b/l30 0 2024-04-26T04:42:20.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/228: chown d1/d17/d36/f40 0 1 2024-04-26T04:42:20.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/229: chown d1/d17/d36/d33/l41 284 1 2024-04-26T04:42:20.210 INFO:tasks.workunit.client.0.smithi090.stdout:3/230: write d1/d9/fa [3474098,89108] 0 2024-04-26T04:42:20.212 INFO:tasks.workunit.client.0.smithi090.stdout:3/231: mknod d1/d17/d36/d33/c50 0 2024-04-26T04:42:20.216 INFO:tasks.workunit.client.0.smithi090.stdout:9/239: dread db/dd/d44/f34 [0,4194304] 0 2024-04-26T04:42:20.216 INFO:tasks.workunit.client.1.smithi104.stdout:7/630: write d2/d4/f13 [821584,39144] 0 2024-04-26T04:42:20.218 INFO:tasks.workunit.client.1.smithi104.stdout:7/631: mkdir d2/d4/d32/d3d/d47/db6/dce 0 2024-04-26T04:42:20.219 INFO:tasks.workunit.client.1.smithi104.stdout:5/669: dwrite d0/d1/d4/f63 [4194304,4194304] 0 2024-04-26T04:42:20.219 INFO:tasks.workunit.client.1.smithi104.stdout:5/670: chown d0/d41/f74 6 1 2024-04-26T04:42:20.220 INFO:tasks.workunit.client.1.smithi104.stdout:5/671: symlink d0/d1/d4/d2a/d6b/d73/ldf 0 2024-04-26T04:42:20.221 INFO:tasks.workunit.client.1.smithi104.stdout:9/575: dwrite d11/d15/d32/d35/f3a [0,4194304] 0 2024-04-26T04:42:20.222 INFO:tasks.workunit.client.1.smithi104.stdout:3/722: dwrite d5/db2/fed [0,4194304] 0 2024-04-26T04:42:20.223 INFO:tasks.workunit.client.1.smithi104.stdout:9/576: symlink d11/d15/d32/d4c/d54/lbc 0 2024-04-26T04:42:20.224 INFO:tasks.workunit.client.1.smithi104.stdout:9/577: write d11/f17 [1491625,97513] 0 2024-04-26T04:42:20.225 INFO:tasks.workunit.client.0.smithi090.stdout:3/232: dread d1/d9/fb [4194304,4194304] 0 2024-04-26T04:42:20.226 INFO:tasks.workunit.client.1.smithi104.stdout:9/578: creat d11/d1d/fbd x:0 0 0 2024-04-26T04:42:20.228 INFO:tasks.workunit.client.0.smithi090.stdout:6/419: dwrite d9/d1c/f47 [4194304,4194304] 0 2024-04-26T04:42:20.228 INFO:tasks.workunit.client.0.smithi090.stdout:0/458: dwrite d0/d4b/f52 [0,4194304] 0 2024-04-26T04:42:20.228 INFO:tasks.workunit.client.0.smithi090.stdout:7/377: dwrite d0/db/d15/f57 [0,4194304] 0 2024-04-26T04:42:20.228 INFO:tasks.workunit.client.1.smithi104.stdout:2/833: dwrite d6/d1b/f71 [0,4194304] 0 2024-04-26T04:42:20.229 INFO:tasks.workunit.client.1.smithi104.stdout:2/834: creat d6/d1b/d68/f110 x:0 0 0 2024-04-26T04:42:20.229 INFO:tasks.workunit.client.1.smithi104.stdout:2/835: creat d6/de/f111 x:0 0 0 2024-04-26T04:42:20.229 INFO:tasks.workunit.client.1.smithi104.stdout:9/579: mkdir d11/d15/d1b/d6d/d9c/dbe 0 2024-04-26T04:42:20.234 INFO:tasks.workunit.client.1.smithi104.stdout:2/836: symlink d6/d8/l112 0 2024-04-26T04:42:20.234 INFO:tasks.workunit.client.1.smithi104.stdout:2/837: chown d6/d1b/d93/faa 10 1 2024-04-26T04:42:20.236 INFO:tasks.workunit.client.1.smithi104.stdout:5/672: dread d0/d1/d64/d26/d36/da7/fa8 [0,4194304] 0 2024-04-26T04:42:20.238 INFO:tasks.workunit.client.1.smithi104.stdout:2/838: unlink d6/d1b/d51/d6a/ldb 0 2024-04-26T04:42:20.238 INFO:tasks.workunit.client.1.smithi104.stdout:2/839: write d6/d1b/fe4 [1021030,85719] 0 2024-04-26T04:42:20.238 INFO:tasks.workunit.client.1.smithi104.stdout:2/840: readlink d6/d1b/d68/d81/dce/lf1 0 2024-04-26T04:42:20.238 INFO:tasks.workunit.client.0.smithi090.stdout:2/389: dwrite db/d11/d14/d25/f3a [0,4194304] 0 2024-04-26T04:42:20.238 INFO:tasks.workunit.client.0.smithi090.stdout:4/342: dwrite f0 [0,4194304] 0 2024-04-26T04:42:20.239 INFO:tasks.workunit.client.0.smithi090.stdout:4/343: write d3/d13/d1d/f38 [142346,55312] 0 2024-04-26T04:42:20.239 INFO:tasks.workunit.client.0.smithi090.stdout:7/378: creat d0/db/d53/f87 x:0 0 0 2024-04-26T04:42:20.239 INFO:tasks.workunit.client.1.smithi104.stdout:9/580: rmdir d11/d15/d32/d35/d7b 39 2024-04-26T04:42:20.240 INFO:tasks.workunit.client.1.smithi104.stdout:3/723: dread d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/fd3 [0,4194304] 0 2024-04-26T04:42:20.241 INFO:tasks.workunit.client.1.smithi104.stdout:5/673: mknod d0/d1/d4/d2a/d2b/d37/d48/ce0 0 2024-04-26T04:42:20.241 INFO:tasks.workunit.client.0.smithi090.stdout:0/459: getdents d0/d4b/d5d/d5c/d8a/d45/d93 0 2024-04-26T04:42:20.241 INFO:tasks.workunit.client.0.smithi090.stdout:0/460: creat d0/d4b/d5d/d48/fa5 x:0 0 0 2024-04-26T04:42:20.243 INFO:tasks.workunit.client.1.smithi104.stdout:2/841: unlink d6/d8/lae 0 2024-04-26T04:42:20.243 INFO:tasks.workunit.client.1.smithi104.stdout:2/842: write d6/d1b/d68/d81/d97/f9a [1012854,35720] 0 2024-04-26T04:42:20.243 INFO:tasks.workunit.client.1.smithi104.stdout:2/843: stat d6/d1b/fe4 0 2024-04-26T04:42:20.245 INFO:tasks.workunit.client.1.smithi104.stdout:9/581: readlink d11/d15/l51 0 2024-04-26T04:42:20.246 INFO:tasks.workunit.client.1.smithi104.stdout:3/724: rename d5/db2/da3/dd0/ddb/f16 to d5/d9/de4/f101 0 2024-04-26T04:42:20.247 INFO:tasks.workunit.client.0.smithi090.stdout:2/390: mkdir db/d11/d14/d25/d86 0 2024-04-26T04:42:20.248 INFO:tasks.workunit.client.1.smithi104.stdout:5/674: truncate d0/d1/d4/d2a/d2b/fb7 163407 0 2024-04-26T04:42:20.250 INFO:tasks.workunit.client.0.smithi090.stdout:4/344: rmdir d3/db/d12/d20/d24/d30 39 2024-04-26T04:42:20.251 INFO:tasks.workunit.client.0.smithi090.stdout:7/379: mknod d0/d1/d4/d43/c88 0 2024-04-26T04:42:20.253 INFO:tasks.workunit.client.1.smithi104.stdout:5/675: creat d0/d1/d64/d26/dd7/fe1 x:0 0 0 2024-04-26T04:42:20.253 INFO:tasks.workunit.client.1.smithi104.stdout:5/676: chown d0/d1/d4/c7e 47701 1 2024-04-26T04:42:20.253 INFO:tasks.workunit.client.1.smithi104.stdout:5/677: write d0/d1/d64/d26/d36/d72/fa3 [614387,73987] 0 2024-04-26T04:42:20.255 INFO:tasks.workunit.client.1.smithi104.stdout:5/678: dread - d0/d1/d4/d2a/d2b/d37/f7c zero size 2024-04-26T04:42:20.255 INFO:tasks.workunit.client.1.smithi104.stdout:7/632: dwrite d2/d4/d32/d3d/d41/d5f/d6d/fa3 [0,4194304] 0 2024-04-26T04:42:20.255 INFO:tasks.workunit.client.1.smithi104.stdout:7/633: read - d2/d4/d32/d3d/d47/db1/fc3 zero size 2024-04-26T04:42:20.255 INFO:tasks.workunit.client.1.smithi104.stdout:7/634: chown d2/d4/d5a/f68 53 1 2024-04-26T04:42:20.255 INFO:tasks.workunit.client.1.smithi104.stdout:7/635: chown d2/d4/d1d/d34/f40 11 1 2024-04-26T04:42:20.255 INFO:tasks.workunit.client.1.smithi104.stdout:7/636: fsync d2/d4/d32/d35/d37/f6f 0 2024-04-26T04:42:20.255 INFO:tasks.workunit.client.0.smithi090.stdout:7/380: rename d0/d1/l78 to d0/db/df/d4d/d64/l89 0 2024-04-26T04:42:20.258 INFO:tasks.workunit.client.1.smithi104.stdout:7/637: creat d2/d4/d32/fcf x:0 0 0 2024-04-26T04:42:20.259 INFO:tasks.workunit.client.1.smithi104.stdout:7/638: chown d2/d4/d32/d5e 1194777 1 2024-04-26T04:42:20.261 INFO:tasks.workunit.client.0.smithi090.stdout:2/391: write db/d11/d14/d27/d2f/f32 [2363403,95176] 0 2024-04-26T04:42:20.261 INFO:tasks.workunit.client.0.smithi090.stdout:2/392: readlink db/d11/d14/d27/d2f/d42/d5c/l55 0 2024-04-26T04:42:20.261 INFO:tasks.workunit.client.0.smithi090.stdout:9/240: dwrite db/dd/d44/f27 [0,4194304] 0 2024-04-26T04:42:20.264 INFO:tasks.workunit.client.0.smithi090.stdout:1/333: rmdir d0/d4 39 2024-04-26T04:42:20.266 INFO:tasks.workunit.client.0.smithi090.stdout:2/393: dread db/d11/d14/d25/d41/f54 [0,4194304] 0 2024-04-26T04:42:20.271 INFO:tasks.workunit.client.1.smithi104.stdout:8/600: sync 2024-04-26T04:42:20.273 INFO:tasks.workunit.client.1.smithi104.stdout:8/601: readlink d0/l4 0 2024-04-26T04:42:20.278 INFO:tasks.workunit.client.1.smithi104.stdout:7/639: write d2/d4/d32/d35/f87 [599903,71308] 0 2024-04-26T04:42:20.279 INFO:tasks.workunit.client.1.smithi104.stdout:7/640: write d2/d4/d1d/fa7 [642115,116539] 0 2024-04-26T04:42:20.280 INFO:tasks.workunit.client.0.smithi090.stdout:1/334: write d0/d4/d7/f2c [3085623,106837] 0 2024-04-26T04:42:20.280 INFO:tasks.workunit.client.1.smithi104.stdout:7/641: creat d2/d4/d32/d3d/d41/d5f/d6c/d89/fd0 x:0 0 0 2024-04-26T04:42:20.281 INFO:tasks.workunit.client.0.smithi090.stdout:3/233: dwrite d1/d17/f2f [0,4194304] 0 2024-04-26T04:42:20.281 INFO:tasks.workunit.client.1.smithi104.stdout:7/642: getdents d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d 0 2024-04-26T04:42:20.283 INFO:tasks.workunit.client.1.smithi104.stdout:7/643: symlink d2/d4/d32/d3d/d47/db1/ld1 0 2024-04-26T04:42:20.285 INFO:tasks.workunit.client.1.smithi104.stdout:7/644: creat d2/d4/d32/d3d/d41/fd2 x:0 0 0 2024-04-26T04:42:20.285 INFO:tasks.workunit.client.0.smithi090.stdout:1/335: rmdir d0 39 2024-04-26T04:42:20.285 INFO:tasks.workunit.client.0.smithi090.stdout:3/234: link d1/d17/c18 d1/d17/d36/d33/d43/c51 0 2024-04-26T04:42:20.287 INFO:tasks.workunit.client.0.smithi090.stdout:3/235: mkdir d1/d17/d52 0 2024-04-26T04:42:20.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/236: creat d1/d17/d36/d2a/d3a/f53 x:0 0 0 2024-04-26T04:42:20.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/237: rename d1/d17/d36 to d1/d17/d36/d2a/d3a/d54 22 2024-04-26T04:42:20.289 INFO:tasks.workunit.client.1.smithi104.stdout:8/602: read d0/d33/fab [488992,26036] 0 2024-04-26T04:42:20.289 INFO:tasks.workunit.client.0.smithi090.stdout:3/238: rmdir d1/d17/d36 39 2024-04-26T04:42:20.290 INFO:tasks.workunit.client.0.smithi090.stdout:3/239: truncate d1/d17/f25 4172155 0 2024-04-26T04:42:20.301 INFO:tasks.workunit.client.1.smithi104.stdout:9/582: dwrite d11/d15/d32/d35/f3a [4194304,4194304] 0 2024-04-26T04:42:20.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/420: dwrite d9/d1c/d4d/d6d/f98 [0,4194304] 0 2024-04-26T04:42:20.302 INFO:tasks.workunit.client.1.smithi104.stdout:2/844: dwrite d6/d8/d5a/f9c [0,4194304] 0 2024-04-26T04:42:20.302 INFO:tasks.workunit.client.0.smithi090.stdout:3/240: dread d1/d17/f2f [0,4194304] 0 2024-04-26T04:42:20.303 INFO:tasks.workunit.client.0.smithi090.stdout:3/241: chown d1/d17/d36/d2a/l2d 1 1 2024-04-26T04:42:20.303 INFO:tasks.workunit.client.0.smithi090.stdout:3/242: dread - d1/d17/f4f zero size 2024-04-26T04:42:20.306 INFO:tasks.workunit.client.1.smithi104.stdout:9/583: dread f10 [0,4194304] 0 2024-04-26T04:42:20.307 INFO:tasks.workunit.client.1.smithi104.stdout:2/845: dread d6/d1b/d68/f95 [0,4194304] 0 2024-04-26T04:42:20.307 INFO:tasks.workunit.client.1.smithi104.stdout:2/846: creat d6/de/f113 x:0 0 0 2024-04-26T04:42:20.307 INFO:tasks.workunit.client.1.smithi104.stdout:9/584: symlink d11/d1d/d34/db3/lbf 0 2024-04-26T04:42:20.309 INFO:tasks.workunit.client.1.smithi104.stdout:2/847: rmdir d6/d1b/d68/d81/dfc 39 2024-04-26T04:42:20.310 INFO:tasks.workunit.client.0.smithi090.stdout:6/421: getdents d9/d1c/d31 0 2024-04-26T04:42:20.310 INFO:tasks.workunit.client.0.smithi090.stdout:6/422: fsync d9/d21/f9b 0 2024-04-26T04:42:20.310 INFO:tasks.workunit.client.1.smithi104.stdout:2/848: rmdir d6/d1b/d93 39 2024-04-26T04:42:20.310 INFO:tasks.workunit.client.1.smithi104.stdout:3/725: dwrite d5/f23 [0,4194304] 0 2024-04-26T04:42:20.311 INFO:tasks.workunit.client.1.smithi104.stdout:8/603: dwrite d0/d1c/d27/d47/d67/d76/daf/fb8 [0,4194304] 0 2024-04-26T04:42:20.311 INFO:tasks.workunit.client.0.smithi090.stdout:9/241: dwrite db/f3d [0,4194304] 0 2024-04-26T04:42:20.311 INFO:tasks.workunit.client.0.smithi090.stdout:9/242: fdatasync db/f1b 0 2024-04-26T04:42:20.311 INFO:tasks.workunit.client.0.smithi090.stdout:9/243: creat db/dd/d18/d31/f52 x:0 0 0 2024-04-26T04:42:20.312 INFO:tasks.workunit.client.1.smithi104.stdout:2/849: mknod d6/d1b/d68/d81/c114 0 2024-04-26T04:42:20.312 INFO:tasks.workunit.client.0.smithi090.stdout:7/381: dwrite d0/d1/d4/f2e [4194304,4194304] 0 2024-04-26T04:42:20.313 INFO:tasks.workunit.client.0.smithi090.stdout:2/394: dwrite db/d11/d14/d27/d28/f6e [0,4194304] 0 2024-04-26T04:42:20.313 INFO:tasks.workunit.client.0.smithi090.stdout:3/243: write d1/d9/f12 [5571462,107572] 0 2024-04-26T04:42:20.314 INFO:tasks.workunit.client.1.smithi104.stdout:7/645: dwrite d2/d4/d32/fcc [0,4194304] 0 2024-04-26T04:42:20.314 INFO:tasks.workunit.client.1.smithi104.stdout:7/646: readlink d2/d4/d32/d3d/l92 0 2024-04-26T04:42:20.314 INFO:tasks.workunit.client.1.smithi104.stdout:7/647: fsync d2/d4/d1d/d34/fc6 0 2024-04-26T04:42:20.314 INFO:tasks.workunit.client.1.smithi104.stdout:7/648: dread - d2/d4/d32/fcf zero size 2024-04-26T04:42:20.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/423: rename f1 to d9/d1c/d4d/d6d/d66/f9e 0 2024-04-26T04:42:20.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/424: chown d9/d21/d3b 3733 1 2024-04-26T04:42:20.318 INFO:tasks.workunit.client.1.smithi104.stdout:3/726: unlink d5/db2/l27 0 2024-04-26T04:42:20.319 INFO:tasks.workunit.client.0.smithi090.stdout:7/382: symlink d0/db/d15/d29/l8a 0 2024-04-26T04:42:20.321 INFO:tasks.workunit.client.1.smithi104.stdout:8/604: truncate d0/d1c/d2d/f32 4058647 0 2024-04-26T04:42:20.322 INFO:tasks.workunit.client.1.smithi104.stdout:2/850: mknod d6/d8/d5a/c115 0 2024-04-26T04:42:20.323 INFO:tasks.workunit.client.0.smithi090.stdout:2/395: rmdir db/d11/d14/d25/d2d/d48 39 2024-04-26T04:42:20.325 INFO:tasks.workunit.client.0.smithi090.stdout:6/425: mkdir d9/d21/d40/d4c/d6f/d9f 0 2024-04-26T04:42:20.325 INFO:tasks.workunit.client.0.smithi090.stdout:6/426: stat d9/d21/d3b/d86 0 2024-04-26T04:42:20.326 INFO:tasks.workunit.client.1.smithi104.stdout:7/649: mkdir d2/d4/d32/d35/d44/d5b/db2/dd3 0 2024-04-26T04:42:20.326 INFO:tasks.workunit.client.1.smithi104.stdout:7/650: chown d2/d4/d32/d3d/d41/d5f/d6d/fa3 99058511 1 2024-04-26T04:42:20.326 INFO:tasks.workunit.client.1.smithi104.stdout:7/651: fdatasync d2/d4/d1d/d34/d7f/faf 0 2024-04-26T04:42:20.326 INFO:tasks.workunit.client.1.smithi104.stdout:7/652: creat d2/d4/d32/d35/d44/d5b/fd4 x:0 0 0 2024-04-26T04:42:20.326 INFO:tasks.workunit.client.1.smithi104.stdout:7/653: chown d2/d4/d32/d35/f87 5 1 2024-04-26T04:42:20.326 INFO:tasks.workunit.client.1.smithi104.stdout:7/654: readlink d2/la 0 2024-04-26T04:42:20.326 INFO:tasks.workunit.client.1.smithi104.stdout:3/727: mknod d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/c102 0 2024-04-26T04:42:20.327 INFO:tasks.workunit.client.1.smithi104.stdout:2/851: symlink d6/d1b/d68/d81/dfc/l116 0 2024-04-26T04:42:20.327 INFO:tasks.workunit.client.0.smithi090.stdout:7/383: creat d0/db/d53/d5e/f8b x:0 0 0 2024-04-26T04:42:20.327 INFO:tasks.workunit.client.0.smithi090.stdout:6/427: mknod d9/d1c/d4d/d6d/d66/ca0 0 2024-04-26T04:42:20.328 INFO:tasks.workunit.client.0.smithi090.stdout:7/384: creat d0/d1/d4/d43/d4a/f8c x:0 0 0 2024-04-26T04:42:20.329 INFO:tasks.workunit.client.1.smithi104.stdout:7/655: creat d2/d4/d1d/d34/db0/fd5 x:0 0 0 2024-04-26T04:42:20.331 INFO:tasks.workunit.client.1.smithi104.stdout:3/728: unlink d5/c51 0 2024-04-26T04:42:20.331 INFO:tasks.workunit.client.1.smithi104.stdout:7/656: chown d2/d4/c12 4013 1 2024-04-26T04:42:20.332 INFO:tasks.workunit.client.0.smithi090.stdout:6/428: unlink d9/d1c/d4d/f8e 0 2024-04-26T04:42:20.333 INFO:tasks.workunit.client.1.smithi104.stdout:7/657: symlink d2/d4/d5a/ld6 0 2024-04-26T04:42:20.334 INFO:tasks.workunit.client.1.smithi104.stdout:3/729: rmdir d5/db2/da3/dd0/ddb/d32/dd9 39 2024-04-26T04:42:20.335 INFO:tasks.workunit.client.1.smithi104.stdout:8/605: write d0/d1c/d27/d47/d67/d76/daf/fb8 [1142879,41383] 0 2024-04-26T04:42:20.336 INFO:tasks.workunit.client.1.smithi104.stdout:7/658: creat d2/d4/d32/d3d/d41/d7e/fd7 x:0 0 0 2024-04-26T04:42:20.336 INFO:tasks.workunit.client.1.smithi104.stdout:7/659: chown d2/d4/d32/d3d/d41/dbd 3 1 2024-04-26T04:42:20.336 INFO:tasks.workunit.client.1.smithi104.stdout:3/730: mknod d5/db2/d46/db7/de6/c103 0 2024-04-26T04:42:20.339 INFO:tasks.workunit.client.1.smithi104.stdout:8/606: truncate d0/d50/f55 535965 0 2024-04-26T04:42:20.339 INFO:tasks.workunit.client.1.smithi104.stdout:8/607: write d0/d1c/f24 [3856152,40961] 0 2024-04-26T04:42:20.340 INFO:tasks.workunit.client.1.smithi104.stdout:8/608: write d0/d33/d35/d6d/f74 [228798,55905] 0 2024-04-26T04:42:20.348 INFO:tasks.workunit.client.0.smithi090.stdout:7/385: write d0/db/df/f70 [2901450,70315] 0 2024-04-26T04:42:20.351 INFO:tasks.workunit.client.1.smithi104.stdout:7/660: write d2/d4/d32/d60/f82 [3998587,67652] 0 2024-04-26T04:42:20.351 INFO:tasks.workunit.client.1.smithi104.stdout:7/661: truncate d2/d4/d32/d60/f8c 819511 0 2024-04-26T04:42:20.352 INFO:tasks.workunit.client.0.smithi090.stdout:7/386: mkdir d0/db/df/d1c/d23/d8d 0 2024-04-26T04:42:20.353 INFO:tasks.workunit.client.0.smithi090.stdout:3/244: dread d1/d9/f12 [0,4194304] 0 2024-04-26T04:42:20.353 INFO:tasks.workunit.client.0.smithi090.stdout:3/245: chown d1/d6/d1b 20838 1 2024-04-26T04:42:20.353 INFO:tasks.workunit.client.0.smithi090.stdout:7/387: mkdir d0/db/d15/d30/d8e 0 2024-04-26T04:42:20.355 INFO:tasks.workunit.client.0.smithi090.stdout:3/246: symlink d1/d17/d36/l55 0 2024-04-26T04:42:20.356 INFO:tasks.workunit.client.1.smithi104.stdout:8/609: dread d0/f2f [4194304,4194304] 0 2024-04-26T04:42:20.356 INFO:tasks.workunit.client.1.smithi104.stdout:7/662: unlink d2/d4/d32/d35/f87 0 2024-04-26T04:42:20.356 INFO:tasks.workunit.client.1.smithi104.stdout:8/610: creat d0/d1c/d27/d47/d67/d76/fbf x:0 0 0 2024-04-26T04:42:20.356 INFO:tasks.workunit.client.1.smithi104.stdout:8/611: read - d0/d1c/d27/f90 zero size 2024-04-26T04:42:20.356 INFO:tasks.workunit.client.1.smithi104.stdout:8/612: fdatasync d0/d33/da6/fa8 0 2024-04-26T04:42:20.357 INFO:tasks.workunit.client.1.smithi104.stdout:8/613: truncate d0/d1c/d2d/f6a 741468 0 2024-04-26T04:42:20.357 INFO:tasks.workunit.client.0.smithi090.stdout:7/388: mknod d0/c8f 0 2024-04-26T04:42:20.361 INFO:tasks.workunit.client.1.smithi104.stdout:5/679: dwrite d0/d1/d4/d2a/d2b/fb7 [0,4194304] 0 2024-04-26T04:42:20.362 INFO:tasks.workunit.client.1.smithi104.stdout:9/585: dwrite d11/d15/d1b/f60 [4194304,4194304] 0 2024-04-26T04:42:20.362 INFO:tasks.workunit.client.0.smithi090.stdout:9/244: dwrite db/dd/d18/d31/d4c/f4d [0,4194304] 0 2024-04-26T04:42:20.364 INFO:tasks.workunit.client.0.smithi090.stdout:9/245: mknod db/dd/d18/c53 0 2024-04-26T04:42:20.365 INFO:tasks.workunit.client.0.smithi090.stdout:2/396: dwrite db/f10 [0,4194304] 0 2024-04-26T04:42:20.366 INFO:tasks.workunit.client.1.smithi104.stdout:7/663: write d2/d4/d32/d3d/d41/d5f/f8d [1594075,59143] 0 2024-04-26T04:42:20.367 INFO:tasks.workunit.client.1.smithi104.stdout:7/664: write d2/d4/d32/d3d/d41/d5f/d6d/faa [988273,54577] 0 2024-04-26T04:42:20.367 INFO:tasks.workunit.client.1.smithi104.stdout:7/665: write d2/d4/d32/d3d/d41/d5f/d6c/f98 [92485,106169] 0 2024-04-26T04:42:20.367 INFO:tasks.workunit.client.0.smithi090.stdout:6/429: dwrite d9/d1c/d4d/d6d/d92/f29 [0,4194304] 0 2024-04-26T04:42:20.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/246: symlink db/dd/d18/d38/d39/l54 0 2024-04-26T04:42:20.368 INFO:tasks.workunit.client.1.smithi104.stdout:9/586: creat d11/d15/d1b/d6d/fc0 x:0 0 0 2024-04-26T04:42:20.368 INFO:tasks.workunit.client.1.smithi104.stdout:9/587: fsync d11/d1d/f49 0 2024-04-26T04:42:20.368 INFO:tasks.workunit.client.1.smithi104.stdout:9/588: truncate d11/d15/d32/d35/d7b/fae 723011 0 2024-04-26T04:42:20.369 INFO:tasks.workunit.client.0.smithi090.stdout:9/247: mknod db/d16/c55 0 2024-04-26T04:42:20.369 INFO:tasks.workunit.client.0.smithi090.stdout:9/248: fsync db/dd/d18/d31/d4c/f4f 0 2024-04-26T04:42:20.370 INFO:tasks.workunit.client.0.smithi090.stdout:9/249: creat db/dd/d44/f56 x:0 0 0 2024-04-26T04:42:20.371 INFO:tasks.workunit.client.1.smithi104.stdout:5/680: rmdir d0/d1/d4/d2a/d2b/d5c 39 2024-04-26T04:42:20.373 INFO:tasks.workunit.client.0.smithi090.stdout:7/389: write d0/db/d53/f2d [325916,11398] 0 2024-04-26T04:42:20.374 INFO:tasks.workunit.client.1.smithi104.stdout:9/589: rename d11/d1d/l1e to d11/d1d/d87/lc1 0 2024-04-26T04:42:20.374 INFO:tasks.workunit.client.1.smithi104.stdout:3/731: dwrite d5/f22 [0,4194304] 0 2024-04-26T04:42:20.374 INFO:tasks.workunit.client.1.smithi104.stdout:9/590: chown d11/d15/d32/d35/l8b 0 1 2024-04-26T04:42:20.374 INFO:tasks.workunit.client.1.smithi104.stdout:9/591: fdatasync d11/d53/fb6 0 2024-04-26T04:42:20.376 INFO:tasks.workunit.client.0.smithi090.stdout:9/250: rename db/dd/d18/d31/d4c to db/dd/d18/d57 0 2024-04-26T04:42:20.376 INFO:tasks.workunit.client.1.smithi104.stdout:8/614: dread d0/d1c/d2d/f8c [0,4194304] 0 2024-04-26T04:42:20.377 INFO:tasks.workunit.client.1.smithi104.stdout:8/615: fdatasync d0/d33/d51/fa2 0 2024-04-26T04:42:20.377 INFO:tasks.workunit.client.1.smithi104.stdout:9/592: truncate d11/d1d/d34/f6c 542371 0 2024-04-26T04:42:20.378 INFO:tasks.workunit.client.0.smithi090.stdout:9/251: mkdir db/dd/d18/d31/d58 0 2024-04-26T04:42:20.385 INFO:tasks.workunit.client.1.smithi104.stdout:5/681: dread d0/f66 [0,4194304] 0 2024-04-26T04:42:20.386 INFO:tasks.workunit.client.1.smithi104.stdout:8/616: getdents d0/d1c/d2d 0 2024-04-26T04:42:20.387 INFO:tasks.workunit.client.0.smithi090.stdout:3/247: dwrite d1/f3c [0,4194304] 0 2024-04-26T04:42:20.389 INFO:tasks.workunit.client.1.smithi104.stdout:9/593: symlink d11/d1d/d71/d94/lc2 0 2024-04-26T04:42:20.389 INFO:tasks.workunit.client.1.smithi104.stdout:9/594: stat d11/d1d/d34/c45 0 2024-04-26T04:42:20.389 INFO:tasks.workunit.client.1.smithi104.stdout:9/595: truncate f10 828417 0 2024-04-26T04:42:20.389 INFO:tasks.workunit.client.1.smithi104.stdout:9/596: chown d11/d15/d32/d35/d7b 32154 1 2024-04-26T04:42:20.389 INFO:tasks.workunit.client.1.smithi104.stdout:9/597: chown d11/d1d/c9d 26101 1 2024-04-26T04:42:20.389 INFO:tasks.workunit.client.1.smithi104.stdout:3/732: dread d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/faa [0,4194304] 0 2024-04-26T04:42:20.391 INFO:tasks.workunit.client.0.smithi090.stdout:7/390: dread d0/db/d53/fa [0,4194304] 0 2024-04-26T04:42:20.391 INFO:tasks.workunit.client.0.smithi090.stdout:7/391: write d0/f45 [1429477,119160] 0 2024-04-26T04:42:20.391 INFO:tasks.workunit.client.0.smithi090.stdout:3/248: mkdir d1/d17/d36/d33/d56 0 2024-04-26T04:42:20.392 INFO:tasks.workunit.client.1.smithi104.stdout:9/598: dread d11/f52 [0,4194304] 0 2024-04-26T04:42:20.392 INFO:tasks.workunit.client.1.smithi104.stdout:9/599: fdatasync d11/d15/d32/d35/f57 0 2024-04-26T04:42:20.393 INFO:tasks.workunit.client.0.smithi090.stdout:7/392: creat d0/d1/d4/d3e/f90 x:0 0 0 2024-04-26T04:42:20.394 INFO:tasks.workunit.client.0.smithi090.stdout:3/249: symlink d1/d6/l57 0 2024-04-26T04:42:20.395 INFO:tasks.workunit.client.1.smithi104.stdout:1/710: sync 2024-04-26T04:42:20.395 INFO:tasks.workunit.client.1.smithi104.stdout:1/711: fsync d0/d6/d2e/f46 0 2024-04-26T04:42:20.395 INFO:tasks.workunit.client.1.smithi104.stdout:0/582: sync 2024-04-26T04:42:20.395 INFO:tasks.workunit.client.1.smithi104.stdout:4/701: sync 2024-04-26T04:42:20.396 INFO:tasks.workunit.client.1.smithi104.stdout:6/765: sync 2024-04-26T04:42:20.396 INFO:tasks.workunit.client.0.smithi090.stdout:3/250: mkdir d1/d6/d1b/d58 0 2024-04-26T04:42:20.397 INFO:tasks.workunit.client.1.smithi104.stdout:5/682: creat d0/d1/d64/d26/d36/da7/fe2 x:0 0 0 2024-04-26T04:42:20.397 INFO:tasks.workunit.client.1.smithi104.stdout:5/683: write d0/d1/d1e/d98/fbc [698622,19073] 0 2024-04-26T04:42:20.398 INFO:tasks.workunit.client.0.smithi090.stdout:3/251: mkdir d1/d59 0 2024-04-26T04:42:20.398 INFO:tasks.workunit.client.0.smithi090.stdout:3/252: dread - d1/d6/d2c/f49 zero size 2024-04-26T04:42:20.398 INFO:tasks.workunit.client.1.smithi104.stdout:0/583: chown d0/f42 10671 1 2024-04-26T04:42:20.399 INFO:tasks.workunit.client.1.smithi104.stdout:6/766: dread d0/d9/d13/d41/d82/dc8/f84 [0,4194304] 0 2024-04-26T04:42:20.400 INFO:tasks.workunit.client.1.smithi104.stdout:6/767: truncate d0/d9/d13/d41/d96/ded/ffe 4834396 0 2024-04-26T04:42:20.400 INFO:tasks.workunit.client.1.smithi104.stdout:6/768: fsync d0/d9/d13/d41/d82/dc8/dba/daa/fdf 0 2024-04-26T04:42:20.400 INFO:tasks.workunit.client.1.smithi104.stdout:3/733: dread d5/d9/f26 [0,4194304] 0 2024-04-26T04:42:20.400 INFO:tasks.workunit.client.1.smithi104.stdout:6/769: write d0/d9/d13/d41/d96/dac/dc4/ff2 [14401,94466] 0 2024-04-26T04:42:20.400 INFO:tasks.workunit.client.1.smithi104.stdout:6/770: read - d0/d9/d13/d44/d47/fe7 zero size 2024-04-26T04:42:20.400 INFO:tasks.workunit.client.1.smithi104.stdout:6/771: fsync d0/d9/d13/d41/d96/fb0 0 2024-04-26T04:42:20.401 INFO:tasks.workunit.client.1.smithi104.stdout:4/702: symlink d4/d20/d55/d56/lf8 0 2024-04-26T04:42:20.401 INFO:tasks.workunit.client.1.smithi104.stdout:4/703: chown d4/d7/d1d/d2e/d74/dbd/dd3/fe9 28869588 1 2024-04-26T04:42:20.401 INFO:tasks.workunit.client.0.smithi090.stdout:2/397: dwrite db/f10 [0,4194304] 0 2024-04-26T04:42:20.402 INFO:tasks.workunit.client.0.smithi090.stdout:9/252: dwrite db/d16/f46 [0,4194304] 0 2024-04-26T04:42:20.402 INFO:tasks.workunit.client.1.smithi104.stdout:5/684: mknod d0/d1/d4/d2a/d6b/d91/dc7/ce3 0 2024-04-26T04:42:20.402 INFO:tasks.workunit.client.1.smithi104.stdout:5/685: stat d0/d1/d4/d2a/d2b/d37/d48/l7a 0 2024-04-26T04:42:20.403 INFO:tasks.workunit.client.1.smithi104.stdout:5/686: chown d0/d1/d1e/d98/dd4 8885 1 2024-04-26T04:42:20.403 INFO:tasks.workunit.client.1.smithi104.stdout:5/687: read d0/d41/f4d [60717,74870] 0 2024-04-26T04:42:20.403 INFO:tasks.workunit.client.1.smithi104.stdout:5/688: readlink d0/d1/d25/lac 0 2024-04-26T04:42:20.403 INFO:tasks.workunit.client.1.smithi104.stdout:8/617: write d0/d33/d51/fa2 [1018009,117720] 0 2024-04-26T04:42:20.403 INFO:tasks.workunit.client.1.smithi104.stdout:0/584: write d0/f91 [333396,85142] 0 2024-04-26T04:42:20.404 INFO:tasks.workunit.client.1.smithi104.stdout:0/585: write d0/d4/f89 [310550,16283] 0 2024-04-26T04:42:20.405 INFO:tasks.workunit.client.1.smithi104.stdout:4/704: truncate d4/d7/d1d/d1e/d24/d48/f4e 2286748 0 2024-04-26T04:42:20.405 INFO:tasks.workunit.client.1.smithi104.stdout:4/705: readlink d4/d7/d1d/d1e/d15/la4 0 2024-04-26T04:42:20.405 INFO:tasks.workunit.client.1.smithi104.stdout:4/706: write d4/d20/d55/d56/feb [582969,92637] 0 2024-04-26T04:42:20.407 INFO:tasks.workunit.client.1.smithi104.stdout:1/712: dread d0/d6/d19/d21/d28/f85 [0,4194304] 0 2024-04-26T04:42:20.407 INFO:tasks.workunit.client.1.smithi104.stdout:1/713: stat d0/d6/d19/d21/c63 0 2024-04-26T04:42:20.407 INFO:tasks.workunit.client.1.smithi104.stdout:0/586: symlink d0/d2/d10/d84/lb5 0 2024-04-26T04:42:20.410 INFO:tasks.workunit.client.0.smithi090.stdout:2/398: mknod db/d11/d14/d25/d86/c87 0 2024-04-26T04:42:20.411 INFO:tasks.workunit.client.0.smithi090.stdout:9/253: rmdir db/dd/d18/d38/d39 39 2024-04-26T04:42:20.412 INFO:tasks.workunit.client.1.smithi104.stdout:1/714: truncate d0/d6/d19/d1b/da5/fa6 3557239 0 2024-04-26T04:42:20.413 INFO:tasks.workunit.client.0.smithi090.stdout:3/253: rmdir d1/d6/d2c 39 2024-04-26T04:42:20.416 INFO:tasks.workunit.client.1.smithi104.stdout:3/734: rmdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/dc2 0 2024-04-26T04:42:20.416 INFO:tasks.workunit.client.1.smithi104.stdout:3/735: chown d5/db2/d46/d95 0 1 2024-04-26T04:42:20.416 INFO:tasks.workunit.client.0.smithi090.stdout:2/399: link db/d11/d14/d1b/d20/c23 db/d11/d14/d25/d2d/d48/c88 0 2024-04-26T04:42:20.417 INFO:tasks.workunit.client.0.smithi090.stdout:2/400: truncate db/d11/d14/d25/d2d/f36 935162 0 2024-04-26T04:42:20.417 INFO:tasks.workunit.client.1.smithi104.stdout:4/707: rmdir d4/d7/d1d/d1e/d24/d59/db3 39 2024-04-26T04:42:20.418 INFO:tasks.workunit.client.0.smithi090.stdout:9/254: link db/dd/d18/l4b db/dd/d18/l59 0 2024-04-26T04:42:20.420 INFO:tasks.workunit.client.0.smithi090.stdout:3/254: mknod d1/d17/d36/d33/d42/c5a 0 2024-04-26T04:42:20.420 INFO:tasks.workunit.client.0.smithi090.stdout:3/255: rename d1/d17/d36/d33 to d1/d17/d36/d33/d56/d5b 22 2024-04-26T04:42:20.421 INFO:tasks.workunit.client.0.smithi090.stdout:3/256: chown d1/d17/d36/d33/l41 529728522 1 2024-04-26T04:42:20.422 INFO:tasks.workunit.client.1.smithi104.stdout:1/715: creat d0/d6/d19/d1b/da7/db8/fe9 x:0 0 0 2024-04-26T04:42:20.422 INFO:tasks.workunit.client.1.smithi104.stdout:1/716: readlink d0/l23 0 2024-04-26T04:42:20.422 INFO:tasks.workunit.client.1.smithi104.stdout:1/717: stat d0/d6/d19/d1b/fd7 0 2024-04-26T04:42:20.422 INFO:tasks.workunit.client.1.smithi104.stdout:3/736: mkdir d5/d9/d86/d104 0 2024-04-26T04:42:20.422 INFO:tasks.workunit.client.1.smithi104.stdout:3/737: readlink d5/d9/d80/lff 0 2024-04-26T04:42:20.423 INFO:tasks.workunit.client.0.smithi090.stdout:3/257: read d1/d9/f38 [379569,86926] 0 2024-04-26T04:42:20.423 INFO:tasks.workunit.client.0.smithi090.stdout:3/258: chown d1/d17/d36/d2a/l2d 3204 1 2024-04-26T04:42:20.424 INFO:tasks.workunit.client.1.smithi104.stdout:4/708: symlink d4/d7/d1d/d2e/d74/dbd/lf9 0 2024-04-26T04:42:20.425 INFO:tasks.workunit.client.0.smithi090.stdout:8/387: sync 2024-04-26T04:42:20.425 INFO:tasks.workunit.client.0.smithi090.stdout:8/388: write dc/d34/d4d/f4e [897918,78998] 0 2024-04-26T04:42:20.426 INFO:tasks.workunit.client.1.smithi104.stdout:1/718: symlink d0/d6/d19/d21/d66/d20/d2a/d56/d8c/lea 0 2024-04-26T04:42:20.426 INFO:tasks.workunit.client.0.smithi090.stdout:8/389: rename dc/d1f to dc/d1f/d49/d3b/d86 22 2024-04-26T04:42:20.427 INFO:tasks.workunit.client.1.smithi104.stdout:3/738: mknod d5/db2/da3/dd0/ddb/d32/d48/c105 0 2024-04-26T04:42:20.427 INFO:tasks.workunit.client.1.smithi104.stdout:3/739: stat d5/d9/d80/da2 0 2024-04-26T04:42:20.428 INFO:tasks.workunit.client.1.smithi104.stdout:5/689: write d0/d1/d1e/f2e [1208447,94419] 0 2024-04-26T04:42:20.428 INFO:tasks.workunit.client.0.smithi090.stdout:8/390: mkdir dc/d34/d87 0 2024-04-26T04:42:20.431 INFO:tasks.workunit.client.1.smithi104.stdout:4/709: link d4/d7/d1d/d8e/fc3 d4/d7/d1d/d1e/d8f/db6/ffa 0 2024-04-26T04:42:20.431 INFO:tasks.workunit.client.1.smithi104.stdout:4/710: chown d4/d7/d1d/d1e/d8f/db6/ccb 296469 1 2024-04-26T04:42:20.432 INFO:tasks.workunit.client.0.smithi090.stdout:7/393: dwrite d0/db/d53/f16 [0,4194304] 0 2024-04-26T04:42:20.432 INFO:tasks.workunit.client.0.smithi090.stdout:7/394: chown d0/db/df/l35 24 1 2024-04-26T04:42:20.432 INFO:tasks.workunit.client.1.smithi104.stdout:8/618: dwrite d0/d1c/d2d/f32 [4194304,4194304] 0 2024-04-26T04:42:20.432 INFO:tasks.workunit.client.1.smithi104.stdout:6/772: dwrite d0/d9/d13/d41/d96/dac/dc4/fcf [0,4194304] 0 2024-04-26T04:42:20.433 INFO:tasks.workunit.client.0.smithi090.stdout:7/395: rename d0/db/df/f42 to d0/db/d15/f91 0 2024-04-26T04:42:20.433 INFO:tasks.workunit.client.0.smithi090.stdout:7/396: write d0/db/d15/f7e [882968,37437] 0 2024-04-26T04:42:20.433 INFO:tasks.workunit.client.0.smithi090.stdout:7/397: chown d0/db/d15/d30/d8e 50912542 1 2024-04-26T04:42:20.434 INFO:tasks.workunit.client.0.smithi090.stdout:7/398: creat d0/db/df/d4d/d64/f92 x:0 0 0 2024-04-26T04:42:20.435 INFO:tasks.workunit.client.1.smithi104.stdout:3/740: symlink d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/l106 0 2024-04-26T04:42:20.439 INFO:tasks.workunit.client.1.smithi104.stdout:5/690: mknod d0/d89/ce4 0 2024-04-26T04:42:20.440 INFO:tasks.workunit.client.0.smithi090.stdout:8/391: dread dc/d1f/d49/f28 [0,4194304] 0 2024-04-26T04:42:20.440 INFO:tasks.workunit.client.1.smithi104.stdout:4/711: write d4/d7/d1d/d1e/d8f/db6/f87 [367034,82489] 0 2024-04-26T04:42:20.445 INFO:tasks.workunit.client.0.smithi090.stdout:7/399: write d0/d1/d4/f10 [638954,13334] 0 2024-04-26T04:42:20.447 INFO:tasks.workunit.client.0.smithi090.stdout:7/400: creat d0/d1/d4/d43/f93 x:0 0 0 2024-04-26T04:42:20.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/401: symlink d0/d1/d4/d43/d65/l94 0 2024-04-26T04:42:20.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/402: write d0/db/df/d1c/f7a [103362,61971] 0 2024-04-26T04:42:20.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/403: creat d0/db/d15/d30/d4f/f95 x:0 0 0 2024-04-26T04:42:20.448 INFO:tasks.workunit.client.1.smithi104.stdout:5/691: creat d0/d1/d4/d2a/d2b/d37/d99/fe5 x:0 0 0 2024-04-26T04:42:20.448 INFO:tasks.workunit.client.1.smithi104.stdout:8/619: creat d0/d9a/fc0 x:0 0 0 2024-04-26T04:42:20.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/404: symlink d0/db/d15/d29/l96 0 2024-04-26T04:42:20.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/405: rmdir d0/d1/d4 39 2024-04-26T04:42:20.450 INFO:tasks.workunit.client.0.smithi090.stdout:8/392: write dc/d1f/f21 [730340,20561] 0 2024-04-26T04:42:20.451 INFO:tasks.workunit.client.0.smithi090.stdout:2/401: dwrite db/d11/d14/d27/d28/f6e [0,4194304] 0 2024-04-26T04:42:20.451 INFO:tasks.workunit.client.0.smithi090.stdout:2/402: fdatasync db/d11/d14/d27/d2f/d42/d5c/f4f 0 2024-04-26T04:42:20.451 INFO:tasks.workunit.client.0.smithi090.stdout:7/406: creat d0/db/df/f97 x:0 0 0 2024-04-26T04:42:20.454 INFO:tasks.workunit.client.1.smithi104.stdout:8/620: dread d0/fe [0,4194304] 0 2024-04-26T04:42:20.454 INFO:tasks.workunit.client.1.smithi104.stdout:8/621: chown d0/d1c/d27/d47/d67/d76/ca7 52220428 1 2024-04-26T04:42:20.454 INFO:tasks.workunit.client.1.smithi104.stdout:8/622: truncate d0/d33/d35/d6d/f9b 544588 0 2024-04-26T04:42:20.455 INFO:tasks.workunit.client.0.smithi090.stdout:7/407: dread d0/f3c [0,4194304] 0 2024-04-26T04:42:20.456 INFO:tasks.workunit.client.1.smithi104.stdout:4/712: write d4/d20/d55/d56/da3/fc4 [2885275,103984] 0 2024-04-26T04:42:20.457 INFO:tasks.workunit.client.1.smithi104.stdout:8/623: dread d0/d33/d51/f71 [0,4194304] 0 2024-04-26T04:42:20.457 INFO:tasks.workunit.client.1.smithi104.stdout:8/624: write d0/d33/d35/f87 [3890525,81228] 0 2024-04-26T04:42:20.458 INFO:tasks.workunit.client.1.smithi104.stdout:4/713: creat d4/ffb x:0 0 0 2024-04-26T04:42:20.459 INFO:tasks.workunit.client.0.smithi090.stdout:3/259: dwrite d1/d17/d36/f24 [0,4194304] 0 2024-04-26T04:42:20.459 INFO:tasks.workunit.client.0.smithi090.stdout:3/260: write d1/d6/d2c/f49 [97443,104887] 0 2024-04-26T04:42:20.460 INFO:tasks.workunit.client.1.smithi104.stdout:8/625: fsync d0/f49 0 2024-04-26T04:42:20.460 INFO:tasks.workunit.client.1.smithi104.stdout:4/714: dread d4/d7/d1d/d1e/d15/dd8/fef [0,4194304] 0 2024-04-26T04:42:20.460 INFO:tasks.workunit.client.1.smithi104.stdout:4/715: truncate d4/d7/fe8 187880 0 2024-04-26T04:42:20.460 INFO:tasks.workunit.client.1.smithi104.stdout:4/716: stat d4/d7/fe8 0 2024-04-26T04:42:20.461 INFO:tasks.workunit.client.1.smithi104.stdout:8/626: truncate d0/f1a 682830 0 2024-04-26T04:42:20.462 INFO:tasks.workunit.client.1.smithi104.stdout:4/717: creat d4/d7/d1d/d1e/d15/df3/ffc x:0 0 0 2024-04-26T04:42:20.462 INFO:tasks.workunit.client.1.smithi104.stdout:4/718: dread - d4/d7/d1d/d1e/d24/d59/db8/fe3 zero size 2024-04-26T04:42:20.463 INFO:tasks.workunit.client.1.smithi104.stdout:4/719: write d4/d7/d1d/d1e/d24/d48/fdb [996974,7854] 0 2024-04-26T04:42:20.463 INFO:tasks.workunit.client.1.smithi104.stdout:8/627: symlink d0/d33/d35/d6d/lc1 0 2024-04-26T04:42:20.464 INFO:tasks.workunit.client.1.smithi104.stdout:6/773: dwrite d0/d9/d13/d41/d96/fb0 [0,4194304] 0 2024-04-26T04:42:20.464 INFO:tasks.workunit.client.1.smithi104.stdout:1/719: dwrite d0/d6/d19/d21/d66/d20/d2a/f78 [0,4194304] 0 2024-04-26T04:42:20.464 INFO:tasks.workunit.client.1.smithi104.stdout:4/720: mkdir d4/d20/d55/d56/da3/dea/dfd 0 2024-04-26T04:42:20.466 INFO:tasks.workunit.client.1.smithi104.stdout:8/628: write d0/fe [3662230,100502] 0 2024-04-26T04:42:20.471 INFO:tasks.workunit.client.1.smithi104.stdout:5/692: dwrite d0/d1/d4/d2a/d6b/d73/da4/fd5 [0,4194304] 0 2024-04-26T04:42:20.471 INFO:tasks.workunit.client.1.smithi104.stdout:5/693: fsync d0/d89/fa2 0 2024-04-26T04:42:20.477 INFO:tasks.workunit.client.0.smithi090.stdout:3/261: write d1/d9/f20 [811414,68804] 0 2024-04-26T04:42:20.482 INFO:tasks.workunit.client.1.smithi104.stdout:1/720: unlink d0/d6/d19/d21/d66/de4/l57 0 2024-04-26T04:42:20.482 INFO:tasks.workunit.client.1.smithi104.stdout:1/721: chown d0/d6/d19/d21/d66/de4/fcd 1021748476 1 2024-04-26T04:42:20.482 INFO:tasks.workunit.client.1.smithi104.stdout:1/722: write d0/d6/d19/d21/d59/fc8 [618652,38582] 0 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:1/723: fdatasync d0/d6/d19/d21/d28/f85 0 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:1/724: truncate d0/d6/d19/d21/d66/d20/d2a/f6f 2905290 0 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:8/629: dread d0/d1c/f4c [0,4194304] 0 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:8/630: chown d0/d1c/d27/d47/d67/c97 232415 1 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:4/721: symlink d4/d20/d55/d56/da3/dea/dfd/lfe 0 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:4/722: write d4/d7/d1d/d1e/f5b [94399,122861] 0 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:4/723: fdatasync d4/d7/d1d/d1e/d15/f22 0 2024-04-26T04:42:20.483 INFO:tasks.workunit.client.1.smithi104.stdout:5/694: mkdir d0/d1/de6 0 2024-04-26T04:42:20.486 INFO:tasks.workunit.client.1.smithi104.stdout:1/725: creat d0/d6/d19/d21/feb x:0 0 0 2024-04-26T04:42:20.487 INFO:tasks.workunit.client.1.smithi104.stdout:6/774: dwrite d0/d9/d13/d41/d96/dac/dc4/fcf [0,4194304] 0 2024-04-26T04:42:20.487 INFO:tasks.workunit.client.1.smithi104.stdout:6/775: stat d0/d9/d13/d41/d96/dac/db3/fe4 0 2024-04-26T04:42:20.488 INFO:tasks.workunit.client.1.smithi104.stdout:6/776: read d0/d9/d13/d41/d96/dac/f4e [129277,111919] 0 2024-04-26T04:42:20.489 INFO:tasks.workunit.client.1.smithi104.stdout:4/724: unlink d4/d7/d1d/lbe 0 2024-04-26T04:42:20.489 INFO:tasks.workunit.client.1.smithi104.stdout:4/725: write d4/d7/d1d/d1e/d24/d59/db8/fe6 [968701,20971] 0 2024-04-26T04:42:20.492 INFO:tasks.workunit.client.1.smithi104.stdout:1/726: rmdir d0/d6/d2e/d32/d4c/dd0 0 2024-04-26T04:42:20.494 INFO:tasks.workunit.client.1.smithi104.stdout:1/727: dread d0/d6/d2e/d32/d4c/fb0 [0,4194304] 0 2024-04-26T04:42:20.495 INFO:tasks.workunit.client.1.smithi104.stdout:4/726: symlink d4/d20/d55/d56/da3/dea/dfd/lff 0 2024-04-26T04:42:20.498 INFO:tasks.workunit.client.0.smithi090.stdout:8/393: mknod dc/d1f/c88 0 2024-04-26T04:42:20.499 INFO:tasks.workunit.client.0.smithi090.stdout:2/403: rmdir db/d11/d14/d25/d2d/d48 39 2024-04-26T04:42:20.499 INFO:tasks.workunit.client.1.smithi104.stdout:1/728: rename d0/d6/d2e/d32/d4d/da1 to d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/dec 0 2024-04-26T04:42:20.499 INFO:tasks.workunit.client.1.smithi104.stdout:6/777: write d0/ff3 [3209383,71933] 0 2024-04-26T04:42:20.501 INFO:tasks.workunit.client.1.smithi104.stdout:4/727: unlink d4/d7/d1d/d2e/faa 0 2024-04-26T04:42:20.501 INFO:tasks.workunit.client.1.smithi104.stdout:1/729: read d0/d6/d19/d21/f6d [722113,28956] 0 2024-04-26T04:42:20.502 INFO:tasks.workunit.client.1.smithi104.stdout:6/778: rmdir d0/d9/d13/d41/d82/dc8/dba/daa/dda 39 2024-04-26T04:42:20.502 INFO:tasks.workunit.client.1.smithi104.stdout:6/779: stat d0/d9/d13/d41/d96/dac/dec/c92 0 2024-04-26T04:42:20.504 INFO:tasks.workunit.client.1.smithi104.stdout:1/730: symlink d0/d6/led 0 2024-04-26T04:42:20.504 INFO:tasks.workunit.client.1.smithi104.stdout:1/731: stat d0/d6/d49/f87 0 2024-04-26T04:42:20.504 INFO:tasks.workunit.client.1.smithi104.stdout:1/732: stat d0/d6/d19/d21/d66/f83 0 2024-04-26T04:42:20.504 INFO:tasks.workunit.client.1.smithi104.stdout:1/733: chown d0/d6/d19/d21/d66/de4/d7e/dc6 1796010 1 2024-04-26T04:42:20.506 INFO:tasks.workunit.client.1.smithi104.stdout:1/734: truncate d0/d6/d19/d21/f6d 221458 0 2024-04-26T04:42:20.510 INFO:tasks.workunit.client.0.smithi090.stdout:3/262: mkdir d1/d5c 0 2024-04-26T04:42:20.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/394: rmdir dc/d1f/d49/d3b/d3d 39 2024-04-26T04:42:20.512 INFO:tasks.workunit.client.0.smithi090.stdout:8/395: stat dc/d5a/d5d/c72 0 2024-04-26T04:42:20.513 INFO:tasks.workunit.client.0.smithi090.stdout:3/263: mkdir d1/d17/d36/d2a/d3a/d5d 0 2024-04-26T04:42:20.513 INFO:tasks.workunit.client.0.smithi090.stdout:3/264: read d1/d17/d36/f40 [1058837,130157] 0 2024-04-26T04:42:20.516 INFO:tasks.workunit.client.0.smithi090.stdout:3/265: mkdir d1/d59/d5e 0 2024-04-26T04:42:20.522 INFO:tasks.workunit.client.0.smithi090.stdout:3/266: read d1/d9/fa [3252484,106503] 0 2024-04-26T04:42:20.524 INFO:tasks.workunit.client.0.smithi090.stdout:3/267: creat d1/d6/d2c/f5f x:0 0 0 2024-04-26T04:42:20.524 INFO:tasks.workunit.client.0.smithi090.stdout:3/268: write d1/d17/d36/d2a/d3a/f53 [463351,47504] 0 2024-04-26T04:42:20.526 INFO:tasks.workunit.client.0.smithi090.stdout:3/269: rmdir d1/d17/d36/d2a 39 2024-04-26T04:42:20.526 INFO:tasks.workunit.client.0.smithi090.stdout:3/270: read d1/d6/d1b/f35 [514179,67460] 0 2024-04-26T04:42:20.527 INFO:tasks.workunit.client.1.smithi104.stdout:8/631: dwrite d0/f9 [0,4194304] 0 2024-04-26T04:42:20.527 INFO:tasks.workunit.client.1.smithi104.stdout:8/632: chown d0/f49 18769024 1 2024-04-26T04:42:20.527 INFO:tasks.workunit.client.1.smithi104.stdout:8/633: creat d0/d1c/d27/d47/d67/d76/d7f/fc2 x:0 0 0 2024-04-26T04:42:20.527 INFO:tasks.workunit.client.1.smithi104.stdout:8/634: truncate d0/d1c/f26 1288097 0 2024-04-26T04:42:20.529 INFO:tasks.workunit.client.1.smithi104.stdout:4/728: dwrite d4/d7/d1d/d1e/d24/d48/fdb [0,4194304] 0 2024-04-26T04:42:20.529 INFO:tasks.workunit.client.0.smithi090.stdout:7/408: rename d0/d1/d4/d43 to d0/d1/d4/d98 0 2024-04-26T04:42:20.529 INFO:tasks.workunit.client.0.smithi090.stdout:7/409: getdents d0/db/d15/d30/d8e 0 2024-04-26T04:42:20.530 INFO:tasks.workunit.client.0.smithi090.stdout:7/410: dread - d0/db/d53/f87 zero size 2024-04-26T04:42:20.530 INFO:tasks.workunit.client.1.smithi104.stdout:8/635: creat d0/d1c/d59/d85/fc3 x:0 0 0 2024-04-26T04:42:20.533 INFO:tasks.workunit.client.1.smithi104.stdout:4/729: rename d4/d7/d1d/d2e/f67 to d4/d7/d1d/d1e/d24/d59/f100 0 2024-04-26T04:42:20.533 INFO:tasks.workunit.client.1.smithi104.stdout:4/730: read d4/db/d6b/d7f/f84 [11548,73583] 0 2024-04-26T04:42:20.534 INFO:tasks.workunit.client.1.smithi104.stdout:8/636: mkdir d0/d50/dc4 0 2024-04-26T04:42:20.536 INFO:tasks.workunit.client.1.smithi104.stdout:4/731: rename d4/d7/d1d/d1e/d24/d48/f60 to d4/db/d6b/d86/f101 0 2024-04-26T04:42:20.536 INFO:tasks.workunit.client.1.smithi104.stdout:4/732: dread - d4/f69 zero size 2024-04-26T04:42:20.537 INFO:tasks.workunit.client.1.smithi104.stdout:1/735: dwrite d0/d6/d2e/f84 [0,4194304] 0 2024-04-26T04:42:20.537 INFO:tasks.workunit.client.1.smithi104.stdout:1/736: truncate d0/d6/d19/d1b/da7/db8/fdd 689770 0 2024-04-26T04:42:20.537 INFO:tasks.workunit.client.1.smithi104.stdout:1/737: fdatasync d0/d6/d19/d21/d66/d29/d3f/fe7 0 2024-04-26T04:42:20.538 INFO:tasks.workunit.client.1.smithi104.stdout:8/637: truncate d0/f5 1887155 0 2024-04-26T04:42:20.538 INFO:tasks.workunit.client.1.smithi104.stdout:8/638: rename d0/d1c/d59/d85 to d0/d1c/d59/d85/dc5 22 2024-04-26T04:42:20.540 INFO:tasks.workunit.client.1.smithi104.stdout:8/639: write d0/d1c/d27/f40 [1624233,90013] 0 2024-04-26T04:42:20.540 INFO:tasks.workunit.client.1.smithi104.stdout:8/640: creat d0/d33/d51/fc6 x:0 0 0 2024-04-26T04:42:20.543 INFO:tasks.workunit.client.0.smithi090.stdout:8/396: dwrite dc/f6b [0,4194304] 0 2024-04-26T04:42:20.543 INFO:tasks.workunit.client.0.smithi090.stdout:8/397: creat dc/d1f/d49/d39/d55/d64/f89 x:0 0 0 2024-04-26T04:42:20.543 INFO:tasks.workunit.client.0.smithi090.stdout:8/398: chown dc/d1f/d49/c25 3061 1 2024-04-26T04:42:20.544 INFO:tasks.workunit.client.1.smithi104.stdout:8/641: dread d0/d1c/d27/f72 [0,4194304] 0 2024-04-26T04:42:20.546 INFO:tasks.workunit.client.0.smithi090.stdout:8/399: creat dc/d1f/d49/d3b/d3d/d57/f8a x:0 0 0 2024-04-26T04:42:20.546 INFO:tasks.workunit.client.1.smithi104.stdout:4/733: fsync d4/db/d6b/d86/f101 0 2024-04-26T04:42:20.547 INFO:tasks.workunit.client.0.smithi090.stdout:8/400: truncate dc/d34/f40 380426 0 2024-04-26T04:42:20.548 INFO:tasks.workunit.client.0.smithi090.stdout:3/271: dread d1/d17/d36/d2a/d3a/f53 [0,4194304] 0 2024-04-26T04:42:20.548 INFO:tasks.workunit.client.1.smithi104.stdout:4/734: link d4/d20/fab d4/db/d6b/df0/f102 0 2024-04-26T04:42:20.548 INFO:tasks.workunit.client.1.smithi104.stdout:4/735: stat d4/db/d6b/d7f/fcc 0 2024-04-26T04:42:20.549 INFO:tasks.workunit.client.0.smithi090.stdout:8/401: rmdir dc/d71 39 2024-04-26T04:42:20.549 INFO:tasks.workunit.client.0.smithi090.stdout:8/402: chown dc/d1f/d49/d39/d55/f70 16208 1 2024-04-26T04:42:20.549 INFO:tasks.workunit.client.0.smithi090.stdout:3/272: symlink d1/d17/d52/l60 0 2024-04-26T04:42:20.550 INFO:tasks.workunit.client.1.smithi104.stdout:4/736: creat d4/d20/d55/d56/da3/dea/f103 x:0 0 0 2024-04-26T04:42:20.550 INFO:tasks.workunit.client.1.smithi104.stdout:4/737: fdatasync d4/d7/d1d/d2e/d74/f93 0 2024-04-26T04:42:20.550 INFO:tasks.workunit.client.1.smithi104.stdout:4/738: chown d4/d7/d1d/d2e/d74/cd0 8097 1 2024-04-26T04:42:20.552 INFO:tasks.workunit.client.1.smithi104.stdout:4/739: read - d4/d7/d1d/d1e/d24/d4a/d9e/fcd zero size 2024-04-26T04:42:20.552 INFO:tasks.workunit.client.1.smithi104.stdout:4/740: chown d4/d7/d1d/d2e/d74/dbd/lf9 1081140 1 2024-04-26T04:42:20.552 INFO:tasks.workunit.client.0.smithi090.stdout:3/273: rename d1/d17/d36/d33/d56 to d1/d17/d61 0 2024-04-26T04:42:20.552 INFO:tasks.workunit.client.0.smithi090.stdout:3/274: stat d1/d9/c47 0 2024-04-26T04:42:20.552 INFO:tasks.workunit.client.0.smithi090.stdout:3/275: creat d1/d59/d5e/f62 x:0 0 0 2024-04-26T04:42:20.552 INFO:tasks.workunit.client.0.smithi090.stdout:3/276: chown d1/d9/c16 174 1 2024-04-26T04:42:20.552 INFO:tasks.workunit.client.0.smithi090.stdout:3/277: chown d1/d17/d36/d2a/d3a/f53 4075565 1 2024-04-26T04:42:20.554 INFO:tasks.workunit.client.0.smithi090.stdout:3/278: creat d1/d17/d36/d33/d42/f63 x:0 0 0 2024-04-26T04:42:20.554 INFO:tasks.workunit.client.1.smithi104.stdout:4/741: mknod d4/d7/d1d/c104 0 2024-04-26T04:42:20.554 INFO:tasks.workunit.client.1.smithi104.stdout:4/742: readlink d4/d7/d1d/d1e/d24/l29 0 2024-04-26T04:42:20.554 INFO:tasks.workunit.client.1.smithi104.stdout:4/743: write d4/d7/d1d/d1e/d8f/db6/f9c [226222,123672] 0 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/345: sync 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:5/459: sync 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:9/255: sync 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/346: fsync d3/f48 0 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:9/256: getdents db/dd/d18/d31/d58 0 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/347: chown d3/d13/d1d/d50/c56 122 1 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:1/336: sync 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:6/430: sync 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:6/431: chown d9/d1c/d4d/d6d/c82 109081110 1 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:6/432: readlink d9/d1c/d4d/d58/l65 0 2024-04-26T04:42:20.555 INFO:tasks.workunit.client.0.smithi090.stdout:0/461: sync 2024-04-26T04:42:20.556 INFO:tasks.workunit.client.1.smithi104.stdout:4/744: unlink d4/d7/d1d/d1e/d15/dd8/fef 0 2024-04-26T04:42:20.556 INFO:tasks.workunit.client.1.smithi104.stdout:4/745: chown d4/d7/d1d/d8e/fc3 18337 1 2024-04-26T04:42:20.556 INFO:tasks.workunit.client.1.smithi104.stdout:4/746: chown d4/c64 18109090 1 2024-04-26T04:42:20.556 INFO:tasks.workunit.client.0.smithi090.stdout:5/460: rename d4/d1a/df/d36/d42 to d4/d1a/d2a/d79/d90 0 2024-04-26T04:42:20.557 INFO:tasks.workunit.client.0.smithi090.stdout:5/461: write d4/d1a/d2a/d79/d90/d49/f7d [322512,28249] 0 2024-04-26T04:42:20.557 INFO:tasks.workunit.client.1.smithi104.stdout:4/747: mkdir d4/d7/d1d/d1e/d105 0 2024-04-26T04:42:20.558 INFO:tasks.workunit.client.0.smithi090.stdout:9/257: creat db/d16/f5a x:0 0 0 2024-04-26T04:42:20.558 INFO:tasks.workunit.client.0.smithi090.stdout:9/258: write db/f47 [773854,48791] 0 2024-04-26T04:42:20.558 INFO:tasks.workunit.client.0.smithi090.stdout:8/403: write dc/d1f/d49/d3b/f4a [794918,80433] 0 2024-04-26T04:42:20.558 INFO:tasks.workunit.client.0.smithi090.stdout:8/404: fdatasync dc/d1f/d49/d39/d55/d64/f7a 0 2024-04-26T04:42:20.558 INFO:tasks.workunit.client.0.smithi090.stdout:7/411: dwrite d0/db/df/d4d/f5a [0,4194304] 0 2024-04-26T04:42:20.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/348: symlink d3/d13/d1d/d50/d51/l66 0 2024-04-26T04:42:20.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/349: write d3/db/f23 [219174,4013] 0 2024-04-26T04:42:20.559 INFO:tasks.workunit.client.0.smithi090.stdout:1/337: rmdir d0/d4/d7/d4b/d53 39 2024-04-26T04:42:20.561 INFO:tasks.workunit.client.0.smithi090.stdout:6/433: mknod d9/d3c/ca1 0 2024-04-26T04:42:20.562 INFO:tasks.workunit.client.0.smithi090.stdout:0/462: mknod d0/d4b/d5d/d5c/d8a/d7c/ca6 0 2024-04-26T04:42:20.564 INFO:tasks.workunit.client.0.smithi090.stdout:5/462: rename d4/d1a/df/d36/d6b to d4/d1a/d2a/d79/d90/d56/d71/d91 0 2024-04-26T04:42:20.565 INFO:tasks.workunit.client.1.smithi104.stdout:4/748: read d4/db/fe [3691470,11791] 0 2024-04-26T04:42:20.565 INFO:tasks.workunit.client.1.smithi104.stdout:4/749: dread - d4/db/d6b/d7f/fcc zero size 2024-04-26T04:42:20.569 INFO:tasks.workunit.client.1.smithi104.stdout:4/750: getdents d4/d7/d1d/d1e/d8f/db6/d7d 0 2024-04-26T04:42:20.569 INFO:tasks.workunit.client.1.smithi104.stdout:4/751: fdatasync d4/d20/d55/d56/f3e 0 2024-04-26T04:42:20.570 INFO:tasks.workunit.client.0.smithi090.stdout:0/463: dread d0/d4b/d5d/d5c/f5e [0,4194304] 0 2024-04-26T04:42:20.575 INFO:tasks.workunit.client.0.smithi090.stdout:3/279: dwrite d1/d9/f12 [0,4194304] 0 2024-04-26T04:42:20.575 INFO:tasks.workunit.client.0.smithi090.stdout:3/280: truncate d1/d6/d1b/f35 5215038 0 2024-04-26T04:42:20.578 INFO:tasks.workunit.client.1.smithi104.stdout:4/752: dread f2 [0,4194304] 0 2024-04-26T04:42:20.592 INFO:tasks.workunit.client.1.smithi104.stdout:4/753: dwrite d4/d20/f36 [0,4194304] 0 2024-04-26T04:42:20.592 INFO:tasks.workunit.client.1.smithi104.stdout:4/754: write d4/db/d6b/fc9 [7160517,31888] 0 2024-04-26T04:42:20.595 INFO:tasks.workunit.client.1.smithi104.stdout:4/755: rmdir d4/d7/d1d/d1e/d24/d59/db8 39 2024-04-26T04:42:20.595 INFO:tasks.workunit.client.1.smithi104.stdout:4/756: truncate d4/d7/d1d/d1e/d24/d48/fce 1121523 0 2024-04-26T04:42:20.596 INFO:tasks.workunit.client.1.smithi104.stdout:4/757: symlink d4/d20/d55/d56/da3/l106 0 2024-04-26T04:42:20.596 INFO:tasks.workunit.client.1.smithi104.stdout:4/758: write d4/d7/d1d/d1e/f61 [69656,3105] 0 2024-04-26T04:42:20.599 INFO:tasks.workunit.client.1.smithi104.stdout:4/759: rmdir d4/d7/d1d/d1e/d15 39 2024-04-26T04:42:20.600 INFO:tasks.workunit.client.1.smithi104.stdout:4/760: symlink d4/d7/d1d/d1e/d8f/db6/l107 0 2024-04-26T04:42:20.600 INFO:tasks.workunit.client.1.smithi104.stdout:4/761: read d4/db/fe [90791,71795] 0 2024-04-26T04:42:20.606 INFO:tasks.workunit.client.1.smithi104.stdout:4/762: dread d4/d7/d1d/d1e/d24/d48/f5c [0,4194304] 0 2024-04-26T04:42:20.608 INFO:tasks.workunit.client.1.smithi104.stdout:4/763: link d4/d7/d1d/d1e/d15/cd6 d4/d7/d1d/d1e/d24/d59/db8/c108 0 2024-04-26T04:42:20.614 INFO:tasks.workunit.client.1.smithi104.stdout:4/764: write d4/d7/d1d/d1e/d24/d4a/fec [377180,79893] 0 2024-04-26T04:42:20.616 INFO:tasks.workunit.client.1.smithi104.stdout:4/765: rename d4/d20/cd2 to d4/d7/d1d/d1e/d15/dd8/c109 0 2024-04-26T04:42:20.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/259: mknod db/dd/d18/d31/d58/c5b 0 2024-04-26T04:42:20.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/260: readlink db/d16/l1c 0 2024-04-26T04:42:20.621 INFO:tasks.workunit.client.0.smithi090.stdout:8/405: creat dc/d1f/d49/d3b/f8b x:0 0 0 2024-04-26T04:42:20.622 INFO:tasks.workunit.client.0.smithi090.stdout:8/406: stat dc/d1f/d49/d39/c61 0 2024-04-26T04:42:20.623 INFO:tasks.workunit.client.0.smithi090.stdout:7/412: truncate d0/db/df/d1c/d23/f26 1507864 0 2024-04-26T04:42:20.624 INFO:tasks.workunit.client.0.smithi090.stdout:4/350: creat d3/db/d12/d20/f67 x:0 0 0 2024-04-26T04:42:20.624 INFO:tasks.workunit.client.0.smithi090.stdout:4/351: dread - d3/db/d12/d20/d45/d53/f65 zero size 2024-04-26T04:42:20.627 INFO:tasks.workunit.client.0.smithi090.stdout:1/338: creat d0/d55/f67 x:0 0 0 2024-04-26T04:42:20.630 INFO:tasks.workunit.client.0.smithi090.stdout:0/464: mknod d0/d4b/d5d/d5c/d8a/d64/ca7 0 2024-04-26T04:42:20.639 INFO:tasks.workunit.client.0.smithi090.stdout:3/281: rename d1/f8 to d1/d59/f64 0 2024-04-26T04:42:20.639 INFO:tasks.workunit.client.1.smithi104.stdout:4/766: dwrite d4/d7/d1d/d1e/f14 [0,4194304] 0 2024-04-26T04:42:20.640 INFO:tasks.workunit.client.0.smithi090.stdout:8/407: dwrite dc/d1f/d49/d3b/d3d/d57/f60 [0,4194304] 0 2024-04-26T04:42:20.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/339: write d0/d4/d7/d17/f21 [1009387,10765] 0 2024-04-26T04:42:20.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/340: readlink d0/d4/l41 0 2024-04-26T04:42:20.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/341: write d0/d4/d7/d17/d22/f5c [846994,123271] 0 2024-04-26T04:42:20.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/342: write d0/f2 [3226420,97616] 0 2024-04-26T04:42:20.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/343: write d0/d4/f15 [2547767,52720] 0 2024-04-26T04:42:20.640 INFO:tasks.workunit.client.0.smithi090.stdout:1/344: fsync d0/d4/d7/d17/f3c 0 2024-04-26T04:42:20.641 INFO:tasks.workunit.client.0.smithi090.stdout:1/345: fdatasync d0/d4/f20 0 2024-04-26T04:42:20.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/261: mknod db/dd/d18/d31/c5c 0 2024-04-26T04:42:20.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/262: rename db/dd to db/dd/d18/d5d 22 2024-04-26T04:42:20.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/263: write db/d16/f5a [899430,45702] 0 2024-04-26T04:42:20.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/264: write db/dd/d18/d31/f4e [458430,127821] 0 2024-04-26T04:42:20.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/265: creat db/dd/d44/f5e x:0 0 0 2024-04-26T04:42:20.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/266: truncate db/dd/d44/f34 4714463 0 2024-04-26T04:42:20.644 INFO:tasks.workunit.client.0.smithi090.stdout:7/413: symlink d0/db/d53/d5e/l99 0 2024-04-26T04:42:20.644 INFO:tasks.workunit.client.0.smithi090.stdout:7/414: dread - d0/d1/f71 zero size 2024-04-26T04:42:20.648 INFO:tasks.workunit.client.0.smithi090.stdout:6/434: rmdir d9/d1c 39 2024-04-26T04:42:20.650 INFO:tasks.workunit.client.0.smithi090.stdout:3/282: creat d1/d17/d36/f65 x:0 0 0 2024-04-26T04:42:20.650 INFO:tasks.workunit.client.0.smithi090.stdout:3/283: write d1/d17/d36/d2a/d3a/f4b [3334831,45064] 0 2024-04-26T04:42:20.650 INFO:tasks.workunit.client.0.smithi090.stdout:3/284: write d1/d59/d5e/f62 [702893,19276] 0 2024-04-26T04:42:20.650 INFO:tasks.workunit.client.0.smithi090.stdout:0/465: dwrite d0/de/f36 [0,4194304] 0 2024-04-26T04:42:20.650 INFO:tasks.workunit.client.0.smithi090.stdout:9/267: dread db/f26 [4194304,4194304] 0 2024-04-26T04:42:20.652 INFO:tasks.workunit.client.0.smithi090.stdout:8/408: link dc/c18 dc/d1f/d49/d39/d55/d64/c8c 0 2024-04-26T04:42:20.652 INFO:tasks.workunit.client.0.smithi090.stdout:8/409: readlink dc/l11 0 2024-04-26T04:42:20.654 INFO:tasks.workunit.client.0.smithi090.stdout:7/415: creat d0/d1/d4/d98/f9a x:0 0 0 2024-04-26T04:42:20.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/352: getdents d3/db/d12/d20 0 2024-04-26T04:42:20.658 INFO:tasks.workunit.client.0.smithi090.stdout:6/435: unlink d9/d1c/d4d/l9c 0 2024-04-26T04:42:20.660 INFO:tasks.workunit.client.0.smithi090.stdout:9/268: getdents db/dd/d18/d38/d39 0 2024-04-26T04:42:20.661 INFO:tasks.workunit.client.0.smithi090.stdout:8/410: link dc/d1f/f36 dc/d1f/d49/d39/d55/d64/f8d 0 2024-04-26T04:42:20.662 INFO:tasks.workunit.client.0.smithi090.stdout:1/346: rename d0/d4/d7/d17/d22/f2f to d0/d4/d7/d17/d22/f68 0 2024-04-26T04:42:20.664 INFO:tasks.workunit.client.0.smithi090.stdout:4/353: rename d3/d13/d1d/l62 to d3/db/d12/d20/d24/l68 0 2024-04-26T04:42:20.664 INFO:tasks.workunit.client.0.smithi090.stdout:4/354: creat d3/db/d12/d20/d45/d53/f69 x:0 0 0 2024-04-26T04:42:20.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/347: mkdir d0/d4/d7/d69 0 2024-04-26T04:42:20.668 INFO:tasks.workunit.client.0.smithi090.stdout:9/269: write db/f35 [1212318,4432] 0 2024-04-26T04:42:20.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/466: dwrite d0/df/f25 [0,4194304] 0 2024-04-26T04:42:20.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/467: truncate d0/d4b/d5d/d5c/d8a/d45/d53/fa3 665371 0 2024-04-26T04:42:20.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/468: write d0/de/f2c [273731,61624] 0 2024-04-26T04:42:20.669 INFO:tasks.workunit.client.0.smithi090.stdout:0/469: write d0/d4b/d5d/d48/d7d/f8f [450645,107364] 0 2024-04-26T04:42:20.670 INFO:tasks.workunit.client.0.smithi090.stdout:0/470: chown d0/de/l22 2 1 2024-04-26T04:42:20.670 INFO:tasks.workunit.client.0.smithi090.stdout:0/471: write d0/d4b/d5d/d5c/d8a/d45/f9b [942876,10584] 0 2024-04-26T04:42:20.670 INFO:tasks.workunit.client.0.smithi090.stdout:4/355: mkdir d3/db/d12/d20/d2d/d58/d6a 0 2024-04-26T04:42:20.671 INFO:tasks.workunit.client.0.smithi090.stdout:1/348: unlink d0/f1c 0 2024-04-26T04:42:20.674 INFO:tasks.workunit.client.0.smithi090.stdout:6/436: dread d9/f14 [8388608,4194304] 0 2024-04-26T04:42:20.674 INFO:tasks.workunit.client.0.smithi090.stdout:6/437: readlink d9/d1c/d4d/d6d/d92/l27 0 2024-04-26T04:42:20.675 INFO:tasks.workunit.client.1.smithi104.stdout:9/600: sync 2024-04-26T04:42:20.675 INFO:tasks.workunit.client.1.smithi104.stdout:2/852: sync 2024-04-26T04:42:20.675 INFO:tasks.workunit.client.1.smithi104.stdout:7/666: sync 2024-04-26T04:42:20.676 INFO:tasks.workunit.client.1.smithi104.stdout:9/601: symlink d11/d15/d32/d4c/d7c/lc3 0 2024-04-26T04:42:20.677 INFO:tasks.workunit.client.1.smithi104.stdout:9/602: write d11/d15/d32/d35/d7b/fae [1616875,104819] 0 2024-04-26T04:42:20.677 INFO:tasks.workunit.client.1.smithi104.stdout:9/603: stat d11/l3e 0 2024-04-26T04:42:20.677 INFO:tasks.workunit.client.1.smithi104.stdout:7/667: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/dd8 0 2024-04-26T04:42:20.678 INFO:tasks.workunit.client.1.smithi104.stdout:7/668: write d2/d4/d1d/f90 [4343503,2300] 0 2024-04-26T04:42:20.678 INFO:tasks.workunit.client.1.smithi104.stdout:7/669: chown d2/d4/d32/d35/d44/c59 4164 1 2024-04-26T04:42:20.679 INFO:tasks.workunit.client.1.smithi104.stdout:9/604: link d11/d15/d1b/d6d/fc0 d11/d15/d1b/d6d/fc4 0 2024-04-26T04:42:20.679 INFO:tasks.workunit.client.1.smithi104.stdout:9/605: creat d11/d15/d32/d4c/d54/fc5 x:0 0 0 2024-04-26T04:42:20.679 INFO:tasks.workunit.client.1.smithi104.stdout:9/606: creat d11/fc6 x:0 0 0 2024-04-26T04:42:20.680 INFO:tasks.workunit.client.0.smithi090.stdout:0/472: link d0/d4b/d5d/d5c/d8a/d45/f55 d0/d4b/d5d/d5c/fa8 0 2024-04-26T04:42:20.683 INFO:tasks.workunit.client.0.smithi090.stdout:1/349: creat d0/d4/d7/d4b/d53/f6a x:0 0 0 2024-04-26T04:42:20.688 INFO:tasks.workunit.client.0.smithi090.stdout:6/438: mkdir d9/d1c/da2 0 2024-04-26T04:42:20.689 INFO:tasks.workunit.client.0.smithi090.stdout:8/411: dwrite dc/f30 [0,4194304] 0 2024-04-26T04:42:20.690 INFO:tasks.workunit.client.0.smithi090.stdout:8/412: write dc/d1f/f67 [974677,107213] 0 2024-04-26T04:42:20.690 INFO:tasks.workunit.client.0.smithi090.stdout:8/413: write dc/d1f/d49/d39/d55/f70 [859389,62] 0 2024-04-26T04:42:20.691 INFO:tasks.workunit.client.0.smithi090.stdout:6/439: creat d9/d1c/d4d/d58/d94/fa3 x:0 0 0 2024-04-26T04:42:20.692 INFO:tasks.workunit.client.1.smithi104.stdout:2/853: write d6/de/d1d/f80 [3447943,129607] 0 2024-04-26T04:42:20.692 INFO:tasks.workunit.client.1.smithi104.stdout:2/854: chown d6/d1b/d93/f9d 278397880 1 2024-04-26T04:42:20.692 INFO:tasks.workunit.client.0.smithi090.stdout:0/473: mknod d0/d4b/d5d/ca9 0 2024-04-26T04:42:20.695 INFO:tasks.workunit.client.0.smithi090.stdout:6/440: creat d9/d1c/d31/d69/fa4 x:0 0 0 2024-04-26T04:42:20.695 INFO:tasks.workunit.client.0.smithi090.stdout:6/441: chown d9/d21 0 1 2024-04-26T04:42:20.696 INFO:tasks.workunit.client.0.smithi090.stdout:0/474: creat d0/d4b/d5d/d5c/d8a/d45/d93/faa x:0 0 0 2024-04-26T04:42:20.698 INFO:tasks.workunit.client.0.smithi090.stdout:6/442: creat d9/d21/d40/d4c/d6f/fa5 x:0 0 0 2024-04-26T04:42:20.700 INFO:tasks.workunit.client.0.smithi090.stdout:6/443: symlink d9/d1c/da2/la6 0 2024-04-26T04:42:20.700 INFO:tasks.workunit.client.0.smithi090.stdout:6/444: stat d9/d1c/d4d/d6d/c5b 0 2024-04-26T04:42:20.702 INFO:tasks.workunit.client.0.smithi090.stdout:6/445: rename d9/f23 to d9/d21/d40/d4c/d6f/fa7 0 2024-04-26T04:42:20.703 INFO:tasks.workunit.client.1.smithi104.stdout:2/855: dread d6/f62 [0,4194304] 0 2024-04-26T04:42:20.704 INFO:tasks.workunit.client.1.smithi104.stdout:2/856: read d6/d1b/d51/f58 [343623,97817] 0 2024-04-26T04:42:20.709 INFO:tasks.workunit.client.0.smithi090.stdout:8/414: dread dc/d1f/d49/d39/d55/d64/f3e [4194304,4194304] 0 2024-04-26T04:42:20.711 INFO:tasks.workunit.client.0.smithi090.stdout:8/415: mknod dc/d1f/d49/d39/d55/c8e 0 2024-04-26T04:42:20.712 INFO:tasks.workunit.client.0.smithi090.stdout:3/285: truncate d1/d59/d5e/f62 98177 0 2024-04-26T04:42:20.713 INFO:tasks.workunit.client.0.smithi090.stdout:3/286: dread - d1/d17/d36/f65 zero size 2024-04-26T04:42:20.713 INFO:tasks.workunit.client.0.smithi090.stdout:4/356: dwrite d3/db/d12/d20/d24/d30/f5a [0,4194304] 0 2024-04-26T04:42:20.713 INFO:tasks.workunit.client.1.smithi104.stdout:7/670: dwrite d2/d4/d1d/f90 [0,4194304] 0 2024-04-26T04:42:20.713 INFO:tasks.workunit.client.1.smithi104.stdout:7/671: getdents d2/d4/d32/d3d/d47/db1 0 2024-04-26T04:42:20.714 INFO:tasks.workunit.client.0.smithi090.stdout:7/416: dwrite d0/db/d15/d30/f32 [0,4194304] 0 2024-04-26T04:42:20.714 INFO:tasks.workunit.client.0.smithi090.stdout:7/417: write d0/db/d15/d30/d4f/f76 [1043650,94408] 0 2024-04-26T04:42:20.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/416: rename dc/d5a/d5d/c7d to dc/c8f 0 2024-04-26T04:42:20.716 INFO:tasks.workunit.client.1.smithi104.stdout:7/672: symlink d2/d4/d32/d3d/d41/d5f/d6d/ld9 0 2024-04-26T04:42:20.721 INFO:tasks.workunit.client.0.smithi090.stdout:4/357: write d3/db/d12/d20/d24/d30/f49 [3369476,115813] 0 2024-04-26T04:42:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:20 smithi090 ceph-mon[26648]: pgmap v252: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 177 MiB/s rd, 227 MiB/s wr, 632 op/s 2024-04-26T04:42:20.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/270: truncate db/dd/d18/d31/f4e 442104 0 2024-04-26T04:42:20.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/271: write db/dd/d18/d57/f4f [520579,30069] 0 2024-04-26T04:42:20.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/272: chown db/dd/d18/d31/f4a 51 1 2024-04-26T04:42:20.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/273: write db/f3d [4267158,13050] 0 2024-04-26T04:42:20.722 INFO:tasks.workunit.client.0.smithi090.stdout:4/358: dread d3/db/d12/f28 [0,4194304] 0 2024-04-26T04:42:20.723 INFO:tasks.workunit.client.0.smithi090.stdout:3/287: link d1/d17/d36/f65 d1/d17/d36/d2a/d3a/f66 0 2024-04-26T04:42:20.726 INFO:tasks.workunit.client.0.smithi090.stdout:6/446: dwrite d9/f15 [0,4194304] 0 2024-04-26T04:42:20.726 INFO:tasks.workunit.client.0.smithi090.stdout:6/447: fsync d9/d1c/d31/d79/f7e 0 2024-04-26T04:42:20.734 INFO:tasks.workunit.client.1.smithi104.stdout:2/857: dwrite d6/d1b/d93/fc4 [0,4194304] 0 2024-04-26T04:42:20.738 INFO:tasks.workunit.client.1.smithi104.stdout:2/858: rmdir d6/d1b/d51 39 2024-04-26T04:42:20.738 INFO:tasks.workunit.client.1.smithi104.stdout:2/859: mknod d6/d1b/d51/dec/c117 0 2024-04-26T04:42:20.738 INFO:tasks.workunit.client.1.smithi104.stdout:2/860: creat d6/d1b/d68/d81/f118 x:0 0 0 2024-04-26T04:42:20.740 INFO:tasks.workunit.client.1.smithi104.stdout:2/861: creat d6/d1b/d68/d81/dfc/f119 x:0 0 0 2024-04-26T04:42:20.741 INFO:tasks.workunit.client.1.smithi104.stdout:2/862: write d6/de/d1d/f8f [371570,38203] 0 2024-04-26T04:42:20.743 INFO:tasks.workunit.client.0.smithi090.stdout:6/448: dwrite d9/d1c/d4d/d6d/d92/f26 [0,4194304] 0 2024-04-26T04:42:20.750 INFO:tasks.workunit.client.0.smithi090.stdout:6/449: read d9/d1c/d31/d79/f7e [3529716,62306] 0 2024-04-26T04:42:20.750 INFO:tasks.workunit.client.0.smithi090.stdout:6/450: chown d9/d21/d3b/l99 3624202 1 2024-04-26T04:42:20.753 INFO:tasks.workunit.client.0.smithi090.stdout:0/475: rmdir d0/d4b/d5d/d5c/d8a/d45/d93 39 2024-04-26T04:42:20.756 INFO:tasks.workunit.client.0.smithi090.stdout:7/418: creat d0/db/d15/d30/f9b x:0 0 0 2024-04-26T04:42:20.756 INFO:tasks.workunit.client.0.smithi090.stdout:7/419: fdatasync d0/db/d53/f16 0 2024-04-26T04:42:20.756 INFO:tasks.workunit.client.0.smithi090.stdout:8/417: mknod dc/d34/c90 0 2024-04-26T04:42:20.757 INFO:tasks.workunit.client.0.smithi090.stdout:4/359: creat d3/d13/d64/f6b x:0 0 0 2024-04-26T04:42:20.759 INFO:tasks.workunit.client.0.smithi090.stdout:3/288: rename d1/f29 to d1/d17/d36/f67 0 2024-04-26T04:42:20.761 INFO:tasks.workunit.client.0.smithi090.stdout:9/274: dwrite db/dd/d18/d31/f52 [0,4194304] 0 2024-04-26T04:42:20.761 INFO:tasks.workunit.client.0.smithi090.stdout:9/275: chown db 2 1 2024-04-26T04:42:20.761 INFO:tasks.workunit.client.0.smithi090.stdout:7/420: unlink d0/d1/d4/f47 0 2024-04-26T04:42:20.761 INFO:tasks.workunit.client.0.smithi090.stdout:7/421: write d0/db/d15/d30/f9b [940062,110871] 0 2024-04-26T04:42:20.761 INFO:tasks.workunit.client.0.smithi090.stdout:7/422: write d0/db/d53/f2d [4355147,95818] 0 2024-04-26T04:42:20.762 INFO:tasks.workunit.client.1.smithi104.stdout:2/863: dwrite d6/d1b/fe4 [0,4194304] 0 2024-04-26T04:42:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:20 smithi104 ceph-mon[32369]: pgmap v252: 65 pgs: 65 active+clean; 1.6 GiB data, 4.6 GiB used, 532 GiB / 536 GiB avail; 177 MiB/s rd, 227 MiB/s wr, 632 op/s 2024-04-26T04:42:20.763 INFO:tasks.workunit.client.0.smithi090.stdout:4/360: mkdir d3/d13/d6c 0 2024-04-26T04:42:20.763 INFO:tasks.workunit.client.0.smithi090.stdout:4/361: creat d3/f6d x:0 0 0 2024-04-26T04:42:20.764 INFO:tasks.workunit.client.1.smithi104.stdout:2/864: unlink d6/d1b/d68/d81/dce/lf1 0 2024-04-26T04:42:20.765 INFO:tasks.workunit.client.1.smithi104.stdout:2/865: rename d6/d1b/d68/d81/d97/lef to d6/d1b/d68/d81/dfc/l11a 0 2024-04-26T04:42:20.766 INFO:tasks.workunit.client.0.smithi090.stdout:6/451: rename d9/d1c/d4d/f85 to d9/d1c/d4d/d6d/d92/fa8 0 2024-04-26T04:42:20.766 INFO:tasks.workunit.client.0.smithi090.stdout:6/452: read - d9/d21/d40/d4c/d6f/fa5 zero size 2024-04-26T04:42:20.766 INFO:tasks.workunit.client.0.smithi090.stdout:6/453: write d9/d21/d3b/f76 [458914,97315] 0 2024-04-26T04:42:20.768 INFO:tasks.workunit.client.0.smithi090.stdout:3/289: mkdir d1/d17/d36/d33/d68 0 2024-04-26T04:42:20.768 INFO:tasks.workunit.client.0.smithi090.stdout:3/290: chown d1/d17/d36/c26 22 1 2024-04-26T04:42:20.771 INFO:tasks.workunit.client.0.smithi090.stdout:9/276: unlink db/dd/d18/c51 0 2024-04-26T04:42:20.775 INFO:tasks.workunit.client.0.smithi090.stdout:4/362: link d3/db/d12/d20/d2d/d2e/f35 d3/db/d12/d20/d45/d53/f6e 0 2024-04-26T04:42:20.777 INFO:tasks.workunit.client.0.smithi090.stdout:7/423: write d0/db/d15/d29/f44 [4173253,58163] 0 2024-04-26T04:42:20.777 INFO:tasks.workunit.client.0.smithi090.stdout:8/418: dwrite dc/d1f/d49/d39/d55/f6c [0,4194304] 0 2024-04-26T04:42:20.777 INFO:tasks.workunit.client.0.smithi090.stdout:8/419: chown dc/d1f/f35 336 1 2024-04-26T04:42:20.778 INFO:tasks.workunit.client.0.smithi090.stdout:0/476: rename d0/d4b/d5d/d5c/d8a/d45/d84 to d0/d4b/d5d/d5c/d8a/d45/d53/dab 0 2024-04-26T04:42:20.778 INFO:tasks.workunit.client.0.smithi090.stdout:0/477: readlink d0/df/l57 0 2024-04-26T04:42:20.779 INFO:tasks.workunit.client.0.smithi090.stdout:6/454: mknod d9/d21/d3b/ca9 0 2024-04-26T04:42:20.780 INFO:tasks.workunit.client.0.smithi090.stdout:3/291: symlink d1/d17/d36/d33/d43/l69 0 2024-04-26T04:42:20.781 INFO:tasks.workunit.client.0.smithi090.stdout:9/277: truncate db/dd/d44/f27 4120292 0 2024-04-26T04:42:20.782 INFO:tasks.workunit.client.0.smithi090.stdout:4/363: creat d3/db/d12/d20/d2d/f6f x:0 0 0 2024-04-26T04:42:20.787 INFO:tasks.workunit.client.1.smithi104.stdout:2/866: dwrite d6/f8b [0,4194304] 0 2024-04-26T04:42:20.787 INFO:tasks.workunit.client.0.smithi090.stdout:5/463: sync 2024-04-26T04:42:20.788 INFO:tasks.workunit.client.0.smithi090.stdout:2/404: sync 2024-04-26T04:42:20.790 INFO:tasks.workunit.client.1.smithi104.stdout:2/867: unlink d6/d1b/d51/d6a/cee 0 2024-04-26T04:42:20.790 INFO:tasks.workunit.client.0.smithi090.stdout:8/420: dread dc/d1f/d49/d39/d55/d64/f44 [0,4194304] 0 2024-04-26T04:42:20.794 INFO:tasks.workunit.client.0.smithi090.stdout:7/424: truncate d0/db/df/d1c/d23/f26 111445 0 2024-04-26T04:42:20.794 INFO:tasks.workunit.client.0.smithi090.stdout:7/425: write d0/d1/d4/d98/f93 [531566,61684] 0 2024-04-26T04:42:20.794 INFO:tasks.workunit.client.1.smithi104.stdout:0/587: sync 2024-04-26T04:42:20.794 INFO:tasks.workunit.client.1.smithi104.stdout:0/588: write d0/d2/d21/d38/f4f [625942,19325] 0 2024-04-26T04:42:20.794 INFO:tasks.workunit.client.1.smithi104.stdout:0/589: chown d0/d2/d21/d49 18 1 2024-04-26T04:42:20.796 INFO:tasks.workunit.client.0.smithi090.stdout:0/478: mkdir d0/d4b/d5d/d5c/d8a/d45/d93/dac 0 2024-04-26T04:42:20.796 INFO:tasks.workunit.client.0.smithi090.stdout:0/479: creat d0/d4b/fad x:0 0 0 2024-04-26T04:42:20.796 INFO:tasks.workunit.client.1.smithi104.stdout:0/590: truncate d0/d2/d21/d38/f34 1624004 0 2024-04-26T04:42:20.796 INFO:tasks.workunit.client.1.smithi104.stdout:0/591: readlink d0/d2/d21/lac 0 2024-04-26T04:42:20.797 INFO:tasks.workunit.client.1.smithi104.stdout:0/592: creat d0/d2/d10/d84/fb6 x:0 0 0 2024-04-26T04:42:20.797 INFO:tasks.workunit.client.1.smithi104.stdout:0/593: truncate d0/d2/d21/d27/d1e/d57/d6c/d85/fa2 37850 0 2024-04-26T04:42:20.797 INFO:tasks.workunit.client.1.smithi104.stdout:0/594: truncate d0/d2/d21/d27/f83 284842 0 2024-04-26T04:42:20.799 INFO:tasks.workunit.client.0.smithi090.stdout:6/455: creat d9/d1c/da2/faa x:0 0 0 2024-04-26T04:42:20.800 INFO:tasks.workunit.client.0.smithi090.stdout:2/405: dread db/d59/f62 [0,4194304] 0 2024-04-26T04:42:20.800 INFO:tasks.workunit.client.0.smithi090.stdout:2/406: write db/d11/d14/f4a [442476,125462] 0 2024-04-26T04:42:20.800 INFO:tasks.workunit.client.0.smithi090.stdout:2/407: write db/d11/d14/d25/f4e [1737909,54167] 0 2024-04-26T04:42:20.800 INFO:tasks.workunit.client.0.smithi090.stdout:2/408: chown db/f76 3790 1 2024-04-26T04:42:20.801 INFO:tasks.workunit.client.0.smithi090.stdout:3/292: rmdir d1/d17/d52 39 2024-04-26T04:42:20.803 INFO:tasks.workunit.client.0.smithi090.stdout:9/278: mkdir db/d5f 0 2024-04-26T04:42:20.805 INFO:tasks.workunit.client.0.smithi090.stdout:7/426: dread d0/db/d15/d30/d4f/f76 [0,4194304] 0 2024-04-26T04:42:20.806 INFO:tasks.workunit.client.0.smithi090.stdout:4/364: mknod d3/d13/d1d/c70 0 2024-04-26T04:42:20.807 INFO:tasks.workunit.client.1.smithi104.stdout:0/595: dread d0/d2/d21/d27/f30 [0,4194304] 0 2024-04-26T04:42:20.808 INFO:tasks.workunit.client.0.smithi090.stdout:5/464: symlink d4/d1a/d2a/d79/l92 0 2024-04-26T04:42:20.808 INFO:tasks.workunit.client.0.smithi090.stdout:5/465: stat d4/d1a/d1f/d53/d59 0 2024-04-26T04:42:20.810 INFO:tasks.workunit.client.0.smithi090.stdout:6/456: mkdir d9/d1c/d4d/d58/dab 0 2024-04-26T04:42:20.810 INFO:tasks.workunit.client.0.smithi090.stdout:6/457: dread - d9/d1c/d31/d69/fa4 zero size 2024-04-26T04:42:20.813 INFO:tasks.workunit.client.0.smithi090.stdout:2/409: link db/d11/d14/d1b/d20/c3b db/dc/d6f/c89 0 2024-04-26T04:42:20.815 INFO:tasks.workunit.client.0.smithi090.stdout:9/279: creat db/dd/f60 x:0 0 0 2024-04-26T04:42:20.816 INFO:tasks.workunit.client.0.smithi090.stdout:7/427: link d0/d1/d4/l5b d0/d1/d4/d98/d4a/l9c 0 2024-04-26T04:42:20.817 INFO:tasks.workunit.client.0.smithi090.stdout:3/293: dread d1/d6/d1b/f35 [0,4194304] 0 2024-04-26T04:42:20.818 INFO:tasks.workunit.client.0.smithi090.stdout:4/365: creat d3/db/d12/d20/d2d/d58/d6a/f71 x:0 0 0 2024-04-26T04:42:20.819 INFO:tasks.workunit.client.1.smithi104.stdout:2/868: dwrite d6/d1b/d68/d81/d97/d99/fa8 [0,4194304] 0 2024-04-26T04:42:20.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/466: mknod d4/d1a/d2a/d79/d90/d56/d80/c93 0 2024-04-26T04:42:20.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/467: fsync d4/d1a/d2a/d79/d90/d56/f75 0 2024-04-26T04:42:20.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/468: truncate d4/d1a/df/d4a/f89 399410 0 2024-04-26T04:42:20.822 INFO:tasks.workunit.client.0.smithi090.stdout:5/469: fdatasync d4/d1a/df/d4a/f55 0 2024-04-26T04:42:20.822 INFO:tasks.workunit.client.0.smithi090.stdout:7/428: dread d0/db/df/d1c/d23/f31 [0,4194304] 0 2024-04-26T04:42:20.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/410: symlink db/d11/d14/d25/d86/l8a 0 2024-04-26T04:42:20.824 INFO:tasks.workunit.client.1.smithi104.stdout:2/869: rename d6/d1b/d68/d81/d97/d99 to d6/d1b/d51/d6a/d11b 0 2024-04-26T04:42:20.824 INFO:tasks.workunit.client.1.smithi104.stdout:2/870: getdents d6/d1b/d68/d105 0 2024-04-26T04:42:20.826 INFO:tasks.workunit.client.1.smithi104.stdout:2/871: rmdir d6 39 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.0.smithi090.stdout:4/366: dread d3/db/d12/d20/d24/f52 [0,4194304] 0 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.0.smithi090.stdout:9/280: mknod db/dd/d18/d57/c61 0 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.0.smithi090.stdout:5/470: mkdir d4/d1a/d2a/d79/d90/d94 0 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.0.smithi090.stdout:5/471: chown d4/d1a/d2a/d79/d90/c6e 254177493 1 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.0.smithi090.stdout:9/281: write db/f35 [2977605,34782] 0 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.0.smithi090.stdout:5/472: write d4/d1a/df/d36/d61/f87 [41898,84243] 0 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.0.smithi090.stdout:8/421: dwrite dc/d5a/f6a [0,4194304] 0 2024-04-26T04:42:20.828 INFO:tasks.workunit.client.1.smithi104.stdout:2/872: symlink d6/d8/d5a/dd1/l11c 0 2024-04-26T04:42:20.832 INFO:tasks.workunit.client.1.smithi104.stdout:2/873: link f2 d6/d1b/d68/d81/dce/f11d 0 2024-04-26T04:42:20.832 INFO:tasks.workunit.client.1.smithi104.stdout:2/874: chown d6/d1b/d68/d81/d97/f9a 492361 1 2024-04-26T04:42:20.832 INFO:tasks.workunit.client.0.smithi090.stdout:4/367: mkdir d3/db/d12/d20/d24/d43/d72 0 2024-04-26T04:42:20.832 INFO:tasks.workunit.client.0.smithi090.stdout:3/294: dread d1/f3c [0,4194304] 0 2024-04-26T04:42:20.832 INFO:tasks.workunit.client.0.smithi090.stdout:9/282: creat db/dd/d18/d31/d58/f62 x:0 0 0 2024-04-26T04:42:20.833 INFO:tasks.workunit.client.0.smithi090.stdout:3/295: write d1/d17/d36/d2a/d3a/f66 [346745,93751] 0 2024-04-26T04:42:20.833 INFO:tasks.workunit.client.1.smithi104.stdout:2/875: symlink d6/d1b/d68/d105/l11e 0 2024-04-26T04:42:20.833 INFO:tasks.workunit.client.1.smithi104.stdout:2/876: creat d6/d1b/d68/d81/f11f x:0 0 0 2024-04-26T04:42:20.833 INFO:tasks.workunit.client.0.smithi090.stdout:5/473: truncate d4/d1a/d3f/f65 46673 0 2024-04-26T04:42:20.833 INFO:tasks.workunit.client.0.smithi090.stdout:5/474: write d4/d1a/df/d72/f77 [977528,109525] 0 2024-04-26T04:42:20.834 INFO:tasks.workunit.client.0.smithi090.stdout:5/475: write d4/f1e [9275145,55117] 0 2024-04-26T04:42:20.834 INFO:tasks.workunit.client.0.smithi090.stdout:1/350: sync 2024-04-26T04:42:20.836 INFO:tasks.workunit.client.0.smithi090.stdout:8/422: symlink dc/d34/l91 0 2024-04-26T04:42:20.836 INFO:tasks.workunit.client.0.smithi090.stdout:8/423: chown dc/d1f/d49/d4f/d58 225767072 1 2024-04-26T04:42:20.836 INFO:tasks.workunit.client.0.smithi090.stdout:8/424: chown dc/d5a/l76 412221834 1 2024-04-26T04:42:20.837 INFO:tasks.workunit.client.0.smithi090.stdout:4/368: symlink d3/db/d12/d20/d2d/d2e/l73 0 2024-04-26T04:42:20.838 INFO:tasks.workunit.client.0.smithi090.stdout:4/369: write d3/db/d12/d20/d45/d53/f6e [1014614,38206] 0 2024-04-26T04:42:20.838 INFO:tasks.workunit.client.0.smithi090.stdout:4/370: fsync d3/db/d12/f28 0 2024-04-26T04:42:20.840 INFO:tasks.workunit.client.0.smithi090.stdout:5/476: mkdir d4/d1a/d1f/d53/d59/d95 0 2024-04-26T04:42:20.843 INFO:tasks.workunit.client.0.smithi090.stdout:4/371: mkdir d3/db/d12/d20/d2d/d58/d6a/d74 0 2024-04-26T04:42:20.844 INFO:tasks.workunit.client.0.smithi090.stdout:5/477: creat d4/d1a/d2a/d79/d90/f96 x:0 0 0 2024-04-26T04:42:20.845 INFO:tasks.workunit.client.1.smithi104.stdout:0/596: dwrite d0/d4/f29 [0,4194304] 0 2024-04-26T04:42:20.847 INFO:tasks.workunit.client.0.smithi090.stdout:5/478: creat d4/d1a/d45/f97 x:0 0 0 2024-04-26T04:42:20.847 INFO:tasks.workunit.client.0.smithi090.stdout:5/479: readlink d4/d1a/d3f/l2e 0 2024-04-26T04:42:20.847 INFO:tasks.workunit.client.0.smithi090.stdout:6/458: dwrite d9/d1c/d31/f41 [0,4194304] 0 2024-04-26T04:42:20.849 INFO:tasks.workunit.client.0.smithi090.stdout:2/411: dread db/d11/d14/d25/d2d/f6a [0,4194304] 0 2024-04-26T04:42:20.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/296: dread d1/d9/fa [0,4194304] 0 2024-04-26T04:42:20.849 INFO:tasks.workunit.client.0.smithi090.stdout:3/297: creat d1/d17/d36/f6a x:0 0 0 2024-04-26T04:42:20.853 INFO:tasks.workunit.client.0.smithi090.stdout:5/480: mknod d4/d1a/df/d72/c98 0 2024-04-26T04:42:20.854 INFO:tasks.workunit.client.0.smithi090.stdout:3/298: dread d1/d6/d2c/f49 [0,4194304] 0 2024-04-26T04:42:20.856 INFO:tasks.workunit.client.0.smithi090.stdout:0/480: dwrite d0/d4b/d5d/d5c/d8a/f13 [0,4194304] 0 2024-04-26T04:42:20.857 INFO:tasks.workunit.client.0.smithi090.stdout:1/351: dread d0/d4/d7/d17/d22/d31/f34 [0,4194304] 0 2024-04-26T04:42:20.857 INFO:tasks.workunit.client.0.smithi090.stdout:1/352: chown d0/d4/d43/d4d/d58 16309824 1 2024-04-26T04:42:20.857 INFO:tasks.workunit.client.0.smithi090.stdout:1/353: write d0/d4/d7/d17/f18 [2104397,15383] 0 2024-04-26T04:42:20.857 INFO:tasks.workunit.client.0.smithi090.stdout:1/354: fdatasync d0/d4/d7/d17/d22/d31/f39 0 2024-04-26T04:42:20.857 INFO:tasks.workunit.client.0.smithi090.stdout:2/412: mkdir db/d11/d14/d27/d2f/d70/d8b 0 2024-04-26T04:42:20.857 INFO:tasks.workunit.client.0.smithi090.stdout:2/413: fsync db/d11/d14/d27/d28/f56 0 2024-04-26T04:42:20.859 INFO:tasks.workunit.client.0.smithi090.stdout:5/481: getdents d4/d1a/d2a/d79/d90/d49 0 2024-04-26T04:42:20.860 INFO:tasks.workunit.client.0.smithi090.stdout:6/459: getdents d9/d1c/d31/d69 0 2024-04-26T04:42:20.861 INFO:tasks.workunit.client.0.smithi090.stdout:2/414: dread db/d11/d14/d25/f49 [0,4194304] 0 2024-04-26T04:42:20.863 INFO:tasks.workunit.client.0.smithi090.stdout:3/299: mknod d1/d17/d36/c6b 0 2024-04-26T04:42:20.865 INFO:tasks.workunit.client.1.smithi104.stdout:2/877: dwrite d6/de/d1d/f42 [0,4194304] 0 2024-04-26T04:42:20.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/283: dwrite db/dd/d18/d31/f3c [0,4194304] 0 2024-04-26T04:42:20.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/284: fdatasync db/f35 0 2024-04-26T04:42:20.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/285: chown db/f13 110436 1 2024-04-26T04:42:20.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/286: truncate db/f41 619797 0 2024-04-26T04:42:20.866 INFO:tasks.workunit.client.1.smithi104.stdout:0/597: dwrite d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fa9 [0,4194304] 0 2024-04-26T04:42:20.867 INFO:tasks.workunit.client.1.smithi104.stdout:0/598: readlink d0/d4/d3b/l98 0 2024-04-26T04:42:20.873 INFO:tasks.workunit.client.1.smithi104.stdout:0/599: mkdir d0/d2/d10/d3d/db7 0 2024-04-26T04:42:20.873 INFO:tasks.workunit.client.1.smithi104.stdout:0/600: chown d0/d2/d21/d38/f34 354487 1 2024-04-26T04:42:20.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/481: rename d0/l9d to d0/d4b/d5d/d5c/d8a/d45/d53/d8e/lae 0 2024-04-26T04:42:20.876 INFO:tasks.workunit.client.0.smithi090.stdout:5/482: symlink d4/d1a/d1f/d53/l99 0 2024-04-26T04:42:20.876 INFO:tasks.workunit.client.0.smithi090.stdout:5/483: read d4/f13 [46516,13656] 0 2024-04-26T04:42:20.877 INFO:tasks.workunit.client.1.smithi104.stdout:0/601: rename d0/d2/d10/la5 to d0/d2/d21/d49/d5e/d9c/d53/d5d/lb8 0 2024-04-26T04:42:20.881 INFO:tasks.workunit.client.0.smithi090.stdout:3/300: mknod d1/d17/d36/d2a/d3a/d5d/c6c 0 2024-04-26T04:42:20.881 INFO:tasks.workunit.client.0.smithi090.stdout:3/301: chown d1/d9/fb 40 1 2024-04-26T04:42:20.881 INFO:tasks.workunit.client.0.smithi090.stdout:3/302: truncate d1/d17/f4f 155411 0 2024-04-26T04:42:20.884 INFO:tasks.workunit.client.0.smithi090.stdout:9/287: write db/f1b [2707688,129523] 0 2024-04-26T04:42:20.884 INFO:tasks.workunit.client.0.smithi090.stdout:9/288: readlink db/d16/l1f 0 2024-04-26T04:42:20.884 INFO:tasks.workunit.client.0.smithi090.stdout:9/289: write db/f1b [1373566,37262] 0 2024-04-26T04:42:20.886 INFO:tasks.workunit.client.1.smithi104.stdout:3/741: sync 2024-04-26T04:42:20.887 INFO:tasks.workunit.client.0.smithi090.stdout:1/355: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:20.887 INFO:tasks.workunit.client.0.smithi090.stdout:5/484: rename d4/f13 to d4/d1a/d2a/d79/d90/d49/f9a 0 2024-04-26T04:42:20.888 INFO:tasks.workunit.client.0.smithi090.stdout:3/303: dread d1/f27 [0,4194304] 0 2024-04-26T04:42:20.888 INFO:tasks.workunit.client.1.smithi104.stdout:3/742: symlink d5/d9/d80/dbc/l107 0 2024-04-26T04:42:20.888 INFO:tasks.workunit.client.1.smithi104.stdout:3/743: chown d5/db2/da3/dd0/ddb/d32/d48/c105 429923594 1 2024-04-26T04:42:20.889 INFO:tasks.workunit.client.1.smithi104.stdout:3/744: creat d5/db2/d46/d95/f108 x:0 0 0 2024-04-26T04:42:20.890 INFO:tasks.workunit.client.0.smithi090.stdout:2/415: dwrite db/d11/d14/d25/d2d/d48/d5b/f7d [0,4194304] 0 2024-04-26T04:42:20.890 INFO:tasks.workunit.client.0.smithi090.stdout:2/416: chown db 13 1 2024-04-26T04:42:20.891 INFO:tasks.workunit.client.0.smithi090.stdout:1/356: mkdir d0/d4/d7/d4b/d53/d6b 0 2024-04-26T04:42:20.892 INFO:tasks.workunit.client.1.smithi104.stdout:0/602: dread d0/d2/d21/d38/f36 [0,4194304] 0 2024-04-26T04:42:20.893 INFO:tasks.workunit.client.0.smithi090.stdout:5/485: truncate d4/d1a/d2a/d79/d90/d49/f82 3530245 0 2024-04-26T04:42:20.895 INFO:tasks.workunit.client.0.smithi090.stdout:3/304: creat d1/d59/f6d x:0 0 0 2024-04-26T04:42:20.895 INFO:tasks.workunit.client.0.smithi090.stdout:6/460: dwrite d9/d21/d3b/f46 [0,4194304] 0 2024-04-26T04:42:20.895 INFO:tasks.workunit.client.0.smithi090.stdout:6/461: chown d9/d21/d40/d4c/f90 12025 1 2024-04-26T04:42:20.895 INFO:tasks.workunit.client.0.smithi090.stdout:6/462: truncate d9/d21/d40/d4c/f7a 4181727 0 2024-04-26T04:42:20.895 INFO:tasks.workunit.client.0.smithi090.stdout:6/463: chown d9/d1c/d31/f54 2288 1 2024-04-26T04:42:20.895 INFO:tasks.workunit.client.1.smithi104.stdout:0/603: rename d0/d2/d10/d84/lb5 to d0/d2/da0/lb9 0 2024-04-26T04:42:20.897 INFO:tasks.workunit.client.1.smithi104.stdout:0/604: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dba 0 2024-04-26T04:42:20.897 INFO:tasks.workunit.client.0.smithi090.stdout:2/417: mkdir db/d11/d14/d25/d2d/d48/d81/d8c 0 2024-04-26T04:42:20.898 INFO:tasks.workunit.client.1.smithi104.stdout:0/605: mknod d0/d4/cbb 0 2024-04-26T04:42:20.898 INFO:tasks.workunit.client.0.smithi090.stdout:2/418: dread db/d11/d14/d27/d2f/d42/d5c/f4d [0,4194304] 0 2024-04-26T04:42:20.899 INFO:tasks.workunit.client.0.smithi090.stdout:1/357: unlink d0/d4/d7/d17/d22/d31/l4e 0 2024-04-26T04:42:20.899 INFO:tasks.workunit.client.1.smithi104.stdout:0/606: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fbc x:0 0 0 2024-04-26T04:42:20.900 INFO:tasks.workunit.client.0.smithi090.stdout:3/305: dread d1/d9/fa [0,4194304] 0 2024-04-26T04:42:20.900 INFO:tasks.workunit.client.0.smithi090.stdout:5/486: mkdir d4/d1a/df/d36/d61/d9b 0 2024-04-26T04:42:20.901 INFO:tasks.workunit.client.1.smithi104.stdout:0/607: rename d0/d2/d21/d38/f4f to d0/d2/d21/d27/d1e/d57/d6c/d85/fbd 0 2024-04-26T04:42:20.902 INFO:tasks.workunit.client.1.smithi104.stdout:3/745: dread f4 [0,4194304] 0 2024-04-26T04:42:20.903 INFO:tasks.workunit.client.0.smithi090.stdout:1/358: creat d0/d4/d7/d17/d22/d31/f6c x:0 0 0 2024-04-26T04:42:20.903 INFO:tasks.workunit.client.1.smithi104.stdout:0/608: getdents d0/d2/d21/d27/d1e 0 2024-04-26T04:42:20.904 INFO:tasks.workunit.client.1.smithi104.stdout:3/746: creat d5/db2/da3/dd0/ddb/f109 x:0 0 0 2024-04-26T04:42:20.904 INFO:tasks.workunit.client.0.smithi090.stdout:3/306: creat d1/d5c/f6e x:0 0 0 2024-04-26T04:42:20.905 INFO:tasks.workunit.client.0.smithi090.stdout:1/359: dread d0/d4/d43/f47 [0,4194304] 0 2024-04-26T04:42:20.905 INFO:tasks.workunit.client.1.smithi104.stdout:0/609: mkdir d0/d2/d21/d49/d5e/d9c/dbe 0 2024-04-26T04:42:20.905 INFO:tasks.workunit.client.1.smithi104.stdout:0/610: write d0/d2/d21/d27/faa [891974,52137] 0 2024-04-26T04:42:20.905 INFO:tasks.workunit.client.1.smithi104.stdout:3/747: creat d5/db2/da3/dd0/ddb/d69/f10a x:0 0 0 2024-04-26T04:42:20.905 INFO:tasks.workunit.client.1.smithi104.stdout:3/748: fsync d5/d9/dd5/ff1 0 2024-04-26T04:42:20.906 INFO:tasks.workunit.client.1.smithi104.stdout:0/611: creat d0/d2/d21/d49/d5e/d9c/dbe/fbf x:0 0 0 2024-04-26T04:42:20.907 INFO:tasks.workunit.client.1.smithi104.stdout:3/749: symlink d5/db2/da3/dd0/ddb/d32/l10b 0 2024-04-26T04:42:20.907 INFO:tasks.workunit.client.1.smithi104.stdout:3/750: write f3 [1838891,87466] 0 2024-04-26T04:42:20.907 INFO:tasks.workunit.client.1.smithi104.stdout:3/751: fsync d5/d9/f26 0 2024-04-26T04:42:20.907 INFO:tasks.workunit.client.1.smithi104.stdout:2/878: dwrite d6/d8/f5b [0,4194304] 0 2024-04-26T04:42:20.908 INFO:tasks.workunit.client.1.smithi104.stdout:0/612: mkdir d0/d2/d10/d3d/dc0 0 2024-04-26T04:42:20.909 INFO:tasks.workunit.client.0.smithi090.stdout:0/482: dwrite d0/d4b/f8c [0,4194304] 0 2024-04-26T04:42:20.909 INFO:tasks.workunit.client.1.smithi104.stdout:2/879: creat d6/d1b/d51/f120 x:0 0 0 2024-04-26T04:42:20.909 INFO:tasks.workunit.client.1.smithi104.stdout:2/880: truncate d6/d1b/d75/f100 929750 0 2024-04-26T04:42:20.910 INFO:tasks.workunit.client.1.smithi104.stdout:2/881: creat d6/d1b/f121 x:0 0 0 2024-04-26T04:42:20.910 INFO:tasks.workunit.client.0.smithi090.stdout:3/307: mknod d1/d59/d5e/c6f 0 2024-04-26T04:42:20.911 INFO:tasks.workunit.client.1.smithi104.stdout:0/613: mkdir d0/d2/d10/d3d/db7/dc1 0 2024-04-26T04:42:20.912 INFO:tasks.workunit.client.0.smithi090.stdout:1/360: unlink d0/d4/d7/l3b 0 2024-04-26T04:42:20.912 INFO:tasks.workunit.client.0.smithi090.stdout:1/361: write d0/d4/d7/d17/f3c [1947723,89544] 0 2024-04-26T04:42:20.913 INFO:tasks.workunit.client.0.smithi090.stdout:9/290: dwrite db/dd/d18/d57/f4f [0,4194304] 0 2024-04-26T04:42:20.913 INFO:tasks.workunit.client.0.smithi090.stdout:9/291: write db/d16/f5a [1530616,70483] 0 2024-04-26T04:42:20.914 INFO:tasks.workunit.client.1.smithi104.stdout:2/882: rename d6/d8/d3e/f2a to d6/d1b/d68/d81/dfc/f122 0 2024-04-26T04:42:20.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/483: dread d0/df/f67 [0,4194304] 0 2024-04-26T04:42:20.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/484: creat d0/d4b/d5d/d5c/d8a/d45/d53/faf x:0 0 0 2024-04-26T04:42:20.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/485: dread - d0/d4b/d5d/f6a zero size 2024-04-26T04:42:20.915 INFO:tasks.workunit.client.0.smithi090.stdout:0/486: fdatasync d0/d4b/d5d/d5c/f9c 0 2024-04-26T04:42:20.917 INFO:tasks.workunit.client.0.smithi090.stdout:9/292: symlink db/l63 0 2024-04-26T04:42:20.918 INFO:tasks.workunit.client.1.smithi104.stdout:2/883: link d6/de/d85/fa6 d6/d1b/d93/f123 0 2024-04-26T04:42:20.918 INFO:tasks.workunit.client.1.smithi104.stdout:2/884: read - d6/de/d85/fa6 zero size 2024-04-26T04:42:20.919 INFO:tasks.workunit.client.0.smithi090.stdout:0/487: rmdir d0/d4b/d5d/d5c/d8a/d45/d93/dac 0 2024-04-26T04:42:20.920 INFO:tasks.workunit.client.0.smithi090.stdout:0/488: dread - d0/d4b/d5d/d5c/d8a/d45/d93/fa1 zero size 2024-04-26T04:42:20.922 INFO:tasks.workunit.client.0.smithi090.stdout:9/293: rename db/f14 to db/d16/f64 0 2024-04-26T04:42:20.922 INFO:tasks.workunit.client.0.smithi090.stdout:9/294: fsync db/dd/d44/f2c 0 2024-04-26T04:42:20.922 INFO:tasks.workunit.client.0.smithi090.stdout:3/308: write d1/d17/f25 [1989566,114065] 0 2024-04-26T04:42:20.926 INFO:tasks.workunit.client.0.smithi090.stdout:3/309: unlink d1/d6/c13 0 2024-04-26T04:42:20.926 INFO:tasks.workunit.client.1.smithi104.stdout:0/614: dread d0/f2f [0,4194304] 0 2024-04-26T04:42:20.928 INFO:tasks.workunit.client.1.smithi104.stdout:0/615: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/dc2 0 2024-04-26T04:42:20.928 INFO:tasks.workunit.client.1.smithi104.stdout:0/616: chown d0/d2/d21/d27/c70 978005955 1 2024-04-26T04:42:20.928 INFO:tasks.workunit.client.1.smithi104.stdout:0/617: truncate d0/f42 4905361 0 2024-04-26T04:42:20.929 INFO:tasks.workunit.client.0.smithi090.stdout:3/310: mkdir d1/d17/d36/d33/d42/d70 0 2024-04-26T04:42:20.929 INFO:tasks.workunit.client.0.smithi090.stdout:3/311: chown d1/d17/d36/f32 7 1 2024-04-26T04:42:20.929 INFO:tasks.workunit.client.0.smithi090.stdout:3/312: chown d1/d17/d36/d33/d42/c5a 162 1 2024-04-26T04:42:20.929 INFO:tasks.workunit.client.0.smithi090.stdout:3/313: chown d1/d6/l57 4118930 1 2024-04-26T04:42:20.930 INFO:tasks.workunit.client.1.smithi104.stdout:0/618: link d0/d4/l35 d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/lc3 0 2024-04-26T04:42:20.930 INFO:tasks.workunit.client.1.smithi104.stdout:0/619: write d0/d2/d21/f52 [4655111,7423] 0 2024-04-26T04:42:20.930 INFO:tasks.workunit.client.0.smithi090.stdout:3/314: mkdir d1/d17/d36/d33/d68/d71 0 2024-04-26T04:42:20.931 INFO:tasks.workunit.client.1.smithi104.stdout:2/885: dread d6/d8/df6/dfe/f10c [0,4194304] 0 2024-04-26T04:42:20.931 INFO:tasks.workunit.client.1.smithi104.stdout:2/886: chown d6/d8/d5a/l79 405554 1 2024-04-26T04:42:20.931 INFO:tasks.workunit.client.1.smithi104.stdout:0/620: dread d0/d2/d21/d27/f83 [0,4194304] 0 2024-04-26T04:42:20.932 INFO:tasks.workunit.client.0.smithi090.stdout:2/419: dwrite db/d59/d66/d77/f79 [0,4194304] 0 2024-04-26T04:42:20.932 INFO:tasks.workunit.client.0.smithi090.stdout:2/420: dread - db/f84 zero size 2024-04-26T04:42:20.934 INFO:tasks.workunit.client.1.smithi104.stdout:2/887: creat d6/de/d8e/f124 x:0 0 0 2024-04-26T04:42:20.935 INFO:tasks.workunit.client.0.smithi090.stdout:2/421: creat db/d11/d14/d27/d2f/f8d x:0 0 0 2024-04-26T04:42:20.935 INFO:tasks.workunit.client.0.smithi090.stdout:2/422: readlink l5 0 2024-04-26T04:42:20.935 INFO:tasks.workunit.client.0.smithi090.stdout:3/315: read d1/f3c [819807,105390] 0 2024-04-26T04:42:20.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/464: truncate d9/d21/d3b/f46 1583747 0 2024-04-26T04:42:20.937 INFO:tasks.workunit.client.0.smithi090.stdout:3/316: mknod d1/d9/c72 0 2024-04-26T04:42:20.937 INFO:tasks.workunit.client.0.smithi090.stdout:3/317: fsync d1/d9/f12 0 2024-04-26T04:42:20.939 INFO:tasks.workunit.client.1.smithi104.stdout:0/621: dread d0/d2/d21/d38/f34 [0,4194304] 0 2024-04-26T04:42:20.940 INFO:tasks.workunit.client.0.smithi090.stdout:6/465: rename d9/d21/c88 to d9/d1c/d4d/d6d/d66/cac 0 2024-04-26T04:42:20.940 INFO:tasks.workunit.client.1.smithi104.stdout:5/695: sync 2024-04-26T04:42:20.941 INFO:tasks.workunit.client.1.smithi104.stdout:5/696: creat d0/d1/d64/d26/fe7 x:0 0 0 2024-04-26T04:42:20.941 INFO:tasks.workunit.client.1.smithi104.stdout:5/697: creat d0/d1/d64/d26/d36/da7/fe8 x:0 0 0 2024-04-26T04:42:20.941 INFO:tasks.workunit.client.1.smithi104.stdout:0/622: truncate d0/d2/d21/f45 1002786 0 2024-04-26T04:42:20.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/318: mknod d1/d17/d61/c73 0 2024-04-26T04:42:20.943 INFO:tasks.workunit.client.0.smithi090.stdout:6/466: symlink d9/d1c/d4d/d58/lad 0 2024-04-26T04:42:20.944 INFO:tasks.workunit.client.1.smithi104.stdout:5/698: creat d0/d1/d1e/d98/dd4/fe9 x:0 0 0 2024-04-26T04:42:20.944 INFO:tasks.workunit.client.1.smithi104.stdout:5/699: chown d0/d1/d64/d26/d36/f43 104271226 1 2024-04-26T04:42:20.945 INFO:tasks.workunit.client.1.smithi104.stdout:5/700: creat d0/d1/d64/d26/d36/d72/fea x:0 0 0 2024-04-26T04:42:20.946 INFO:tasks.workunit.client.1.smithi104.stdout:3/752: dwrite d5/db2/da3/dd0/ddb/d32/d48/f78 [0,4194304] 0 2024-04-26T04:42:20.947 INFO:tasks.workunit.client.0.smithi090.stdout:2/423: read db/d11/d14/d25/f49 [78930,117292] 0 2024-04-26T04:42:20.947 INFO:tasks.workunit.client.0.smithi090.stdout:2/424: getdents db/d11/d14/d25/d2d/d48/d81/d8c 0 2024-04-26T04:42:20.947 INFO:tasks.workunit.client.0.smithi090.stdout:2/425: truncate db/d11/d14/f4a 874830 0 2024-04-26T04:42:20.948 INFO:tasks.workunit.client.0.smithi090.stdout:2/426: write db/d11/d14/d27/d2f/d70/d71/f83 [457955,49646] 0 2024-04-26T04:42:20.948 INFO:tasks.workunit.client.0.smithi090.stdout:9/295: dread db/f41 [0,4194304] 0 2024-04-26T04:42:20.948 INFO:tasks.workunit.client.0.smithi090.stdout:9/296: creat db/dd/d18/d31/d58/f65 x:0 0 0 2024-04-26T04:42:20.948 INFO:tasks.workunit.client.0.smithi090.stdout:0/489: dwrite d0/d4b/d5d/d5c/d8a/d45/f9b [0,4194304] 0 2024-04-26T04:42:20.948 INFO:tasks.workunit.client.0.smithi090.stdout:0/490: getdents d0/d4b/d5d/d5c/d8a/d45/d53/dab 0 2024-04-26T04:42:20.948 INFO:tasks.workunit.client.0.smithi090.stdout:3/319: link d1/l3f d1/d5c/l74 0 2024-04-26T04:42:20.949 INFO:tasks.workunit.client.0.smithi090.stdout:1/362: dwrite d0/d4/d7/d17/d22/f23 [0,4194304] 0 2024-04-26T04:42:20.949 INFO:tasks.workunit.client.0.smithi090.stdout:1/363: write d0/d4/d7/d17/f21 [3388253,45398] 0 2024-04-26T04:42:20.949 INFO:tasks.workunit.client.0.smithi090.stdout:6/467: mkdir d9/d21/d40/d4c/d6f/d9f/dae 0 2024-04-26T04:42:20.949 INFO:tasks.workunit.client.0.smithi090.stdout:6/468: chown d9/d1c/d4d/d6d/d92 50 1 2024-04-26T04:42:20.950 INFO:tasks.workunit.client.0.smithi090.stdout:2/427: creat db/dc/f8e x:0 0 0 2024-04-26T04:42:20.950 INFO:tasks.workunit.client.0.smithi090.stdout:2/428: write db/d11/d14/d27/d2f/d70/d71/f83 [1347532,62280] 0 2024-04-26T04:42:20.950 INFO:tasks.workunit.client.0.smithi090.stdout:2/429: stat db/f33 0 2024-04-26T04:42:20.952 INFO:tasks.workunit.client.0.smithi090.stdout:9/297: symlink db/dd/d18/l66 0 2024-04-26T04:42:20.954 INFO:tasks.workunit.client.0.smithi090.stdout:9/298: write db/dd/d18/f36 [899434,90110] 0 2024-04-26T04:42:20.954 INFO:tasks.workunit.client.0.smithi090.stdout:2/430: write db/d11/d14/d25/f4e [499104,25540] 0 2024-04-26T04:42:20.956 INFO:tasks.workunit.client.1.smithi104.stdout:2/888: dwrite d6/de/d1d/f8f [0,4194304] 0 2024-04-26T04:42:20.956 INFO:tasks.workunit.client.1.smithi104.stdout:2/889: write d6/d8/d5a/fb3 [1758372,75711] 0 2024-04-26T04:42:20.956 INFO:tasks.workunit.client.1.smithi104.stdout:2/890: readlink d6/de/d8e/l5e 0 2024-04-26T04:42:20.956 INFO:tasks.workunit.client.1.smithi104.stdout:2/891: fdatasync d6/d8/d3e/fa3 0 2024-04-26T04:42:20.958 INFO:tasks.workunit.client.0.smithi090.stdout:0/491: mknod d0/d4b/d5d/d5c/d8a/d45/cb0 0 2024-04-26T04:42:20.971 INFO:tasks.workunit.client.0.smithi090.stdout:3/320: mkdir d1/d17/d36/d33/d68/d71/d75 0 2024-04-26T04:42:20.972 INFO:tasks.workunit.client.0.smithi090.stdout:3/321: dread d1/d17/f4f [0,4194304] 0 2024-04-26T04:42:20.972 INFO:tasks.workunit.client.0.smithi090.stdout:3/322: fdatasync d1/d6/d1b/f35 0 2024-04-26T04:42:20.972 INFO:tasks.workunit.client.0.smithi090.stdout:3/323: truncate d1/d17/d36/f24 4981201 0 2024-04-26T04:42:20.972 INFO:tasks.workunit.client.0.smithi090.stdout:2/431: creat db/d11/d14/d25/d2d/d48/d81/f8f x:0 0 0 2024-04-26T04:42:20.973 INFO:tasks.workunit.client.0.smithi090.stdout:0/492: creat d0/d4b/d5d/fb1 x:0 0 0 2024-04-26T04:42:20.973 INFO:tasks.workunit.client.0.smithi090.stdout:0/493: creat d0/d4b/d5d/d48/d7d/fb2 x:0 0 0 2024-04-26T04:42:20.973 INFO:tasks.workunit.client.0.smithi090.stdout:6/469: mknod d9/caf 0 2024-04-26T04:42:20.974 INFO:tasks.workunit.client.0.smithi090.stdout:3/324: mkdir d1/d17/d36/d33/d68/d76 0 2024-04-26T04:42:20.974 INFO:tasks.workunit.client.0.smithi090.stdout:2/432: rmdir db/d11/d14/d25/d86 39 2024-04-26T04:42:20.976 INFO:tasks.workunit.client.0.smithi090.stdout:0/494: unlink d0/d4b/d5d/d48/d7d/c8b 0 2024-04-26T04:42:20.976 INFO:tasks.workunit.client.0.smithi090.stdout:0/495: chown d0/d4b/d5d/d5c/d8a/d45 14362405 1 2024-04-26T04:42:20.977 INFO:tasks.workunit.client.0.smithi090.stdout:6/470: unlink d9/d21/d3b/l99 0 2024-04-26T04:42:20.977 INFO:tasks.workunit.client.0.smithi090.stdout:6/471: write d9/d1c/d31/d79/f7e [930934,44567] 0 2024-04-26T04:42:20.979 INFO:tasks.workunit.client.0.smithi090.stdout:2/433: mknod db/d11/d14/d27/d2f/d42/d5c/c90 0 2024-04-26T04:42:20.979 INFO:tasks.workunit.client.0.smithi090.stdout:3/325: getdents d1/d59 0 2024-04-26T04:42:20.980 INFO:tasks.workunit.client.0.smithi090.stdout:3/326: dread - d1/d59/f6d zero size 2024-04-26T04:42:20.980 INFO:tasks.workunit.client.0.smithi090.stdout:3/327: write d1/d17/d36/f24 [5453285,21521] 0 2024-04-26T04:42:20.980 INFO:tasks.workunit.client.0.smithi090.stdout:3/328: stat d1/d17/d36/d2a/d3a/f4b 0 2024-04-26T04:42:20.980 INFO:tasks.workunit.client.0.smithi090.stdout:2/434: creat db/d11/d14/d25/d2d/f91 x:0 0 0 2024-04-26T04:42:20.982 INFO:tasks.workunit.client.0.smithi090.stdout:0/496: dread d0/fc [0,4194304] 0 2024-04-26T04:42:20.982 INFO:tasks.workunit.client.0.smithi090.stdout:3/329: rename d1/d6/d1b/l30 to d1/d6/d2c/l77 0 2024-04-26T04:42:20.982 INFO:tasks.workunit.client.0.smithi090.stdout:3/330: stat d1/d17/d36/d33/d42 0 2024-04-26T04:42:20.982 INFO:tasks.workunit.client.0.smithi090.stdout:3/331: chown d1/d9/fa 4460 1 2024-04-26T04:42:20.982 INFO:tasks.workunit.client.0.smithi090.stdout:3/332: chown d1 105 1 2024-04-26T04:42:20.983 INFO:tasks.workunit.client.0.smithi090.stdout:2/435: mkdir db/d11/d14/d63/d92 0 2024-04-26T04:42:20.983 INFO:tasks.workunit.client.0.smithi090.stdout:0/497: mkdir d0/d4b/db3 0 2024-04-26T04:42:20.984 INFO:tasks.workunit.client.0.smithi090.stdout:3/333: mkdir d1/d6/d2c/d78 0 2024-04-26T04:42:20.984 INFO:tasks.workunit.client.0.smithi090.stdout:3/334: creat d1/d6/f79 x:0 0 0 2024-04-26T04:42:20.985 INFO:tasks.workunit.client.0.smithi090.stdout:0/498: unlink d0/d4b/d5d/d5c/d8a/d45/d53/d8e/l97 0 2024-04-26T04:42:20.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/499: creat d0/d4b/d5d/d5c/d8a/d45/fb4 x:0 0 0 2024-04-26T04:42:20.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/500: creat d0/d4b/d5d/d5c/d8a/d45/fb5 x:0 0 0 2024-04-26T04:42:20.986 INFO:tasks.workunit.client.0.smithi090.stdout:1/364: dwrite d0/d4/d7/d17/d22/d31/f6c [0,4194304] 0 2024-04-26T04:42:20.986 INFO:tasks.workunit.client.0.smithi090.stdout:1/365: write d0/d4/d43/f47 [5501,47772] 0 2024-04-26T04:42:20.986 INFO:tasks.workunit.client.0.smithi090.stdout:1/366: chown d0/d4/d43/f54 16211411 1 2024-04-26T04:42:20.986 INFO:tasks.workunit.client.0.smithi090.stdout:1/367: creat d0/d4/d7/d17/d22/d31/f6d x:0 0 0 2024-04-26T04:42:20.988 INFO:tasks.workunit.client.1.smithi104.stdout:0/623: dwrite d0/fa [0,4194304] 0 2024-04-26T04:42:20.988 INFO:tasks.workunit.client.1.smithi104.stdout:0/624: readlink d0/d2/d10/d3d/l87 0 2024-04-26T04:42:20.990 INFO:tasks.workunit.client.1.smithi104.stdout:8/642: sync 2024-04-26T04:42:20.990 INFO:tasks.workunit.client.1.smithi104.stdout:4/767: sync 2024-04-26T04:42:20.990 INFO:tasks.workunit.client.1.smithi104.stdout:7/673: sync 2024-04-26T04:42:20.991 INFO:tasks.workunit.client.1.smithi104.stdout:8/643: chown d0/d33/d35/l3d 10 1 2024-04-26T04:42:20.991 INFO:tasks.workunit.client.1.smithi104.stdout:9/607: sync 2024-04-26T04:42:20.991 INFO:tasks.workunit.client.1.smithi104.stdout:4/768: creat d4/d7/d1d/d8e/f10a x:0 0 0 2024-04-26T04:42:20.991 INFO:tasks.workunit.client.1.smithi104.stdout:6/780: sync 2024-04-26T04:42:20.991 INFO:tasks.workunit.client.1.smithi104.stdout:6/781: creat d0/d9/d13/d41/d82/dc8/d80/f100 x:0 0 0 2024-04-26T04:42:20.992 INFO:tasks.workunit.client.1.smithi104.stdout:1/738: sync 2024-04-26T04:42:20.992 INFO:tasks.workunit.client.1.smithi104.stdout:8/644: unlink d0/d1c/d2d/l54 0 2024-04-26T04:42:20.992 INFO:tasks.workunit.client.1.smithi104.stdout:8/645: readlink d0/d1c/d27/d47/d67/d76/d7f/lb5 0 2024-04-26T04:42:20.993 INFO:tasks.workunit.client.0.smithi090.stdout:0/501: dread d0/d4b/d5d/f42 [0,4194304] 0 2024-04-26T04:42:20.994 INFO:tasks.workunit.client.1.smithi104.stdout:7/674: mkdir d2/d4/d32/d35/dbe/dda 0 2024-04-26T04:42:20.994 INFO:tasks.workunit.client.0.smithi090.stdout:9/299: dwrite db/dd/d18/d31/f4e [0,4194304] 0 2024-04-26T04:42:20.995 INFO:tasks.workunit.client.1.smithi104.stdout:3/753: dwrite d5/db2/fed [0,4194304] 0 2024-04-26T04:42:20.995 INFO:tasks.workunit.client.1.smithi104.stdout:2/892: dwrite d6/d1b/d51/fe6 [0,4194304] 0 2024-04-26T04:42:20.995 INFO:tasks.workunit.client.1.smithi104.stdout:2/893: readlink d6/de/d8e/l5e 0 2024-04-26T04:42:20.995 INFO:tasks.workunit.client.1.smithi104.stdout:2/894: creat d6/d1b/d68/f125 x:0 0 0 2024-04-26T04:42:20.996 INFO:tasks.workunit.client.0.smithi090.stdout:1/368: link d0/d4/d7/d17/d22/f66 d0/d4/d7/d4b/d53/f6e 0 2024-04-26T04:42:20.996 INFO:tasks.workunit.client.1.smithi104.stdout:5/701: dwrite d0/d1/d64/d26/dd7/fe1 [0,4194304] 0 2024-04-26T04:42:20.997 INFO:tasks.workunit.client.0.smithi090.stdout:0/502: mknod d0/d4b/d5d/d5c/d8a/cb6 0 2024-04-26T04:42:20.999 INFO:tasks.workunit.client.0.smithi090.stdout:9/300: mkdir db/dd/d18/d38/d39/d67 0 2024-04-26T04:42:20.999 INFO:tasks.workunit.client.0.smithi090.stdout:9/301: chown f9 3 1 2024-04-26T04:42:21.000 INFO:tasks.workunit.client.1.smithi104.stdout:1/739: rename d0/d6/lba to d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/dec/lee 0 2024-04-26T04:42:21.000 INFO:tasks.workunit.client.1.smithi104.stdout:7/675: mknod d2/d4/d1d/d34/d7f/cdb 0 2024-04-26T04:42:21.001 INFO:tasks.workunit.client.0.smithi090.stdout:1/369: creat d0/d4/d7/d49/f6f x:0 0 0 2024-04-26T04:42:21.005 INFO:tasks.workunit.client.0.smithi090.stdout:0/503: unlink d0/fc 0 2024-04-26T04:42:21.005 INFO:tasks.workunit.client.0.smithi090.stdout:0/504: chown d0/df/f77 2 1 2024-04-26T04:42:21.006 INFO:tasks.workunit.client.0.smithi090.stdout:0/505: dread - d0/d4b/d5d/f6a zero size 2024-04-26T04:42:21.007 INFO:tasks.workunit.client.1.smithi104.stdout:4/769: rmdir d4/d7/d1d/d1e/d24/d59 39 2024-04-26T04:42:21.007 INFO:tasks.workunit.client.0.smithi090.stdout:9/302: rename db/dd/d18/d57/f4f to db/dd/d18/d57/f68 0 2024-04-26T04:42:21.008 INFO:tasks.workunit.client.1.smithi104.stdout:6/782: dread d0/d9/d13/d44/d71/f21 [0,4194304] 0 2024-04-26T04:42:21.008 INFO:tasks.workunit.client.0.smithi090.stdout:3/335: dread d1/d17/f25 [0,4194304] 0 2024-04-26T04:42:21.009 INFO:tasks.workunit.client.0.smithi090.stdout:3/336: truncate d1/d59/f6d 726360 0 2024-04-26T04:42:21.010 INFO:tasks.workunit.client.0.smithi090.stdout:0/506: link d0/d4b/d5d/d5c/d8a/d45/d93/faa d0/d4b/d5d/d5c/d8a/d45/d93/fb7 0 2024-04-26T04:42:21.011 INFO:tasks.workunit.client.1.smithi104.stdout:5/702: rename d0/d1/c87 to d0/d89/db2/ceb 0 2024-04-26T04:42:21.011 INFO:tasks.workunit.client.1.smithi104.stdout:1/740: getdents d0/d6/d2e/d32/d4c 0 2024-04-26T04:42:21.013 INFO:tasks.workunit.client.0.smithi090.stdout:9/303: symlink db/l69 0 2024-04-26T04:42:21.014 INFO:tasks.workunit.client.1.smithi104.stdout:4/770: rename d4/d7/fdf to d4/d7/d1d/d1e/d8f/db6/f10b 0 2024-04-26T04:42:21.016 INFO:tasks.workunit.client.0.smithi090.stdout:0/507: symlink d0/d4b/d5d/d5c/d8a/lb8 0 2024-04-26T04:42:21.018 INFO:tasks.workunit.client.0.smithi090.stdout:9/304: truncate db/dd/d18/d31/f52 617033 0 2024-04-26T04:42:21.019 INFO:tasks.workunit.client.1.smithi104.stdout:4/771: rename d4/d7/d1d/d1e/d24/l4f to d4/db/d6b/l10c 0 2024-04-26T04:42:21.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/508: symlink d0/d4b/d5d/d5c/d8a/d45/lb9 0 2024-04-26T04:42:21.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/509: write d0/d4b/d5d/d5c/d8a/d45/d53/faf [293021,10640] 0 2024-04-26T04:42:21.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/510: chown d0/d4b/f52 345702131 1 2024-04-26T04:42:21.023 INFO:tasks.workunit.client.1.smithi104.stdout:4/772: rename d4/d7/d1d/d1e/f14 to d4/d7/d1d/d2e/d74/dbd/dd3/f10d 0 2024-04-26T04:42:21.024 INFO:tasks.workunit.client.0.smithi090.stdout:9/305: mkdir db/dd/d18/d31/d6a 0 2024-04-26T04:42:21.026 INFO:tasks.workunit.client.1.smithi104.stdout:4/773: creat d4/db/f10e x:0 0 0 2024-04-26T04:42:21.027 INFO:tasks.workunit.client.0.smithi090.stdout:9/306: mkdir db/dd/d18/d38/d39/d6b 0 2024-04-26T04:42:21.027 INFO:tasks.workunit.client.0.smithi090.stdout:9/307: creat db/dd/d18/d31/d58/f6c x:0 0 0 2024-04-26T04:42:21.030 INFO:tasks.workunit.client.0.smithi090.stdout:9/308: symlink db/d16/l6d 0 2024-04-26T04:42:21.031 INFO:tasks.workunit.client.1.smithi104.stdout:5/703: write d0/d1/d4/d2a/d6b/d91/dbd/fc5 [1875001,21912] 0 2024-04-26T04:42:21.031 INFO:tasks.workunit.client.1.smithi104.stdout:4/774: rename d4/d7/d1d/d1e/d15/cd6 to d4/d7/d1d/d1e/d24/d48/c10f 0 2024-04-26T04:42:21.033 INFO:tasks.workunit.client.0.smithi090.stdout:9/309: mknod db/dd/d18/d31/c6e 0 2024-04-26T04:42:21.033 INFO:tasks.workunit.client.0.smithi090.stdout:9/310: chown db/dd/d44/c28 38 1 2024-04-26T04:42:21.033 INFO:tasks.workunit.client.0.smithi090.stdout:9/311: read - db/dd/d18/d31/d58/f6c zero size 2024-04-26T04:42:21.035 INFO:tasks.workunit.client.0.smithi090.stdout:6/472: dwrite d9/d1c/da2/faa [0,4194304] 0 2024-04-26T04:42:21.036 INFO:tasks.workunit.client.1.smithi104.stdout:5/704: rename d0/d1/d64/f85 to d0/d1/d4/d2a/d2b/d37/d99/fec 0 2024-04-26T04:42:21.036 INFO:tasks.workunit.client.1.smithi104.stdout:5/705: stat d0/d1/l82 0 2024-04-26T04:42:21.037 INFO:tasks.workunit.client.1.smithi104.stdout:4/775: mknod d4/db/d6b/c110 0 2024-04-26T04:42:21.037 INFO:tasks.workunit.client.1.smithi104.stdout:4/776: chown d4/d7/d1d/d1e/d15/dd8 21481 1 2024-04-26T04:42:21.037 INFO:tasks.workunit.client.0.smithi090.stdout:2/436: dwrite db/d11/d14/d1b/d20/f24 [0,4194304] 0 2024-04-26T04:42:21.041 INFO:tasks.workunit.client.0.smithi090.stdout:6/473: mkdir d9/d77/db0 0 2024-04-26T04:42:21.043 INFO:tasks.workunit.client.1.smithi104.stdout:0/625: dwrite d0/d2/d21/d27/d1e/d57/d6c/d85/fbd [0,4194304] 0 2024-04-26T04:42:21.043 INFO:tasks.workunit.client.1.smithi104.stdout:0/626: chown d0/d2/d21/ca4 18694 1 2024-04-26T04:42:21.053 INFO:tasks.workunit.client.0.smithi090.stdout:2/437: rename db/f33 to db/d11/d14/d25/d41/f93 0 2024-04-26T04:42:21.056 INFO:tasks.workunit.client.0.smithi090.stdout:2/438: mknod db/d11/d14/d25/d2d/c94 0 2024-04-26T04:42:21.059 INFO:tasks.workunit.client.0.smithi090.stdout:2/439: mkdir db/d11/d14/d25/d65/d95 0 2024-04-26T04:42:21.060 INFO:tasks.workunit.client.0.smithi090.stdout:2/440: dread db/d11/d14/d1b/f43 [0,4194304] 0 2024-04-26T04:42:21.060 INFO:tasks.workunit.client.0.smithi090.stdout:2/441: stat db/d11/d14/d27/f38 0 2024-04-26T04:42:21.060 INFO:tasks.workunit.client.0.smithi090.stdout:2/442: dread - db/d11/d14/d1b/f6d zero size 2024-04-26T04:42:21.060 INFO:tasks.workunit.client.0.smithi090.stdout:6/474: write d9/d1c/d4d/d6d/f93 [1267379,125198] 0 2024-04-26T04:42:21.062 INFO:tasks.workunit.client.0.smithi090.stdout:2/443: unlink db/dc/f12 0 2024-04-26T04:42:21.063 INFO:tasks.workunit.client.1.smithi104.stdout:7/676: dwrite d2/d4/d32/d3d/fb4 [0,4194304] 0 2024-04-26T04:42:21.063 INFO:tasks.workunit.client.1.smithi104.stdout:7/677: write d2/f56 [881106,33691] 0 2024-04-26T04:42:21.066 INFO:tasks.workunit.client.0.smithi090.stdout:6/475: rename d9/l39 to d9/d1c/d4d/lb1 0 2024-04-26T04:42:21.075 INFO:tasks.workunit.client.1.smithi104.stdout:2/895: dwrite d6/de/d85/f91 [0,4194304] 0 2024-04-26T04:42:21.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/444: dread db/d11/d14/d1b/f51 [0,4194304] 0 2024-04-26T04:42:21.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/445: chown db/d11/d14/d27/d2f/d42/c57 2788205 1 2024-04-26T04:42:21.081 INFO:tasks.workunit.client.0.smithi090.stdout:6/476: dread d9/d21/f48 [0,4194304] 0 2024-04-26T04:42:21.093 INFO:tasks.workunit.client.0.smithi090.stdout:6/477: dread d9/d1c/d31/d79/f7e [0,4194304] 0 2024-04-26T04:42:21.093 INFO:tasks.workunit.client.0.smithi090.stdout:6/478: getdents d9/d1c/d31 0 2024-04-26T04:42:21.093 INFO:tasks.workunit.client.0.smithi090.stdout:3/337: dwrite d1/d6/f14 [0,4194304] 0 2024-04-26T04:42:21.093 INFO:tasks.workunit.client.0.smithi090.stdout:3/338: dread - d1/d5c/f6e zero size 2024-04-26T04:42:21.093 INFO:tasks.workunit.client.0.smithi090.stdout:3/339: symlink d1/d17/d36/d33/d42/d70/l7a 0 2024-04-26T04:42:21.094 INFO:tasks.workunit.client.0.smithi090.stdout:6/479: write d9/d3c/f64 [186259,81701] 0 2024-04-26T04:42:21.094 INFO:tasks.workunit.client.0.smithi090.stdout:3/340: mknod d1/d17/d36/d2a/d3a/d5d/c7b 0 2024-04-26T04:42:21.098 INFO:tasks.workunit.client.0.smithi090.stdout:3/341: getdents d1/d17/d36 0 2024-04-26T04:42:21.098 INFO:tasks.workunit.client.0.smithi090.stdout:3/342: fsync d1/d9/f20 0 2024-04-26T04:42:21.098 INFO:tasks.workunit.client.0.smithi090.stdout:3/343: dread - d1/d17/d36/d33/d42/f63 zero size 2024-04-26T04:42:21.098 INFO:tasks.workunit.client.0.smithi090.stdout:0/511: dwrite d0/d4b/f52 [0,4194304] 0 2024-04-26T04:42:21.098 INFO:tasks.workunit.client.0.smithi090.stdout:9/312: dwrite db/dd/f60 [0,4194304] 0 2024-04-26T04:42:21.098 INFO:tasks.workunit.client.0.smithi090.stdout:3/344: dread d1/d59/f6d [0,4194304] 0 2024-04-26T04:42:21.098 INFO:tasks.workunit.client.0.smithi090.stdout:3/345: chown d1/d17/d36/d2a/d3a/f4b 204 1 2024-04-26T04:42:21.099 INFO:tasks.workunit.client.1.smithi104.stdout:6/783: dwrite d0/d9/d13/d44/d71/fb9 [0,4194304] 0 2024-04-26T04:42:21.099 INFO:tasks.workunit.client.1.smithi104.stdout:1/741: dwrite d0/d6/fc0 [0,4194304] 0 2024-04-26T04:42:21.099 INFO:tasks.workunit.client.0.smithi090.stdout:9/313: truncate db/dd/d18/f43 841798 0 2024-04-26T04:42:21.100 INFO:tasks.workunit.client.1.smithi104.stdout:5/706: dwrite d0/d1/d4/d2a/fd2 [0,4194304] 0 2024-04-26T04:42:21.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/480: dread d9/d21/d40/d4c/f59 [0,4194304] 0 2024-04-26T04:42:21.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/481: write d9/d21/d40/d4c/f7b [593829,31466] 0 2024-04-26T04:42:21.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/482: creat d9/d21/d40/d4c/d6f/fb2 x:0 0 0 2024-04-26T04:42:21.101 INFO:tasks.workunit.client.0.smithi090.stdout:6/483: creat d9/d3c/fb3 x:0 0 0 2024-04-26T04:42:21.101 INFO:tasks.workunit.client.0.smithi090.stdout:9/314: creat db/d5f/f6f x:0 0 0 2024-04-26T04:42:21.101 INFO:tasks.workunit.client.0.smithi090.stdout:9/315: write db/dd/d18/d38/d39/f3b [4226898,88604] 0 2024-04-26T04:42:21.102 INFO:tasks.workunit.client.1.smithi104.stdout:3/754: dwrite d5/d9/d80/dbc/fde [0,4194304] 0 2024-04-26T04:42:21.102 INFO:tasks.workunit.client.1.smithi104.stdout:3/755: write d5/db2/da3/dd0/ddb/d32/d47/ff9 [627477,43235] 0 2024-04-26T04:42:21.102 INFO:tasks.workunit.client.1.smithi104.stdout:0/627: dwrite d0/d2/d21/d49/d5e/d9c/f86 [0,4194304] 0 2024-04-26T04:42:21.102 INFO:tasks.workunit.client.1.smithi104.stdout:3/756: write d5/db2/da3/dd0/ddb/d32/f5a [937897,86367] 0 2024-04-26T04:42:21.102 INFO:tasks.workunit.client.0.smithi090.stdout:0/512: dread d0/d4b/d5d/d5c/f5e [0,4194304] 0 2024-04-26T04:42:21.102 INFO:tasks.workunit.client.0.smithi090.stdout:0/513: creat d0/d4b/d5d/d5c/d8a/d45/d53/fba x:0 0 0 2024-04-26T04:42:21.105 INFO:tasks.workunit.client.1.smithi104.stdout:4/777: dwrite d4/d7/d1d/d1e/d15/df3/ffc [0,4194304] 0 2024-04-26T04:42:21.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/484: mkdir d9/d3c/db4 0 2024-04-26T04:42:21.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/485: readlink d9/d1c/d4d/d6d/l67 0 2024-04-26T04:42:21.109 INFO:tasks.workunit.client.0.smithi090.stdout:9/316: mkdir db/dd/d18/d31/d6a/d70 0 2024-04-26T04:42:21.114 INFO:tasks.workunit.client.0.smithi090.stdout:9/317: symlink db/d16/l71 0 2024-04-26T04:42:21.117 INFO:tasks.workunit.client.0.smithi090.stdout:6/486: rename d9/d1c/d4d/d58/d94 to d9/d77/db0/db5 0 2024-04-26T04:42:21.119 INFO:tasks.workunit.client.0.smithi090.stdout:8/425: sync 2024-04-26T04:42:21.119 INFO:tasks.workunit.client.0.smithi090.stdout:7/429: sync 2024-04-26T04:42:21.120 INFO:tasks.workunit.client.0.smithi090.stdout:5/487: sync 2024-04-26T04:42:21.120 INFO:tasks.workunit.client.0.smithi090.stdout:4/372: sync 2024-04-26T04:42:21.120 INFO:tasks.workunit.client.0.smithi090.stdout:7/430: stat d0/d1/d4/f6b 0 2024-04-26T04:42:21.120 INFO:tasks.workunit.client.0.smithi090.stdout:8/426: chown dc/c80 5724 1 2024-04-26T04:42:21.121 INFO:tasks.workunit.client.0.smithi090.stdout:4/373: creat d3/db/d12/d3b/f75 x:0 0 0 2024-04-26T04:42:21.123 INFO:tasks.workunit.client.0.smithi090.stdout:7/431: mknod d0/db/d15/d30/d8e/c9d 0 2024-04-26T04:42:21.123 INFO:tasks.workunit.client.0.smithi090.stdout:7/432: fsync d0/db/df/f97 0 2024-04-26T04:42:21.124 INFO:tasks.workunit.client.1.smithi104.stdout:2/896: symlink d6/d8/d5a/l126 0 2024-04-26T04:42:21.126 INFO:tasks.workunit.client.0.smithi090.stdout:8/427: symlink dc/l92 0 2024-04-26T04:42:21.127 INFO:tasks.workunit.client.0.smithi090.stdout:4/374: truncate d3/db/d12/d20/d24/f32 2419388 0 2024-04-26T04:42:21.127 INFO:tasks.workunit.client.0.smithi090.stdout:4/375: write d3/db/d12/d20/f67 [360383,64623] 0 2024-04-26T04:42:21.128 INFO:tasks.workunit.client.1.smithi104.stdout:6/784: stat d0/d9/d13/d44/d47/c50 0 2024-04-26T04:42:21.128 INFO:tasks.workunit.client.1.smithi104.stdout:6/785: chown d0/d9/d13 0 1 2024-04-26T04:42:21.130 INFO:tasks.workunit.client.0.smithi090.stdout:7/433: mknod d0/db/df/d4d/d64/d66/c9e 0 2024-04-26T04:42:21.130 INFO:tasks.workunit.client.0.smithi090.stdout:2/446: rmdir db/dc 39 2024-04-26T04:42:21.134 INFO:tasks.workunit.client.0.smithi090.stdout:8/428: rename dc/f6b to dc/d34/f93 0 2024-04-26T04:42:21.135 INFO:tasks.workunit.client.0.smithi090.stdout:5/488: dread d4/d1a/d3f/f52 [0,4194304] 0 2024-04-26T04:42:21.135 INFO:tasks.workunit.client.0.smithi090.stdout:5/489: write d4/d1a/df/d72/f77 [1691858,5236] 0 2024-04-26T04:42:21.135 INFO:tasks.workunit.client.0.smithi090.stdout:4/376: creat d3/db/d12/d20/f76 x:0 0 0 2024-04-26T04:42:21.135 INFO:tasks.workunit.client.0.smithi090.stdout:4/377: fdatasync d3/f9 0 2024-04-26T04:42:21.136 INFO:tasks.workunit.client.1.smithi104.stdout:1/742: readlink d0/l79 0 2024-04-26T04:42:21.137 INFO:tasks.workunit.client.0.smithi090.stdout:8/429: creat dc/d71/f94 x:0 0 0 2024-04-26T04:42:21.138 INFO:tasks.workunit.client.0.smithi090.stdout:5/490: rmdir d4/d1a/d3f 39 2024-04-26T04:42:21.138 INFO:tasks.workunit.client.0.smithi090.stdout:5/491: stat d4/d1a/d45/c25 0 2024-04-26T04:42:21.138 INFO:tasks.workunit.client.0.smithi090.stdout:6/487: dread d9/d1c/f47 [4194304,4194304] 0 2024-04-26T04:42:21.138 INFO:tasks.workunit.client.0.smithi090.stdout:6/488: fdatasync d9/f17 0 2024-04-26T04:42:21.139 INFO:tasks.workunit.client.0.smithi090.stdout:0/514: dwrite d0/de/f39 [0,4194304] 0 2024-04-26T04:42:21.140 INFO:tasks.workunit.client.0.smithi090.stdout:4/378: creat d3/db/d12/d20/d2d/d2e/f77 x:0 0 0 2024-04-26T04:42:21.140 INFO:tasks.workunit.client.0.smithi090.stdout:4/379: getdents d3/db/d60/d5c 0 2024-04-26T04:42:21.141 INFO:tasks.workunit.client.0.smithi090.stdout:5/492: mknod d4/d1a/d2a/c9c 0 2024-04-26T04:42:21.143 INFO:tasks.workunit.client.0.smithi090.stdout:5/493: symlink d4/l9d 0 2024-04-26T04:42:21.144 INFO:tasks.workunit.client.1.smithi104.stdout:5/707: dwrite d0/f4e [0,4194304] 0 2024-04-26T04:42:21.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/494: rename d4/d1a/d3f/f44 to d4/d1a/d1f/d53/d59/d95/f9e 0 2024-04-26T04:42:21.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/495: write d4/d1a/f19 [3287608,53525] 0 2024-04-26T04:42:21.147 INFO:tasks.workunit.client.0.smithi090.stdout:9/318: dwrite db/d16/f46 [0,4194304] 0 2024-04-26T04:42:21.150 INFO:tasks.workunit.client.0.smithi090.stdout:8/430: write dc/d1f/d49/d39/d55/f6c [2615798,69668] 0 2024-04-26T04:42:21.152 INFO:tasks.workunit.client.0.smithi090.stdout:6/489: dread d9/d1c/d4d/d6d/f84 [0,4194304] 0 2024-04-26T04:42:21.156 INFO:tasks.workunit.client.0.smithi090.stdout:8/431: rmdir dc/d1f/d49/d39/d55 39 2024-04-26T04:42:21.158 INFO:tasks.workunit.client.0.smithi090.stdout:5/496: dread d4/d1a/d1f/f2b [4194304,4194304] 0 2024-04-26T04:42:21.161 INFO:tasks.workunit.client.0.smithi090.stdout:5/497: mknod d4/d1a/c9f 0 2024-04-26T04:42:21.162 INFO:tasks.workunit.client.1.smithi104.stdout:2/897: creat d6/d8/d5a/f127 x:0 0 0 2024-04-26T04:42:21.162 INFO:tasks.workunit.client.1.smithi104.stdout:2/898: chown d6/d1b/f121 16863644 1 2024-04-26T04:42:21.162 INFO:tasks.workunit.client.1.smithi104.stdout:2/899: dread - d6/f33 zero size 2024-04-26T04:42:21.163 INFO:tasks.workunit.client.0.smithi090.stdout:7/434: dwrite d0/d1/d4/f6b [0,4194304] 0 2024-04-26T04:42:21.163 INFO:tasks.workunit.client.0.smithi090.stdout:8/432: rename dc/d1f/d49/d3b/d3d/d57 to dc/d1f/d49/d4f/d58/d95 0 2024-04-26T04:42:21.164 INFO:tasks.workunit.client.1.smithi104.stdout:5/708: dread d0/d1/d64/d26/d36/fcc [0,4194304] 0 2024-04-26T04:42:21.165 INFO:tasks.workunit.client.1.smithi104.stdout:6/786: creat d0/d9/d13/d44/f101 x:0 0 0 2024-04-26T04:42:21.167 INFO:tasks.workunit.client.0.smithi090.stdout:7/435: mknod d0/c9f 0 2024-04-26T04:42:21.167 INFO:tasks.workunit.client.0.smithi090.stdout:7/436: chown d0/d1/d4/d98/d4a/l9c 388700681 1 2024-04-26T04:42:21.168 INFO:tasks.workunit.client.0.smithi090.stdout:9/319: dread db/dd/f60 [0,4194304] 0 2024-04-26T04:42:21.168 INFO:tasks.workunit.client.1.smithi104.stdout:1/743: rename d0/d6/d19/d1b/da7/db8/fdd to d0/d6/d2e/d32/d4c/fef 0 2024-04-26T04:42:21.170 INFO:tasks.workunit.client.0.smithi090.stdout:8/433: creat dc/d1f/d49/d4f/f96 x:0 0 0 2024-04-26T04:42:21.170 INFO:tasks.workunit.client.0.smithi090.stdout:7/437: truncate d0/db/df/d4d/d64/d66/f7c 300187 0 2024-04-26T04:42:21.170 INFO:tasks.workunit.client.0.smithi090.stdout:7/438: write d0/db/df/f97 [420362,112992] 0 2024-04-26T04:42:21.171 INFO:tasks.workunit.client.1.smithi104.stdout:3/757: getdents d5 0 2024-04-26T04:42:21.172 INFO:tasks.workunit.client.0.smithi090.stdout:7/439: dread d0/db/d15/d30/d4f/f76 [0,4194304] 0 2024-04-26T04:42:21.174 INFO:tasks.workunit.client.0.smithi090.stdout:7/440: truncate d0/db/d15/f91 470381 0 2024-04-26T04:42:21.177 INFO:tasks.workunit.client.1.smithi104.stdout:2/900: mkdir d6/de/d1d/dcf/d128 0 2024-04-26T04:42:21.179 INFO:tasks.workunit.client.0.smithi090.stdout:4/380: dwrite d3/db/d12/f28 [0,4194304] 0 2024-04-26T04:42:21.179 INFO:tasks.workunit.client.0.smithi090.stdout:4/381: creat d3/db/d12/d3b/f78 x:0 0 0 2024-04-26T04:42:21.180 INFO:tasks.workunit.client.0.smithi090.stdout:4/382: fdatasync d3/db/f19 0 2024-04-26T04:42:21.180 INFO:tasks.workunit.client.0.smithi090.stdout:4/383: truncate d3/db/d12/d20/f76 3846 0 2024-04-26T04:42:21.180 INFO:tasks.workunit.client.0.smithi090.stdout:4/384: creat d3/d13/d64/f79 x:0 0 0 2024-04-26T04:42:21.180 INFO:tasks.workunit.client.1.smithi104.stdout:5/709: link d0/f18 d0/d41/fed 0 2024-04-26T04:42:21.180 INFO:tasks.workunit.client.1.smithi104.stdout:5/710: creat d0/d1/d64/d26/d36/da7/fee x:0 0 0 2024-04-26T04:42:21.185 INFO:tasks.workunit.client.0.smithi090.stdout:8/434: read dc/f30 [1107283,10462] 0 2024-04-26T04:42:21.186 INFO:tasks.workunit.client.1.smithi104.stdout:1/744: creat d0/d6/d19/d21/d66/d20/da3/dde/ff0 x:0 0 0 2024-04-26T04:42:21.187 INFO:tasks.workunit.client.1.smithi104.stdout:1/745: fdatasync d0/d6/d19/d21/d28/f85 0 2024-04-26T04:42:21.191 INFO:tasks.workunit.client.0.smithi090.stdout:6/490: dwrite d9/d21/d40/d4c/f7a [0,4194304] 0 2024-04-26T04:42:21.192 INFO:tasks.workunit.client.0.smithi090.stdout:0/515: dwrite d0/df/f25 [0,4194304] 0 2024-04-26T04:42:21.193 INFO:tasks.workunit.client.0.smithi090.stdout:8/435: write dc/f1e [1551971,24324] 0 2024-04-26T04:42:21.193 INFO:tasks.workunit.client.0.smithi090.stdout:8/436: write dc/d1f/d49/d39/f81 [956439,38202] 0 2024-04-26T04:42:21.193 INFO:tasks.workunit.client.0.smithi090.stdout:8/437: stat dc/d1f/d49/d39/d55/d64/f7a 0 2024-04-26T04:42:21.193 INFO:tasks.workunit.client.1.smithi104.stdout:2/901: creat d6/d1b/d75/f129 x:0 0 0 2024-04-26T04:42:21.194 INFO:tasks.workunit.client.0.smithi090.stdout:9/320: dwrite db/dd/f60 [0,4194304] 0 2024-04-26T04:42:21.194 INFO:tasks.workunit.client.0.smithi090.stdout:0/516: symlink d0/d4b/d5d/d5c/d8a/d45/d53/dab/lbb 0 2024-04-26T04:42:21.197 INFO:tasks.workunit.client.1.smithi104.stdout:1/746: symlink d0/d6/d19/d21/d66/de4/d81/lf1 0 2024-04-26T04:42:21.198 INFO:tasks.workunit.client.0.smithi090.stdout:9/321: truncate db/dd/d18/d57/f68 788335 0 2024-04-26T04:42:21.198 INFO:tasks.workunit.client.0.smithi090.stdout:9/322: stat db/d16/l1f 0 2024-04-26T04:42:21.199 INFO:tasks.workunit.client.1.smithi104.stdout:3/758: rmdir d5/db2/d46/d95 39 2024-04-26T04:42:21.200 INFO:tasks.workunit.client.1.smithi104.stdout:5/711: dread d0/d1/d1e/f2e [0,4194304] 0 2024-04-26T04:42:21.201 INFO:tasks.workunit.client.1.smithi104.stdout:6/787: rename d0/d9/d13/d44/d47/da4/da6 to d0/dd8/de0/d102 0 2024-04-26T04:42:21.201 INFO:tasks.workunit.client.1.smithi104.stdout:6/788: chown d0/d9/d13/d44/f9a 90131 1 2024-04-26T04:42:21.202 INFO:tasks.workunit.client.0.smithi090.stdout:0/517: mknod d0/d4b/d6d/cbc 0 2024-04-26T04:42:21.202 INFO:tasks.workunit.client.0.smithi090.stdout:0/518: creat d0/d4b/fbd x:0 0 0 2024-04-26T04:42:21.203 INFO:tasks.workunit.client.1.smithi104.stdout:3/759: symlink d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/l10c 0 2024-04-26T04:42:21.204 INFO:tasks.workunit.client.0.smithi090.stdout:6/491: dread d9/d1c/d31/f54 [0,4194304] 0 2024-04-26T04:42:21.205 INFO:tasks.workunit.client.1.smithi104.stdout:5/712: mkdir d0/d1/d4/d2a/d2b/def 0 2024-04-26T04:42:21.209 INFO:tasks.workunit.client.0.smithi090.stdout:7/441: dwrite d0/db/d15/f57 [4194304,4194304] 0 2024-04-26T04:42:21.209 INFO:tasks.workunit.client.0.smithi090.stdout:0/519: rename d0/d4b/d5d/l40 to d0/d4b/d5d/d48/d7d/lbe 0 2024-04-26T04:42:21.209 INFO:tasks.workunit.client.0.smithi090.stdout:0/520: stat d0/de/f36 0 2024-04-26T04:42:21.209 INFO:tasks.workunit.client.0.smithi090.stdout:0/521: truncate d0/de/f39 4749787 0 2024-04-26T04:42:21.210 INFO:tasks.workunit.client.1.smithi104.stdout:6/789: symlink d0/d9/d13/d41/d96/dac/df1/l103 0 2024-04-26T04:42:21.212 INFO:tasks.workunit.client.0.smithi090.stdout:7/442: mknod d0/db/ca0 0 2024-04-26T04:42:21.213 INFO:tasks.workunit.client.0.smithi090.stdout:8/438: dread dc/d1f/d49/d39/d55/d64/f3e [0,4194304] 0 2024-04-26T04:42:21.213 INFO:tasks.workunit.client.0.smithi090.stdout:4/385: dwrite d3/d13/d64/f79 [0,4194304] 0 2024-04-26T04:42:21.213 INFO:tasks.workunit.client.1.smithi104.stdout:5/713: creat d0/d7b/ff0 x:0 0 0 2024-04-26T04:42:21.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/443: creat d0/db/fa1 x:0 0 0 2024-04-26T04:42:21.215 INFO:tasks.workunit.client.1.smithi104.stdout:2/902: dwrite d6/d1b/d93/f123 [0,4194304] 0 2024-04-26T04:42:21.215 INFO:tasks.workunit.client.1.smithi104.stdout:5/714: mknod d0/d1/de6/cf1 0 2024-04-26T04:42:21.217 INFO:tasks.workunit.client.0.smithi090.stdout:8/439: mkdir dc/d1f/d49/d3b/d3d/d97 0 2024-04-26T04:42:21.221 INFO:tasks.workunit.client.0.smithi090.stdout:8/440: symlink dc/d1f/l98 0 2024-04-26T04:42:21.223 INFO:tasks.workunit.client.0.smithi090.stdout:8/441: mkdir dc/d1f/d49/d39/d99 0 2024-04-26T04:42:21.223 INFO:tasks.workunit.client.0.smithi090.stdout:8/442: read dc/f30 [2911146,3447] 0 2024-04-26T04:42:21.223 INFO:tasks.workunit.client.0.smithi090.stdout:8/443: fdatasync dc/d5a/d5d/f66 0 2024-04-26T04:42:21.223 INFO:tasks.workunit.client.0.smithi090.stdout:8/444: stat dc/d1f/d49/d39/d55/d64/f89 0 2024-04-26T04:42:21.223 INFO:tasks.workunit.client.1.smithi104.stdout:5/715: mkdir d0/d41/d4f/df2 0 2024-04-26T04:42:21.223 INFO:tasks.workunit.client.1.smithi104.stdout:5/716: fsync d0/d1/d4/d2a/d2b/d37/d99/fc0 0 2024-04-26T04:42:21.224 INFO:tasks.workunit.client.0.smithi090.stdout:0/522: write d0/d4b/f52 [3143642,116266] 0 2024-04-26T04:42:21.224 INFO:tasks.workunit.client.0.smithi090.stdout:8/445: mknod dc/d1f/d49/d39/d99/c9a 0 2024-04-26T04:42:21.229 INFO:tasks.workunit.client.0.smithi090.stdout:0/523: rename d0/d4b/d5d/d5c/d8a/d45/d53/l70 to d0/de/lbf 0 2024-04-26T04:42:21.231 INFO:tasks.workunit.client.0.smithi090.stdout:0/524: mknod d0/df/cc0 0 2024-04-26T04:42:21.231 INFO:tasks.workunit.client.0.smithi090.stdout:0/525: chown d0/d4b/d5d/d5c/d8a/d64/ca4 56 1 2024-04-26T04:42:21.231 INFO:tasks.workunit.client.0.smithi090.stdout:0/526: chown d0/de/f89 1478530844 1 2024-04-26T04:42:21.235 INFO:tasks.workunit.client.0.smithi090.stdout:0/527: write d0/d4b/d5d/d5c/d8a/f3e [1195501,19610] 0 2024-04-26T04:42:21.235 INFO:tasks.workunit.client.0.smithi090.stdout:0/528: readlink d0/d4b/d5d/d5c/d8a/d45/d53/l5b 0 2024-04-26T04:42:21.235 INFO:tasks.workunit.client.0.smithi090.stdout:9/323: dwrite db/dd/d44/f2c [0,4194304] 0 2024-04-26T04:42:21.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/529: write d0/d4b/d5d/d5c/d8a/d45/d53/faf [161538,19059] 0 2024-04-26T04:42:21.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/530: write d0/d4b/d5d/d5c/d8a/d45/d53/fba [33586,81087] 0 2024-04-26T04:42:21.236 INFO:tasks.workunit.client.0.smithi090.stdout:0/531: dread - d0/d4b/d5d/fb1 zero size 2024-04-26T04:42:21.238 INFO:tasks.workunit.client.0.smithi090.stdout:9/324: link db/dd/d44/l30 db/dd/d18/d31/d6a/l72 0 2024-04-26T04:42:21.243 INFO:tasks.workunit.client.1.smithi104.stdout:5/717: dread d0/f4e [0,4194304] 0 2024-04-26T04:42:21.244 INFO:tasks.workunit.client.1.smithi104.stdout:5/718: symlink d0/d89/lf3 0 2024-04-26T04:42:21.246 INFO:tasks.workunit.client.1.smithi104.stdout:2/903: read d6/d1b/d93/fc4 [4033642,101771] 0 2024-04-26T04:42:21.246 INFO:tasks.workunit.client.1.smithi104.stdout:5/719: symlink d0/d1/d64/d26/ddc/lf4 0 2024-04-26T04:42:21.246 INFO:tasks.workunit.client.1.smithi104.stdout:2/904: stat d6/de/c5c 0 2024-04-26T04:42:21.246 INFO:tasks.workunit.client.1.smithi104.stdout:5/720: write d0/d1/d64/f31 [9952,119564] 0 2024-04-26T04:42:21.247 INFO:tasks.workunit.client.1.smithi104.stdout:5/721: rmdir d0/d1 39 2024-04-26T04:42:21.252 INFO:tasks.workunit.client.1.smithi104.stdout:1/747: rename d0/d6/d19/d21/d66/d20/d2a/d98 to d0/d6/d19/d21/d58/df2 0 2024-04-26T04:42:21.256 INFO:tasks.workunit.client.0.smithi090.stdout:0/532: dread d0/d4b/d5d/d5c/d8a/f13 [0,4194304] 0 2024-04-26T04:42:21.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/760: rename d5/db2/da3/dd0/ddb/d32/l9c to d5/db2/da3/l10d 0 2024-04-26T04:42:21.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/761: read - d5/db2/da3/dd0/ddb/d32/d48/f67 zero size 2024-04-26T04:42:21.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/762: truncate d5/db2/da3/dd0/ddb/d69/f10a 374185 0 2024-04-26T04:42:21.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/763: chown d5/db2/da3/dd0 15272604 1 2024-04-26T04:42:21.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/764: chown d5/db2/d46/db7/de6 11353 1 2024-04-26T04:42:21.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/765: readlink d5/db2/da3/dd0/ddb/d32/d48/dac/la6 0 2024-04-26T04:42:21.257 INFO:tasks.workunit.client.1.smithi104.stdout:3/766: chown d5/d9/d80/dbc/ldd 1 1 2024-04-26T04:42:21.258 INFO:tasks.workunit.client.0.smithi090.stdout:9/325: dread db/dd/d44/f27 [0,4194304] 0 2024-04-26T04:42:21.258 INFO:tasks.workunit.client.0.smithi090.stdout:9/326: write db/dd/d18/f43 [1289953,95558] 0 2024-04-26T04:42:21.259 INFO:tasks.workunit.client.1.smithi104.stdout:1/748: mkdir d0/d6/d2e/d32/d6a/df3 0 2024-04-26T04:42:21.259 INFO:tasks.workunit.client.1.smithi104.stdout:1/749: read - d0/d6/d19/d1b/da7/db8/fe6 zero size 2024-04-26T04:42:21.259 INFO:tasks.workunit.client.0.smithi090.stdout:0/533: rmdir d0/d4b/d6d 39 2024-04-26T04:42:21.260 INFO:tasks.workunit.client.0.smithi090.stdout:9/327: symlink db/dd/d18/d38/d39/d6b/l73 0 2024-04-26T04:42:21.262 INFO:tasks.workunit.client.1.smithi104.stdout:7/678: rmdir d2/d4/d32/d35/d44 39 2024-04-26T04:42:21.262 INFO:tasks.workunit.client.1.smithi104.stdout:3/767: mknod d5/d9/dd1/c10e 0 2024-04-26T04:42:21.262 INFO:tasks.workunit.client.1.smithi104.stdout:3/768: truncate d5/db2/d72/f99 5182832 0 2024-04-26T04:42:21.262 INFO:tasks.workunit.client.1.smithi104.stdout:5/722: write d0/d41/fed [3778844,54009] 0 2024-04-26T04:42:21.263 INFO:tasks.workunit.client.0.smithi090.stdout:4/386: dwrite d3/db/d12/d3b/f78 [0,4194304] 0 2024-04-26T04:42:21.263 INFO:tasks.workunit.client.0.smithi090.stdout:0/534: unlink d0/d4b/d5d/d5c/d8a/d45/f55 0 2024-04-26T04:42:21.263 INFO:tasks.workunit.client.1.smithi104.stdout:1/750: dread d0/f14 [4194304,4194304] 0 2024-04-26T04:42:21.264 INFO:tasks.workunit.client.0.smithi090.stdout:9/328: mknod db/dd/c74 0 2024-04-26T04:42:21.265 INFO:tasks.workunit.client.1.smithi104.stdout:3/769: rename d5/db2/f73 to d5/db2/da3/dd0/ddb/d3a/df5/f10f 0 2024-04-26T04:42:21.265 INFO:tasks.workunit.client.1.smithi104.stdout:3/770: truncate d5/db2/da3/dd0/ddb/d69/fb6 48253 0 2024-04-26T04:42:21.266 INFO:tasks.workunit.client.0.smithi090.stdout:7/444: dwrite d0/f45 [0,4194304] 0 2024-04-26T04:42:21.266 INFO:tasks.workunit.client.0.smithi090.stdout:7/445: creat d0/db/d15/d29/fa2 x:0 0 0 2024-04-26T04:42:21.266 INFO:tasks.workunit.client.0.smithi090.stdout:8/446: dwrite dc/d1f/d49/d3b/f85 [0,4194304] 0 2024-04-26T04:42:21.267 INFO:tasks.workunit.client.0.smithi090.stdout:8/447: chown dc/f5c 1075452803 1 2024-04-26T04:42:21.267 INFO:tasks.workunit.client.0.smithi090.stdout:4/387: mkdir d3/db/d12/d20/d24/d43/d72/d7a 0 2024-04-26T04:42:21.267 INFO:tasks.workunit.client.0.smithi090.stdout:4/388: chown d3/f9 3781818 1 2024-04-26T04:42:21.267 INFO:tasks.workunit.client.0.smithi090.stdout:6/492: dwrite d9/d21/d3b/f7c [0,4194304] 0 2024-04-26T04:42:21.267 INFO:tasks.workunit.client.0.smithi090.stdout:6/493: fsync d9/d21/d40/d4c/f59 0 2024-04-26T04:42:21.268 INFO:tasks.workunit.client.0.smithi090.stdout:6/494: truncate d9/d1c/d4d/d6d/d92/fa8 364876 0 2024-04-26T04:42:21.268 INFO:tasks.workunit.client.0.smithi090.stdout:6/495: creat d9/d21/d40/d4c/d6f/fb6 x:0 0 0 2024-04-26T04:42:21.268 INFO:tasks.workunit.client.1.smithi104.stdout:7/679: rmdir d2/d4/d32/d35/dbe 39 2024-04-26T04:42:21.268 INFO:tasks.workunit.client.1.smithi104.stdout:7/680: dread - d2/d4/d32/d3d/d41/d7e/fd7 zero size 2024-04-26T04:42:21.268 INFO:tasks.workunit.client.1.smithi104.stdout:7/681: chown d2/d4/d32/d3d/d41/d5f/d6c/d94 165 1 2024-04-26T04:42:21.268 INFO:tasks.workunit.client.1.smithi104.stdout:2/905: dwrite d6/d8/d3e/fd6 [0,4194304] 0 2024-04-26T04:42:21.268 INFO:tasks.workunit.client.0.smithi090.stdout:0/535: mkdir d0/d4b/d5d/d48/dc1 0 2024-04-26T04:42:21.269 INFO:tasks.workunit.client.0.smithi090.stdout:9/329: symlink db/dd/d18/d31/d6a/d70/l75 0 2024-04-26T04:42:21.269 INFO:tasks.workunit.client.0.smithi090.stdout:9/330: fdatasync db/dd/d18/f2a 0 2024-04-26T04:42:21.270 INFO:tasks.workunit.client.0.smithi090.stdout:6/496: dread d9/d21/d40/d4c/f59 [0,4194304] 0 2024-04-26T04:42:21.270 INFO:tasks.workunit.client.0.smithi090.stdout:6/497: readlink d9/d21/d40/d4c/l96 0 2024-04-26T04:42:21.270 INFO:tasks.workunit.client.1.smithi104.stdout:6/790: symlink d0/d9/d13/d44/d47/l104 0 2024-04-26T04:42:21.270 INFO:tasks.workunit.client.1.smithi104.stdout:6/791: chown d0/d9/l4d 225801084 1 2024-04-26T04:42:21.276 INFO:tasks.workunit.client.1.smithi104.stdout:1/751: write d0/d6/d19/d21/d66/d20/d2a/f31 [4024406,51249] 0 2024-04-26T04:42:21.276 INFO:tasks.workunit.client.1.smithi104.stdout:1/752: dread - d0/d6/d19/fae zero size 2024-04-26T04:42:21.277 INFO:tasks.workunit.client.0.smithi090.stdout:7/446: mkdir d0/d1/da3 0 2024-04-26T04:42:21.278 INFO:tasks.workunit.client.0.smithi090.stdout:7/447: truncate d0/db/d15/d29/f18 428099 0 2024-04-26T04:42:21.278 INFO:tasks.workunit.client.1.smithi104.stdout:2/906: creat d6/d1b/d68/d105/f12a x:0 0 0 2024-04-26T04:42:21.279 INFO:tasks.workunit.client.0.smithi090.stdout:8/448: symlink dc/d5a/d5d/l9b 0 2024-04-26T04:42:21.279 INFO:tasks.workunit.client.0.smithi090.stdout:4/389: dread d3/db/d12/d20/d24/d30/f49 [0,4194304] 0 2024-04-26T04:42:21.280 INFO:tasks.workunit.client.1.smithi104.stdout:6/792: creat d0/dd8/de0/d102/f105 x:0 0 0 2024-04-26T04:42:21.282 INFO:tasks.workunit.client.1.smithi104.stdout:2/907: link d6/de/d1d/f83 d6/d1b/d68/d81/f12b 0 2024-04-26T04:42:21.283 INFO:tasks.workunit.client.1.smithi104.stdout:2/908: chown d6/d1b/d68/fa1 7 1 2024-04-26T04:42:21.284 INFO:tasks.workunit.client.1.smithi104.stdout:6/793: rename d0/ce1 to d0/d9/d13/dcd/dd9/c106 0 2024-04-26T04:42:21.285 INFO:tasks.workunit.client.1.smithi104.stdout:3/771: dread d5/db2/da3/dd0/ddb/d3a/f57 [0,4194304] 0 2024-04-26T04:42:21.285 INFO:tasks.workunit.client.1.smithi104.stdout:3/772: fdatasync d5/f25 0 2024-04-26T04:42:21.285 INFO:tasks.workunit.client.1.smithi104.stdout:6/794: mknod d0/d9/d13/d41/d82/c107 0 2024-04-26T04:42:21.286 INFO:tasks.workunit.client.1.smithi104.stdout:6/795: creat d0/d9/d13/f108 x:0 0 0 2024-04-26T04:42:21.287 INFO:tasks.workunit.client.1.smithi104.stdout:3/773: link d5/d9/fe d5/db2/da3/dd0/ddb/d32/dd9/f110 0 2024-04-26T04:42:21.292 INFO:tasks.workunit.client.1.smithi104.stdout:6/796: rename d0/d9/d13/d44/f97 to d0/d9/d13/dcd/dd9/f109 0 2024-04-26T04:42:21.292 INFO:tasks.workunit.client.1.smithi104.stdout:1/753: dread d0/d6/d19/d21/d59/f6e [0,4194304] 0 2024-04-26T04:42:21.292 INFO:tasks.workunit.client.1.smithi104.stdout:1/754: write d0/d6/fc0 [4552997,7293] 0 2024-04-26T04:42:21.293 INFO:tasks.workunit.client.1.smithi104.stdout:7/682: dwrite d2/f56 [0,4194304] 0 2024-04-26T04:42:21.293 INFO:tasks.workunit.client.1.smithi104.stdout:7/683: creat d2/d4/d32/d3d/d41/d5f/d6c/d89/fdc x:0 0 0 2024-04-26T04:42:21.293 INFO:tasks.workunit.client.1.smithi104.stdout:7/684: read - d2/d4/d32/d3d/d41/fd2 zero size 2024-04-26T04:42:21.293 INFO:tasks.workunit.client.1.smithi104.stdout:7/685: chown d2/d4/d32/l48 84326 1 2024-04-26T04:42:21.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/797: link d0/d9/d13/d44/d71/f69 d0/d9/d13/dcd/dd9/f10a 0 2024-04-26T04:42:21.298 INFO:tasks.workunit.client.0.smithi090.stdout:0/536: mkdir d0/d4b/d5d/d5c/d8a/d45/d53/dc2 0 2024-04-26T04:42:21.298 INFO:tasks.workunit.client.1.smithi104.stdout:1/755: mknod d0/d6/d2e/d32/cf4 0 2024-04-26T04:42:21.299 INFO:tasks.workunit.client.1.smithi104.stdout:1/756: write d0/d6/d19/d1b/fd7 [344247,110361] 0 2024-04-26T04:42:21.300 INFO:tasks.workunit.client.0.smithi090.stdout:9/331: rename db/dd/d44/f27 to db/dd/d18/d38/d39/d3f/f76 0 2024-04-26T04:42:21.300 INFO:tasks.workunit.client.0.smithi090.stdout:9/332: chown db/f41 185 1 2024-04-26T04:42:21.301 INFO:tasks.workunit.client.1.smithi104.stdout:6/798: dread d0/d9/d13/d41/d96/dac/f4e [0,4194304] 0 2024-04-26T04:42:21.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/498: mkdir d9/d1c/d31/d69/db7 0 2024-04-26T04:42:21.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/449: creat dc/d1f/d49/d3b/d3d/f9c x:0 0 0 2024-04-26T04:42:21.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/450: chown dc/f1e 1553 1 2024-04-26T04:42:21.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/451: creat dc/d1f/d49/d4f/f9d x:0 0 0 2024-04-26T04:42:21.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/452: creat dc/d34/d4d/f9e x:0 0 0 2024-04-26T04:42:21.303 INFO:tasks.workunit.client.0.smithi090.stdout:8/453: creat dc/d1f/d49/d4f/f9f x:0 0 0 2024-04-26T04:42:21.304 INFO:tasks.workunit.client.1.smithi104.stdout:9/608: sync 2024-04-26T04:42:21.304 INFO:tasks.workunit.client.1.smithi104.stdout:7/686: mkdir d2/d4/d32/d3d/d47/db6/dce/ddd 0 2024-04-26T04:42:21.305 INFO:tasks.workunit.client.1.smithi104.stdout:4/778: sync 2024-04-26T04:42:21.306 INFO:tasks.workunit.client.1.smithi104.stdout:0/628: sync 2024-04-26T04:42:21.306 INFO:tasks.workunit.client.1.smithi104.stdout:4/779: write d4/d7/d1d/d1e/f61 [272620,108060] 0 2024-04-26T04:42:21.306 INFO:tasks.workunit.client.1.smithi104.stdout:3/774: getdents d5/db2/da3/dd0/ddb 0 2024-04-26T04:42:21.306 INFO:tasks.workunit.client.1.smithi104.stdout:4/780: chown d4/d7/d1d/d1e/d24/d59/cd1 209433 1 2024-04-26T04:42:21.306 INFO:tasks.workunit.client.1.smithi104.stdout:8/646: sync 2024-04-26T04:42:21.306 INFO:tasks.workunit.client.1.smithi104.stdout:8/647: read d0/d33/d51/f71 [109751,16851] 0 2024-04-26T04:42:21.308 INFO:tasks.workunit.client.1.smithi104.stdout:7/687: dread d2/d4/d32/d3d/d41/d5f/d6d/faa [0,4194304] 0 2024-04-26T04:42:21.308 INFO:tasks.workunit.client.1.smithi104.stdout:2/909: dwrite d6/d8/d5a/fb0 [4194304,4194304] 0 2024-04-26T04:42:21.309 INFO:tasks.workunit.client.0.smithi090.stdout:7/448: dwrite d0/db/d53/f17 [0,4194304] 0 2024-04-26T04:42:21.309 INFO:tasks.workunit.client.1.smithi104.stdout:9/609: symlink d11/d53/d84/lc7 0 2024-04-26T04:42:21.310 INFO:tasks.workunit.client.1.smithi104.stdout:6/799: write d0/d9/d13/d44/d71/d3f/fe6 [71567,60898] 0 2024-04-26T04:42:21.310 INFO:tasks.workunit.client.1.smithi104.stdout:9/610: dread - d11/d15/d32/d4c/d7c/fb4 zero size 2024-04-26T04:42:21.310 INFO:tasks.workunit.client.0.smithi090.stdout:4/390: rename d3/db/d12/d20/d24/c55 to d3/db/d12/d3b/c7b 0 2024-04-26T04:42:21.310 INFO:tasks.workunit.client.0.smithi090.stdout:4/391: stat d3/d13/d1d/c70 0 2024-04-26T04:42:21.310 INFO:tasks.workunit.client.0.smithi090.stdout:4/392: truncate d3/db/d12/d20/f67 995117 0 2024-04-26T04:42:21.310 INFO:tasks.workunit.client.0.smithi090.stdout:4/393: write d3/db/d12/d20/d2d/d2e/f77 [709317,68647] 0 2024-04-26T04:42:21.311 INFO:tasks.workunit.client.1.smithi104.stdout:0/629: mknod d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/cc4 0 2024-04-26T04:42:21.314 INFO:tasks.workunit.client.1.smithi104.stdout:4/781: rename d4/d20/l32 to d4/d7/d1d/d2e/d74/dbd/dee/l111 0 2024-04-26T04:42:21.315 INFO:tasks.workunit.client.0.smithi090.stdout:8/454: mknod dc/d1f/d49/d39/ca0 0 2024-04-26T04:42:21.319 INFO:tasks.workunit.client.1.smithi104.stdout:3/775: write d5/f43 [1469129,121098] 0 2024-04-26T04:42:21.319 INFO:tasks.workunit.client.1.smithi104.stdout:3/776: truncate d5/d9/ff 366695 0 2024-04-26T04:42:21.320 INFO:tasks.workunit.client.1.smithi104.stdout:2/910: unlink d6/d8/d3e/l5d 0 2024-04-26T04:42:21.320 INFO:tasks.workunit.client.1.smithi104.stdout:2/911: truncate d6/d8/d3e/f10d 1018527 0 2024-04-26T04:42:21.322 INFO:tasks.workunit.client.0.smithi090.stdout:4/394: link d3/f36 d3/db/d12/d20/d24/d43/d72/f7c 0 2024-04-26T04:42:21.322 INFO:tasks.workunit.client.0.smithi090.stdout:4/395: read d3/f27 [611106,74899] 0 2024-04-26T04:42:21.323 INFO:tasks.workunit.client.0.smithi090.stdout:9/333: getdents db/dd 0 2024-04-26T04:42:21.324 INFO:tasks.workunit.client.1.smithi104.stdout:6/800: truncate d0/d9/dc6/f9e 646476 0 2024-04-26T04:42:21.326 INFO:tasks.workunit.client.1.smithi104.stdout:9/611: link d11/d1d/d71/d94/ca0 d11/d1d/d71/d94/cc8 0 2024-04-26T04:42:21.326 INFO:tasks.workunit.client.1.smithi104.stdout:0/630: dread d0/d2/d10/d3d/f9e [0,4194304] 0 2024-04-26T04:42:21.326 INFO:tasks.workunit.client.1.smithi104.stdout:9/612: write d11/d1d/d34/d69/f78 [811628,60957] 0 2024-04-26T04:42:21.326 INFO:tasks.workunit.client.1.smithi104.stdout:9/613: dread - d11/d53/f86 zero size 2024-04-26T04:42:21.327 INFO:tasks.workunit.client.0.smithi090.stdout:4/396: rename d3/db/d12/d20/f76 to d3/f7d 0 2024-04-26T04:42:21.329 INFO:tasks.workunit.client.0.smithi090.stdout:4/397: mkdir d3/d13/d6c/d7e 0 2024-04-26T04:42:21.330 INFO:tasks.workunit.client.0.smithi090.stdout:8/455: dread dc/d1f/d49/f20 [0,4194304] 0 2024-04-26T04:42:21.332 INFO:tasks.workunit.client.0.smithi090.stdout:8/456: mknod dc/d34/d87/ca1 0 2024-04-26T04:42:21.332 INFO:tasks.workunit.client.0.smithi090.stdout:8/457: chown dc/d1f/d49/d39/d55/f70 6574404 1 2024-04-26T04:42:21.332 INFO:tasks.workunit.client.0.smithi090.stdout:8/458: truncate dc/d5a/d5d/f66 675964 0 2024-04-26T04:42:21.332 INFO:tasks.workunit.client.0.smithi090.stdout:8/459: truncate dc/d1f/f35 1455472 0 2024-04-26T04:42:21.333 INFO:tasks.workunit.client.1.smithi104.stdout:4/782: creat d4/d7/d1d/d1e/d8f/db6/d7d/f112 x:0 0 0 2024-04-26T04:42:21.336 INFO:tasks.workunit.client.1.smithi104.stdout:8/648: link d0/l15 d0/d33/d35/d6d/lc7 0 2024-04-26T04:42:21.336 INFO:tasks.workunit.client.1.smithi104.stdout:8/649: chown d0/d33/c9d 0 1 2024-04-26T04:42:21.337 INFO:tasks.workunit.client.0.smithi090.stdout:9/334: write db/dd/d18/d57/f4d [963961,84005] 0 2024-04-26T04:42:21.338 INFO:tasks.workunit.client.1.smithi104.stdout:3/777: creat d5/db2/da3/dd0/ddb/f111 x:0 0 0 2024-04-26T04:42:21.338 INFO:tasks.workunit.client.1.smithi104.stdout:2/912: dread d6/d1b/d51/d6a/f7d [0,4194304] 0 2024-04-26T04:42:21.338 INFO:tasks.workunit.client.1.smithi104.stdout:6/801: creat d0/d9/d13/d41/d96/ded/f10b x:0 0 0 2024-04-26T04:42:21.338 INFO:tasks.workunit.client.1.smithi104.stdout:6/802: truncate d0/d9/f49 1579903 0 2024-04-26T04:42:21.339 INFO:tasks.workunit.client.0.smithi090.stdout:8/460: rename dc/f1a to dc/d1f/d49/d3b/d3d/d97/fa2 0 2024-04-26T04:42:21.339 INFO:tasks.workunit.client.0.smithi090.stdout:8/461: dread - dc/d1f/d49/d4f/f7f zero size 2024-04-26T04:42:21.339 INFO:tasks.workunit.client.1.smithi104.stdout:1/757: dwrite d0/d6/d2e/d32/f62 [0,4194304] 0 2024-04-26T04:42:21.340 INFO:tasks.workunit.client.1.smithi104.stdout:1/758: chown d0/d6/d49/f87 49 1 2024-04-26T04:42:21.340 INFO:tasks.workunit.client.1.smithi104.stdout:1/759: chown d0/d6/d19/d21/d66/d20/c6b 3 1 2024-04-26T04:42:21.340 INFO:tasks.workunit.client.1.smithi104.stdout:1/760: write d0/d6/d19/d21/d66/d20/d2a/f2c [3297446,49901] 0 2024-04-26T04:42:21.340 INFO:tasks.workunit.client.1.smithi104.stdout:1/761: fsync d0/d6/d19/d21/d66/de4/fcd 0 2024-04-26T04:42:21.341 INFO:tasks.workunit.client.0.smithi090.stdout:6/499: dwrite d9/d1c/d31/f95 [0,4194304] 0 2024-04-26T04:42:21.341 INFO:tasks.workunit.client.0.smithi090.stdout:6/500: chown d9/d3c 6629007 1 2024-04-26T04:42:21.341 INFO:tasks.workunit.client.1.smithi104.stdout:0/631: rename d0/d4/d3b/c58 to d0/d2/cc5 0 2024-04-26T04:42:21.341 INFO:tasks.workunit.client.1.smithi104.stdout:3/778: write d5/db2/d46/f4d [147982,39011] 0 2024-04-26T04:42:21.341 INFO:tasks.workunit.client.1.smithi104.stdout:3/779: chown d5/db2/d46/d95/f108 31 1 2024-04-26T04:42:21.342 INFO:tasks.workunit.client.1.smithi104.stdout:4/783: mkdir d4/d7/d1d/d1e/d15/d113 0 2024-04-26T04:42:21.342 INFO:tasks.workunit.client.0.smithi090.stdout:8/462: read dc/d34/f40 [231171,73238] 0 2024-04-26T04:42:21.343 INFO:tasks.workunit.client.0.smithi090.stdout:8/463: dread - dc/d1f/d49/d4f/f9f zero size 2024-04-26T04:42:21.344 INFO:tasks.workunit.client.0.smithi090.stdout:7/449: dwrite d0/d1/f52 [4194304,4194304] 0 2024-04-26T04:42:21.345 INFO:tasks.workunit.client.0.smithi090.stdout:8/464: mknod dc/d1f/d49/d3b/d3d/d97/ca3 0 2024-04-26T04:42:21.345 INFO:tasks.workunit.client.0.smithi090.stdout:6/501: dread d9/d3c/f64 [0,4194304] 0 2024-04-26T04:42:21.347 INFO:tasks.workunit.client.1.smithi104.stdout:8/650: creat d0/d33/d35/fc8 x:0 0 0 2024-04-26T04:42:21.347 INFO:tasks.workunit.client.1.smithi104.stdout:8/651: chown d0/d1c/d27/d47/c62 302491 1 2024-04-26T04:42:21.347 INFO:tasks.workunit.client.1.smithi104.stdout:2/913: symlink d6/d8/l12c 0 2024-04-26T04:42:21.347 INFO:tasks.workunit.client.1.smithi104.stdout:6/803: dread d0/d9/d13/f91 [0,4194304] 0 2024-04-26T04:42:21.347 INFO:tasks.workunit.client.1.smithi104.stdout:1/762: creat d0/d6/d19/d21/d66/d20/ff5 x:0 0 0 2024-04-26T04:42:21.349 INFO:tasks.workunit.client.1.smithi104.stdout:0/632: link d0/d2/c3f d0/d2/d21/d27/d1e/cc6 0 2024-04-26T04:42:21.349 INFO:tasks.workunit.client.1.smithi104.stdout:4/784: symlink d4/d7/d1d/d2e/d74/dbd/dd3/l114 0 2024-04-26T04:42:21.351 INFO:tasks.workunit.client.1.smithi104.stdout:2/914: unlink d6/d8/fb 0 2024-04-26T04:42:21.351 INFO:tasks.workunit.client.1.smithi104.stdout:2/915: creat d6/d8/f12d x:0 0 0 2024-04-26T04:42:21.359 INFO:tasks.workunit.client.0.smithi090.stdout:8/465: dread dc/d5a/f6a [0,4194304] 0 2024-04-26T04:42:21.360 INFO:tasks.workunit.client.0.smithi090.stdout:8/466: creat dc/d34/d87/fa4 x:0 0 0 2024-04-26T04:42:21.365 INFO:tasks.workunit.client.1.smithi104.stdout:6/804: mkdir d0/d9/d25/d10c 0 2024-04-26T04:42:21.367 INFO:tasks.workunit.client.1.smithi104.stdout:7/688: dwrite d2/d4/d32/fcc [0,4194304] 0 2024-04-26T04:42:21.368 INFO:tasks.workunit.client.1.smithi104.stdout:8/652: link d0/f2e d0/d1c/d27/d47/db1/fc9 0 2024-04-26T04:42:21.370 INFO:tasks.workunit.client.1.smithi104.stdout:6/805: dread d0/d9/d13/d41/d96/dac/dc4/ff2 [0,4194304] 0 2024-04-26T04:42:21.371 INFO:tasks.workunit.client.1.smithi104.stdout:2/916: link d6/d8/d3e/f7a d6/de/d1d/dcf/f12e 0 2024-04-26T04:42:21.371 INFO:tasks.workunit.client.1.smithi104.stdout:6/806: write d0/d9/d13/dcd/dd9/f10a [121040,18392] 0 2024-04-26T04:42:21.374 INFO:tasks.workunit.client.1.smithi104.stdout:7/689: rmdir d2/d4/d32/d35/dbe 39 2024-04-26T04:42:21.375 INFO:tasks.workunit.client.1.smithi104.stdout:8/653: link d0/d33/d35/f87 d0/d1c/d2d/fca 0 2024-04-26T04:42:21.378 INFO:tasks.workunit.client.1.smithi104.stdout:2/917: symlink d6/d1b/d68/d81/l12f 0 2024-04-26T04:42:21.378 INFO:tasks.workunit.client.1.smithi104.stdout:2/918: chown d6/d8/d5a/c10e 3 1 2024-04-26T04:42:21.380 INFO:tasks.workunit.client.0.smithi090.stdout:9/335: dwrite db/dd/d18/d31/f52 [0,4194304] 0 2024-04-26T04:42:21.380 INFO:tasks.workunit.client.0.smithi090.stdout:4/398: dwrite d3/f7d [0,4194304] 0 2024-04-26T04:42:21.382 INFO:tasks.workunit.client.0.smithi090.stdout:9/336: mknod db/dd/d18/d31/d6a/c77 0 2024-04-26T04:42:21.383 INFO:tasks.workunit.client.1.smithi104.stdout:9/614: dwrite d11/f44 [0,4194304] 0 2024-04-26T04:42:21.383 INFO:tasks.workunit.client.1.smithi104.stdout:7/690: symlink d2/d4/d32/d3d/d47/db6/dce/ddd/lde 0 2024-04-26T04:42:21.383 INFO:tasks.workunit.client.1.smithi104.stdout:1/763: dwrite d0/d6/d19/d21/d66/d29/d3f/fe7 [0,4194304] 0 2024-04-26T04:42:21.384 INFO:tasks.workunit.client.0.smithi090.stdout:7/450: dwrite d0/db/d15/d29/fa2 [0,4194304] 0 2024-04-26T04:42:21.384 INFO:tasks.workunit.client.0.smithi090.stdout:7/451: truncate d0/db/df/f97 674215 0 2024-04-26T04:42:21.384 INFO:tasks.workunit.client.0.smithi090.stdout:7/452: fdatasync d0/db/d53/f87 0 2024-04-26T04:42:21.385 INFO:tasks.workunit.client.1.smithi104.stdout:8/654: rename d0/d33/d35/d6d/f74 to d0/d33/d35/d6d/da4/fcb 0 2024-04-26T04:42:21.385 INFO:tasks.workunit.client.1.smithi104.stdout:8/655: chown d0/d1c/d27/l5f 4506 1 2024-04-26T04:42:21.385 INFO:tasks.workunit.client.1.smithi104.stdout:8/656: fdatasync d0/d1c/d27/f40 0 2024-04-26T04:42:21.385 INFO:tasks.workunit.client.1.smithi104.stdout:8/657: truncate d0/d1c/d27/d47/d67/d76/d7f/fc2 80033 0 2024-04-26T04:42:21.389 INFO:tasks.workunit.client.1.smithi104.stdout:9/615: mknod d11/d15/d32/d4c/d54/cc9 0 2024-04-26T04:42:21.393 INFO:tasks.workunit.client.0.smithi090.stdout:9/337: unlink db/f26 0 2024-04-26T04:42:21.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/453: dread d0/db/d15/d29/fa2 [0,4194304] 0 2024-04-26T04:42:21.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/454: write d0/db/d53/d5e/f8b [249766,78335] 0 2024-04-26T04:42:21.395 INFO:tasks.workunit.client.1.smithi104.stdout:1/764: creat d0/d6/d19/d21/d66/ff6 x:0 0 0 2024-04-26T04:42:21.395 INFO:tasks.workunit.client.1.smithi104.stdout:1/765: write d0/d6/d19/d21/d66/d20/d2a/fcc [308414,128220] 0 2024-04-26T04:42:21.395 INFO:tasks.workunit.client.1.smithi104.stdout:9/616: dread d11/d15/d32/f9a [0,4194304] 0 2024-04-26T04:42:21.396 INFO:tasks.workunit.client.1.smithi104.stdout:8/658: symlink d0/d33/d35/d6d/da4/lcc 0 2024-04-26T04:42:21.397 INFO:tasks.workunit.client.1.smithi104.stdout:2/919: dwrite d6/d1b/d51/d6a/f6f [4194304,4194304] 0 2024-04-26T04:42:21.398 INFO:tasks.workunit.client.0.smithi090.stdout:7/455: dread d0/db/df/d1c/d23/f60 [0,4194304] 0 2024-04-26T04:42:21.398 INFO:tasks.workunit.client.1.smithi104.stdout:1/766: mkdir d0/d6/d19/d21/d66/d29/df7 0 2024-04-26T04:42:21.399 INFO:tasks.workunit.client.1.smithi104.stdout:6/807: dwrite d0/d9/d13/d41/d96/dac/f67 [0,4194304] 0 2024-04-26T04:42:21.399 INFO:tasks.workunit.client.1.smithi104.stdout:6/808: chown d0/d9/d13/d41/d82/dc8/f84 14261 1 2024-04-26T04:42:21.399 INFO:tasks.workunit.client.1.smithi104.stdout:6/809: write d0/d9/d13/d41/d82/dc8/fd6 [1457015,71288] 0 2024-04-26T04:42:21.399 INFO:tasks.workunit.client.1.smithi104.stdout:6/810: fsync d0/f9d 0 2024-04-26T04:42:21.399 INFO:tasks.workunit.client.0.smithi090.stdout:9/338: unlink f3 0 2024-04-26T04:42:21.400 INFO:tasks.workunit.client.1.smithi104.stdout:9/617: mknod d11/d15/d1b/d6d/d9c/dbe/cca 0 2024-04-26T04:42:21.400 INFO:tasks.workunit.client.0.smithi090.stdout:7/456: creat d0/d1/da3/fa4 x:0 0 0 2024-04-26T04:42:21.401 INFO:tasks.workunit.client.0.smithi090.stdout:9/339: rmdir db/dd/d18/d38/d39/d3f 39 2024-04-26T04:42:21.401 INFO:tasks.workunit.client.1.smithi104.stdout:7/691: dread d2/d4/f33 [0,4194304] 0 2024-04-26T04:42:21.402 INFO:tasks.workunit.client.1.smithi104.stdout:2/920: rename d6/de/d8e/f69 to d6/d1b/d68/d105/f130 0 2024-04-26T04:42:21.404 INFO:tasks.workunit.client.0.smithi090.stdout:7/457: rename d0/d1/d4/l58 to d0/d1/da3/la5 0 2024-04-26T04:42:21.404 INFO:tasks.workunit.client.0.smithi090.stdout:7/458: chown d0/f8 70288746 1 2024-04-26T04:42:21.404 INFO:tasks.workunit.client.0.smithi090.stdout:7/459: creat d0/db/df/d1c/fa6 x:0 0 0 2024-04-26T04:42:21.404 INFO:tasks.workunit.client.0.smithi090.stdout:7/460: chown d0/db/d15/d29/c3f 122601669 1 2024-04-26T04:42:21.405 INFO:tasks.workunit.client.1.smithi104.stdout:6/811: unlink d0/d9/d13/dcd/dd9/cf0 0 2024-04-26T04:42:21.406 INFO:tasks.workunit.client.0.smithi090.stdout:9/340: symlink db/dd/d18/d31/d58/l78 0 2024-04-26T04:42:21.409 INFO:tasks.workunit.client.1.smithi104.stdout:1/767: dread d0/d6/d19/d21/d66/f83 [0,4194304] 0 2024-04-26T04:42:21.409 INFO:tasks.workunit.client.1.smithi104.stdout:1/768: write d0/d6/d19/d21/d66/d20/ff5 [799867,34338] 0 2024-04-26T04:42:21.409 INFO:tasks.workunit.client.0.smithi090.stdout:7/461: mknod d0/db/d15/d30/ca7 0 2024-04-26T04:42:21.409 INFO:tasks.workunit.client.0.smithi090.stdout:7/462: fsync d0/db/df/f70 0 2024-04-26T04:42:21.410 INFO:tasks.workunit.client.1.smithi104.stdout:9/618: mknod d11/d15/d1b/d6d/ccb 0 2024-04-26T04:42:21.410 INFO:tasks.workunit.client.1.smithi104.stdout:9/619: fsync d11/f89 0 2024-04-26T04:42:21.410 INFO:tasks.workunit.client.0.smithi090.stdout:4/399: truncate d3/f7d 870946 0 2024-04-26T04:42:21.411 INFO:tasks.workunit.client.0.smithi090.stdout:7/463: mknod d0/d1/ca8 0 2024-04-26T04:42:21.411 INFO:tasks.workunit.client.0.smithi090.stdout:7/464: creat d0/db/d53/d5e/fa9 x:0 0 0 2024-04-26T04:42:21.413 INFO:tasks.workunit.client.0.smithi090.stdout:4/400: creat d3/db/d12/d20/d45/f7f x:0 0 0 2024-04-26T04:42:21.417 INFO:tasks.workunit.client.0.smithi090.stdout:7/465: creat d0/db/df/d4d/d64/d66/faa x:0 0 0 2024-04-26T04:42:21.419 INFO:tasks.workunit.client.1.smithi104.stdout:6/812: unlink d0/d9/d13/d44/f101 0 2024-04-26T04:42:21.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/466: symlink d0/db/d15/d30/lab 0 2024-04-26T04:42:21.421 INFO:tasks.workunit.client.1.smithi104.stdout:1/769: rename d0/d6/d2e/d32/d6a to d0/d6/d19/d21/d66/d20/d2a/dbc/df8 0 2024-04-26T04:42:21.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/467: unlink d0/db/d15/d30/lab 0 2024-04-26T04:42:21.422 INFO:tasks.workunit.client.1.smithi104.stdout:9/620: creat d11/d1d/d71/fcc x:0 0 0 2024-04-26T04:42:21.422 INFO:tasks.workunit.client.1.smithi104.stdout:9/621: chown ld 38 1 2024-04-26T04:42:21.423 INFO:tasks.workunit.client.0.smithi090.stdout:7/468: dread d0/db/df/d1c/f7a [0,4194304] 0 2024-04-26T04:42:21.424 INFO:tasks.workunit.client.0.smithi090.stdout:7/469: fdatasync d0/db/df/d4d/f5a 0 2024-04-26T04:42:21.424 INFO:tasks.workunit.client.1.smithi104.stdout:1/770: creat d0/d6/d19/d21/d66/de4/d7e/ff9 x:0 0 0 2024-04-26T04:42:21.425 INFO:tasks.workunit.client.1.smithi104.stdout:7/692: dwrite d2/d4/d32/d3d/d47/f61 [0,4194304] 0 2024-04-26T04:42:21.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/470: symlink d0/d1/d4/lac 0 2024-04-26T04:42:21.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/471: truncate d0/db/df/f97 1717022 0 2024-04-26T04:42:21.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/472: write d0/db/d15/d30/d4f/f95 [503165,62576] 0 2024-04-26T04:42:21.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/473: fsync d0/db/d53/fa 0 2024-04-26T04:42:21.426 INFO:tasks.workunit.client.1.smithi104.stdout:1/771: symlink d0/d6/d19/d21/d66/d20/d2a/lfa 0 2024-04-26T04:42:21.427 INFO:tasks.workunit.client.0.smithi090.stdout:9/341: dwrite db/dd/d18/f3a [0,4194304] 0 2024-04-26T04:42:21.427 INFO:tasks.workunit.client.0.smithi090.stdout:9/342: creat db/dd/d18/d38/f79 x:0 0 0 2024-04-26T04:42:21.427 INFO:tasks.workunit.client.1.smithi104.stdout:7/693: mknod d2/d4/d5a/cdf 0 2024-04-26T04:42:21.430 INFO:tasks.workunit.client.1.smithi104.stdout:1/772: unlink d0/d6/d19/d21/d58/lb9 0 2024-04-26T04:42:21.430 INFO:tasks.workunit.client.0.smithi090.stdout:7/474: rename d0/db/d15/l36 to d0/db/df/lad 0 2024-04-26T04:42:21.431 INFO:tasks.workunit.client.1.smithi104.stdout:7/694: mkdir d2/d4/d32/d3d/d41/d7e/de0 0 2024-04-26T04:42:21.433 INFO:tasks.workunit.client.1.smithi104.stdout:1/773: creat d0/d6/d19/d21/d66/d29/df7/ffb x:0 0 0 2024-04-26T04:42:21.433 INFO:tasks.workunit.client.1.smithi104.stdout:1/774: read d0/d6/d19/d21/d66/d20/d2a/f31 [2767966,105220] 0 2024-04-26T04:42:21.433 INFO:tasks.workunit.client.0.smithi090.stdout:9/343: write db/f41 [190802,17398] 0 2024-04-26T04:42:21.435 INFO:tasks.workunit.client.0.smithi090.stdout:1/370: sync 2024-04-26T04:42:21.435 INFO:tasks.workunit.client.0.smithi090.stdout:3/346: sync 2024-04-26T04:42:21.435 INFO:tasks.workunit.client.0.smithi090.stdout:1/371: stat d0/d4/d7/d17/d22/d31/f39 0 2024-04-26T04:42:21.436 INFO:tasks.workunit.client.1.smithi104.stdout:7/695: creat d2/d4/d32/d35/d44/d5b/db2/fe1 x:0 0 0 2024-04-26T04:42:21.436 INFO:tasks.workunit.client.1.smithi104.stdout:7/696: readlink d2/l83 0 2024-04-26T04:42:21.436 INFO:tasks.workunit.client.0.smithi090.stdout:3/347: dread d1/d9/f38 [0,4194304] 0 2024-04-26T04:42:21.439 INFO:tasks.workunit.client.0.smithi090.stdout:7/475: link d0/d1/d4/d3e/l68 d0/db/df/d1c/d23/lae 0 2024-04-26T04:42:21.442 INFO:tasks.workunit.client.0.smithi090.stdout:4/401: dwrite d3/db/d12/d3b/f78 [0,4194304] 0 2024-04-26T04:42:21.444 INFO:tasks.workunit.client.0.smithi090.stdout:3/348: rename d1/d6/d2c/f49 to d1/d17/d52/f7c 0 2024-04-26T04:42:21.444 INFO:tasks.workunit.client.1.smithi104.stdout:6/813: dwrite d0/d9/d13/d41/d96/dac/dc4/fcf [0,4194304] 0 2024-04-26T04:42:21.447 INFO:tasks.workunit.client.0.smithi090.stdout:7/476: mkdir d0/db/df/d4d/daf 0 2024-04-26T04:42:21.449 INFO:tasks.workunit.client.0.smithi090.stdout:1/372: symlink d0/d4/d7/d17/d22/l70 0 2024-04-26T04:42:21.450 INFO:tasks.workunit.client.0.smithi090.stdout:4/402: truncate d3/f7d 527962 0 2024-04-26T04:42:21.451 INFO:tasks.workunit.client.0.smithi090.stdout:1/373: creat d0/d4/d7/d69/f71 x:0 0 0 2024-04-26T04:42:21.481 INFO:tasks.workunit.client.1.smithi104.stdout:9/622: dwrite d11/d1d/f49 [4194304,4194304] 0 2024-04-26T04:42:21.481 INFO:tasks.workunit.client.1.smithi104.stdout:9/623: creat d11/d15/d1b/d6d/fcd x:0 0 0 2024-04-26T04:42:21.483 INFO:tasks.workunit.client.1.smithi104.stdout:9/624: dread d11/f20 [0,4194304] 0 2024-04-26T04:42:21.483 INFO:tasks.workunit.client.1.smithi104.stdout:9/625: chown d11/d90/f99 7 1 2024-04-26T04:42:21.483 INFO:tasks.workunit.client.1.smithi104.stdout:9/626: readlink d11/d15/l7f 0 2024-04-26T04:42:21.483 INFO:tasks.workunit.client.1.smithi104.stdout:9/627: creat d11/d1d/d34/d69/fce x:0 0 0 2024-04-26T04:42:21.483 INFO:tasks.workunit.client.1.smithi104.stdout:9/628: creat d11/d53/d84/fcf x:0 0 0 2024-04-26T04:42:21.484 INFO:tasks.workunit.client.1.smithi104.stdout:9/629: dread d11/d15/f6a [0,4194304] 0 2024-04-26T04:42:21.485 INFO:tasks.workunit.client.1.smithi104.stdout:9/630: symlink d11/d15/d32/d4c/ld0 0 2024-04-26T04:42:21.486 INFO:tasks.workunit.client.1.smithi104.stdout:9/631: creat d11/d15/da3/fd1 x:0 0 0 2024-04-26T04:42:21.486 INFO:tasks.workunit.client.1.smithi104.stdout:9/632: fdatasync d11/d15/d32/d4c/d54/fc5 0 2024-04-26T04:42:21.486 INFO:tasks.workunit.client.1.smithi104.stdout:9/633: fdatasync d11/d1d/d34/f6c 0 2024-04-26T04:42:21.487 INFO:tasks.workunit.client.1.smithi104.stdout:9/634: symlink d11/d15/d32/d35/d7b/ld2 0 2024-04-26T04:42:21.487 INFO:tasks.workunit.client.1.smithi104.stdout:9/635: dread - d11/f95 zero size 2024-04-26T04:42:21.487 INFO:tasks.workunit.client.1.smithi104.stdout:9/636: fsync d11/d15/d32/d4c/d54/fc5 0 2024-04-26T04:42:21.487 INFO:tasks.workunit.client.1.smithi104.stdout:9/637: creat d11/fd3 x:0 0 0 2024-04-26T04:42:21.493 INFO:tasks.workunit.client.1.smithi104.stdout:1/775: dwrite d0/d6/fc0 [0,4194304] 0 2024-04-26T04:42:21.493 INFO:tasks.workunit.client.1.smithi104.stdout:1/776: dread - d0/d6/d19/d21/d59/f94 zero size 2024-04-26T04:42:21.494 INFO:tasks.workunit.client.1.smithi104.stdout:7/697: dwrite d2/d4/d32/d3d/d47/f8a [0,4194304] 0 2024-04-26T04:42:21.496 INFO:tasks.workunit.client.0.smithi090.stdout:3/349: dwrite d1/d59/d5e/f62 [0,4194304] 0 2024-04-26T04:42:21.496 INFO:tasks.workunit.client.0.smithi090.stdout:3/350: write d1/d17/d36/f32 [17554,24887] 0 2024-04-26T04:42:21.496 INFO:tasks.workunit.client.0.smithi090.stdout:3/351: stat d1/d17/d36/d2a/d3a/d5d 0 2024-04-26T04:42:21.497 INFO:tasks.workunit.client.0.smithi090.stdout:1/374: dwrite d0/d4/f20 [0,4194304] 0 2024-04-26T04:42:21.497 INFO:tasks.workunit.client.0.smithi090.stdout:9/344: symlink db/dd/d18/d38/d39/l7a 0 2024-04-26T04:42:21.499 INFO:tasks.workunit.client.1.smithi104.stdout:7/698: truncate d2/d4/f6e 3621219 0 2024-04-26T04:42:21.501 INFO:tasks.workunit.client.1.smithi104.stdout:6/814: rename d0/d9/d13/d44 to d0/d9/d13/d41/d96/ded/d10d 0 2024-04-26T04:42:21.502 INFO:tasks.workunit.client.0.smithi090.stdout:7/477: dwrite d0/d1/d4/f6b [4194304,4194304] 0 2024-04-26T04:42:21.505 INFO:tasks.workunit.client.0.smithi090.stdout:4/403: truncate d3/f7d 703228 0 2024-04-26T04:42:21.507 INFO:tasks.workunit.client.1.smithi104.stdout:7/699: getdents d2/d4/d1d/d34/d7f 0 2024-04-26T04:42:21.507 INFO:tasks.workunit.client.0.smithi090.stdout:1/375: symlink d0/d4/d62/l72 0 2024-04-26T04:42:21.509 INFO:tasks.workunit.client.1.smithi104.stdout:7/700: mkdir d2/d4/d32/d3d/d41/de2 0 2024-04-26T04:42:21.510 INFO:tasks.workunit.client.1.smithi104.stdout:7/701: creat d2/d4/fe3 x:0 0 0 2024-04-26T04:42:21.510 INFO:tasks.workunit.client.0.smithi090.stdout:7/478: mkdir d0/db/df/d4d/d64/db0 0 2024-04-26T04:42:21.511 INFO:tasks.workunit.client.0.smithi090.stdout:4/404: creat d3/db/d12/d20/d24/d43/d72/f80 x:0 0 0 2024-04-26T04:42:21.512 INFO:tasks.workunit.client.0.smithi090.stdout:4/405: creat d3/d13/f81 x:0 0 0 2024-04-26T04:42:21.513 INFO:tasks.workunit.client.0.smithi090.stdout:9/345: symlink db/dd/d18/d38/d39/d6b/l7b 0 2024-04-26T04:42:21.513 INFO:tasks.workunit.client.1.smithi104.stdout:9/638: dwrite d11/d15/d1b/f72 [0,4194304] 0 2024-04-26T04:42:21.514 INFO:tasks.workunit.client.1.smithi104.stdout:9/639: dread - d11/d15/d32/d4c/d54/fc5 zero size 2024-04-26T04:42:21.514 INFO:tasks.workunit.client.0.smithi090.stdout:7/479: mkdir d0/d1/d4/d98/db1 0 2024-04-26T04:42:21.515 INFO:tasks.workunit.client.1.smithi104.stdout:6/815: dread d0/d9/d13/f20 [0,4194304] 0 2024-04-26T04:42:21.517 INFO:tasks.workunit.client.0.smithi090.stdout:4/406: truncate d3/db/d12/d20/f67 375554 0 2024-04-26T04:42:21.517 INFO:tasks.workunit.client.0.smithi090.stdout:4/407: creat d3/db/d12/d3b/f82 x:0 0 0 2024-04-26T04:42:21.517 INFO:tasks.workunit.client.1.smithi104.stdout:6/816: creat d0/d9/d13/d41/d82/dc8/d80/dca/f10e x:0 0 0 2024-04-26T04:42:21.518 INFO:tasks.workunit.client.1.smithi104.stdout:6/817: chown d0/d9/d13/d41/d82/dc8/dba/daa 772214700 1 2024-04-26T04:42:21.518 INFO:tasks.workunit.client.1.smithi104.stdout:6/818: stat d0/d9/d13/d41/d82/dc8/f84 0 2024-04-26T04:42:21.519 INFO:tasks.workunit.client.0.smithi090.stdout:1/376: rename d0/d4/d7/d17/d22/f23 to d0/d4/d7/d17/d22/f73 0 2024-04-26T04:42:21.519 INFO:tasks.workunit.client.0.smithi090.stdout:1/377: fdatasync d0/d4/d7/d4b/f5b 0 2024-04-26T04:42:21.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/346: link db/d16/l1f db/d5f/l7c 0 2024-04-26T04:42:21.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/347: read db/dd/d18/d57/f68 [384091,90965] 0 2024-04-26T04:42:21.521 INFO:tasks.workunit.client.0.smithi090.stdout:7/480: mkdir d0/d1/d4/d98/d65/db2 0 2024-04-26T04:42:21.521 INFO:tasks.workunit.client.0.smithi090.stdout:7/481: write d0/db/d53/f16 [921405,5083] 0 2024-04-26T04:42:21.522 INFO:tasks.workunit.client.1.smithi104.stdout:9/640: dread d11/d15/d32/d35/f3a [4194304,4194304] 0 2024-04-26T04:42:21.522 INFO:tasks.workunit.client.1.smithi104.stdout:5/723: sync 2024-04-26T04:42:21.523 INFO:tasks.workunit.client.0.smithi090.stdout:4/408: creat d3/db/d12/d20/d24/d43/d72/d7a/f83 x:0 0 0 2024-04-26T04:42:21.523 INFO:tasks.workunit.client.0.smithi090.stdout:4/409: write d3/db/d12/d3b/f75 [94674,22418] 0 2024-04-26T04:42:21.523 INFO:tasks.workunit.client.0.smithi090.stdout:9/348: dread db/dd/d18/f36 [0,4194304] 0 2024-04-26T04:42:21.525 INFO:tasks.workunit.client.1.smithi104.stdout:9/641: creat d11/d1d/d87/fd4 x:0 0 0 2024-04-26T04:42:21.526 INFO:tasks.workunit.client.1.smithi104.stdout:9/642: mkdir d11/d15/d1b/d6d/dd5 0 2024-04-26T04:42:21.527 INFO:tasks.workunit.client.0.smithi090.stdout:7/482: unlink d0/d1/d4/d98/f9a 0 2024-04-26T04:42:21.527 INFO:tasks.workunit.client.0.smithi090.stdout:5/498: sync 2024-04-26T04:42:21.527 INFO:tasks.workunit.client.0.smithi090.stdout:2/447: sync 2024-04-26T04:42:21.527 INFO:tasks.workunit.client.0.smithi090.stdout:5/499: stat d4/d1a/d1f/f3d 0 2024-04-26T04:42:21.527 INFO:tasks.workunit.client.0.smithi090.stdout:2/448: chown db/d11/d14/d27/f7b 1 1 2024-04-26T04:42:21.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/410: mkdir d3/db/d12/d20/d84 0 2024-04-26T04:42:21.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/411: dread - d3/db/d12/d20/d45/d53/f65 zero size 2024-04-26T04:42:21.528 INFO:tasks.workunit.client.1.smithi104.stdout:9/643: mkdir d11/d15/d32/dd6 0 2024-04-26T04:42:21.535 INFO:tasks.workunit.client.0.smithi090.stdout:1/378: rename d0/f5a to d0/d4/d7/d17/d22/f74 0 2024-04-26T04:42:21.537 INFO:tasks.workunit.client.0.smithi090.stdout:7/483: mkdir d0/db/df/d4d/d7b/db3 0 2024-04-26T04:42:21.539 INFO:tasks.workunit.client.0.smithi090.stdout:2/449: dread db/d59/f62 [0,4194304] 0 2024-04-26T04:42:21.541 INFO:tasks.workunit.client.0.smithi090.stdout:9/349: mknod db/dd/d18/d38/d39/c7d 0 2024-04-26T04:42:21.544 INFO:tasks.workunit.client.0.smithi090.stdout:1/379: mkdir d0/d4/d7/d4a/d75 0 2024-04-26T04:42:21.544 INFO:tasks.workunit.client.0.smithi090.stdout:1/380: readlink d0/d4/d7/d17/l42 0 2024-04-26T04:42:21.545 INFO:tasks.workunit.client.0.smithi090.stdout:4/412: write d3/db/d12/d20/d24/f32 [1182917,16011] 0 2024-04-26T04:42:21.545 INFO:tasks.workunit.client.0.smithi090.stdout:7/484: link d0/db/d15/d30/l54 d0/d1/da3/lb4 0 2024-04-26T04:42:21.546 INFO:tasks.workunit.client.0.smithi090.stdout:2/450: mkdir db/d59/d66/d96 0 2024-04-26T04:42:21.546 INFO:tasks.workunit.client.1.smithi104.stdout:1/777: dwrite d0/d6/d19/d21/d66/f41 [0,4194304] 0 2024-04-26T04:42:21.547 INFO:tasks.workunit.client.0.smithi090.stdout:1/381: getdents d0/d4/d62 0 2024-04-26T04:42:21.548 INFO:tasks.workunit.client.0.smithi090.stdout:7/485: creat d0/d1/d4/d98/db1/fb5 x:0 0 0 2024-04-26T04:42:21.549 INFO:tasks.workunit.client.1.smithi104.stdout:1/778: mknod d0/d6/d19/d21/d66/de4/d7e/dc6/cfc 0 2024-04-26T04:42:21.549 INFO:tasks.workunit.client.1.smithi104.stdout:1/779: chown d0/d6/d19/d1b/d35/c7d 200196 1 2024-04-26T04:42:21.549 INFO:tasks.workunit.client.0.smithi090.stdout:2/451: rename db/dc/l29 to db/d11/d14/d27/d2f/d42/d5c/l97 0 2024-04-26T04:42:21.550 INFO:tasks.workunit.client.1.smithi104.stdout:1/780: mknod d0/d6/d49/cfd 0 2024-04-26T04:42:21.550 INFO:tasks.workunit.client.1.smithi104.stdout:1/781: write d0/d6/d19/d1b/f7f [101398,114078] 0 2024-04-26T04:42:21.550 INFO:tasks.workunit.client.0.smithi090.stdout:1/382: symlink d0/d4/d7/d49/l76 0 2024-04-26T04:42:21.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/452: read db/d11/d14/f4a [350706,83759] 0 2024-04-26T04:42:21.551 INFO:tasks.workunit.client.0.smithi090.stdout:1/383: creat d0/d4/d7/d4a/f77 x:0 0 0 2024-04-26T04:42:21.551 INFO:tasks.workunit.client.0.smithi090.stdout:1/384: dread - d0/d4/d7/d17/d22/f74 zero size 2024-04-26T04:42:21.552 INFO:tasks.workunit.client.0.smithi090.stdout:2/453: mkdir db/d11/d14/d1b/d20/d4c/d98 0 2024-04-26T04:42:21.552 INFO:tasks.workunit.client.0.smithi090.stdout:1/385: mkdir d0/d4/d62/d78 0 2024-04-26T04:42:21.553 INFO:tasks.workunit.client.1.smithi104.stdout:1/782: rename d0/f80 to d0/d6/d19/d21/d66/d20/d2a/d56/d67/ffe 0 2024-04-26T04:42:21.553 INFO:tasks.workunit.client.1.smithi104.stdout:1/783: chown d0/d6/c2b 252874549 1 2024-04-26T04:42:21.553 INFO:tasks.workunit.client.1.smithi104.stdout:1/784: stat d0/d6/d19/d21/d66/f83 0 2024-04-26T04:42:21.553 INFO:tasks.workunit.client.0.smithi090.stdout:2/454: symlink db/d11/d14/d25/d65/d95/l99 0 2024-04-26T04:42:21.554 INFO:tasks.workunit.client.0.smithi090.stdout:1/386: creat d0/d4/d7/d17/f79 x:0 0 0 2024-04-26T04:42:21.554 INFO:tasks.workunit.client.1.smithi104.stdout:1/785: link d0/d6/d9e/lc1 d0/d6/d49/lff 0 2024-04-26T04:42:21.556 INFO:tasks.workunit.client.0.smithi090.stdout:2/455: creat db/d11/d14/d25/f9a x:0 0 0 2024-04-26T04:42:21.557 INFO:tasks.workunit.client.1.smithi104.stdout:1/786: symlink d0/d6/d19/d21/d66/d20/da3/dde/l100 0 2024-04-26T04:42:21.557 INFO:tasks.workunit.client.1.smithi104.stdout:1/787: chown d0/d6/d19/d21/d66/de4/d81/f8b 860794 1 2024-04-26T04:42:21.557 INFO:tasks.workunit.client.1.smithi104.stdout:1/788: dread - d0/d6/d19/d1b/da7/db8/fe6 zero size 2024-04-26T04:42:21.560 INFO:tasks.workunit.client.1.smithi104.stdout:1/789: unlink d0/d6/ff 0 2024-04-26T04:42:21.560 INFO:tasks.workunit.client.1.smithi104.stdout:1/790: dread - d0/d6/d19/d21/d66/ff6 zero size 2024-04-26T04:42:21.560 INFO:tasks.workunit.client.1.smithi104.stdout:1/791: chown d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/fb6 83699 1 2024-04-26T04:42:21.564 INFO:tasks.workunit.client.0.smithi090.stdout:9/350: dread db/dd/d44/f34 [0,4194304] 0 2024-04-26T04:42:21.575 INFO:tasks.workunit.client.1.smithi104.stdout:9/644: dwrite d11/d15/d32/d4c/d54/f77 [0,4194304] 0 2024-04-26T04:42:21.575 INFO:tasks.workunit.client.1.smithi104.stdout:9/645: truncate d11/fc6 821585 0 2024-04-26T04:42:21.576 INFO:tasks.workunit.client.1.smithi104.stdout:9/646: write d11/d1d/d34/d69/f78 [651054,14373] 0 2024-04-26T04:42:21.577 INFO:tasks.workunit.client.1.smithi104.stdout:9/647: getdents d11/d53/d84 0 2024-04-26T04:42:21.578 INFO:tasks.workunit.client.1.smithi104.stdout:9/648: rmdir d11/d90 39 2024-04-26T04:42:21.586 INFO:tasks.workunit.client.1.smithi104.stdout:5/724: dwrite d0/d41/d4f/fa0 [0,4194304] 0 2024-04-26T04:42:21.586 INFO:tasks.workunit.client.1.smithi104.stdout:6/819: dwrite d0/dd8/fe9 [0,4194304] 0 2024-04-26T04:42:21.586 INFO:tasks.workunit.client.0.smithi090.stdout:3/352: dwrite d1/d17/d52/f7c [0,4194304] 0 2024-04-26T04:42:21.589 INFO:tasks.workunit.client.0.smithi090.stdout:7/486: dwrite d0/db/df/d4d/f5a [0,4194304] 0 2024-04-26T04:42:21.590 INFO:tasks.workunit.client.0.smithi090.stdout:1/387: dwrite d0/d4/d7/d49/f6f [0,4194304] 0 2024-04-26T04:42:21.590 INFO:tasks.workunit.client.0.smithi090.stdout:2/456: dwrite db/f76 [0,4194304] 0 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:6/820: unlink d0/d9/d13/d41/d82/dc8/dba/fdb 0 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:7/702: dwrite d2/d4/d32/d3d/d47/f93 [0,4194304] 0 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:7/703: write d2/d4/d32/d35/d44/fa4 [994811,91008] 0 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:7/704: dread - d2/d4/d1d/d34/db0/fd5 zero size 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:7/705: chown d2/d4/d32/d3d/d41/d5f/d6c/d89/c96 3758007 1 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:7/706: fsync d2/d4/d32/f76 0 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:7/707: stat d2/d4/d32/d3d/d47/lc8 0 2024-04-26T04:42:21.591 INFO:tasks.workunit.client.1.smithi104.stdout:7/708: chown d2/d4/d32/d3d/d47/db6/dce/ddd 0 1 2024-04-26T04:42:21.592 INFO:tasks.workunit.client.0.smithi090.stdout:4/413: dread d3/db/d12/d20/d24/f32 [0,4194304] 0 2024-04-26T04:42:21.592 INFO:tasks.workunit.client.0.smithi090.stdout:4/414: stat d3/db/f23 0 2024-04-26T04:42:21.592 INFO:tasks.workunit.client.1.smithi104.stdout:5/725: mkdir d0/d1/d1e/df5 0 2024-04-26T04:42:21.592 INFO:tasks.workunit.client.1.smithi104.stdout:9/649: dread d11/d15/d32/d35/f80 [0,4194304] 0 2024-04-26T04:42:21.596 INFO:tasks.workunit.client.1.smithi104.stdout:6/821: mknod d0/d9/d13/d41/d96/ded/d10d/d71/da5/c10f 0 2024-04-26T04:42:21.598 INFO:tasks.workunit.client.0.smithi090.stdout:7/487: dread d0/f45 [0,4194304] 0 2024-04-26T04:42:21.600 INFO:tasks.workunit.client.0.smithi090.stdout:4/415: rmdir d3/db/d12/d20/d84 0 2024-04-26T04:42:21.600 INFO:tasks.workunit.client.1.smithi104.stdout:7/709: mkdir d2/d4/d1d/d34/db0/de4 0 2024-04-26T04:42:21.600 INFO:tasks.workunit.client.1.smithi104.stdout:7/710: truncate d2/d4/d32/d35/d44/d5b/fd4 472518 0 2024-04-26T04:42:21.602 INFO:tasks.workunit.client.0.smithi090.stdout:7/488: rename d0/db/d15/f57 to d0/d1/d4/d98/d65/db2/fb6 0 2024-04-26T04:42:21.603 INFO:tasks.workunit.client.0.smithi090.stdout:2/457: dread db/d11/d14/d25/f3a [0,4194304] 0 2024-04-26T04:42:21.604 INFO:tasks.workunit.client.1.smithi104.stdout:5/726: unlink d0/d1/d1e/d98/fae 0 2024-04-26T04:42:21.604 INFO:tasks.workunit.client.0.smithi090.stdout:1/388: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:21.604 INFO:tasks.workunit.client.1.smithi104.stdout:9/650: rmdir d11/d15/d98 39 2024-04-26T04:42:21.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/489: truncate d0/db/d53/f17 1250729 0 2024-04-26T04:42:21.607 INFO:tasks.workunit.client.1.smithi104.stdout:7/711: mkdir d2/d4/d1d/d34/db0/de4/de5 0 2024-04-26T04:42:21.607 INFO:tasks.workunit.client.1.smithi104.stdout:7/712: creat d2/d4/d32/d3d/d41/d5f/d6d/fe6 x:0 0 0 2024-04-26T04:42:21.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/389: mknod d0/d4/d7/c7a 0 2024-04-26T04:42:21.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/390: stat d0/d4/d7/d4a/c4f 0 2024-04-26T04:42:21.608 INFO:tasks.workunit.client.0.smithi090.stdout:1/391: fsync d0/d55/f63 0 2024-04-26T04:42:21.609 INFO:tasks.workunit.client.1.smithi104.stdout:5/727: creat d0/d1/de6/ff6 x:0 0 0 2024-04-26T04:42:21.611 INFO:tasks.workunit.client.1.smithi104.stdout:9/651: unlink d11/d15/d32/d35/l5b 0 2024-04-26T04:42:21.611 INFO:tasks.workunit.client.1.smithi104.stdout:9/652: write d11/d15/d32/f56 [331515,108931] 0 2024-04-26T04:42:21.612 INFO:tasks.workunit.client.0.smithi090.stdout:3/353: dwrite d1/d6/d1b/f35 [0,4194304] 0 2024-04-26T04:42:21.614 INFO:tasks.workunit.client.0.smithi090.stdout:3/354: mkdir d1/d17/d52/d7d 0 2024-04-26T04:42:21.616 INFO:tasks.workunit.client.0.smithi090.stdout:2/458: write db/f76 [1795413,92536] 0 2024-04-26T04:42:21.616 INFO:tasks.workunit.client.0.smithi090.stdout:7/490: rename d0/db/d15/d30 to d0/db/d53/db7 0 2024-04-26T04:42:21.616 INFO:tasks.workunit.client.0.smithi090.stdout:7/491: chown d0/d1/d4/d98/d4a/l9c 364853 1 2024-04-26T04:42:21.616 INFO:tasks.workunit.client.1.smithi104.stdout:5/728: creat d0/d1/d4/d2a/d2b/d37/ff7 x:0 0 0 2024-04-26T04:42:21.618 INFO:tasks.workunit.client.0.smithi090.stdout:3/355: truncate d1/d59/d5e/f62 1092894 0 2024-04-26T04:42:21.618 INFO:tasks.workunit.client.0.smithi090.stdout:4/416: dwrite d3/db/d12/f4c [0,4194304] 0 2024-04-26T04:42:21.618 INFO:tasks.workunit.client.0.smithi090.stdout:4/417: write d3/db/d12/f40 [445301,92854] 0 2024-04-26T04:42:21.619 INFO:tasks.workunit.client.1.smithi104.stdout:9/653: mkdir d11/d15/d1b/d6d/d9c/dd7 0 2024-04-26T04:42:21.621 INFO:tasks.workunit.client.0.smithi090.stdout:2/459: mknod db/d11/d14/d27/d2f/d70/c9b 0 2024-04-26T04:42:21.623 INFO:tasks.workunit.client.0.smithi090.stdout:3/356: mkdir d1/d17/d36/d2a/d3a/d5d/d7e 0 2024-04-26T04:42:21.626 INFO:tasks.workunit.client.0.smithi090.stdout:4/418: mkdir d3/db/d12/d20/d2d/d58/d6a/d74/d85 0 2024-04-26T04:42:21.626 INFO:tasks.workunit.client.0.smithi090.stdout:4/419: truncate d3/f6d 264534 0 2024-04-26T04:42:21.627 INFO:tasks.workunit.client.0.smithi090.stdout:2/460: getdents db/d11/d14/d63 0 2024-04-26T04:42:21.627 INFO:tasks.workunit.client.0.smithi090.stdout:2/461: creat db/d11/d14/d27/d2f/d70/d71/f9c x:0 0 0 2024-04-26T04:42:21.628 INFO:tasks.workunit.client.1.smithi104.stdout:3/780: sync 2024-04-26T04:42:21.628 INFO:tasks.workunit.client.1.smithi104.stdout:3/781: readlink d5/db2/da3/dd0/ddb/l9a 0 2024-04-26T04:42:21.628 INFO:tasks.workunit.client.1.smithi104.stdout:3/782: write d5/db2/da3/fc4 [684441,84144] 0 2024-04-26T04:42:21.629 INFO:tasks.workunit.client.0.smithi090.stdout:3/357: mknod d1/d6/d1b/c7f 0 2024-04-26T04:42:21.631 INFO:tasks.workunit.client.0.smithi090.stdout:4/420: creat d3/db/d12/d20/d2d/d58/d6a/d74/f86 x:0 0 0 2024-04-26T04:42:21.631 INFO:tasks.workunit.client.0.smithi090.stdout:4/421: write d3/db/d12/f4e [1146801,30286] 0 2024-04-26T04:42:21.631 INFO:tasks.workunit.client.0.smithi090.stdout:4/422: write d3/db/d12/d20/d24/d43/d72/f80 [714526,7895] 0 2024-04-26T04:42:21.633 INFO:tasks.workunit.client.0.smithi090.stdout:8/467: sync 2024-04-26T04:42:21.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/502: sync 2024-04-26T04:42:21.633 INFO:tasks.workunit.client.0.smithi090.stdout:6/503: readlink d9/l63 0 2024-04-26T04:42:21.633 INFO:tasks.workunit.client.0.smithi090.stdout:8/468: chown dc/d1f/d49/d39/f52 476883583 1 2024-04-26T04:42:21.634 INFO:tasks.workunit.client.0.smithi090.stdout:0/537: sync 2024-04-26T04:42:21.634 INFO:tasks.workunit.client.0.smithi090.stdout:0/538: creat d0/d4b/d5d/d48/d7d/fc3 x:0 0 0 2024-04-26T04:42:21.634 INFO:tasks.workunit.client.0.smithi090.stdout:2/462: getdents db/d11/d14/d25/d65/d95 0 2024-04-26T04:42:21.634 INFO:tasks.workunit.client.0.smithi090.stdout:2/463: chown db/d11/d14/d25/d2d/l64 10817617 1 2024-04-26T04:42:21.634 INFO:tasks.workunit.client.0.smithi090.stdout:2/464: write db/d11/d14/d27/d2f/d42/d5c/f80 [29513,13350] 0 2024-04-26T04:42:21.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/654: write d11/d15/d32/d35/d7b/fa6 [8365944,46247] 0 2024-04-26T04:42:21.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/655: chown d11/d15/d32/d4c/d8a 874 1 2024-04-26T04:42:21.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/656: rename d11/d15/d32/d35 to d11/d15/d32/d35/dd8 22 2024-04-26T04:42:21.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/657: rename d11/d1d/f31 to d11/d15/d32/dd6/fd9 0 2024-04-26T04:42:21.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/658: write d11/d1d/d34/f6c [1104155,86064] 0 2024-04-26T04:42:21.636 INFO:tasks.workunit.client.0.smithi090.stdout:3/358: link d1/d9/f20 d1/d17/d36/d33/f80 0 2024-04-26T04:42:21.636 INFO:tasks.workunit.client.0.smithi090.stdout:3/359: fdatasync d1/d59/f64 0 2024-04-26T04:42:21.637 INFO:tasks.workunit.client.0.smithi090.stdout:4/423: truncate d3/f36 2359086 0 2024-04-26T04:42:21.638 INFO:tasks.workunit.client.1.smithi104.stdout:3/783: dread d5/db2/da3/dd0/ddb/d32/d48/dac/f9f [4194304,4194304] 0 2024-04-26T04:42:21.639 INFO:tasks.workunit.client.0.smithi090.stdout:6/504: creat d9/d1c/d4d/d6d/d66/fb8 x:0 0 0 2024-04-26T04:42:21.641 INFO:tasks.workunit.client.1.smithi104.stdout:3/784: mkdir d5/db2/d46/db7/de6/d112 0 2024-04-26T04:42:21.641 INFO:tasks.workunit.client.1.smithi104.stdout:3/785: write d5/d9/d80/ffd [214299,124345] 0 2024-04-26T04:42:21.641 INFO:tasks.workunit.client.1.smithi104.stdout:5/729: dwrite d0/d1/d4/d2a/d6b/d91/fb0 [0,4194304] 0 2024-04-26T04:42:21.645 INFO:tasks.workunit.client.0.smithi090.stdout:2/465: creat db/d11/d14/d25/d65/f9d x:0 0 0 2024-04-26T04:42:21.647 INFO:tasks.workunit.client.0.smithi090.stdout:3/360: link d1/l7 d1/d17/d36/d33/d43/l81 0 2024-04-26T04:42:21.648 INFO:tasks.workunit.client.0.smithi090.stdout:3/361: truncate d1/d17/f1a 401441 0 2024-04-26T04:42:21.648 INFO:tasks.workunit.client.0.smithi090.stdout:3/362: chown d1/d17/d36/f6a 84155 1 2024-04-26T04:42:21.648 INFO:tasks.workunit.client.1.smithi104.stdout:9/659: getdents d11/d15/d1b/d6d/d9c 0 2024-04-26T04:42:21.652 INFO:tasks.workunit.client.0.smithi090.stdout:2/466: symlink db/d11/d14/d1b/d20/d4c/l9e 0 2024-04-26T04:42:21.654 INFO:tasks.workunit.client.1.smithi104.stdout:9/660: dread d11/f89 [0,4194304] 0 2024-04-26T04:42:21.654 INFO:tasks.workunit.client.1.smithi104.stdout:9/661: dread - d11/d15/d1b/d6d/fc0 zero size 2024-04-26T04:42:21.654 INFO:tasks.workunit.client.0.smithi090.stdout:2/467: dread db/d11/d14/d25/f3e [0,4194304] 0 2024-04-26T04:42:21.655 INFO:tasks.workunit.client.0.smithi090.stdout:2/468: chown db/d11/d14/d27 4126 1 2024-04-26T04:42:21.656 INFO:tasks.workunit.client.0.smithi090.stdout:3/363: write d1/d59/d5e/f62 [2138553,107735] 0 2024-04-26T04:42:21.664 INFO:tasks.workunit.client.0.smithi090.stdout:2/469: dread db/d59/f62 [0,4194304] 0 2024-04-26T04:42:21.667 INFO:tasks.workunit.client.0.smithi090.stdout:2/470: dread db/d11/d14/f4a [0,4194304] 0 2024-04-26T04:42:21.699 INFO:tasks.workunit.client.0.smithi090.stdout:4/424: dwrite d3/f27 [0,4194304] 0 2024-04-26T04:42:21.699 INFO:tasks.workunit.client.0.smithi090.stdout:4/425: creat d3/d13/f87 x:0 0 0 2024-04-26T04:42:21.699 INFO:tasks.workunit.client.0.smithi090.stdout:8/469: dwrite dc/d1f/d49/d4f/d58/d95/f8a [0,4194304] 0 2024-04-26T04:42:21.699 INFO:tasks.workunit.client.0.smithi090.stdout:8/470: chown dc/d1f/d49/d3b/d3d/f9c 108 1 2024-04-26T04:42:21.699 INFO:tasks.workunit.client.0.smithi090.stdout:0/539: dwrite d0/d4b/d5d/d5c/d8a/d45/fb5 [0,4194304] 0 2024-04-26T04:42:21.699 INFO:tasks.workunit.client.0.smithi090.stdout:8/471: creat dc/d1f/d49/d4f/d58/fa5 x:0 0 0 2024-04-26T04:42:21.699 INFO:tasks.workunit.client.0.smithi090.stdout:0/540: dread - d0/df/f77 zero size 2024-04-26T04:42:21.700 INFO:tasks.workunit.client.0.smithi090.stdout:3/364: dwrite d1/d17/d36/f65 [0,4194304] 0 2024-04-26T04:42:21.700 INFO:tasks.workunit.client.1.smithi104.stdout:3/786: dwrite d5/db2/f3c [0,4194304] 0 2024-04-26T04:42:21.700 INFO:tasks.workunit.client.1.smithi104.stdout:5/730: dwrite d0/d1/d4/d2a/d2b/fa1 [0,4194304] 0 2024-04-26T04:42:21.701 INFO:tasks.workunit.client.1.smithi104.stdout:3/787: creat d5/d9/d86/f113 x:0 0 0 2024-04-26T04:42:21.701 INFO:tasks.workunit.client.1.smithi104.stdout:3/788: chown d5/d9/f26 9604 1 2024-04-26T04:42:21.702 INFO:tasks.workunit.client.0.smithi090.stdout:2/471: dwrite db/d11/d14/d1b/d20/f24 [0,4194304] 0 2024-04-26T04:42:21.702 INFO:tasks.workunit.client.0.smithi090.stdout:6/505: dwrite d9/d21/f5f [0,4194304] 0 2024-04-26T04:42:21.702 INFO:tasks.workunit.client.0.smithi090.stdout:3/365: dread d1/d17/f4f [0,4194304] 0 2024-04-26T04:42:21.703 INFO:tasks.workunit.client.0.smithi090.stdout:4/426: mknod d3/db/d12/d20/d24/d43/c88 0 2024-04-26T04:42:21.703 INFO:tasks.workunit.client.0.smithi090.stdout:7/492: dread d0/db/d53/f17 [0,4194304] 0 2024-04-26T04:42:21.703 INFO:tasks.workunit.client.0.smithi090.stdout:7/493: chown d0/db/d53/d5e/fa9 105829 1 2024-04-26T04:42:21.705 INFO:tasks.workunit.client.1.smithi104.stdout:5/731: rename d0/d1/d4/d2a/lcf to d0/d41/d4f/df2/lf8 0 2024-04-26T04:42:21.705 INFO:tasks.workunit.client.1.smithi104.stdout:3/789: dread d5/db2/da3/dd0/ddb/d69/f3e [0,4194304] 0 2024-04-26T04:42:21.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/472: mkdir dc/d1f/d49/d4f/da6 0 2024-04-26T04:42:21.708 INFO:tasks.workunit.client.1.smithi104.stdout:3/790: rename d5/db2/da3/dd0/ddb/d69/f55 to d5/d9/dd1/f114 0 2024-04-26T04:42:21.708 INFO:tasks.workunit.client.1.smithi104.stdout:3/791: write d5/db2/da3/dd0/ddb/d69/fab [43119,1536] 0 2024-04-26T04:42:21.710 INFO:tasks.workunit.client.1.smithi104.stdout:3/792: mkdir d5/d9/d86/d115 0 2024-04-26T04:42:21.710 INFO:tasks.workunit.client.1.smithi104.stdout:3/793: write d5/fb3 [322037,44197] 0 2024-04-26T04:42:21.712 INFO:tasks.workunit.client.1.smithi104.stdout:3/794: unlink d5/db2/da3/dd0/ddb/d32/l10b 0 2024-04-26T04:42:21.715 INFO:tasks.workunit.client.1.smithi104.stdout:3/795: unlink d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/fbd 0 2024-04-26T04:42:21.717 INFO:tasks.workunit.client.0.smithi090.stdout:4/427: mknod d3/db/d60/d5c/c89 0 2024-04-26T04:42:21.717 INFO:tasks.workunit.client.0.smithi090.stdout:4/428: creat d3/db/d12/d20/d24/d43/d72/f8a x:0 0 0 2024-04-26T04:42:21.717 INFO:tasks.workunit.client.0.smithi090.stdout:4/429: fdatasync d3/db/d12/d20/d24/f32 0 2024-04-26T04:42:21.718 INFO:tasks.workunit.client.0.smithi090.stdout:2/472: read db/dc/f1f [1750343,76926] 0 2024-04-26T04:42:21.718 INFO:tasks.workunit.client.1.smithi104.stdout:3/796: creat d5/db2/da3/dd0/f116 x:0 0 0 2024-04-26T04:42:21.719 INFO:tasks.workunit.client.1.smithi104.stdout:3/797: write d5/db2/da3/dd0/ddb/f111 [376539,23126] 0 2024-04-26T04:42:21.719 INFO:tasks.workunit.client.0.smithi090.stdout:7/494: write d0/db/d53/db7/d4f/f95 [427656,75089] 0 2024-04-26T04:42:21.719 INFO:tasks.workunit.client.0.smithi090.stdout:3/366: dread d1/d9/f12 [0,4194304] 0 2024-04-26T04:42:21.720 INFO:tasks.workunit.client.1.smithi104.stdout:3/798: creat d5/d9/de4/f117 x:0 0 0 2024-04-26T04:42:21.720 INFO:tasks.workunit.client.1.smithi104.stdout:3/799: chown d5/db2/da3/dd0/ddb/d69/f6e 4 1 2024-04-26T04:42:21.722 INFO:tasks.workunit.client.1.smithi104.stdout:3/800: mkdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/d118 0 2024-04-26T04:42:21.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/473: truncate dc/d1f/d49/d39/d55/d64/f3e 5446964 0 2024-04-26T04:42:21.725 INFO:tasks.workunit.client.0.smithi090.stdout:6/506: getdents d9/d21/d40 0 2024-04-26T04:42:21.726 INFO:tasks.workunit.client.1.smithi104.stdout:5/732: dwrite d0/d1/d4/d2a/fd2 [0,4194304] 0 2024-04-26T04:42:21.728 INFO:tasks.workunit.client.1.smithi104.stdout:5/733: creat d0/d1/d1e/d98/dd4/ff9 x:0 0 0 2024-04-26T04:42:21.728 INFO:tasks.workunit.client.1.smithi104.stdout:5/734: creat d0/d1/d4/d2a/d6b/d91/dbd/ffa x:0 0 0 2024-04-26T04:42:21.731 INFO:tasks.workunit.client.0.smithi090.stdout:7/495: rename d0/l62 to d0/d1/d4/d98/d65/db2/lb8 0 2024-04-26T04:42:21.731 INFO:tasks.workunit.client.0.smithi090.stdout:7/496: chown d0/db/df/d1c/d23/d8d 809585709 1 2024-04-26T04:42:21.732 INFO:tasks.workunit.client.1.smithi104.stdout:5/735: unlink d0/d1/d64/d26/d36/da7/fe8 0 2024-04-26T04:42:21.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/367: rmdir d1/d17/d36/d2a 39 2024-04-26T04:42:21.735 INFO:tasks.workunit.client.0.smithi090.stdout:8/474: creat dc/d1f/d49/d4f/d58/d95/fa7 x:0 0 0 2024-04-26T04:42:21.736 INFO:tasks.workunit.client.0.smithi090.stdout:8/475: creat dc/d1f/fa8 x:0 0 0 2024-04-26T04:42:21.736 INFO:tasks.workunit.client.0.smithi090.stdout:3/368: mkdir d1/d17/d36/d2a/d3a/d5d/d82 0 2024-04-26T04:42:21.739 INFO:tasks.workunit.client.0.smithi090.stdout:8/476: symlink dc/d1f/d49/d3b/la9 0 2024-04-26T04:42:21.740 INFO:tasks.workunit.client.0.smithi090.stdout:8/477: write dc/d1f/d49/d4f/d58/d95/fa7 [353503,112235] 0 2024-04-26T04:42:21.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/478: mknod dc/d1f/d49/d4f/da6/caa 0 2024-04-26T04:42:21.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/479: dread - dc/d1f/fa8 zero size 2024-04-26T04:42:21.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/480: stat dc/d5a/f62 0 2024-04-26T04:42:21.747 INFO:tasks.workunit.client.1.smithi104.stdout:5/736: dread d0/d1/d4/d2a/d2b/fb7 [0,4194304] 0 2024-04-26T04:42:21.748 INFO:tasks.workunit.client.0.smithi090.stdout:4/430: dwrite d3/db/d12/d20/d24/d43/d72/f80 [0,4194304] 0 2024-04-26T04:42:21.748 INFO:tasks.workunit.client.0.smithi090.stdout:4/431: truncate d3/db/d12/d20/f2a 1473809 0 2024-04-26T04:42:21.748 INFO:tasks.workunit.client.0.smithi090.stdout:4/432: stat d3/db/d12/d20/d24/d43/d72/f7c 0 2024-04-26T04:42:21.749 INFO:tasks.workunit.client.0.smithi090.stdout:4/433: chown d3/db/d12/l29 5678 1 2024-04-26T04:42:21.749 INFO:tasks.workunit.client.1.smithi104.stdout:5/737: unlink d0/f18 0 2024-04-26T04:42:21.751 INFO:tasks.workunit.client.0.smithi090.stdout:4/434: link f0 d3/db/d12/d20/d2d/d58/d6a/f8b 0 2024-04-26T04:42:21.752 INFO:tasks.workunit.client.0.smithi090.stdout:3/369: write d1/d17/d36/f65 [1633728,35628] 0 2024-04-26T04:42:21.755 INFO:tasks.workunit.client.0.smithi090.stdout:7/497: dread d0/db/d53/db7/d4f/f95 [0,4194304] 0 2024-04-26T04:42:21.756 INFO:tasks.workunit.client.0.smithi090.stdout:7/498: mknod d0/db/df/d4d/d7b/cb9 0 2024-04-26T04:42:21.756 INFO:tasks.workunit.client.0.smithi090.stdout:3/370: dread d1/d9/f38 [0,4194304] 0 2024-04-26T04:42:21.758 INFO:tasks.workunit.client.0.smithi090.stdout:7/499: rename d0/d1/d4/d98/f93 to d0/d1/d4/d98/d65/db2/fba 0 2024-04-26T04:42:21.761 INFO:tasks.workunit.client.0.smithi090.stdout:2/473: dwrite db/d59/f62 [0,4194304] 0 2024-04-26T04:42:21.761 INFO:tasks.workunit.client.0.smithi090.stdout:2/474: fdatasync db/d11/d14/d25/f37 0 2024-04-26T04:42:21.762 INFO:tasks.workunit.client.0.smithi090.stdout:2/475: mknod db/d11/d14/d25/d65/d95/c9f 0 2024-04-26T04:42:21.762 INFO:tasks.workunit.client.0.smithi090.stdout:2/476: write db/d11/d14/d25/f49 [4389821,122094] 0 2024-04-26T04:42:21.763 INFO:tasks.workunit.client.0.smithi090.stdout:2/477: truncate db/d11/d14/d25/f9a 931834 0 2024-04-26T04:42:21.763 INFO:tasks.workunit.client.0.smithi090.stdout:7/500: read d0/db/d15/d29/fa2 [2869496,28155] 0 2024-04-26T04:42:21.764 INFO:tasks.workunit.client.0.smithi090.stdout:4/435: read d3/f2c [3494127,89798] 0 2024-04-26T04:42:21.764 INFO:tasks.workunit.client.0.smithi090.stdout:4/436: fdatasync d3/db/d12/d20/d2d/d58/f63 0 2024-04-26T04:42:21.764 INFO:tasks.workunit.client.0.smithi090.stdout:4/437: chown d3/db/d12/d20/d24/f52 455182 1 2024-04-26T04:42:21.764 INFO:tasks.workunit.client.0.smithi090.stdout:2/478: rename db/d11/d14/d25/d86/l8a to db/d11/d14/d25/d2d/d48/d81/la0 0 2024-04-26T04:42:21.765 INFO:tasks.workunit.client.1.smithi104.stdout:3/801: dwrite d5/db2/d46/fe5 [0,4194304] 0 2024-04-26T04:42:21.765 INFO:tasks.workunit.client.0.smithi090.stdout:6/507: dwrite d9/d1c/f2b [4194304,4194304] 0 2024-04-26T04:42:21.766 INFO:tasks.workunit.client.0.smithi090.stdout:7/501: mkdir d0/d1/d4/d98/d65/dbb 0 2024-04-26T04:42:21.766 INFO:tasks.workunit.client.0.smithi090.stdout:7/502: truncate d0/d1/d4/d98/db1/fb5 578185 0 2024-04-26T04:42:21.766 INFO:tasks.workunit.client.0.smithi090.stdout:7/503: write d0/db/d15/f91 [491544,108923] 0 2024-04-26T04:42:21.766 INFO:tasks.workunit.client.1.smithi104.stdout:5/738: dwrite d0/d1/d1e/f2e [4194304,4194304] 0 2024-04-26T04:42:21.767 INFO:tasks.workunit.client.0.smithi090.stdout:4/438: truncate d3/db/f19 1634293 0 2024-04-26T04:42:21.767 INFO:tasks.workunit.client.0.smithi090.stdout:8/481: dread dc/d1f/d49/d4f/d58/d95/fa7 [0,4194304] 0 2024-04-26T04:42:21.768 INFO:tasks.workunit.client.1.smithi104.stdout:3/802: mknod d5/d9/c119 0 2024-04-26T04:42:21.768 INFO:tasks.workunit.client.1.smithi104.stdout:3/803: read - d5/db2/da3/dd0/ddb/d32/d48/fdf zero size 2024-04-26T04:42:21.773 INFO:tasks.workunit.client.1.smithi104.stdout:5/739: mkdir d0/d1/d4/d2a/d6b/dfb 0 2024-04-26T04:42:21.773 INFO:tasks.workunit.client.1.smithi104.stdout:5/740: chown d0/d1/d64/d26/l47 1495 1 2024-04-26T04:42:21.773 INFO:tasks.workunit.client.1.smithi104.stdout:5/741: creat d0/d1/d64/d26/d36/da7/ffc x:0 0 0 2024-04-26T04:42:21.774 INFO:tasks.workunit.client.0.smithi090.stdout:2/479: rename db/d11/d14/d25/d2d/d48/d81/f8f to db/d11/d7f/fa1 0 2024-04-26T04:42:21.774 INFO:tasks.workunit.client.1.smithi104.stdout:3/804: mkdir d5/db2/da3/dd0/ddb/d32/d47/d11a 0 2024-04-26T04:42:21.774 INFO:tasks.workunit.client.1.smithi104.stdout:3/805: write d5/d9/d86/fd2 [375575,123413] 0 2024-04-26T04:42:21.775 INFO:tasks.workunit.client.0.smithi090.stdout:4/439: write d3/db/d12/d20/d2d/d2e/f3a [1400775,33255] 0 2024-04-26T04:42:21.775 INFO:tasks.workunit.client.0.smithi090.stdout:7/504: mkdir d0/db/d15/dbc 0 2024-04-26T04:42:21.775 INFO:tasks.workunit.client.1.smithi104.stdout:5/742: getdents d0/d1/d4/d2a/d6b/d91 0 2024-04-26T04:42:21.776 INFO:tasks.workunit.client.0.smithi090.stdout:2/480: dread db/d11/d14/d25/d2d/f36 [0,4194304] 0 2024-04-26T04:42:21.776 INFO:tasks.workunit.client.0.smithi090.stdout:2/481: chown db/d11/d14/d27/d28/f61 21 1 2024-04-26T04:42:21.778 INFO:tasks.workunit.client.1.smithi104.stdout:3/806: link d5/db2/d46/f4d d5/db2/da3/dd0/ddb/f11b 0 2024-04-26T04:42:21.778 INFO:tasks.workunit.client.1.smithi104.stdout:3/807: chown d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/f58 587266631 1 2024-04-26T04:42:21.778 INFO:tasks.workunit.client.1.smithi104.stdout:5/743: truncate d0/f22 5795157 0 2024-04-26T04:42:21.778 INFO:tasks.workunit.client.1.smithi104.stdout:5/744: chown d0/d1/d64/d26/d36/f9f 184559334 1 2024-04-26T04:42:21.778 INFO:tasks.workunit.client.0.smithi090.stdout:4/440: mkdir d3/d13/d6c/d8c 0 2024-04-26T04:42:21.779 INFO:tasks.workunit.client.1.smithi104.stdout:5/745: mkdir d0/d1/d4/d2a/d2b/d37/d48/dc1/dfd 0 2024-04-26T04:42:21.782 INFO:tasks.workunit.client.1.smithi104.stdout:5/746: rename d0/d1/d4/d2a/d2b/d37/d99 to d0/d1/d64/d26/d36/d72/dfe 0 2024-04-26T04:42:21.782 INFO:tasks.workunit.client.1.smithi104.stdout:5/747: read - d0/d1/d64/d26/d36/da7/ffc zero size 2024-04-26T04:42:21.784 INFO:tasks.workunit.client.1.smithi104.stdout:5/748: rmdir d0/d1/d64/d26/dd7 39 2024-04-26T04:42:21.784 INFO:tasks.workunit.client.1.smithi104.stdout:5/749: readlink d0/d1/l9b 0 2024-04-26T04:42:21.788 INFO:tasks.workunit.client.1.smithi104.stdout:3/808: write d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/f82 [2703438,55862] 0 2024-04-26T04:42:21.788 INFO:tasks.workunit.client.1.smithi104.stdout:3/809: write d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/fe0 [842098,96758] 0 2024-04-26T04:42:21.790 INFO:tasks.workunit.client.0.smithi090.stdout:3/371: dwrite d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:21.790 INFO:tasks.workunit.client.0.smithi090.stdout:3/372: truncate d1/d17/f1a 1344702 0 2024-04-26T04:42:21.790 INFO:tasks.workunit.client.0.smithi090.stdout:3/373: chown d1/d17/d36/d33/d43/l81 1687 1 2024-04-26T04:42:21.790 INFO:tasks.workunit.client.0.smithi090.stdout:3/374: truncate d1/d59/d5e/f62 2927307 0 2024-04-26T04:42:21.791 INFO:tasks.workunit.client.1.smithi104.stdout:3/810: creat d5/d9/dd1/f11c x:0 0 0 2024-04-26T04:42:21.791 INFO:tasks.workunit.client.0.smithi090.stdout:4/441: dread d3/db/d12/d20/d45/d53/f6e [0,4194304] 0 2024-04-26T04:42:21.794 INFO:tasks.workunit.client.0.smithi090.stdout:6/508: dwrite d9/d21/d40/d4c/f90 [0,4194304] 0 2024-04-26T04:42:21.796 INFO:tasks.workunit.client.1.smithi104.stdout:3/811: mknod d5/db2/da3/df3/c11d 0 2024-04-26T04:42:21.798 INFO:tasks.workunit.client.1.smithi104.stdout:3/812: rename d5/db2/da3/dd0/ddb/d69/l33 to d5/db2/d46/d95/l11e 0 2024-04-26T04:42:21.800 INFO:tasks.workunit.client.1.smithi104.stdout:3/813: unlink d5/db2/d60/fa7 0 2024-04-26T04:42:21.801 INFO:tasks.workunit.client.1.smithi104.stdout:5/750: write d0/d1/d4/d2a/d2b/fa1 [3660110,61097] 0 2024-04-26T04:42:21.801 INFO:tasks.workunit.client.1.smithi104.stdout:5/751: readlink d0/d41/d4f/l9a 0 2024-04-26T04:42:21.801 INFO:tasks.workunit.client.1.smithi104.stdout:5/752: readlink d0/d1/d4/d2a/d2b/d37/d48/l7a 0 2024-04-26T04:42:21.802 INFO:tasks.workunit.client.1.smithi104.stdout:3/814: dread d5/fb3 [0,4194304] 0 2024-04-26T04:42:21.804 INFO:tasks.workunit.client.0.smithi090.stdout:3/375: write d1/d9/fb [3725301,50345] 0 2024-04-26T04:42:21.805 INFO:tasks.workunit.client.0.smithi090.stdout:3/376: write d1/d17/d36/d33/d42/f63 [731861,130138] 0 2024-04-26T04:42:21.813 INFO:tasks.workunit.client.1.smithi104.stdout:0/633: sync 2024-04-26T04:42:21.813 INFO:tasks.workunit.client.1.smithi104.stdout:5/753: dread d0/d41/f74 [0,4194304] 0 2024-04-26T04:42:21.813 INFO:tasks.workunit.client.1.smithi104.stdout:4/785: sync 2024-04-26T04:42:21.827 INFO:tasks.workunit.client.1.smithi104.stdout:4/786: write d4/d7/d1d/d8e/fc3 [3165954,47257] 0 2024-04-26T04:42:21.827 INFO:tasks.workunit.client.1.smithi104.stdout:4/787: fsync d4/d7/f1f 0 2024-04-26T04:42:21.828 INFO:tasks.workunit.client.1.smithi104.stdout:4/788: write d4/d7/d1d/d2e/fba [814586,110003] 0 2024-04-26T04:42:21.829 INFO:tasks.workunit.client.1.smithi104.stdout:4/789: mknod d4/d7/d1d/c115 0 2024-04-26T04:42:21.848 INFO:tasks.workunit.client.0.smithi090.stdout:4/442: dwrite d3/db/d12/f4c [0,4194304] 0 2024-04-26T04:42:21.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/505: mkdir d0/db/df/d4d/daf/dbd 0 2024-04-26T04:42:21.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/506: chown d0/db/df/d1c/d23/f82 1809 1 2024-04-26T04:42:21.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/507: chown d0/db/l21 13848973 1 2024-04-26T04:42:21.850 INFO:tasks.workunit.client.0.smithi090.stdout:2/482: mknod db/d11/d14/d27/d2f/d70/ca2 0 2024-04-26T04:42:21.850 INFO:tasks.workunit.client.0.smithi090.stdout:3/377: dwrite d1/d17/d36/f32 [0,4194304] 0 2024-04-26T04:42:21.850 INFO:tasks.workunit.client.0.smithi090.stdout:8/482: rmdir dc/d71 39 2024-04-26T04:42:21.850 INFO:tasks.workunit.client.0.smithi090.stdout:8/483: read - dc/d1f/d49/d4f/f7f zero size 2024-04-26T04:42:21.851 INFO:tasks.workunit.client.1.smithi104.stdout:3/815: dwrite d5/db2/da3/dd0/ddb/f21 [0,4194304] 0 2024-04-26T04:42:21.851 INFO:tasks.workunit.client.1.smithi104.stdout:3/816: getdents d5/db2/da3/dd0/ddb/d3a 0 2024-04-26T04:42:21.851 INFO:tasks.workunit.client.1.smithi104.stdout:5/754: dwrite d0/d41/d4f/fb6 [0,4194304] 0 2024-04-26T04:42:21.851 INFO:tasks.workunit.client.1.smithi104.stdout:5/755: write d0/d1/d4/d2a/d2b/d37/f9c [4980479,17711] 0 2024-04-26T04:42:21.852 INFO:tasks.workunit.client.1.smithi104.stdout:0/634: dwrite d0/d2/d10/d3d/f8d [4194304,4194304] 0 2024-04-26T04:42:21.852 INFO:tasks.workunit.client.1.smithi104.stdout:0/635: write d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fa9 [4559395,130663] 0 2024-04-26T04:42:21.853 INFO:tasks.workunit.client.1.smithi104.stdout:4/790: dwrite d4/d7/d1d/d8e/f10a [0,4194304] 0 2024-04-26T04:42:21.853 INFO:tasks.workunit.client.1.smithi104.stdout:4/791: dread - d4/d20/d55/d56/da3/dea/f103 zero size 2024-04-26T04:42:21.854 INFO:tasks.workunit.client.1.smithi104.stdout:5/756: rename d0/d89/fa2 to d0/d1/d4/d2a/d2b/d37/d48/fff 0 2024-04-26T04:42:21.854 INFO:tasks.workunit.client.1.smithi104.stdout:5/757: fdatasync d0/d1/d4/d2a/d6b/d73/d95/fa5 0 2024-04-26T04:42:21.854 INFO:tasks.workunit.client.1.smithi104.stdout:5/758: truncate d0/d1/d3e/f51 2019154 0 2024-04-26T04:42:21.860 INFO:tasks.workunit.client.1.smithi104.stdout:3/817: dread d5/fbb [0,4194304] 0 2024-04-26T04:42:21.860 INFO:tasks.workunit.client.1.smithi104.stdout:3/818: creat d5/db2/da3/dd0/ddb/d69/f11f x:0 0 0 2024-04-26T04:42:21.860 INFO:tasks.workunit.client.1.smithi104.stdout:3/819: readlink d5/d9/d86/lf6 0 2024-04-26T04:42:21.861 INFO:tasks.workunit.client.1.smithi104.stdout:4/792: mknod d4/d7/c116 0 2024-04-26T04:42:21.861 INFO:tasks.workunit.client.1.smithi104.stdout:0/636: mkdir d0/d2/d21/d27/d1e/d57/d6c/dc7 0 2024-04-26T04:42:21.861 INFO:tasks.workunit.client.1.smithi104.stdout:0/637: fsync d0/d2/d21/d27/f6d 0 2024-04-26T04:42:21.863 INFO:tasks.workunit.client.1.smithi104.stdout:4/793: mknod d4/d7/d1d/d1e/d24/d59/dc1/c117 0 2024-04-26T04:42:21.865 INFO:tasks.workunit.client.1.smithi104.stdout:0/638: dread d0/d2/d21/d27/faa [0,4194304] 0 2024-04-26T04:42:21.865 INFO:tasks.workunit.client.1.smithi104.stdout:0/639: fsync d0/d2/d21/d49/f55 0 2024-04-26T04:42:21.865 INFO:tasks.workunit.client.1.smithi104.stdout:4/794: mkdir d4/d118 0 2024-04-26T04:42:21.868 INFO:tasks.workunit.client.1.smithi104.stdout:0/640: rename d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fa9 to d0/d2/d21/d27/d1e/fc8 0 2024-04-26T04:42:21.868 INFO:tasks.workunit.client.1.smithi104.stdout:0/641: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fc9 x:0 0 0 2024-04-26T04:42:21.868 INFO:tasks.workunit.client.1.smithi104.stdout:4/795: mknod d4/d20/d55/d56/da3/c119 0 2024-04-26T04:42:21.869 INFO:tasks.workunit.client.1.smithi104.stdout:3/820: read d5/f24 [799956,2022] 0 2024-04-26T04:42:21.872 INFO:tasks.workunit.client.1.smithi104.stdout:0/642: symlink d0/d2/lca 0 2024-04-26T04:42:21.873 INFO:tasks.workunit.client.1.smithi104.stdout:0/643: mknod d0/d2/d21/d38/d7c/ccb 0 2024-04-26T04:42:21.878 INFO:tasks.workunit.client.1.smithi104.stdout:5/759: dwrite d0/d1/d64/f19 [0,4194304] 0 2024-04-26T04:42:21.879 INFO:tasks.workunit.client.0.smithi090.stdout:8/484: dwrite dc/d1f/fa8 [0,4194304] 0 2024-04-26T04:42:21.879 INFO:tasks.workunit.client.0.smithi090.stdout:8/485: truncate dc/f30 4520993 0 2024-04-26T04:42:21.879 INFO:tasks.workunit.client.0.smithi090.stdout:8/486: creat dc/d1f/d49/d39/fab x:0 0 0 2024-04-26T04:42:21.879 INFO:tasks.workunit.client.0.smithi090.stdout:6/509: symlink d9/d77/d9a/lb9 0 2024-04-26T04:42:21.879 INFO:tasks.workunit.client.1.smithi104.stdout:3/821: dread d5/f54 [0,4194304] 0 2024-04-26T04:42:21.880 INFO:tasks.workunit.client.0.smithi090.stdout:8/487: dread dc/d34/d4d/f4e [0,4194304] 0 2024-04-26T04:42:21.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/443: rename d3/db/d12/d20/d45/f7f to d3/db/d12/d20/d2d/d58/d6a/f8d 0 2024-04-26T04:42:21.881 INFO:tasks.workunit.client.0.smithi090.stdout:6/510: write d9/d21/d40/d4c/f59 [978409,103875] 0 2024-04-26T04:42:21.881 INFO:tasks.workunit.client.0.smithi090.stdout:6/511: chown d9/d1c/d31 23 1 2024-04-26T04:42:21.882 INFO:tasks.workunit.client.0.smithi090.stdout:2/483: creat db/d11/d14/d27/d28/fa3 x:0 0 0 2024-04-26T04:42:21.883 INFO:tasks.workunit.client.1.smithi104.stdout:3/822: dread d5/db2/f91 [0,4194304] 0 2024-04-26T04:42:21.886 INFO:tasks.workunit.client.1.smithi104.stdout:3/823: truncate d5/db2/d60/f7e 3627357 0 2024-04-26T04:42:21.886 INFO:tasks.workunit.client.1.smithi104.stdout:3/824: dread - d5/db2/da3/dd0/ddb/f36 zero size 2024-04-26T04:42:21.887 INFO:tasks.workunit.client.0.smithi090.stdout:3/378: symlink d1/d17/d52/d7d/l83 0 2024-04-26T04:42:21.888 INFO:tasks.workunit.client.1.smithi104.stdout:3/825: creat d5/db2/d46/f120 x:0 0 0 2024-04-26T04:42:21.891 INFO:tasks.workunit.client.1.smithi104.stdout:3/826: symlink d5/db2/da3/dd0/ddb/l121 0 2024-04-26T04:42:21.891 INFO:tasks.workunit.client.1.smithi104.stdout:3/827: read - d5/d9/dd1/f11c zero size 2024-04-26T04:42:21.892 INFO:tasks.workunit.client.1.smithi104.stdout:3/828: rename d5/db2/da3/fc3 to d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/f122 0 2024-04-26T04:42:21.894 INFO:tasks.workunit.client.0.smithi090.stdout:4/444: link d3/d13/f81 d3/db/d12/d20/d2d/d58/d6a/d74/d85/f8e 0 2024-04-26T04:42:21.896 INFO:tasks.workunit.client.0.smithi090.stdout:6/512: unlink d9/d77/c91 0 2024-04-26T04:42:21.897 INFO:tasks.workunit.client.0.smithi090.stdout:2/484: link db/d11/d14/d25/d65/d95/l99 db/d11/d14/d1b/d20/la4 0 2024-04-26T04:42:21.897 INFO:tasks.workunit.client.0.smithi090.stdout:2/485: creat db/d11/d14/d27/d28/fa5 x:0 0 0 2024-04-26T04:42:21.899 INFO:tasks.workunit.client.0.smithi090.stdout:4/445: mknod d3/d13/d64/c8f 0 2024-04-26T04:42:21.901 INFO:tasks.workunit.client.0.smithi090.stdout:2/486: rename db/d11/d14/d25/f3a to db/d59/d66/d69/fa6 0 2024-04-26T04:42:21.902 INFO:tasks.workunit.client.0.smithi090.stdout:3/379: dread d1/d17/d36/f24 [0,4194304] 0 2024-04-26T04:42:21.902 INFO:tasks.workunit.client.0.smithi090.stdout:3/380: write d1/d6/f79 [454547,126414] 0 2024-04-26T04:42:21.902 INFO:tasks.workunit.client.0.smithi090.stdout:3/381: chown d1/d6/d1b/f35 54453336 1 2024-04-26T04:42:21.903 INFO:tasks.workunit.client.0.smithi090.stdout:2/487: mkdir db/d11/d14/d1b/d20/da7 0 2024-04-26T04:42:21.903 INFO:tasks.workunit.client.0.smithi090.stdout:4/446: getdents d3/db/d12/d20/d24/d43/d72 0 2024-04-26T04:42:21.903 INFO:tasks.workunit.client.0.smithi090.stdout:4/447: readlink d3/db/d12/l22 0 2024-04-26T04:42:21.904 INFO:tasks.workunit.client.0.smithi090.stdout:4/448: write d3/db/d12/d20/d2d/d58/d6a/d74/f86 [834394,48943] 0 2024-04-26T04:42:21.905 INFO:tasks.workunit.client.0.smithi090.stdout:2/488: dread db/d11/d14/d25/f3e [0,4194304] 0 2024-04-26T04:42:21.905 INFO:tasks.workunit.client.0.smithi090.stdout:4/449: creat d3/d13/d6c/f90 x:0 0 0 2024-04-26T04:42:21.909 INFO:tasks.workunit.client.0.smithi090.stdout:4/450: rename d3/d13/d1d/f38 to d3/db/d12/d20/d24/d43/f91 0 2024-04-26T04:42:21.912 INFO:tasks.workunit.client.0.smithi090.stdout:2/489: rmdir db/d11/d14/d25/d2d/d48/d81/d8c 0 2024-04-26T04:42:21.912 INFO:tasks.workunit.client.0.smithi090.stdout:2/490: fsync db/d11/d14/f4a 0 2024-04-26T04:42:21.912 INFO:tasks.workunit.client.1.smithi104.stdout:3/829: write d5/d9/d80/dbc/fde [1708348,61951] 0 2024-04-26T04:42:21.918 INFO:tasks.workunit.client.1.smithi104.stdout:3/830: rename d5/d9/d80/ffd to d5/db2/da3/dd0/ddb/d3a/f123 0 2024-04-26T04:42:21.918 INFO:tasks.workunit.client.1.smithi104.stdout:3/831: mkdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/d124 0 2024-04-26T04:42:21.919 INFO:tasks.workunit.client.1.smithi104.stdout:3/832: unlink d5/db2/da3/dd0/ddb/d32/f77 0 2024-04-26T04:42:21.923 INFO:tasks.workunit.client.0.smithi090.stdout:6/513: dwrite d9/d1c/d4d/d6d/d92/f29 [8388608,4194304] 0 2024-04-26T04:42:21.925 INFO:tasks.workunit.client.0.smithi090.stdout:6/514: creat d9/d1c/d31/d79/fba x:0 0 0 2024-04-26T04:42:21.927 INFO:tasks.workunit.client.0.smithi090.stdout:3/382: dwrite d1/d9/f38 [0,4194304] 0 2024-04-26T04:42:21.927 INFO:tasks.workunit.client.0.smithi090.stdout:6/515: write d9/d3c/f61 [1198142,66029] 0 2024-04-26T04:42:21.931 INFO:tasks.workunit.client.0.smithi090.stdout:6/516: mknod d9/d77/cbb 0 2024-04-26T04:42:21.934 INFO:tasks.workunit.client.0.smithi090.stdout:6/517: mkdir d9/d1c/d31/d69/dbc 0 2024-04-26T04:42:21.935 INFO:tasks.workunit.client.0.smithi090.stdout:6/518: mknod d9/d1c/d31/d69/dbc/cbd 0 2024-04-26T04:42:21.935 INFO:tasks.workunit.client.0.smithi090.stdout:6/519: fdatasync d9/d77/db0/db5/fa3 0 2024-04-26T04:42:21.935 INFO:tasks.workunit.client.0.smithi090.stdout:6/520: write d9/d1c/d31/d79/fba [710948,20221] 0 2024-04-26T04:42:21.935 INFO:tasks.workunit.client.0.smithi090.stdout:6/521: chown d9/d1c/d31 153115411 1 2024-04-26T04:42:21.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/522: rename d9/d21/f9b to d9/d77/db0/db5/fbe 0 2024-04-26T04:42:21.937 INFO:tasks.workunit.client.0.smithi090.stdout:6/523: symlink d9/d1c/d31/d79/lbf 0 2024-04-26T04:42:21.938 INFO:tasks.workunit.client.1.smithi104.stdout:3/833: dwrite d5/f25 [0,4194304] 0 2024-04-26T04:42:21.953 INFO:tasks.workunit.client.0.smithi090.stdout:6/524: dwrite d9/d1c/f2b [0,4194304] 0 2024-04-26T04:42:21.953 INFO:tasks.workunit.client.0.smithi090.stdout:6/525: mkdir d9/d77/dc0 0 2024-04-26T04:42:21.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/526: getdents d9/d1c/d4d/d58 0 2024-04-26T04:42:21.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/527: stat d9/d1c/da2/faa 0 2024-04-26T04:42:21.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/528: chown d9/d3c/f61 12702 1 2024-04-26T04:42:21.955 INFO:tasks.workunit.client.0.smithi090.stdout:6/529: rename d9/d21/d40/d4c/f7a to d9/d21/d40/d4c/d6f/d9f/dae/fc1 0 2024-04-26T04:42:21.969 INFO:tasks.workunit.client.0.smithi090.stdout:6/530: dwrite d9/d1c/d31/d79/f7e [4194304,4194304] 0 2024-04-26T04:42:21.969 INFO:tasks.workunit.client.0.smithi090.stdout:6/531: fdatasync d9/d21/f5f 0 2024-04-26T04:42:21.984 INFO:tasks.workunit.client.0.smithi090.stdout:6/532: dwrite d9/d1c/d31/d69/fa4 [0,4194304] 0 2024-04-26T04:42:21.984 INFO:tasks.workunit.client.0.smithi090.stdout:6/533: creat d9/d21/fc2 x:0 0 0 2024-04-26T04:42:21.984 INFO:tasks.workunit.client.0.smithi090.stdout:6/534: stat d9/d21/d40/d4c/c56 0 2024-04-26T04:42:21.986 INFO:tasks.workunit.client.0.smithi090.stdout:6/535: chown d9/c10 190 1 2024-04-26T04:42:21.987 INFO:tasks.workunit.client.0.smithi090.stdout:6/536: unlink d9/f12 0 2024-04-26T04:42:22.005 INFO:tasks.workunit.client.0.smithi090.stdout:6/537: dwrite d9/d1c/d4d/d6d/f93 [0,4194304] 0 2024-04-26T04:42:22.006 INFO:tasks.workunit.client.1.smithi104.stdout:5/760: dwrite d0/d1/d64/d26/d36/d72/dfe/fec [0,4194304] 0 2024-04-26T04:42:22.006 INFO:tasks.workunit.client.0.smithi090.stdout:8/488: dwrite dc/d1f/d49/d39/d55/d64/f8d [4194304,4194304] 0 2024-04-26T04:42:22.008 INFO:tasks.workunit.client.1.smithi104.stdout:0/644: dwrite d0/d2/d21/d49/d5e/d9c/d53/fa6 [0,4194304] 0 2024-04-26T04:42:22.008 INFO:tasks.workunit.client.1.smithi104.stdout:0/645: stat d0/d2/d21/d27/f7b 0 2024-04-26T04:42:22.009 INFO:tasks.workunit.client.0.smithi090.stdout:4/451: dwrite d3/db/d12/d20/d2d/f6f [0,4194304] 0 2024-04-26T04:42:22.009 INFO:tasks.workunit.client.0.smithi090.stdout:4/452: dread - d3/d13/d6c/f90 zero size 2024-04-26T04:42:22.009 INFO:tasks.workunit.client.0.smithi090.stdout:4/453: creat d3/db/d12/d20/d45/f92 x:0 0 0 2024-04-26T04:42:22.009 INFO:tasks.workunit.client.0.smithi090.stdout:4/454: write d3/db/d12/d20/f67 [536831,85071] 0 2024-04-26T04:42:22.009 INFO:tasks.workunit.client.0.smithi090.stdout:4/455: stat d3/d13/d64/f6b 0 2024-04-26T04:42:22.010 INFO:tasks.workunit.client.1.smithi104.stdout:5/761: rename d0/d1/d4/d2a/d2b/d5c/f8a to d0/d1/d1e/f100 0 2024-04-26T04:42:22.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/489: link dc/d1f/d49/d39/f52 dc/d1f/d49/d3b/d3d/d97/fac 0 2024-04-26T04:42:22.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/490: chown dc/l92 5 1 2024-04-26T04:42:22.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/491: dread - dc/d1f/d49/d39/d55/d64/f89 zero size 2024-04-26T04:42:22.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/492: stat dc/d1f/l48 0 2024-04-26T04:42:22.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/493: chown dc/d1f/d49/d3b/d3d/f65 68981 1 2024-04-26T04:42:22.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/494: write dc/d1f/d49/d3b/d3d/f5e [1183762,85207] 0 2024-04-26T04:42:22.012 INFO:tasks.workunit.client.0.smithi090.stdout:5/500: sync 2024-04-26T04:42:22.012 INFO:tasks.workunit.client.0.smithi090.stdout:9/351: sync 2024-04-26T04:42:22.012 INFO:tasks.workunit.client.0.smithi090.stdout:0/541: sync 2024-04-26T04:42:22.012 INFO:tasks.workunit.client.0.smithi090.stdout:9/352: fdatasync db/dd/d18/d38/f79 0 2024-04-26T04:42:22.012 INFO:tasks.workunit.client.0.smithi090.stdout:1/392: sync 2024-04-26T04:42:22.012 INFO:tasks.workunit.client.1.smithi104.stdout:0/646: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc 0 2024-04-26T04:42:22.014 INFO:tasks.workunit.client.1.smithi104.stdout:5/762: truncate d0/d1/d4/d2a/d2b/d37/f9c 2599828 0 2024-04-26T04:42:22.014 INFO:tasks.workunit.client.1.smithi104.stdout:5/763: readlink d0/d1/d4/d2a/d2b/d37/d48/dc6/ldd 0 2024-04-26T04:42:22.016 INFO:tasks.workunit.client.0.smithi090.stdout:8/495: mkdir dc/d1f/d49/d3b/d3d/d97/dad 0 2024-04-26T04:42:22.016 INFO:tasks.workunit.client.1.smithi104.stdout:5/764: mknod d0/d1/d64/d26/d36/d72/da9/c101 0 2024-04-26T04:42:22.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/542: truncate d0/de/f2c 1042575 0 2024-04-26T04:42:22.019 INFO:tasks.workunit.client.1.smithi104.stdout:5/765: getdents d0/d7b 0 2024-04-26T04:42:22.020 INFO:tasks.workunit.client.0.smithi090.stdout:5/501: mkdir d4/d1a/d1f/d53/d59/d5e/da0 0 2024-04-26T04:42:22.020 INFO:tasks.workunit.client.0.smithi090.stdout:5/502: fsync d4/d1a/df/d4a/f7a 0 2024-04-26T04:42:22.020 INFO:tasks.workunit.client.0.smithi090.stdout:1/393: mkdir d0/d4/d7/d4b/d53/d7b 0 2024-04-26T04:42:22.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/496: stat dc/c80 0 2024-04-26T04:42:22.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/497: fsync dc/d1f/d49/d3b/d3d/d97/fa2 0 2024-04-26T04:42:22.022 INFO:tasks.workunit.client.1.smithi104.stdout:5/766: link d0/d1/d4/d2a/d2b/d37/d48/c53 d0/d1/d4/d2a/d6b/c102 0 2024-04-26T04:42:22.022 INFO:tasks.workunit.client.1.smithi104.stdout:0/647: dread d0/d2/d21/d27/d1e/fc8 [0,4194304] 0 2024-04-26T04:42:22.022 INFO:tasks.workunit.client.1.smithi104.stdout:5/767: write d0/d1/d4/d2a/d2b/fa1 [2716185,79366] 0 2024-04-26T04:42:22.022 INFO:tasks.workunit.client.1.smithi104.stdout:5/768: fdatasync d0/d1/d3e/f51 0 2024-04-26T04:42:22.023 INFO:tasks.workunit.client.0.smithi090.stdout:1/394: unlink d0/d4/d7/d49/l76 0 2024-04-26T04:42:22.023 INFO:tasks.workunit.client.0.smithi090.stdout:0/543: mkdir d0/d4b/d5d/d5c/dc4 0 2024-04-26T04:42:22.023 INFO:tasks.workunit.client.0.smithi090.stdout:0/544: getdents d0/d4b/d5d/d48/dc1 0 2024-04-26T04:42:22.024 INFO:tasks.workunit.client.0.smithi090.stdout:1/395: mkdir d0/d4/d43/d4d/d58/d7c 0 2024-04-26T04:42:22.025 INFO:tasks.workunit.client.1.smithi104.stdout:5/769: symlink d0/d7b/l103 0 2024-04-26T04:42:22.026 INFO:tasks.workunit.client.0.smithi090.stdout:1/396: creat d0/d4/d62/d78/f7d x:0 0 0 2024-04-26T04:42:22.027 INFO:tasks.workunit.client.0.smithi090.stdout:1/397: mknod d0/d4/d43/c7e 0 2024-04-26T04:42:22.028 INFO:tasks.workunit.client.0.smithi090.stdout:1/398: symlink d0/d4/d7/d17/l7f 0 2024-04-26T04:42:22.037 INFO:tasks.workunit.client.1.smithi104.stdout:0/648: dread d0/d2/d21/d27/d1e/d57/f8f [0,4194304] 0 2024-04-26T04:42:22.039 INFO:tasks.workunit.client.1.smithi104.stdout:0/649: creat d0/d2/d10/fcd x:0 0 0 2024-04-26T04:42:22.040 INFO:tasks.workunit.client.1.smithi104.stdout:0/650: unlink d0/d2/d21/d27/d1e/cc6 0 2024-04-26T04:42:22.041 INFO:tasks.workunit.client.1.smithi104.stdout:0/651: dread d0/d2/d21/d27/d1e/d57/d6c/d85/fa2 [0,4194304] 0 2024-04-26T04:42:22.044 INFO:tasks.workunit.client.1.smithi104.stdout:0/652: link d0/c93 d0/d2/d21/d49/d5e/d9c/d53/d5d/d6f/cce 0 2024-04-26T04:42:22.046 INFO:tasks.workunit.client.1.smithi104.stdout:0/653: mkdir d0/d2/d21/d38/dcf 0 2024-04-26T04:42:22.049 INFO:tasks.workunit.client.1.smithi104.stdout:0/654: getdents d0/d2/d21/d27 0 2024-04-26T04:42:22.051 INFO:tasks.workunit.client.0.smithi090.stdout:9/353: dwrite f9 [0,4194304] 0 2024-04-26T04:42:22.051 INFO:tasks.workunit.client.0.smithi090.stdout:9/354: truncate db/dd/d18/d31/f4a 1024779 0 2024-04-26T04:42:22.053 INFO:tasks.workunit.client.0.smithi090.stdout:9/355: rmdir db/dd/d18/d31 39 2024-04-26T04:42:22.057 INFO:tasks.workunit.client.0.smithi090.stdout:6/538: dwrite d9/d21/d40/f74 [0,4194304] 0 2024-04-26T04:42:22.057 INFO:tasks.workunit.client.0.smithi090.stdout:3/383: dwrite d1/d17/d36/f67 [0,4194304] 0 2024-04-26T04:42:22.058 INFO:tasks.workunit.client.0.smithi090.stdout:9/356: dread db/dd/d18/d38/d39/f3b [0,4194304] 0 2024-04-26T04:42:22.059 INFO:tasks.workunit.client.0.smithi090.stdout:8/498: dwrite dc/d1f/d49/d3b/d3d/f65 [0,4194304] 0 2024-04-26T04:42:22.059 INFO:tasks.workunit.client.0.smithi090.stdout:5/503: dwrite d4/d1a/d2a/d79/d90/d56/f6f [0,4194304] 0 2024-04-26T04:42:22.064 INFO:tasks.workunit.client.1.smithi104.stdout:1/792: sync 2024-04-26T04:42:22.064 INFO:tasks.workunit.client.1.smithi104.stdout:8/659: sync 2024-04-26T04:42:22.064 INFO:tasks.workunit.client.1.smithi104.stdout:1/793: fsync d0/d6/d19/fae 0 2024-04-26T04:42:22.064 INFO:tasks.workunit.client.1.smithi104.stdout:2/921: sync 2024-04-26T04:42:22.065 INFO:tasks.workunit.client.0.smithi090.stdout:3/384: symlink d1/d17/d36/l84 0 2024-04-26T04:42:22.065 INFO:tasks.workunit.client.0.smithi090.stdout:9/357: creat db/dd/d18/d38/d39/f7e x:0 0 0 2024-04-26T04:42:22.072 INFO:tasks.workunit.client.1.smithi104.stdout:0/655: dread d0/d2/d21/f52 [0,4194304] 0 2024-04-26T04:42:22.072 INFO:tasks.workunit.client.1.smithi104.stdout:0/656: truncate d0/d2/d21/d27/faa 1378762 0 2024-04-26T04:42:22.072 INFO:tasks.workunit.client.0.smithi090.stdout:3/385: mknod d1/d17/d52/d7d/c85 0 2024-04-26T04:42:22.074 INFO:tasks.workunit.client.0.smithi090.stdout:9/358: rmdir db/dd/d18/d31/d6a/d70 39 2024-04-26T04:42:22.074 INFO:tasks.workunit.client.0.smithi090.stdout:9/359: truncate db/d5f/f6f 895992 0 2024-04-26T04:42:22.074 INFO:tasks.workunit.client.0.smithi090.stdout:9/360: fsync db/dd/d18/d31/d58/f65 0 2024-04-26T04:42:22.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/657: creat d0/d2/d21/d27/d1e/d57/fd0 x:0 0 0 2024-04-26T04:42:22.078 INFO:tasks.workunit.client.0.smithi090.stdout:9/361: creat db/dd/d18/d38/d39/f7f x:0 0 0 2024-04-26T04:42:22.079 INFO:tasks.workunit.client.0.smithi090.stdout:0/545: getdents d0/d4b/d5d/d5c 0 2024-04-26T04:42:22.083 INFO:tasks.workunit.client.0.smithi090.stdout:9/362: rename db/d16/c55 to db/dd/d44/c80 0 2024-04-26T04:42:22.087 INFO:tasks.workunit.client.1.smithi104.stdout:8/660: write d0/d33/f7b [1569197,110097] 0 2024-04-26T04:42:22.087 INFO:tasks.workunit.client.1.smithi104.stdout:8/661: write d0/f2f [9727131,65796] 0 2024-04-26T04:42:22.088 INFO:tasks.workunit.client.1.smithi104.stdout:8/662: read d0/d33/d35/f87 [2844381,110399] 0 2024-04-26T04:42:22.090 INFO:tasks.workunit.client.1.smithi104.stdout:8/663: rename d0/d1c/d2d/l3a to d0/d1c/lcd 0 2024-04-26T04:42:22.091 INFO:tasks.workunit.client.1.smithi104.stdout:8/664: chown d0/d33/l1b 3 1 2024-04-26T04:42:22.093 INFO:tasks.workunit.client.1.smithi104.stdout:8/665: mkdir d0/d9a/dce 0 2024-04-26T04:42:22.094 INFO:tasks.workunit.client.1.smithi104.stdout:8/666: mkdir d0/d50/dc4/dcf 0 2024-04-26T04:42:22.099 INFO:tasks.workunit.client.1.smithi104.stdout:0/658: write d0/d2/d21/d49/d5e/fa7 [409422,14981] 0 2024-04-26T04:42:22.100 INFO:tasks.workunit.client.1.smithi104.stdout:8/667: creat d0/d1c/fd0 x:0 0 0 2024-04-26T04:42:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.105+0000 7f7b33fff700 1 -- 172.21.15.90:0/2007419964 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b380054b0 con 0x7f7b3c106840 2024-04-26T04:42:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.105+0000 7f7b436ac700 1 -- 172.21.15.90:0/2007419964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 msgr2=0x7f7b3c106c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:22.106 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.105+0000 7f7b436ac700 1 --2- 172.21.15.90:0/2007419964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 0x7f7b3c106c50 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7b38009910 tx=0x7f7b38009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:22.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.106+0000 7f7b436ac700 1 -- 172.21.15.90:0/2007419964 shutdown_connections 2024-04-26T04:42:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.106+0000 7f7b436ac700 1 --2- 172.21.15.90:0/2007419964 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 0x7f7b3c107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.106+0000 7f7b436ac700 1 --2- 172.21.15.90:0/2007419964 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 0x7f7b3c106c50 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.106+0000 7f7b436ac700 1 -- 172.21.15.90:0/2007419964 >> 172.21.15.90:0/2007419964 conn(0x7f7b3c075970 msgr2=0x7f7b3c077dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.108+0000 7f7b436ac700 1 -- 172.21.15.90:0/2007419964 shutdown_connections 2024-04-26T04:42:22.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.108+0000 7f7b436ac700 1 -- 172.21.15.90:0/2007419964 wait complete. 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b436ac700 1 Processor -- start 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b436ac700 1 -- start start 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b436ac700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 0x7f7b3c19c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b436ac700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 0x7f7b3c19c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b436ac700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c19cd10 con 0x7f7b3c106840 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b436ac700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b3c19ce50 con 0x7f7b3c107a40 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b41448700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 0x7f7b3c19c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b40c47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 0x7f7b3c19c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b40c47700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 0x7f7b3c19c6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.90:45746/0 (socket says 172.21.15.90:45746) 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.109+0000 7f7b40c47700 1 -- 172.21.15.90:0/3985928980 learned_addr learned my addr 172.21.15.90:0/3985928980 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.110+0000 7f7b40c47700 1 -- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 msgr2=0x7f7b3c19c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:22.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.110+0000 7f7b40c47700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 0x7f7b3c19c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.110+0000 7f7b40c47700 1 -- 172.21.15.90:0/3985928980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b380095f0 con 0x7f7b3c107a40 2024-04-26T04:42:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.110+0000 7f7b40c47700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 0x7f7b3c19c6f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b4b0 tx=0x7f7b2c00b7c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.111+0000 7f7b327fc700 1 -- 172.21.15.90:0/3985928980 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b2c011800 con 0x7f7b3c107a40 2024-04-26T04:42:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.111+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b3c079740 con 0x7f7b3c107a40 2024-04-26T04:42:22.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.111+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b3c079c90 con 0x7f7b3c107a40 2024-04-26T04:42:22.114 INFO:tasks.workunit.client.0.smithi090.stdout:0/546: mkdir d0/d4b/d5d/d5c/d8a/d45/d53/dc5 0 2024-04-26T04:42:22.114 INFO:tasks.workunit.client.0.smithi090.stdout:0/547: write d0/d4b/d5d/d5c/d8a/d45/d53/faf [691315,93636] 0 2024-04-26T04:42:22.114 INFO:tasks.workunit.client.0.smithi090.stdout:0/548: dread - d0/d4b/d5d/d5c/d8a/d45/fb4 zero size 2024-04-26T04:42:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.111+0000 7f7b327fc700 1 -- 172.21.15.90:0/3985928980 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7b2c011e10 con 0x7f7b3c107a40 2024-04-26T04:42:22.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.111+0000 7f7b327fc700 1 -- 172.21.15.90:0/3985928980 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7b2c00f2e0 con 0x7f7b3c107a40 2024-04-26T04:42:22.114 INFO:tasks.workunit.client.0.smithi090.stdout:9/363: symlink db/dd/d18/d31/d6a/d70/l81 0 2024-04-26T04:42:22.115 INFO:tasks.workunit.client.0.smithi090.stdout:9/364: chown db/dd/d18/d31/d58/c5b 358358 1 2024-04-26T04:42:22.115 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.113+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b20005320 con 0x7f7b3c107a40 2024-04-26T04:42:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.114+0000 7f7b327fc700 1 -- 172.21.15.90:0/3985928980 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f7b2c011960 con 0x7f7b3c107a40 2024-04-26T04:42:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.114+0000 7f7b327fc700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b2806c680 0x7f7b2806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.114+0000 7f7b327fc700 1 -- 172.21.15.90:0/3985928980 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f7b2c08bb90 con 0x7f7b3c107a40 2024-04-26T04:42:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.114+0000 7f7b41448700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b2806c680 0x7f7b2806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:22.116 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.115+0000 7f7b41448700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b2806c680 0x7f7b2806eb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006010 tx=0x7f7b3800b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:22.117 INFO:tasks.workunit.client.1.smithi104.stdout:8/668: dread d0/f4d [0,4194304] 0 2024-04-26T04:42:22.117 INFO:tasks.workunit.client.1.smithi104.stdout:8/669: dread d0/d1c/f84 [0,4194304] 0 2024-04-26T04:42:22.120 INFO:tasks.workunit.client.1.smithi104.stdout:2/922: dwrite d6/d8/fea [0,4194304] 0 2024-04-26T04:42:22.120 INFO:tasks.workunit.client.1.smithi104.stdout:2/923: chown d6/d1b/d51/db8 1022517 1 2024-04-26T04:42:22.122 INFO:tasks.workunit.client.0.smithi090.stdout:6/539: dwrite d9/d77/db0/db5/fbe [0,4194304] 0 2024-04-26T04:42:22.122 INFO:tasks.workunit.client.0.smithi090.stdout:6/540: chown d9/d3c/f64 2110641 1 2024-04-26T04:42:22.122 INFO:tasks.workunit.client.1.smithi104.stdout:8/670: rename d0/d9a/l9e to d0/ld1 0 2024-04-26T04:42:22.122 INFO:tasks.workunit.client.1.smithi104.stdout:1/794: dwrite d0/d6/d19/d21/d66/d20/d2a/d56/d67/f76 [0,4194304] 0 2024-04-26T04:42:22.122 INFO:tasks.workunit.client.0.smithi090.stdout:3/386: dwrite d1/d9/fb [4194304,4194304] 0 2024-04-26T04:42:22.123 INFO:tasks.workunit.client.0.smithi090.stdout:3/387: stat d1/d17/d52/d7d 0 2024-04-26T04:42:22.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.121+0000 7f7b327fc700 1 -- 172.21.15.90:0/3985928980 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f7b2c059c80 con 0x7f7b3c107a40 2024-04-26T04:42:22.123 INFO:tasks.workunit.client.0.smithi090.stdout:8/499: dwrite dc/d1f/d49/d3b/f4a [0,4194304] 0 2024-04-26T04:42:22.124 INFO:tasks.workunit.client.0.smithi090.stdout:8/500: readlink dc/d1f/l31 0 2024-04-26T04:42:22.124 INFO:tasks.workunit.client.0.smithi090.stdout:8/501: truncate dc/d1f/d49/d4f/d58/d95/f83 473734 0 2024-04-26T04:42:22.124 INFO:tasks.workunit.client.0.smithi090.stdout:8/502: write dc/d34/d87/fa4 [608983,50476] 0 2024-04-26T04:42:22.124 INFO:tasks.workunit.client.1.smithi104.stdout:0/659: dwrite d0/d2/d10/d3d/f9e [0,4194304] 0 2024-04-26T04:42:22.124 INFO:tasks.workunit.client.1.smithi104.stdout:0/660: fsync d0/d4/f11 0 2024-04-26T04:42:22.124 INFO:tasks.workunit.client.1.smithi104.stdout:2/924: stat d6/d1b/d51/c76 0 2024-04-26T04:42:22.126 INFO:tasks.workunit.client.1.smithi104.stdout:2/925: creat d6/d8/d5a/f131 x:0 0 0 2024-04-26T04:42:22.126 INFO:tasks.workunit.client.1.smithi104.stdout:2/926: truncate d6/d1b/f38 926514 0 2024-04-26T04:42:22.129 INFO:tasks.workunit.client.1.smithi104.stdout:0/661: truncate d0/d2/d21/d27/d1e/fc8 4149028 0 2024-04-26T04:42:22.129 INFO:tasks.workunit.client.1.smithi104.stdout:0/662: truncate d0/d2/d21/d27/d1e/d57/d6c/d85/fa2 138019 0 2024-04-26T04:42:22.131 INFO:tasks.workunit.client.1.smithi104.stdout:2/927: rename d6/d8/d5a/la0 to d6/de/d1d/dcf/l132 0 2024-04-26T04:42:22.131 INFO:tasks.workunit.client.1.smithi104.stdout:0/663: truncate d0/d2/d21/d49/d5e/d9c/f86 6965746 0 2024-04-26T04:42:22.132 INFO:tasks.workunit.client.1.smithi104.stdout:8/671: write d0/d1c/f7e [1470988,67276] 0 2024-04-26T04:42:22.135 INFO:tasks.workunit.client.0.smithi090.stdout:6/541: dread d9/d21/d40/f74 [0,4194304] 0 2024-04-26T04:42:22.136 INFO:tasks.workunit.client.1.smithi104.stdout:8/672: link d0/d33/d35/d6d/f9c d0/d1c/fd2 0 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/388: write d1/d6/f14 [2418752,39572] 0 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/389: chown d1/d6/c2e 360172 1 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/390: chown d1/d9/c72 421 1 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/391: write d1/d17/f25 [2526420,16241] 0 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/392: fsync d1/d9/f38 0 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/393: chown d1/d17/f25 20 1 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/394: fsync d1/d9/f20 0 2024-04-26T04:42:22.137 INFO:tasks.workunit.client.0.smithi090.stdout:3/395: dread - d1/d5c/f6e zero size 2024-04-26T04:42:22.140 INFO:tasks.workunit.client.1.smithi104.stdout:8/673: dread d0/d1c/f26 [0,4194304] 0 2024-04-26T04:42:22.140 INFO:tasks.workunit.client.1.smithi104.stdout:8/674: write d0/d33/d35/d6d/f95 [422852,80077] 0 2024-04-26T04:42:22.140 INFO:tasks.workunit.client.1.smithi104.stdout:8/675: chown d0/d50/dc4/dcf 1 1 2024-04-26T04:42:22.140 INFO:tasks.workunit.client.1.smithi104.stdout:8/676: write d0/d1c/f91 [178277,129361] 0 2024-04-26T04:42:22.143 INFO:tasks.workunit.client.1.smithi104.stdout:8/677: unlink d0/d33/d35/d6d/da4/fcb 0 2024-04-26T04:42:22.143 INFO:tasks.workunit.client.1.smithi104.stdout:8/678: chown d0/d1c 1 1 2024-04-26T04:42:22.143 INFO:tasks.workunit.client.1.smithi104.stdout:8/679: fdatasync d0/d1c/d27/d47/f1d 0 2024-04-26T04:42:22.143 INFO:tasks.workunit.client.1.smithi104.stdout:1/795: dwrite d0/d6/d19/d21/d66/d20/da3/dde/ff0 [0,4194304] 0 2024-04-26T04:42:22.145 INFO:tasks.workunit.client.1.smithi104.stdout:0/664: rename d0/d2/d21/d38 to d0/d2/d21/d27/d1e/d57/d6c/dd1 0 2024-04-26T04:42:22.147 INFO:tasks.workunit.client.1.smithi104.stdout:2/928: write d6/d1b/d68/fd5 [5252835,67105] 0 2024-04-26T04:42:22.151 INFO:tasks.workunit.client.0.smithi090.stdout:3/396: dread d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:22.160 INFO:tasks.workunit.client.1.smithi104.stdout:1/796: link d0/d6/d9e/lc1 d0/d6/d19/d1b/l101 0 2024-04-26T04:42:22.160 INFO:tasks.workunit.client.1.smithi104.stdout:0/665: getdents d0/d2/d21/d49 0 2024-04-26T04:42:22.161 INFO:tasks.workunit.client.1.smithi104.stdout:0/666: write d0/d2/d21/d49/d5e/d9c/f46 [1656299,104770] 0 2024-04-26T04:42:22.164 INFO:tasks.workunit.client.1.smithi104.stdout:2/929: read d6/d8/d3e/f7a [2407801,70076] 0 2024-04-26T04:42:22.164 INFO:tasks.workunit.client.1.smithi104.stdout:2/930: dread - d6/d8/d5a/f131 zero size 2024-04-26T04:42:22.164 INFO:tasks.workunit.client.1.smithi104.stdout:2/931: fdatasync d6/d1b/d51/d6a/fb6 0 2024-04-26T04:42:22.167 INFO:tasks.workunit.client.1.smithi104.stdout:1/797: dread d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/fdc [0,4194304] 0 2024-04-26T04:42:22.168 INFO:tasks.workunit.client.1.smithi104.stdout:0/667: rmdir d0/d2/d10/d3d/db7/dc1 0 2024-04-26T04:42:22.168 INFO:tasks.workunit.client.1.smithi104.stdout:8/680: dwrite d0/d1c/d27/d47/d67/d76/daf/fb8 [4194304,4194304] 0 2024-04-26T04:42:22.168 INFO:tasks.workunit.client.1.smithi104.stdout:8/681: stat d0/d33/da6/fb6 0 2024-04-26T04:42:22.170 INFO:tasks.workunit.client.1.smithi104.stdout:2/932: rename d6/de/d1d/f80 to d6/f133 0 2024-04-26T04:42:22.170 INFO:tasks.workunit.client.1.smithi104.stdout:6/822: sync 2024-04-26T04:42:22.170 INFO:tasks.workunit.client.1.smithi104.stdout:7/713: sync 2024-04-26T04:42:22.171 INFO:tasks.workunit.client.1.smithi104.stdout:1/798: symlink d0/d6/d19/d21/d66/d29/df7/l102 0 2024-04-26T04:42:22.174 INFO:tasks.workunit.client.1.smithi104.stdout:0/668: creat d0/d2/d10/d3d/db7/fd2 x:0 0 0 2024-04-26T04:42:22.181 INFO:tasks.workunit.client.1.smithi104.stdout:1/799: rename d0/d6/d19/d21/d66/de4/d7e/dc6/cfc to d0/d6/d19/d21/d59/c103 0 2024-04-26T04:42:22.185 INFO:tasks.workunit.client.1.smithi104.stdout:2/933: dread d6/de/d85/fa6 [0,4194304] 0 2024-04-26T04:42:22.190 INFO:tasks.workunit.client.1.smithi104.stdout:2/934: rename d6/c9 to d6/d1b/d68/de0/c134 0 2024-04-26T04:42:22.193 INFO:tasks.workunit.client.1.smithi104.stdout:0/669: dread d0/d2/d21/d49/d5e/d9c/d53/fa6 [0,4194304] 0 2024-04-26T04:42:22.195 INFO:tasks.workunit.client.1.smithi104.stdout:0/670: symlink d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/ld3 0 2024-04-26T04:42:22.200 INFO:tasks.workunit.client.1.smithi104.stdout:5/770: sync 2024-04-26T04:42:22.200 INFO:tasks.workunit.client.1.smithi104.stdout:9/662: sync 2024-04-26T04:42:22.200 INFO:tasks.workunit.client.1.smithi104.stdout:4/796: sync 2024-04-26T04:42:22.200 INFO:tasks.workunit.client.1.smithi104.stdout:9/663: chown d11/d1d/d71/d94/ca0 259372156 1 2024-04-26T04:42:22.200 INFO:tasks.workunit.client.1.smithi104.stdout:9/664: chown d11/d1d/d71/l9f 5 1 2024-04-26T04:42:22.200 INFO:tasks.workunit.client.1.smithi104.stdout:5/771: truncate d0/d1/d64/d26/d36/d72/dfe/fc0 24714 0 2024-04-26T04:42:22.201 INFO:tasks.workunit.client.1.smithi104.stdout:9/665: mkdir d11/d1d/d87/dda 0 2024-04-26T04:42:22.202 INFO:tasks.workunit.client.1.smithi104.stdout:9/666: creat d11/d1d/d87/dda/fdb x:0 0 0 2024-04-26T04:42:22.204 INFO:tasks.workunit.client.1.smithi104.stdout:2/935: dread d6/de/d1d/f42 [0,4194304] 0 2024-04-26T04:42:22.209 INFO:tasks.workunit.client.1.smithi104.stdout:2/936: symlink d6/d8/d5a/l135 0 2024-04-26T04:42:22.210 INFO:tasks.workunit.client.1.smithi104.stdout:3/834: sync 2024-04-26T04:42:22.213 INFO:tasks.workunit.client.1.smithi104.stdout:3/835: creat d5/d9/d80/dbc/f125 x:0 0 0 2024-04-26T04:42:22.213 INFO:tasks.workunit.client.1.smithi104.stdout:3/836: write d5/d9/d80/da2/fad [154340,11087] 0 2024-04-26T04:42:22.217 INFO:tasks.workunit.client.1.smithi104.stdout:6/823: dwrite d0/d9/d13/d41/d82/dc8/f60 [4194304,4194304] 0 2024-04-26T04:42:22.219 INFO:tasks.workunit.client.1.smithi104.stdout:6/824: truncate d0/d9/d25/f39 3626324 0 2024-04-26T04:42:22.219 INFO:tasks.workunit.client.1.smithi104.stdout:6/825: chown d0/d9/d13/d41/d96/ded/d10d/d47/fe7 4811123 1 2024-04-26T04:42:22.219 INFO:tasks.workunit.client.1.smithi104.stdout:3/837: dread d5/db2/da3/fc4 [0,4194304] 0 2024-04-26T04:42:22.220 INFO:tasks.workunit.client.1.smithi104.stdout:4/797: dread d4/f37 [0,4194304] 0 2024-04-26T04:42:22.220 INFO:tasks.workunit.client.1.smithi104.stdout:4/798: write d4/d7/d1d/d1e/d24/d59/db8/fe3 [705617,2979] 0 2024-04-26T04:42:22.221 INFO:tasks.workunit.client.1.smithi104.stdout:4/799: write d4/db/d6b/d86/f101 [596630,116177] 0 2024-04-26T04:42:22.222 INFO:tasks.workunit.client.1.smithi104.stdout:6/826: rename d0/d9/d13/d41/d96/ded/f10b to d0/d9/d13/dcd/f110 0 2024-04-26T04:42:22.223 INFO:tasks.workunit.client.1.smithi104.stdout:3/838: symlink d5/db2/d60/l126 0 2024-04-26T04:42:22.238 INFO:tasks.workunit.client.1.smithi104.stdout:4/800: write d4/d7/d1d/d2e/d74/dbd/dd3/f10d [2085049,80129] 0 2024-04-26T04:42:22.238 INFO:tasks.workunit.client.1.smithi104.stdout:1/800: dwrite d0/d6/d19/d21/d66/d20/ff5 [0,4194304] 0 2024-04-26T04:42:22.240 INFO:tasks.workunit.client.1.smithi104.stdout:4/801: mkdir d4/d20/d55/d56/da3/dea/dfd/d11a 0 2024-04-26T04:42:22.240 INFO:tasks.workunit.client.1.smithi104.stdout:4/802: fdatasync d4/d7/d1d/d1e/d24/d59/f6a 0 2024-04-26T04:42:22.240 INFO:tasks.workunit.client.1.smithi104.stdout:7/714: dwrite d2/d4/d32/d35/d44/fa4 [0,4194304] 0 2024-04-26T04:42:22.242 INFO:tasks.workunit.client.1.smithi104.stdout:1/801: mkdir d0/d6/d19/d21/d66/d20/d2a/d56/d67/d104 0 2024-04-26T04:42:22.244 INFO:tasks.workunit.client.1.smithi104.stdout:7/715: mknod d2/ce7 0 2024-04-26T04:42:22.244 INFO:tasks.workunit.client.1.smithi104.stdout:7/716: fdatasync d2/d4/d32/d35/d44/f91 0 2024-04-26T04:42:22.244 INFO:tasks.workunit.client.1.smithi104.stdout:7/717: chown d2/d4/d1d/f90 1041 1 2024-04-26T04:42:22.244 INFO:tasks.workunit.client.1.smithi104.stdout:7/718: creat d2/fe8 x:0 0 0 2024-04-26T04:42:22.245 INFO:tasks.workunit.client.1.smithi104.stdout:7/719: fdatasync d2/d4/d1d/d34/db0/fd5 0 2024-04-26T04:42:22.247 INFO:tasks.workunit.client.1.smithi104.stdout:7/720: symlink d2/d4/le9 0 2024-04-26T04:42:22.250 INFO:tasks.workunit.client.1.smithi104.stdout:9/667: dwrite d11/f12 [0,4194304] 0 2024-04-26T04:42:22.250 INFO:tasks.workunit.client.1.smithi104.stdout:9/668: chown d11/d15/d1b/d6d/d9c/dbe 0 1 2024-04-26T04:42:22.253 INFO:tasks.workunit.client.1.smithi104.stdout:4/803: write d4/db/d6b/fc9 [433266,48278] 0 2024-04-26T04:42:22.254 INFO:tasks.workunit.client.1.smithi104.stdout:0/671: dwrite d0/d4/d3b/f77 [0,4194304] 0 2024-04-26T04:42:22.255 INFO:tasks.workunit.client.1.smithi104.stdout:3/839: dwrite d5/d9/f12 [4194304,4194304] 0 2024-04-26T04:42:22.256 INFO:tasks.workunit.client.1.smithi104.stdout:5/772: dwrite d0/d1/d1e/d98/dd4/fe9 [0,4194304] 0 2024-04-26T04:42:22.256 INFO:tasks.workunit.client.1.smithi104.stdout:2/937: dwrite d6/de/f111 [0,4194304] 0 2024-04-26T04:42:22.256 INFO:tasks.workunit.client.1.smithi104.stdout:2/938: chown d6/d1b/d93/faa 121179 1 2024-04-26T04:42:22.256 INFO:tasks.workunit.client.1.smithi104.stdout:2/939: read d6/d8/d3e/f10d [432955,44814] 0 2024-04-26T04:42:22.262 INFO:tasks.workunit.client.1.smithi104.stdout:6/827: dwrite d0/d9/dc6/f2c [4194304,4194304] 0 2024-04-26T04:42:22.263 INFO:tasks.workunit.client.1.smithi104.stdout:6/828: write d0/d9/dc6/f7c [204292,43668] 0 2024-04-26T04:42:22.263 INFO:tasks.workunit.client.1.smithi104.stdout:6/829: fsync d0/dd8/de0/d102/f105 0 2024-04-26T04:42:22.263 INFO:tasks.workunit.client.1.smithi104.stdout:6/830: write d0/ff3 [2349518,13548] 0 2024-04-26T04:42:22.263 INFO:tasks.workunit.client.1.smithi104.stdout:6/831: chown d0/d9/d13/d41/d82/dc8/d80/f75 1 1 2024-04-26T04:42:22.270 INFO:tasks.workunit.client.1.smithi104.stdout:3/840: dread d5/d9/f12 [8388608,4194304] 0 2024-04-26T04:42:22.271 INFO:tasks.workunit.client.1.smithi104.stdout:5/773: write d0/f6 [324245,56329] 0 2024-04-26T04:42:22.273 INFO:tasks.workunit.client.1.smithi104.stdout:2/940: link d6/de/d85/fcc d6/d1b/f136 0 2024-04-26T04:42:22.281 INFO:tasks.workunit.client.1.smithi104.stdout:7/721: dwrite d2/d4/f11 [0,4194304] 0 2024-04-26T04:42:22.281 INFO:tasks.workunit.client.1.smithi104.stdout:4/804: dwrite d4/d7/d1d/d1e/d24/d4a/d9e/fcd [0,4194304] 0 2024-04-26T04:42:22.281 INFO:tasks.workunit.client.1.smithi104.stdout:4/805: creat d4/d7/d1d/d1e/d24/d4a/d9e/f11b x:0 0 0 2024-04-26T04:42:22.283 INFO:tasks.workunit.client.1.smithi104.stdout:0/672: dread d0/f4d [0,4194304] 0 2024-04-26T04:42:22.284 INFO:tasks.workunit.client.1.smithi104.stdout:2/941: dread d6/d1b/f38 [0,4194304] 0 2024-04-26T04:42:22.284 INFO:tasks.workunit.client.1.smithi104.stdout:2/942: readlink d6/d8/d3e/l3b 0 2024-04-26T04:42:22.284 INFO:tasks.workunit.client.1.smithi104.stdout:2/943: chown d6/d8/f78 4 1 2024-04-26T04:42:22.287 INFO:tasks.workunit.client.1.smithi104.stdout:7/722: truncate d2/d4/d32/d3d/d41/d5f/d6d/fa3 314238 0 2024-04-26T04:42:22.291 INFO:tasks.workunit.client.1.smithi104.stdout:4/806: mkdir d4/d20/d55/d56/da3/dea/d11c 0 2024-04-26T04:42:22.291 INFO:tasks.workunit.client.1.smithi104.stdout:4/807: write d4/db/fe [1049441,24443] 0 2024-04-26T04:42:22.292 INFO:tasks.workunit.client.1.smithi104.stdout:0/673: rmdir d0/d2/d21/d27/d1e 39 2024-04-26T04:42:22.292 INFO:tasks.workunit.client.1.smithi104.stdout:0/674: stat d0/d2/da0 0 2024-04-26T04:42:22.293 INFO:tasks.workunit.client.1.smithi104.stdout:2/944: creat d6/de/d1d/f137 x:0 0 0 2024-04-26T04:42:22.294 INFO:tasks.workunit.client.1.smithi104.stdout:4/808: rmdir d4/d7/d1d/d1e/d15/df3 39 2024-04-26T04:42:22.295 INFO:tasks.workunit.client.1.smithi104.stdout:0/675: creat d0/d2/d21/d27/fd4 x:0 0 0 2024-04-26T04:42:22.296 INFO:tasks.workunit.client.1.smithi104.stdout:2/945: rmdir d6/d1b/d51/db8 39 2024-04-26T04:42:22.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/832: dread d0/d9/d13/d41/d82/dc8/f94 [0,4194304] 0 2024-04-26T04:42:22.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/833: stat d0/d9/d13/d41/d96/ded/d10d 0 2024-04-26T04:42:22.296 INFO:tasks.workunit.client.1.smithi104.stdout:6/834: write d0/d9/d13/d41/d96/ded/d10d/d71/da5/fb7 [964418,113492] 0 2024-04-26T04:42:22.297 INFO:tasks.workunit.client.1.smithi104.stdout:1/802: dwrite d0/d6/d19/d21/d66/d20/da3/dde/ff0 [0,4194304] 0 2024-04-26T04:42:22.297 INFO:tasks.workunit.client.1.smithi104.stdout:1/803: truncate d0/d6/d2e/f86 815183 0 2024-04-26T04:42:22.297 INFO:tasks.workunit.client.1.smithi104.stdout:1/804: read - d0/d6/d2e/d32/fc7 zero size 2024-04-26T04:42:22.299 INFO:tasks.workunit.client.1.smithi104.stdout:2/946: mknod d6/d1b/d51/d6a/dd8/c138 0 2024-04-26T04:42:22.299 INFO:tasks.workunit.client.1.smithi104.stdout:2/947: truncate d6/de/d1d/f4a 1521338 0 2024-04-26T04:42:22.299 INFO:tasks.workunit.client.1.smithi104.stdout:6/835: dread d0/d9/d13/d41/d96/ded/d10d/d71/d3f/fe6 [0,4194304] 0 2024-04-26T04:42:22.299 INFO:tasks.workunit.client.1.smithi104.stdout:4/809: rmdir d4/d20/d55/d56/da3/dbf 39 2024-04-26T04:42:22.301 INFO:tasks.workunit.client.1.smithi104.stdout:3/841: write d5/db2/f34 [1263167,80540] 0 2024-04-26T04:42:22.301 INFO:tasks.workunit.client.1.smithi104.stdout:3/842: truncate d5/db2/f3b 4711202 0 2024-04-26T04:42:22.303 INFO:tasks.workunit.client.1.smithi104.stdout:1/805: truncate d0/d6/d19/d21/d66/de4/d7e/dc6/fce 652822 0 2024-04-26T04:42:22.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/365: truncate db/dd/d44/f34 2522547 0 2024-04-26T04:42:22.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/366: readlink db/dd/d44/l32 0 2024-04-26T04:42:22.305 INFO:tasks.workunit.client.1.smithi104.stdout:2/948: mkdir d6/d1b/d51/d139 0 2024-04-26T04:42:22.306 INFO:tasks.workunit.client.1.smithi104.stdout:2/949: chown d6/d8/d5a/l135 4 1 2024-04-26T04:42:22.306 INFO:tasks.workunit.client.1.smithi104.stdout:2/950: fdatasync d6/d1b/d93/f9d 0 2024-04-26T04:42:22.307 INFO:tasks.workunit.client.1.smithi104.stdout:6/836: getdents d0/d9/d13/d41/d82/dc8 0 2024-04-26T04:42:22.307 INFO:tasks.workunit.client.1.smithi104.stdout:6/837: dread - d0/d9/d13/d41/d82/dc8/d80/f100 zero size 2024-04-26T04:42:22.310 INFO:tasks.workunit.client.1.smithi104.stdout:5/774: dread d0/f6 [0,4194304] 0 2024-04-26T04:42:22.312 INFO:tasks.workunit.client.1.smithi104.stdout:7/723: dwrite d2/d4/d5a/da0/fab [0,4194304] 0 2024-04-26T04:42:22.314 INFO:tasks.workunit.client.0.smithi090.stdout:8/503: creat dc/d71/fae x:0 0 0 2024-04-26T04:42:22.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/542: mkdir d9/d77/db0/dc3 0 2024-04-26T04:42:22.316 INFO:tasks.workunit.client.0.smithi090.stdout:3/397: symlink d1/d17/d36/d2a/d3a/d5d/l86 0 2024-04-26T04:42:22.320 INFO:tasks.workunit.client.1.smithi104.stdout:3/843: mknod d5/db2/da3/dd0/ddb/d32/d48/c127 0 2024-04-26T04:42:22.320 INFO:tasks.workunit.client.1.smithi104.stdout:3/844: getdents d5 0 2024-04-26T04:42:22.320 INFO:tasks.workunit.client.1.smithi104.stdout:2/951: write d6/d8/f5b [676331,2069] 0 2024-04-26T04:42:22.322 INFO:tasks.workunit.client.1.smithi104.stdout:6/838: unlink d0/d9/d13/d41/d82/dc8/dba/daa/fdf 0 2024-04-26T04:42:22.325 INFO:tasks.workunit.client.0.smithi090.stdout:8/504: link dc/d1f/d49/d39/l7e dc/d1f/d49/d39/d99/laf 0 2024-04-26T04:42:22.325 INFO:tasks.workunit.client.0.smithi090.stdout:8/505: write dc/d1f/d49/d39/f7c [169230,86] 0 2024-04-26T04:42:22.326 INFO:tasks.workunit.client.1.smithi104.stdout:1/806: dwrite d0/d6/d2e/d32/d4c/fef [0,4194304] 0 2024-04-26T04:42:22.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/810: truncate d4/db/d6b/fc9 5248789 0 2024-04-26T04:42:22.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/811: creat d4/db/d6b/d7f/f11d x:0 0 0 2024-04-26T04:42:22.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/812: read d4/d20/d55/d56/f94 [2479860,83290] 0 2024-04-26T04:42:22.330 INFO:tasks.workunit.client.0.smithi090.stdout:6/543: dread d9/d1c/f2b [4194304,4194304] 0 2024-04-26T04:42:22.330 INFO:tasks.workunit.client.0.smithi090.stdout:6/544: write d9/d3c/fb3 [841412,110769] 0 2024-04-26T04:42:22.330 INFO:tasks.workunit.client.0.smithi090.stdout:8/506: creat dc/d1f/d49/d39/d55/d64/fb0 x:0 0 0 2024-04-26T04:42:22.330 INFO:tasks.workunit.client.0.smithi090.stdout:6/545: mknod d9/d1c/d31/d69/dbc/cc4 0 2024-04-26T04:42:22.330 INFO:tasks.workunit.client.0.smithi090.stdout:6/546: chown d9 27111 1 2024-04-26T04:42:22.330 INFO:tasks.workunit.client.0.smithi090.stdout:6/547: write d9/d3c/f64 [360825,50119] 0 2024-04-26T04:42:22.331 INFO:tasks.workunit.client.0.smithi090.stdout:6/548: mkdir d9/d1c/dc5 0 2024-04-26T04:42:22.333 INFO:tasks.workunit.client.0.smithi090.stdout:6/549: creat d9/d1c/d4d/d6d/fc6 x:0 0 0 2024-04-26T04:42:22.334 INFO:tasks.workunit.client.0.smithi090.stdout:6/550: write d9/d1c/d31/f3a [625067,117941] 0 2024-04-26T04:42:22.334 INFO:tasks.workunit.client.0.smithi090.stdout:6/551: write d9/d1c/d4d/d58/f8c [4151759,56606] 0 2024-04-26T04:42:22.334 INFO:tasks.workunit.client.1.smithi104.stdout:5/775: dread d0/d41/fed [0,4194304] 0 2024-04-26T04:42:22.337 INFO:tasks.workunit.client.1.smithi104.stdout:3/845: mkdir d5/d9/d80/dbc/dd4/d128 0 2024-04-26T04:42:22.338 INFO:tasks.workunit.client.0.smithi090.stdout:6/552: rename d9/d77/db0/dc3 to d9/d21/d40/dc7 0 2024-04-26T04:42:22.339 INFO:tasks.workunit.client.0.smithi090.stdout:8/507: write dc/d1f/fa8 [326242,129646] 0 2024-04-26T04:42:22.340 INFO:tasks.workunit.client.1.smithi104.stdout:6/839: mkdir d0/d9/d13/d41/d96/ded/d10d/d71/da5/d111 0 2024-04-26T04:42:22.340 INFO:tasks.workunit.client.1.smithi104.stdout:6/840: fdatasync d0/d9/d13/d41/d96/ded/d10d/d71/fb9 0 2024-04-26T04:42:22.340 INFO:tasks.workunit.client.1.smithi104.stdout:2/952: dwrite d6/de/d85/fcc [0,4194304] 0 2024-04-26T04:42:22.341 INFO:tasks.workunit.client.1.smithi104.stdout:2/953: write d6/d8/df6/dfe/f10f [335687,62626] 0 2024-04-26T04:42:22.341 INFO:tasks.workunit.client.1.smithi104.stdout:2/954: write d6/d1b/d75/f10a [223682,70464] 0 2024-04-26T04:42:22.342 INFO:tasks.workunit.client.0.smithi090.stdout:6/553: unlink d9/d1c/d4d/d58/f8c 0 2024-04-26T04:42:22.344 INFO:tasks.workunit.client.1.smithi104.stdout:1/807: creat d0/d6/d2e/d32/d4d/f105 x:0 0 0 2024-04-26T04:42:22.346 INFO:tasks.workunit.client.0.smithi090.stdout:6/554: mkdir d9/d77/d9a/dc8 0 2024-04-26T04:42:22.346 INFO:tasks.workunit.client.0.smithi090.stdout:6/555: fsync d9/d1c/d4d/d6d/f98 0 2024-04-26T04:42:22.346 INFO:tasks.workunit.client.0.smithi090.stdout:6/556: readlink d9/l33 0 2024-04-26T04:42:22.347 INFO:tasks.workunit.client.0.smithi090.stdout:8/508: rmdir dc/d5a/d5d 39 2024-04-26T04:42:22.347 INFO:tasks.workunit.client.0.smithi090.stdout:8/509: write dc/d1f/d49/d39/d55/f6c [3946093,100512] 0 2024-04-26T04:42:22.348 INFO:tasks.workunit.client.1.smithi104.stdout:4/813: symlink d4/d7/d1d/d2e/d74/dbd/dee/l11e 0 2024-04-26T04:42:22.350 INFO:tasks.workunit.client.1.smithi104.stdout:5/776: symlink d0/d1/l104 0 2024-04-26T04:42:22.350 INFO:tasks.workunit.client.0.smithi090.stdout:8/510: mkdir dc/d1f/d49/d4f/da6/db1 0 2024-04-26T04:42:22.351 INFO:tasks.workunit.client.0.smithi090.stdout:8/511: truncate dc/d5a/d5d/f66 598307 0 2024-04-26T04:42:22.352 INFO:tasks.workunit.client.0.smithi090.stdout:8/512: symlink dc/d71/lb2 0 2024-04-26T04:42:22.352 INFO:tasks.workunit.client.1.smithi104.stdout:7/724: rmdir d2/d4/d1d/d34 39 2024-04-26T04:42:22.352 INFO:tasks.workunit.client.1.smithi104.stdout:6/841: dread d0/d9/d13/d41/d96/dac/dec/f9b [0,4194304] 0 2024-04-26T04:42:22.355 INFO:tasks.workunit.client.1.smithi104.stdout:3/846: link d5/db2/f2a d5/db2/da3/dd0/ddb/d3a/df5/f129 0 2024-04-26T04:42:22.357 INFO:tasks.workunit.client.1.smithi104.stdout:2/955: mkdir d6/d1b/d51/d139/d13a 0 2024-04-26T04:42:22.358 INFO:tasks.workunit.client.1.smithi104.stdout:1/808: creat d0/d6/d2e/f106 x:0 0 0 2024-04-26T04:42:22.360 INFO:tasks.workunit.client.1.smithi104.stdout:4/814: creat d4/d7/d1d/d1e/d24/d48/f11f x:0 0 0 2024-04-26T04:42:22.361 INFO:tasks.workunit.client.1.smithi104.stdout:6/842: write d0/d9/d13/d41/d96/dac/dc4/fcf [3358890,93819] 0 2024-04-26T04:42:22.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/513: dread dc/f5c [0,4194304] 0 2024-04-26T04:42:22.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/514: creat dc/d1f/d49/d4f/d58/d95/fb3 x:0 0 0 2024-04-26T04:42:22.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/515: truncate dc/d1f/d49/d3b/d3d/f9c 736343 0 2024-04-26T04:42:22.364 INFO:tasks.workunit.client.1.smithi104.stdout:5/777: mknod d0/d1/d4/d2a/d2b/d37/d48/dc1/dfd/c105 0 2024-04-26T04:42:22.364 INFO:tasks.workunit.client.1.smithi104.stdout:7/725: link d2/d4/d32/d60/l66 d2/d4/d32/d3d/d41/dc0/lea 0 2024-04-26T04:42:22.365 INFO:tasks.workunit.client.1.smithi104.stdout:2/956: creat d6/d1b/d51/d6a/f13b x:0 0 0 2024-04-26T04:42:22.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/557: dwrite d9/d21/d3b/f7c [4194304,4194304] 0 2024-04-26T04:42:22.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/558: truncate d9/d21/d40/d4c/f7b 1488897 0 2024-04-26T04:42:22.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/559: write d9/d21/d40/d4c/d6f/fb6 [374413,56579] 0 2024-04-26T04:42:22.367 INFO:tasks.workunit.client.1.smithi104.stdout:4/815: mknod d4/d7/d1d/d8e/c120 0 2024-04-26T04:42:22.368 INFO:tasks.workunit.client.1.smithi104.stdout:6/843: unlink d0/d9/d13/d41/d96/ded/d10d/d47/f4c 0 2024-04-26T04:42:22.369 INFO:tasks.workunit.client.1.smithi104.stdout:3/847: write d5/db2/da3/dd0/ddb/f21 [996526,9637] 0 2024-04-26T04:42:22.370 INFO:tasks.workunit.client.1.smithi104.stdout:3/848: chown d5/db2/da3/ff2 250 1 2024-04-26T04:42:22.370 INFO:tasks.workunit.client.1.smithi104.stdout:3/849: stat d5/db2/c66 0 2024-04-26T04:42:22.370 INFO:tasks.workunit.client.1.smithi104.stdout:3/850: dread - d5/db2/d60/faf zero size 2024-04-26T04:42:22.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/560: link d9/d1c/d4d/d6d/d66/ca0 d9/d21/d40/d4c/cc9 0 2024-04-26T04:42:22.373 INFO:tasks.workunit.client.1.smithi104.stdout:5/778: rename d0/d1/d64/d26/lca to d0/d1/d64/d26/d36/d72/dfe/l106 0 2024-04-26T04:42:22.375 INFO:tasks.workunit.client.1.smithi104.stdout:7/726: creat d2/d4/d32/d35/dbe/feb x:0 0 0 2024-04-26T04:42:22.377 INFO:tasks.workunit.client.1.smithi104.stdout:2/957: truncate d6/de/d1d/f83 3992121 0 2024-04-26T04:42:22.378 INFO:tasks.workunit.client.1.smithi104.stdout:1/809: getdents d0/d6/d19/d21/d66/d29/df7 0 2024-04-26T04:42:22.378 INFO:tasks.workunit.client.0.smithi090.stdout:8/516: dwrite dc/d1f/f27 [0,4194304] 0 2024-04-26T04:42:22.381 INFO:tasks.workunit.client.1.smithi104.stdout:4/816: symlink d4/d20/d55/d56/da3/dea/dfd/d11a/l121 0 2024-04-26T04:42:22.381 INFO:tasks.workunit.client.0.smithi090.stdout:8/517: symlink dc/d1f/d49/d39/d55/d64/lb4 0 2024-04-26T04:42:22.383 INFO:tasks.workunit.client.1.smithi104.stdout:6/844: link d0/d9/d13/d41/d82/dc8/d80/fc2 d0/d9/d13/d41/d96/dac/dec/dbd/f112 0 2024-04-26T04:42:22.385 INFO:tasks.workunit.client.1.smithi104.stdout:3/851: rename d5/db2/da3/dd0/ddb/d32/d48/f67 to d5/db2/d46/db7/de6/d112/f12a 0 2024-04-26T04:42:22.386 INFO:tasks.workunit.client.1.smithi104.stdout:5/779: mknod d0/d1/d4/d2a/d6b/d73/da4/db5/c107 0 2024-04-26T04:42:22.387 INFO:tasks.workunit.client.1.smithi104.stdout:5/780: truncate d0/d1/d4/d2a/d2b/d37/f7c 897500 0 2024-04-26T04:42:22.387 INFO:tasks.workunit.client.0.smithi090.stdout:6/561: truncate d9/d77/db0/db5/fbe 3805810 0 2024-04-26T04:42:22.388 INFO:tasks.workunit.client.1.smithi104.stdout:1/810: mkdir d0/d6/d19/d21/d66/d20/d107 0 2024-04-26T04:42:22.388 INFO:tasks.workunit.client.1.smithi104.stdout:1/811: fsync d0/d6/d2e/f86 0 2024-04-26T04:42:22.389 INFO:tasks.workunit.client.0.smithi090.stdout:6/562: mkdir d9/d77/db0/db5/dca 0 2024-04-26T04:42:22.389 INFO:tasks.workunit.client.0.smithi090.stdout:7/508: sync 2024-04-26T04:42:22.391 INFO:tasks.workunit.client.1.smithi104.stdout:6/845: symlink d0/d9/d13/d41/d82/df7/dfb/l113 0 2024-04-26T04:42:22.391 INFO:tasks.workunit.client.1.smithi104.stdout:6/846: dread - d0/d9/da8/fea zero size 2024-04-26T04:42:22.391 INFO:tasks.workunit.client.1.smithi104.stdout:6/847: fdatasync d0/d9/d13/d41/f7f 0 2024-04-26T04:42:22.392 INFO:tasks.workunit.client.1.smithi104.stdout:3/852: mknod d5/db2/d46/db7/c12b 0 2024-04-26T04:42:22.394 INFO:tasks.workunit.client.1.smithi104.stdout:5/781: getdents d0/d1/d4/d2a/d2b/d37/d48 0 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/509: creat d0/d1/d4/d98/d4a/fbe x:0 0 0 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:1/399: sync 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:4/456: sync 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:5/504: sync 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/510: chown d0/d1/d4/d3e/f90 1 1 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/511: chown d0/db/fa1 1 1 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/512: write d0/d1/da3/fa4 [1012531,128677] 0 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/513: stat d0/db/d15/dbc 0 2024-04-26T04:42:22.395 INFO:tasks.workunit.client.0.smithi090.stdout:2/491: sync 2024-04-26T04:42:22.397 INFO:tasks.workunit.client.0.smithi090.stdout:1/400: rename d0/d4/d43/d4d to d0/d4/d7/d4a/d75/d80 0 2024-04-26T04:42:22.397 INFO:tasks.workunit.client.1.smithi104.stdout:1/812: rmdir d0/d6/d19/d21/d58/df2 39 2024-04-26T04:42:22.398 INFO:tasks.workunit.client.1.smithi104.stdout:6/848: getdents d0/d9/d13/d41/d82 0 2024-04-26T04:42:22.398 INFO:tasks.workunit.client.1.smithi104.stdout:6/849: write d0/d9/f2a [3411175,10677] 0 2024-04-26T04:42:22.398 INFO:tasks.workunit.client.1.smithi104.stdout:3/853: mkdir d5/db2/da3/dd0/ddb/d12c 0 2024-04-26T04:42:22.399 INFO:tasks.workunit.client.1.smithi104.stdout:6/850: write d0/d9/d13/d41/d82/dc8/f5e [1385731,122560] 0 2024-04-26T04:42:22.399 INFO:tasks.workunit.client.1.smithi104.stdout:6/851: dread - d0/dd8/de0/d102/fe8 zero size 2024-04-26T04:42:22.401 INFO:tasks.workunit.client.0.smithi090.stdout:1/401: mknod d0/d4/d7/d69/c81 0 2024-04-26T04:42:22.404 INFO:tasks.workunit.client.1.smithi104.stdout:1/813: creat d0/d6/d2e/d32/d4c/db1/dc3/f108 x:0 0 0 2024-04-26T04:42:22.405 INFO:tasks.workunit.client.0.smithi090.stdout:1/402: rename d0/d4/d7/d17/d22/d31/f40 to d0/d4/d62/d78/f82 0 2024-04-26T04:42:22.406 INFO:tasks.workunit.client.0.smithi090.stdout:9/367: dread db/dd/d44/f34 [0,4194304] 0 2024-04-26T04:42:22.409 INFO:tasks.workunit.client.0.smithi090.stdout:9/368: mkdir db/dd/d44/d82 0 2024-04-26T04:42:22.409 INFO:tasks.workunit.client.0.smithi090.stdout:9/369: dread - db/dd/d44/f56 zero size 2024-04-26T04:42:22.412 INFO:tasks.workunit.client.0.smithi090.stdout:1/403: dread d0/d4/f15 [0,4194304] 0 2024-04-26T04:42:22.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/404: fsync d0/d4/d7/d49/f5e 0 2024-04-26T04:42:22.414 INFO:tasks.workunit.client.0.smithi090.stdout:9/370: rmdir db/dd/d18/d38/d39/d67 0 2024-04-26T04:42:22.414 INFO:tasks.workunit.client.0.smithi090.stdout:9/371: write db/dd/d44/f5e [887608,119870] 0 2024-04-26T04:42:22.416 INFO:tasks.workunit.client.0.smithi090.stdout:9/372: symlink db/l83 0 2024-04-26T04:42:22.416 INFO:tasks.workunit.client.0.smithi090.stdout:8/518: dwrite dc/d1f/d49/d3b/f6d [0,4194304] 0 2024-04-26T04:42:22.416 INFO:tasks.workunit.client.1.smithi104.stdout:7/727: dwrite d2/d4/d32/d35/d44/fb3 [0,4194304] 0 2024-04-26T04:42:22.416 INFO:tasks.workunit.client.1.smithi104.stdout:1/814: mkdir d0/d6/d19/d21/d66/d20/d2a/d109 0 2024-04-26T04:42:22.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/405: dread d0/f2e [0,4194304] 0 2024-04-26T04:42:22.419 INFO:tasks.workunit.client.0.smithi090.stdout:9/373: creat db/f84 x:0 0 0 2024-04-26T04:42:22.419 INFO:tasks.workunit.client.0.smithi090.stdout:9/374: stat db/dd/d18/d31/f4a 0 2024-04-26T04:42:22.420 INFO:tasks.workunit.client.1.smithi104.stdout:7/728: creat d2/d4/d5a/fec x:0 0 0 2024-04-26T04:42:22.420 INFO:tasks.workunit.client.0.smithi090.stdout:3/398: symlink d1/d6/l87 0 2024-04-26T04:42:22.421 INFO:tasks.workunit.client.0.smithi090.stdout:8/519: symlink dc/d5a/d5d/d73/lb5 0 2024-04-26T04:42:22.424 INFO:tasks.workunit.client.0.smithi090.stdout:2/492: dread db/d11/f17 [0,4194304] 0 2024-04-26T04:42:22.426 INFO:tasks.workunit.client.1.smithi104.stdout:4/817: dwrite d4/db/f10e [0,4194304] 0 2024-04-26T04:42:22.426 INFO:tasks.workunit.client.1.smithi104.stdout:4/818: chown d4/d20/d55/d56/da3/dea/d11c 32555803 1 2024-04-26T04:42:22.429 INFO:tasks.workunit.client.1.smithi104.stdout:4/819: mkdir d4/db/d6b/d86/d122 0 2024-04-26T04:42:22.429 INFO:tasks.workunit.client.1.smithi104.stdout:4/820: fdatasync d4/d7/d1d/d2e/fba 0 2024-04-26T04:42:22.433 INFO:tasks.workunit.client.1.smithi104.stdout:4/821: truncate d4/d7/d1d/d1e/d24/d4a/fac 37130 0 2024-04-26T04:42:22.433 INFO:tasks.workunit.client.1.smithi104.stdout:4/822: write d4/d20/f51 [1702588,25724] 0 2024-04-26T04:42:22.434 INFO:tasks.workunit.client.0.smithi090.stdout:2/493: rename db/d11/d14/d27/d2f/d42/d5c/l50 to db/d11/d14/d27/d2f/d42/d5c/la8 0 2024-04-26T04:42:22.435 INFO:tasks.workunit.client.0.smithi090.stdout:3/399: getdents d1/d17 0 2024-04-26T04:42:22.435 INFO:tasks.workunit.client.0.smithi090.stdout:3/400: truncate d1/d17/f1a 2095037 0 2024-04-26T04:42:22.436 INFO:tasks.workunit.client.1.smithi104.stdout:4/823: symlink d4/d7/d1d/d8e/l123 0 2024-04-26T04:42:22.437 INFO:tasks.workunit.client.1.smithi104.stdout:4/824: rmdir d4/d7/d1d/d1e/d24/d59/db8 39 2024-04-26T04:42:22.437 INFO:tasks.workunit.client.1.smithi104.stdout:4/825: creat d4/d7/d1d/d8e/f124 x:0 0 0 2024-04-26T04:42:22.437 INFO:tasks.workunit.client.0.smithi090.stdout:2/494: symlink db/d11/d14/d1b/d20/d4c/la9 0 2024-04-26T04:42:22.438 INFO:tasks.workunit.client.0.smithi090.stdout:3/401: rename d1/l1c to d1/d17/d36/d2a/l88 0 2024-04-26T04:42:22.439 INFO:tasks.workunit.client.1.smithi104.stdout:4/826: creat d4/d7/d1d/d1e/d24/d4a/f125 x:0 0 0 2024-04-26T04:42:22.440 INFO:tasks.workunit.client.0.smithi090.stdout:2/495: creat db/d59/d66/d96/faa x:0 0 0 2024-04-26T04:42:22.440 INFO:tasks.workunit.client.0.smithi090.stdout:2/496: fsync db/dc/f19 0 2024-04-26T04:42:22.441 INFO:tasks.workunit.client.1.smithi104.stdout:4/827: mknod d4/db/d6b/c126 0 2024-04-26T04:42:22.441 INFO:tasks.workunit.client.0.smithi090.stdout:3/402: creat d1/d17/d36/d33/f89 x:0 0 0 2024-04-26T04:42:22.442 INFO:tasks.workunit.client.0.smithi090.stdout:2/497: rename db/d11/d14/d27/d28/f6e to db/d11/d14/d27/fab 0 2024-04-26T04:42:22.443 INFO:tasks.workunit.client.0.smithi090.stdout:3/403: mknod d1/d17/d36/d33/d68/c8a 0 2024-04-26T04:42:22.443 INFO:tasks.workunit.client.0.smithi090.stdout:3/404: getdents d1/d17/d36/d33/d68/d71/d75 0 2024-04-26T04:42:22.444 INFO:tasks.workunit.client.0.smithi090.stdout:2/498: unlink db/cf 0 2024-04-26T04:42:22.444 INFO:tasks.workunit.client.0.smithi090.stdout:3/405: rmdir d1/d6 39 2024-04-26T04:42:22.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/406: truncate d1/d9/fa 1725777 0 2024-04-26T04:42:22.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/407: read - d1/d17/d36/d33/f89 zero size 2024-04-26T04:42:22.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/408: write d1/d17/d36/f6a [747345,95926] 0 2024-04-26T04:42:22.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/409: chown d1/d17/d61 58044 1 2024-04-26T04:42:22.446 INFO:tasks.workunit.client.0.smithi090.stdout:3/410: chown d1/d9/c72 576465 1 2024-04-26T04:42:22.447 INFO:tasks.workunit.client.0.smithi090.stdout:3/411: mknod d1/d17/d52/c8b 0 2024-04-26T04:42:22.447 INFO:tasks.workunit.client.0.smithi090.stdout:3/412: chown d1/d6 127 1 2024-04-26T04:42:22.456 INFO:tasks.workunit.client.1.smithi104.stdout:4/828: dread d4/d7/d1d/d2e/f7e [0,4194304] 0 2024-04-26T04:42:22.456 INFO:tasks.workunit.client.1.smithi104.stdout:4/829: readlink d4/d7/d1d/d8e/l123 0 2024-04-26T04:42:22.456 INFO:tasks.workunit.client.1.smithi104.stdout:4/830: chown d4/d7/d1d/d1e/d24/d59/db8/fe6 386408 1 2024-04-26T04:42:22.456 INFO:tasks.workunit.client.1.smithi104.stdout:4/831: truncate d4/d20/d55/d56/da3/dbf/fc5 528833 0 2024-04-26T04:42:22.457 INFO:tasks.workunit.client.0.smithi090.stdout:2/499: dread db/d11/d14/d27/d28/f3d [0,4194304] 0 2024-04-26T04:42:22.461 INFO:tasks.workunit.client.0.smithi090.stdout:5/505: dwrite d4/d1a/d3f/f65 [0,4194304] 0 2024-04-26T04:42:22.464 INFO:tasks.workunit.client.0.smithi090.stdout:5/506: creat d4/d1a/d2a/d79/d90/d56/d71/fa1 x:0 0 0 2024-04-26T04:42:22.465 INFO:tasks.workunit.client.0.smithi090.stdout:5/507: symlink d4/d1a/df/d36/d61/la2 0 2024-04-26T04:42:22.472 INFO:tasks.workunit.client.0.smithi090.stdout:2/500: write db/d11/d14/d27/fab [375187,88486] 0 2024-04-26T04:42:22.474 INFO:tasks.workunit.client.0.smithi090.stdout:2/501: creat db/d11/d14/d63/fac x:0 0 0 2024-04-26T04:42:22.474 INFO:tasks.workunit.client.0.smithi090.stdout:2/502: chown db/d11/d7f 39888568 1 2024-04-26T04:42:22.475 INFO:tasks.workunit.client.0.smithi090.stdout:2/503: write db/d11/d14/d27/d2f/d42/d5c/f4d [184006,51503] 0 2024-04-26T04:42:22.476 INFO:tasks.workunit.client.0.smithi090.stdout:2/504: mknod db/d59/d66/cad 0 2024-04-26T04:42:22.476 INFO:tasks.workunit.client.0.smithi090.stdout:2/505: chown db/d11 811 1 2024-04-26T04:42:22.477 INFO:tasks.workunit.client.0.smithi090.stdout:4/457: dwrite f0 [4194304,4194304] 0 2024-04-26T04:42:22.477 INFO:tasks.workunit.client.0.smithi090.stdout:4/458: chown d3/db/d12/d3b/c7b 14862 1 2024-04-26T04:42:22.478 INFO:tasks.workunit.client.1.smithi104.stdout:7/729: dwrite d2/d4/d32/d3d/d41/d5f/d6d/faa [0,4194304] 0 2024-04-26T04:42:22.478 INFO:tasks.workunit.client.1.smithi104.stdout:5/782: dwrite d0/d1/d1e/f56 [0,4194304] 0 2024-04-26T04:42:22.478 INFO:tasks.workunit.client.1.smithi104.stdout:5/783: chown d0/d1/d4/d2a/cc3 26337037 1 2024-04-26T04:42:22.479 INFO:tasks.workunit.client.1.smithi104.stdout:1/815: dwrite d0/d6/d19/d21/d66/d20/d2a/f2c [0,4194304] 0 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:7/514: dwrite d0/d1/d4/f3b [4194304,4194304] 0 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/406: dwrite d0/d4/d7/f2c [0,4194304] 0 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/407: chown d0/l5 2141 1 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/408: chown d0/d4/d7/d17/d22/d31/f34 778367001 1 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/409: readlink d0/d4/d7/d4a/d75/d80/l59 0 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/410: creat d0/d55/f83 x:0 0 0 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:1/411: readlink d0/d4/d7/d17/d22/l70 0 2024-04-26T04:42:22.480 INFO:tasks.workunit.client.0.smithi090.stdout:5/508: dread d4/d1a/df/d4a/f51 [0,4194304] 0 2024-04-26T04:42:22.481 INFO:tasks.workunit.client.0.smithi090.stdout:9/375: dwrite db/dd/d18/d57/f68 [0,4194304] 0 2024-04-26T04:42:22.482 INFO:tasks.workunit.client.0.smithi090.stdout:3/413: dwrite d1/d17/d36/d2a/d3a/f4b [0,4194304] 0 2024-04-26T04:42:22.483 INFO:tasks.workunit.client.0.smithi090.stdout:4/459: mkdir d3/db/d12/d20/d2d/d93 0 2024-04-26T04:42:22.485 INFO:tasks.workunit.client.1.smithi104.stdout:4/832: dwrite d4/d7/d1d/d1e/d24/d4a/d9e/fcd [0,4194304] 0 2024-04-26T04:42:22.486 INFO:tasks.workunit.client.0.smithi090.stdout:7/515: symlink d0/d1/d4/d98/d4a/lbf 0 2024-04-26T04:42:22.487 INFO:tasks.workunit.client.1.smithi104.stdout:5/784: mkdir d0/d41/d4f/df2/d108 0 2024-04-26T04:42:22.493 INFO:tasks.workunit.client.0.smithi090.stdout:5/509: creat d4/d1a/d2a/d79/fa3 x:0 0 0 2024-04-26T04:42:22.493 INFO:tasks.workunit.client.0.smithi090.stdout:5/510: chown d4/d1a/df/d4a/f89 281 1 2024-04-26T04:42:22.493 INFO:tasks.workunit.client.0.smithi090.stdout:4/460: dread d3/db/d12/d20/d2d/d2e/f3a [0,4194304] 0 2024-04-26T04:42:22.493 INFO:tasks.workunit.client.0.smithi090.stdout:4/461: write d3/db/d12/d20/d45/d53/f6e [5181449,84986] 0 2024-04-26T04:42:22.493 INFO:tasks.workunit.client.0.smithi090.stdout:2/506: dread db/dc/f19 [0,4194304] 0 2024-04-26T04:42:22.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.493+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7b20000bf0 con 0x7f7b2806c680 2024-04-26T04:42:22.498 INFO:tasks.workunit.client.0.smithi090.stdout:7/516: mkdir d0/d1/d4/d98/d65/dbb/dc0 0 2024-04-26T04:42:22.498 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.495+0000 7f7b327fc700 1 -- 172.21.15.90:0/3985928980 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f7b20000bf0 con 0x7f7b2806c680 2024-04-26T04:42:22.499 INFO:tasks.workunit.client.0.smithi090.stdout:5/511: creat d4/d1a/df/d4a/fa4 x:0 0 0 2024-04-26T04:42:22.499 INFO:tasks.workunit.client.1.smithi104.stdout:1/816: read d0/d6/d19/f3a [3798478,32423] 0 2024-04-26T04:42:22.499 INFO:tasks.workunit.client.1.smithi104.stdout:1/817: readlink d0/d6/d19/d1b/l53 0 2024-04-26T04:42:22.499 INFO:tasks.workunit.client.1.smithi104.stdout:1/818: creat d0/d6/d19/d1b/da7/db8/f10a x:0 0 0 2024-04-26T04:42:22.499 INFO:tasks.workunit.client.1.smithi104.stdout:1/819: write d0/d6/d2e/d32/d4c/db1/dc3/f108 [1047700,85771] 0 2024-04-26T04:42:22.499 INFO:tasks.workunit.client.1.smithi104.stdout:5/785: dread d0/d1/d4/d2a/d2b/fb7 [0,4194304] 0 2024-04-26T04:42:22.499 INFO:tasks.workunit.client.1.smithi104.stdout:5/786: fdatasync d0/d1/d64/f92 0 2024-04-26T04:42:22.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b2806c680 msgr2=0x7f7b2806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:22.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b2806c680 0x7f7b2806eb30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7b38006010 tx=0x7f7b3800b540 comp rx=0 tx=0).stop 2024-04-26T04:42:22.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 msgr2=0x7f7b3c19c6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:22.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 0x7f7b3c19c6f0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b4b0 tx=0x7f7b2c00b7c0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 shutdown_connections 2024-04-26T04:42:22.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f7b2806c680 0x7f7b2806eb30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7b3c106840 0x7f7b3c19c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 --2- 172.21.15.90:0/3985928980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7b3c107a40 0x7f7b3c19c6f0 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7b2c00b4b0 tx=0x7f7b2c00b7c0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 >> 172.21.15.90:0/3985928980 conn(0x7f7b3c075970 msgr2=0x7f7b3c10ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:22.501 INFO:tasks.workunit.client.0.smithi090.stdout:4/462: link d3/d13/d6c/f90 d3/d13/d6c/d7e/f94 0 2024-04-26T04:42:22.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 shutdown_connections 2024-04-26T04:42:22.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.499+0000 7f7b436ac700 1 -- 172.21.15.90:0/3985928980 wait complete. 2024-04-26T04:42:22.502 INFO:tasks.workunit.client.0.smithi090.stdout:1/412: dread d0/d4/d7/d17/d22/d31/f34 [0,4194304] 0 2024-04-26T04:42:22.502 INFO:tasks.workunit.client.0.smithi090.stdout:2/507: truncate db/d11/d14/d25/f9a 35248 0 2024-04-26T04:42:22.502 INFO:tasks.workunit.client.1.smithi104.stdout:1/820: dread d0/d6/d19/d21/d66/de4/d7e/dc6/fce [0,4194304] 0 2024-04-26T04:42:22.503 INFO:tasks.workunit.client.1.smithi104.stdout:4/833: rmdir d4/d20/d55/d56/da3/dea/dfd 39 2024-04-26T04:42:22.503 INFO:tasks.workunit.client.0.smithi090.stdout:3/414: dread d1/f27 [0,4194304] 0 2024-04-26T04:42:22.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/517: write d0/db/f19 [2417951,21088] 0 2024-04-26T04:42:22.504 INFO:tasks.workunit.client.0.smithi090.stdout:0/549: sync 2024-04-26T04:42:22.504 INFO:tasks.workunit.client.1.smithi104.stdout:7/730: dwrite d2/d4/d32/d60/f8c [0,4194304] 0 2024-04-26T04:42:22.504 INFO:tasks.workunit.client.0.smithi090.stdout:7/518: truncate d0/db/df/d4d/d64/d66/f7c 139778 0 2024-04-26T04:42:22.505 INFO:tasks.workunit.client.0.smithi090.stdout:7/519: write d0/db/df/d4d/d64/d66/faa [648490,41644] 0 2024-04-26T04:42:22.505 INFO:tasks.workunit.client.0.smithi090.stdout:0/550: getdents d0/d4b/d5d/d5c/d8a/d45/d53/dab 0 2024-04-26T04:42:22.508 INFO:tasks.workunit.client.1.smithi104.stdout:5/787: unlink d0/d41/f4d 0 2024-04-26T04:42:22.508 INFO:tasks.workunit.client.0.smithi090.stdout:7/520: link d0/d1/d4/d3e/f90 d0/db/df/d4d/fc1 0 2024-04-26T04:42:22.510 INFO:tasks.workunit.client.0.smithi090.stdout:2/508: dread db/d11/d14/d25/d41/f93 [0,4194304] 0 2024-04-26T04:42:22.510 INFO:tasks.workunit.client.0.smithi090.stdout:2/509: write db/d11/d14/d27/d2f/f32 [4582666,10524] 0 2024-04-26T04:42:22.511 INFO:tasks.workunit.client.0.smithi090.stdout:0/551: truncate d0/df/f3c 3442927 0 2024-04-26T04:42:22.512 INFO:tasks.workunit.client.0.smithi090.stdout:0/552: fdatasync d0/d4b/d5d/d5c/d8a/d45/f96 0 2024-04-26T04:42:22.513 INFO:tasks.workunit.client.0.smithi090.stdout:2/510: unlink db/d59/d66/d77/f79 0 2024-04-26T04:42:22.513 INFO:tasks.workunit.client.0.smithi090.stdout:2/511: fdatasync db/d11/d14/d1b/f6d 0 2024-04-26T04:42:22.514 INFO:tasks.workunit.client.0.smithi090.stdout:1/413: write d0/d4/f48 [1231338,11437] 0 2024-04-26T04:42:22.514 INFO:tasks.workunit.client.1.smithi104.stdout:1/821: dread d0/f14 [8388608,4194304] 0 2024-04-26T04:42:22.515 INFO:tasks.workunit.client.1.smithi104.stdout:7/731: truncate d2/d4/d32/fcc 4021342 0 2024-04-26T04:42:22.518 INFO:tasks.workunit.client.1.smithi104.stdout:7/732: mknod d2/d4/d32/d3d/d41/ced 0 2024-04-26T04:42:22.518 INFO:tasks.workunit.client.1.smithi104.stdout:7/733: chown d2/d4/d1d/d34/c72 2 1 2024-04-26T04:42:22.519 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:42:22.519 INFO:tasks.workunit.client.0.smithi090.stdout:2/512: creat db/d11/d14/d1b/d20/d4c/fae x:0 0 0 2024-04-26T04:42:22.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/376: dwrite db/dd/d44/f2c [4194304,4194304] 0 2024-04-26T04:42:22.520 INFO:tasks.workunit.client.0.smithi090.stdout:0/553: link d0/de/f78 d0/d4b/d6d/fc6 0 2024-04-26T04:42:22.522 INFO:tasks.workunit.client.1.smithi104.stdout:5/788: dread d0/d1/d64/d26/d36/da7/fa8 [0,4194304] 0 2024-04-26T04:42:22.522 INFO:tasks.workunit.client.0.smithi090.stdout:1/414: mknod d0/d4/d7/d49/c84 0 2024-04-26T04:42:22.522 INFO:tasks.workunit.client.1.smithi104.stdout:5/789: chown d0/d1/de6 37340697 1 2024-04-26T04:42:22.523 INFO:tasks.workunit.client.1.smithi104.stdout:4/834: rmdir d4/d7/d1d/d1e/d24/d4a 39 2024-04-26T04:42:22.524 INFO:tasks.workunit.client.0.smithi090.stdout:4/463: read d3/db/d12/d3b/f78 [892261,9410] 0 2024-04-26T04:42:22.527 INFO:tasks.workunit.client.0.smithi090.stdout:2/513: creat db/d11/d14/d25/d65/d95/faf x:0 0 0 2024-04-26T04:42:22.530 INFO:tasks.workunit.client.0.smithi090.stdout:3/415: write d1/d17/d36/f32 [3274008,92367] 0 2024-04-26T04:42:22.530 INFO:tasks.workunit.client.0.smithi090.stdout:9/377: unlink db/c50 0 2024-04-26T04:42:22.533 INFO:tasks.workunit.client.0.smithi090.stdout:0/554: rename d0/d4b/d5d/l72 to d0/d4b/d5d/d5c/lc7 0 2024-04-26T04:42:22.533 INFO:tasks.workunit.client.0.smithi090.stdout:0/555: truncate d0/d4b/fbd 156677 0 2024-04-26T04:42:22.535 INFO:tasks.workunit.client.1.smithi104.stdout:5/790: rename d0/d1/d64/d26/d36/d72 to d0/d1/d4/d90/d109 0 2024-04-26T04:42:22.535 INFO:tasks.workunit.client.1.smithi104.stdout:5/791: dread - d0/d1/d64/d26/d36/da7/fe2 zero size 2024-04-26T04:42:22.536 INFO:tasks.workunit.client.0.smithi090.stdout:1/415: mkdir d0/d4/d7/d4b/d53/d7b/d85 0 2024-04-26T04:42:22.536 INFO:tasks.workunit.client.0.smithi090.stdout:1/416: fsync d0/d4/d7/d4b/f4c 0 2024-04-26T04:42:22.536 INFO:tasks.workunit.client.0.smithi090.stdout:1/417: truncate d0/d4/d7/d4a/f56 821453 0 2024-04-26T04:42:22.536 INFO:tasks.workunit.client.0.smithi090.stdout:1/418: chown d0/d4/d7/d4a/d75 4 1 2024-04-26T04:42:22.536 INFO:tasks.workunit.client.0.smithi090.stdout:7/521: dread d0/db/d53/f56 [0,4194304] 0 2024-04-26T04:42:22.537 INFO:tasks.workunit.client.1.smithi104.stdout:8/682: sync 2024-04-26T04:42:22.540 INFO:tasks.workunit.client.0.smithi090.stdout:9/378: unlink la 0 2024-04-26T04:42:22.540 INFO:tasks.workunit.client.0.smithi090.stdout:9/379: chown db/d16/f33 3620 1 2024-04-26T04:42:22.543 INFO:tasks.workunit.client.0.smithi090.stdout:2/514: dread db/d11/d14/d1b/d20/f24 [0,4194304] 0 2024-04-26T04:42:22.546 INFO:tasks.workunit.client.0.smithi090.stdout:0/556: rename d0/d4b/d5d/d48/d7d/f8f to d0/d4b/d5d/d5c/d8a/d45/d53/dab/fc8 0 2024-04-26T04:42:22.547 INFO:tasks.workunit.client.0.smithi090.stdout:0/557: dread d0/d4b/d5d/d5c/d8a/d45/d53/fba [0,4194304] 0 2024-04-26T04:42:22.547 INFO:tasks.workunit.client.0.smithi090.stdout:0/558: creat d0/d4b/d5d/d5c/d8a/d7c/fc9 x:0 0 0 2024-04-26T04:42:22.547 INFO:tasks.workunit.client.0.smithi090.stdout:0/559: chown d0/d4b/d5d/d5c/d8a/d7c 1232 1 2024-04-26T04:42:22.549 INFO:tasks.workunit.client.0.smithi090.stdout:3/416: rename d1/d9/c72 to d1/d17/d36/d33/d42/c8c 0 2024-04-26T04:42:22.549 INFO:tasks.workunit.client.0.smithi090.stdout:3/417: readlink d1/d6/d1b/l34 0 2024-04-26T04:42:22.549 INFO:tasks.workunit.client.1.smithi104.stdout:8/683: symlink d0/d50/dc4/ld3 0 2024-04-26T04:42:22.553 INFO:tasks.workunit.client.0.smithi090.stdout:7/522: rename d0/db/df/l2c to d0/db/d53/d5e/lc2 0 2024-04-26T04:42:22.553 INFO:tasks.workunit.client.1.smithi104.stdout:8/684: mkdir d0/d1c/d27/d47/d67/d76/daf/dd4 0 2024-04-26T04:42:22.557 INFO:tasks.workunit.client.0.smithi090.stdout:3/418: symlink d1/d17/d36/d2a/d3a/l8d 0 2024-04-26T04:42:22.557 INFO:tasks.workunit.client.0.smithi090.stdout:3/419: fdatasync d1/d59/d5e/f62 0 2024-04-26T04:42:22.559 INFO:tasks.workunit.client.0.smithi090.stdout:3/420: creat d1/d17/d36/d33/d68/d76/f8e x:0 0 0 2024-04-26T04:42:22.563 INFO:tasks.workunit.client.1.smithi104.stdout:1/822: dwrite d0/d6/d19/d21/d59/f94 [0,4194304] 0 2024-04-26T04:42:22.566 INFO:tasks.workunit.client.1.smithi104.stdout:1/823: creat d0/d6/d19/d21/d66/d20/d2a/f10b x:0 0 0 2024-04-26T04:42:22.567 INFO:tasks.workunit.client.1.smithi104.stdout:4/835: dwrite d4/d7/d1d/d1e/f5b [0,4194304] 0 2024-04-26T04:42:22.573 INFO:tasks.workunit.client.1.smithi104.stdout:4/836: truncate d4/d7/d1d/d1e/d24/d4a/fac 178553 0 2024-04-26T04:42:22.574 INFO:tasks.workunit.client.1.smithi104.stdout:1/824: rename d0/d6/d19/d1b to d0/d6/d9e/d10c 0 2024-04-26T04:42:22.576 INFO:tasks.workunit.client.1.smithi104.stdout:1/825: rmdir d0/d6/d19/d21/d66/d20/d2a/d56/d8c 39 2024-04-26T04:42:22.582 INFO:tasks.workunit.client.0.smithi090.stdout:1/419: dwrite d0/d4/d7/d4b/d53/f6a [0,4194304] 0 2024-04-26T04:42:22.584 INFO:tasks.workunit.client.1.smithi104.stdout:4/837: dread d4/db/f10e [0,4194304] 0 2024-04-26T04:42:22.585 INFO:tasks.workunit.client.1.smithi104.stdout:4/838: getdents d4/d7/d1d/d2e/d74 0 2024-04-26T04:42:22.585 INFO:tasks.workunit.client.0.smithi090.stdout:1/420: creat d0/f86 x:0 0 0 2024-04-26T04:42:22.588 INFO:tasks.workunit.client.1.smithi104.stdout:4/839: dread d4/d20/d55/d56/f31 [0,4194304] 0 2024-04-26T04:42:22.588 INFO:tasks.workunit.client.1.smithi104.stdout:8/685: dwrite d0/d1c/fd0 [0,4194304] 0 2024-04-26T04:42:22.592 INFO:tasks.workunit.client.0.smithi090.stdout:1/421: dread d0/d4/d7/d17/d22/f5c [0,4194304] 0 2024-04-26T04:42:22.593 INFO:tasks.workunit.client.1.smithi104.stdout:1/826: write d0/d6/d9e/d10c/da5/fa6 [1883016,111971] 0 2024-04-26T04:42:22.593 INFO:tasks.workunit.client.0.smithi090.stdout:1/422: dread d0/d4/d7/d17/f3d [0,4194304] 0 2024-04-26T04:42:22.594 INFO:tasks.workunit.client.0.smithi090.stdout:1/423: creat d0/d4/f87 x:0 0 0 2024-04-26T04:42:22.594 INFO:tasks.workunit.client.0.smithi090.stdout:1/424: read d0/f2e [358205,105586] 0 2024-04-26T04:42:22.595 INFO:tasks.workunit.client.0.smithi090.stdout:1/425: readlink d0/d4/l41 0 2024-04-26T04:42:22.596 INFO:tasks.workunit.client.1.smithi104.stdout:4/840: rename d4/d7/d1d/d1e/d8f/db6/f9c to d4/d7/d1d/d1e/d15/dd8/f127 0 2024-04-26T04:42:22.596 INFO:tasks.workunit.client.1.smithi104.stdout:4/841: read d4/db/d6b/d86/f101 [460464,101765] 0 2024-04-26T04:42:22.597 INFO:tasks.workunit.client.0.smithi090.stdout:1/426: fsync d0/f86 0 2024-04-26T04:42:22.598 INFO:tasks.workunit.client.0.smithi090.stdout:1/427: stat d0/d4/cd 0 2024-04-26T04:42:22.599 INFO:tasks.workunit.client.1.smithi104.stdout:8/686: mkdir d0/d1c/d27/d47/dd5 0 2024-04-26T04:42:22.603 INFO:tasks.workunit.client.1.smithi104.stdout:4/842: dread d4/d7/f53 [0,4194304] 0 2024-04-26T04:42:22.603 INFO:tasks.workunit.client.1.smithi104.stdout:4/843: chown d4/d20/d55/d56/la0 648 1 2024-04-26T04:42:22.604 INFO:tasks.workunit.client.1.smithi104.stdout:1/827: unlink d0/l79 0 2024-04-26T04:42:22.605 INFO:tasks.workunit.client.1.smithi104.stdout:8/687: truncate d0/d1c/d27/d47/d67/d76/daf/fb8 6044328 0 2024-04-26T04:42:22.606 INFO:tasks.workunit.client.1.smithi104.stdout:1/828: dread d0/d6/d9e/d10c/fd7 [0,4194304] 0 2024-04-26T04:42:22.606 INFO:tasks.workunit.client.1.smithi104.stdout:1/829: creat d0/d6/d19/d21/d66/d20/f10d x:0 0 0 2024-04-26T04:42:22.607 INFO:tasks.workunit.client.1.smithi104.stdout:4/844: creat d4/d7/d1d/d1e/d24/d59/dc1/f128 x:0 0 0 2024-04-26T04:42:22.607 INFO:tasks.workunit.client.1.smithi104.stdout:4/845: read d4/db/f4b [3407640,14540] 0 2024-04-26T04:42:22.609 INFO:tasks.workunit.client.1.smithi104.stdout:1/830: rmdir d0/d6/d9e/d10c/da5 39 2024-04-26T04:42:22.611 INFO:tasks.workunit.client.1.smithi104.stdout:1/831: getdents d0/d6/d2e/d32/d4d 0 2024-04-26T04:42:22.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/832: getdents d0/d6/d19/d21/d66/d20/d2a/dbc/df8/df3 0 2024-04-26T04:42:22.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/833: chown d0/d6/ca 0 1 2024-04-26T04:42:22.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/834: write d0/d6/d19/d21/d66/de4/fcd [693492,93257] 0 2024-04-26T04:42:22.613 INFO:tasks.workunit.client.0.smithi090.stdout:1/428: dwrite d0/d4/d7/d17/d22/f52 [0,4194304] 0 2024-04-26T04:42:22.613 INFO:tasks.workunit.client.0.smithi090.stdout:1/429: write d0/d4/d7/d69/f71 [751330,50632] 0 2024-04-26T04:42:22.614 INFO:tasks.workunit.client.1.smithi104.stdout:1/835: unlink d0/d6/d49/f68 0 2024-04-26T04:42:22.622 INFO:tasks.workunit.client.0.smithi090.stdout:1/430: dread d0/d4/f9 [0,4194304] 0 2024-04-26T04:42:22.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/431: write d0/d4/d7/d17/d22/d31/f39 [3927276,99336] 0 2024-04-26T04:42:22.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/432: stat d0/d55 0 2024-04-26T04:42:22.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/433: dread - d0/d4/d7/d4b/f5b zero size 2024-04-26T04:42:22.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/434: unlink d0/d4/d7/d17/d22/d31/c44 0 2024-04-26T04:42:22.633 INFO:tasks.workunit.client.0.smithi090.stdout:1/435: symlink d0/d4/d7/d49/l88 0 2024-04-26T04:42:22.643 INFO:tasks.workunit.client.0.smithi090.stdout:1/436: dread d0/d4/f20 [0,4194304] 0 2024-04-26T04:42:22.644 INFO:tasks.workunit.client.0.smithi090.stdout:1/437: unlink d0/f2e 0 2024-04-26T04:42:22.644 INFO:tasks.workunit.client.0.smithi090.stdout:1/438: truncate d0/d4/d7/d17/d22/f73 4353148 0 2024-04-26T04:42:22.645 INFO:tasks.workunit.client.0.smithi090.stdout:1/439: truncate d0/d4/d7/d17/d22/f74 501178 0 2024-04-26T04:42:22.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/464: dwrite f0 [8388608,4194304] 0 2024-04-26T04:42:22.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/465: truncate d3/db/d12/d20/d2d/d58/d6a/f71 337618 0 2024-04-26T04:42:22.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/466: write d3/db/d12/d3b/f75 [794621,94735] 0 2024-04-26T04:42:22.655 INFO:tasks.workunit.client.1.smithi104.stdout:5/792: dwrite d0/d1/d64/d26/fb8 [0,4194304] 0 2024-04-26T04:42:22.655 INFO:tasks.workunit.client.1.smithi104.stdout:5/793: write d0/d1/d4/d90/d109/fbe [141437,65202] 0 2024-04-26T04:42:22.657 INFO:tasks.workunit.client.1.smithi104.stdout:5/794: truncate d0/d1/d4/d2a/d6b/d73/d95/fba 941050 0 2024-04-26T04:42:22.658 INFO:tasks.workunit.client.1.smithi104.stdout:5/795: creat d0/d1/d4/d2a/d6b/d91/dbd/f10a x:0 0 0 2024-04-26T04:42:22.660 INFO:tasks.workunit.client.1.smithi104.stdout:5/796: creat d0/d89/db2/f10b x:0 0 0 2024-04-26T04:42:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 -- 172.21.15.90:0/3496697359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 msgr2=0x7f1618100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:22.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 --2- 172.21.15.90:0/3496697359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f1618100840 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f1608009910 tx=0x7f1608009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 -- 172.21.15.90:0/3496697359 shutdown_connections 2024-04-26T04:42:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 --2- 172.21.15.90:0/3496697359 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f1618100840 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 --2- 172.21.15.90:0/3496697359 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16180ff130 0x7f16180ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 -- 172.21.15.90:0/3496697359 >> 172.21.15.90:0/3496697359 conn(0x7f16180fa780 msgr2=0x7f16180fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 -- 172.21.15.90:0/3496697359 shutdown_connections 2024-04-26T04:42:22.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.662+0000 7f161f8f4700 1 -- 172.21.15.90:0/3496697359 wait complete. 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161f8f4700 1 Processor -- start 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161f8f4700 1 -- start start 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161f8f4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16180ff130 0x7f1618103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161f8f4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f16181017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161f8f4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618101cf0 con 0x7f16181003d0 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161f8f4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618101e30 con 0x7f16180ff130 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161ce8f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f16181017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:22.666 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161ce8f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f16181017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52716/0 (socket says 172.21.15.90:52716) 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.663+0000 7f161ce8f700 1 -- 172.21.15.90:0/641884593 learned_addr learned my addr 172.21.15.90:0/641884593 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161d690700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16180ff130 0x7f1618103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161ce8f700 1 -- 172.21.15.90:0/641884593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16180ff130 msgr2=0x7f1618103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161ce8f700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16180ff130 0x7f1618103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161ce8f700 1 -- 172.21.15.90:0/641884593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16080095f0 con 0x7f16181003d0 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161d690700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16180ff130 0x7f1618103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161ce8f700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f16181017b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f160800bd70 tx=0x7f160800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f160e7fc700 1 -- 172.21.15.90:0/641884593 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160801d070 con 0x7f16181003d0 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f160e7fc700 1 -- 172.21.15.90:0/641884593 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1608022470 con 0x7f16181003d0 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16181020b0 con 0x7f16181003d0 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16181025a0 con 0x7f16181003d0 2024-04-26T04:42:22.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.664+0000 7f160e7fc700 1 -- 172.21.15.90:0/641884593 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f160800f670 con 0x7f16181003d0 2024-04-26T04:42:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.665+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1618107c90 con 0x7f16181003d0 2024-04-26T04:42:22.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.666+0000 7f160e7fc700 1 -- 172.21.15.90:0/641884593 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1608004b70 con 0x7f16181003d0 2024-04-26T04:42:22.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.667+0000 7f160e7fc700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f160406c680 0x7f160406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:22.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.667+0000 7f161d690700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f160406c680 0x7f160406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:22.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.667+0000 7f160e7fc700 1 -- 172.21.15.90:0/641884593 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f160808cee0 con 0x7f16181003d0 2024-04-26T04:42:22.669 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.667+0000 7f161d690700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f160406c680 0x7f160406eb30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1614007900 tx=0x7f1614008040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:22.670 INFO:tasks.workunit.client.0.smithi090.stdout:3/421: dread d1/d17/f2f [0,4194304] 0 2024-04-26T04:42:22.672 INFO:tasks.workunit.client.0.smithi090.stdout:3/422: mknod d1/d6/d2c/c8f 0 2024-04-26T04:42:22.673 INFO:tasks.workunit.client.0.smithi090.stdout:3/423: mknod d1/d17/d36/d2a/c90 0 2024-04-26T04:42:22.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:22.672+0000 7f160e7fc700 1 -- 172.21.15.90:0/641884593 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f160805b120 con 0x7f16181003d0 2024-04-26T04:42:22.683 INFO:tasks.workunit.client.0.smithi090.stdout:2/515: dwrite db/dc/f8e [0,4194304] 0 2024-04-26T04:42:22.688 INFO:tasks.workunit.client.0.smithi090.stdout:2/516: rename db/d59/d66/d96 to db/d11/d14/d27/db0 0 2024-04-26T04:42:22.688 INFO:tasks.workunit.client.0.smithi090.stdout:2/517: readlink db/d11/d14/d25/d65/d95/l99 0 2024-04-26T04:42:22.689 INFO:tasks.workunit.client.0.smithi090.stdout:2/518: symlink db/d11/d14/d27/d2f/d70/d71/lb1 0 2024-04-26T04:42:22.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/519: rename db/d11/d14/d1b/l74 to db/d11/d14/d1b/lb2 0 2024-04-26T04:42:22.692 INFO:tasks.workunit.client.0.smithi090.stdout:2/520: truncate db/d59/f78 2593104 0 2024-04-26T04:42:22.692 INFO:tasks.workunit.client.0.smithi090.stdout:2/521: creat db/d11/d14/d27/d2f/d42/d5c/fb3 x:0 0 0 2024-04-26T04:42:22.694 INFO:tasks.workunit.client.0.smithi090.stdout:2/522: mkdir db/d11/d14/d1b/d20/d4c/d98/db4 0 2024-04-26T04:42:22.695 INFO:tasks.workunit.client.1.smithi104.stdout:1/836: dwrite d0/d6/d2e/d32/f62 [0,4194304] 0 2024-04-26T04:42:22.696 INFO:tasks.workunit.client.1.smithi104.stdout:1/837: creat d0/d6/d9e/d10c/da7/db8/f10e x:0 0 0 2024-04-26T04:42:22.697 INFO:tasks.workunit.client.1.smithi104.stdout:1/838: mkdir d0/d6/d9e/d10c/d10f 0 2024-04-26T04:42:22.697 INFO:tasks.workunit.client.1.smithi104.stdout:1/839: getdents d0/d6/d19/d21/d66/de4/d81/d9d 0 2024-04-26T04:42:22.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/840: mkdir d0/d6/d19/d21/d66/de4/d81/d110 0 2024-04-26T04:42:22.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/841: chown d0/d6/d19/d21/d66/d29/d3f/fe7 1 1 2024-04-26T04:42:22.701 INFO:tasks.workunit.client.0.smithi090.stdout:9/380: dwrite db/dd/d18/d38/d39/f7e [0,4194304] 0 2024-04-26T04:42:22.701 INFO:tasks.workunit.client.0.smithi090.stdout:7/523: dwrite d0/db/d53/db7/f3a [0,4194304] 0 2024-04-26T04:42:22.702 INFO:tasks.workunit.client.1.smithi104.stdout:8/688: dwrite d0/d33/d35/f3e [0,4194304] 0 2024-04-26T04:42:22.702 INFO:tasks.workunit.client.0.smithi090.stdout:1/440: dwrite d0/d4/d7/d49/f5e [0,4194304] 0 2024-04-26T04:42:22.703 INFO:tasks.workunit.client.0.smithi090.stdout:1/441: chown d0/d4/c1e 4320559 1 2024-04-26T04:42:22.703 INFO:tasks.workunit.client.0.smithi090.stdout:1/442: fdatasync d0/d4/d7/d4a/f77 0 2024-04-26T04:42:22.704 INFO:tasks.workunit.client.1.smithi104.stdout:5/797: dwrite d0/d1/d1e/f8f [4194304,4194304] 0 2024-04-26T04:42:22.704 INFO:tasks.workunit.client.0.smithi090.stdout:9/381: mkdir db/dd/d44/d82/d85 0 2024-04-26T04:42:22.706 INFO:tasks.workunit.client.0.smithi090.stdout:4/467: dwrite d3/f36 [0,4194304] 0 2024-04-26T04:42:22.707 INFO:tasks.workunit.client.0.smithi090.stdout:3/424: dwrite d1/d17/d36/f65 [0,4194304] 0 2024-04-26T04:42:22.708 INFO:tasks.workunit.client.1.smithi104.stdout:8/689: dread d0/f4d [0,4194304] 0 2024-04-26T04:42:22.710 INFO:tasks.workunit.client.0.smithi090.stdout:7/524: dread d0/db/d53/fa [0,4194304] 0 2024-04-26T04:42:22.711 INFO:tasks.workunit.client.1.smithi104.stdout:1/842: dread d0/d6/d19/d21/d59/f73 [0,4194304] 0 2024-04-26T04:42:22.714 INFO:tasks.workunit.client.1.smithi104.stdout:1/843: creat d0/d6/d19/f111 x:0 0 0 2024-04-26T04:42:22.714 INFO:tasks.workunit.client.1.smithi104.stdout:9/669: sync 2024-04-26T04:42:22.714 INFO:tasks.workunit.client.1.smithi104.stdout:9/670: dread - d11/d53/f86 zero size 2024-04-26T04:42:22.715 INFO:tasks.workunit.client.0.smithi090.stdout:1/443: rmdir d0/d4/d7/d4b/d53/d6b 0 2024-04-26T04:42:22.717 INFO:tasks.workunit.client.1.smithi104.stdout:9/671: unlink d11/d15/d32/l3f 0 2024-04-26T04:42:22.719 INFO:tasks.workunit.client.1.smithi104.stdout:5/798: dread d0/d1/d1e/f8f [0,4194304] 0 2024-04-26T04:42:22.720 INFO:tasks.workunit.client.0.smithi090.stdout:3/425: creat d1/d6/f91 x:0 0 0 2024-04-26T04:42:22.721 INFO:tasks.workunit.client.1.smithi104.stdout:5/799: symlink d0/d1/d4/d2a/d6b/d91/dbd/l10c 0 2024-04-26T04:42:22.722 INFO:tasks.workunit.client.1.smithi104.stdout:5/800: truncate d0/d1/d64/d26/f44 388648 0 2024-04-26T04:42:22.722 INFO:tasks.workunit.client.0.smithi090.stdout:7/525: mknod d0/db/df/d1c/cc3 0 2024-04-26T04:42:22.724 INFO:tasks.workunit.client.1.smithi104.stdout:8/690: dread d0/d1c/d27/d46/f78 [0,4194304] 0 2024-04-26T04:42:22.724 INFO:tasks.workunit.client.1.smithi104.stdout:9/672: dread d11/f89 [0,4194304] 0 2024-04-26T04:42:22.726 INFO:tasks.workunit.client.0.smithi090.stdout:4/468: dwrite d3/f7d [0,4194304] 0 2024-04-26T04:42:22.729 INFO:tasks.workunit.client.0.smithi090.stdout:9/382: rmdir db/d16 39 2024-04-26T04:42:22.730 INFO:tasks.workunit.client.0.smithi090.stdout:9/383: fdatasync db/dd/d44/f34 0 2024-04-26T04:42:22.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/426: truncate d1/d17/d36/f67 2027871 0 2024-04-26T04:42:22.733 INFO:tasks.workunit.client.0.smithi090.stdout:3/427: write d1/d6/d2c/f5f [362355,6045] 0 2024-04-26T04:42:22.738 INFO:tasks.workunit.client.0.smithi090.stdout:4/469: mknod d3/d13/d1d/c95 0 2024-04-26T04:42:22.738 INFO:tasks.workunit.client.0.smithi090.stdout:4/470: fdatasync d3/d13/d6c/d7e/f94 0 2024-04-26T04:42:22.738 INFO:tasks.workunit.client.0.smithi090.stdout:9/384: rename db/dd/d18/d31/d6a/d70/l81 to db/dd/d18/d38/l86 0 2024-04-26T04:42:22.738 INFO:tasks.workunit.client.0.smithi090.stdout:9/385: chown db/d16/l6d 172350559 1 2024-04-26T04:42:22.738 INFO:tasks.workunit.client.0.smithi090.stdout:9/386: write db/dd/d18/d31/d58/f6c [548256,42062] 0 2024-04-26T04:42:22.738 INFO:tasks.workunit.client.0.smithi090.stdout:9/387: creat db/f87 x:0 0 0 2024-04-26T04:42:22.739 INFO:tasks.workunit.client.0.smithi090.stdout:4/471: link d3/d13/d6c/f90 d3/db/d12/d20/d45/f96 0 2024-04-26T04:42:22.739 INFO:tasks.workunit.client.1.smithi104.stdout:8/691: read d0/d1c/fd0 [3730702,68491] 0 2024-04-26T04:42:22.740 INFO:tasks.workunit.client.0.smithi090.stdout:4/472: link d3/db/f19 d3/d13/d1d/d50/d51/f97 0 2024-04-26T04:42:22.741 INFO:tasks.workunit.client.0.smithi090.stdout:4/473: readlink d3/db/d12/d20/l5d 0 2024-04-26T04:42:22.741 INFO:tasks.workunit.client.1.smithi104.stdout:1/844: dwrite d0/d6/d19/d21/d66/d29/d3f/f7b [0,4194304] 0 2024-04-26T04:42:22.743 INFO:tasks.workunit.client.0.smithi090.stdout:3/428: dread d1/d9/fa [0,4194304] 0 2024-04-26T04:42:22.743 INFO:tasks.workunit.client.0.smithi090.stdout:3/429: truncate d1/d6/f79 721642 0 2024-04-26T04:42:22.745 INFO:tasks.workunit.client.0.smithi090.stdout:3/430: getdents d1/d17/d52/d7d 0 2024-04-26T04:42:22.746 INFO:tasks.workunit.client.0.smithi090.stdout:2/523: write db/d59/f78 [283498,84558] 0 2024-04-26T04:42:22.746 INFO:tasks.workunit.client.0.smithi090.stdout:2/524: chown db/d11/d14/d1b/d20/d4c/d98/db4 876 1 2024-04-26T04:42:22.746 INFO:tasks.workunit.client.0.smithi090.stdout:2/525: truncate db/d11/f44 702455 0 2024-04-26T04:42:22.747 INFO:tasks.workunit.client.1.smithi104.stdout:8/692: rename d0/d33/d51/fc6 to d0/d1c/d59/fd6 0 2024-04-26T04:42:22.750 INFO:tasks.workunit.client.0.smithi090.stdout:2/526: dread db/dc/f19 [0,4194304] 0 2024-04-26T04:42:22.750 INFO:tasks.workunit.client.0.smithi090.stdout:2/527: write db/d11/d14/d63/fac [529203,81824] 0 2024-04-26T04:42:22.752 INFO:tasks.workunit.client.1.smithi104.stdout:8/693: dread d0/d1c/d27/d47/f1d [0,4194304] 0 2024-04-26T04:42:22.754 INFO:tasks.workunit.client.0.smithi090.stdout:9/388: dread db/d16/f64 [4194304,4194304] 0 2024-04-26T04:42:22.756 INFO:tasks.workunit.client.1.smithi104.stdout:8/694: mknod d0/d9a/cd7 0 2024-04-26T04:42:22.756 INFO:tasks.workunit.client.1.smithi104.stdout:8/695: chown d0/d33/d35/d6d/da4/cbe 5654 1 2024-04-26T04:42:22.757 INFO:tasks.workunit.client.0.smithi090.stdout:9/389: creat db/d16/f88 x:0 0 0 2024-04-26T04:42:22.757 INFO:tasks.workunit.client.0.smithi090.stdout:9/390: readlink db/dd/l22 0 2024-04-26T04:42:22.757 INFO:tasks.workunit.client.1.smithi104.stdout:8/696: truncate d0/d1c/d27/d46/f70 455486 0 2024-04-26T04:42:22.757 INFO:tasks.workunit.client.1.smithi104.stdout:8/697: creat d0/d1c/d27/d47/d67/d76/d7f/fd8 x:0 0 0 2024-04-26T04:42:22.757 INFO:tasks.workunit.client.1.smithi104.stdout:8/698: fsync d0/f2f 0 2024-04-26T04:42:22.758 INFO:tasks.workunit.client.0.smithi090.stdout:3/431: read d1/f27 [1141133,868] 0 2024-04-26T04:42:22.760 INFO:tasks.workunit.client.0.smithi090.stdout:3/432: unlink d1/d17/f3e 0 2024-04-26T04:42:22.761 INFO:tasks.workunit.client.1.smithi104.stdout:8/699: mknod d0/d33/da6/cd9 0 2024-04-26T04:42:22.762 INFO:tasks.workunit.client.1.smithi104.stdout:8/700: chown d0/d1c/f6e 39578524 1 2024-04-26T04:42:22.762 INFO:tasks.workunit.client.1.smithi104.stdout:8/701: mknod d0/d1c/d27/cda 0 2024-04-26T04:42:22.763 INFO:tasks.workunit.client.0.smithi090.stdout:3/433: getdents d1/d17/d36/d2a/d3a 0 2024-04-26T04:42:22.764 INFO:tasks.workunit.client.0.smithi090.stdout:3/434: truncate d1/d17/f2f 555829 0 2024-04-26T04:42:22.764 INFO:tasks.workunit.client.0.smithi090.stdout:3/435: write d1/d59/f6d [1139805,36881] 0 2024-04-26T04:42:22.764 INFO:tasks.workunit.client.0.smithi090.stdout:3/436: chown d1/d17/d36/d2a/d3a/d5d 40399883 1 2024-04-26T04:42:22.765 INFO:tasks.workunit.client.1.smithi104.stdout:8/702: dread d0/d1c/d27/d47/db1/fc9 [0,4194304] 0 2024-04-26T04:42:22.766 INFO:tasks.workunit.client.0.smithi090.stdout:7/526: dwrite d0/d1/d4/d3e/f90 [0,4194304] 0 2024-04-26T04:42:22.767 INFO:tasks.workunit.client.1.smithi104.stdout:1/845: dread d0/d6/d19/d21/d59/f94 [0,4194304] 0 2024-04-26T04:42:22.769 INFO:tasks.workunit.client.0.smithi090.stdout:9/391: dread db/dd/d44/f2c [4194304,4194304] 0 2024-04-26T04:42:22.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/392: mknod db/dd/d18/d38/d39/d3f/c89 0 2024-04-26T04:42:22.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/393: fdatasync db/dd/d18/d31/d58/f65 0 2024-04-26T04:42:22.783 INFO:tasks.workunit.client.0.smithi090.stdout:9/394: dread db/f41 [0,4194304] 0 2024-04-26T04:42:22.783 INFO:tasks.workunit.client.0.smithi090.stdout:9/395: fsync db/dd/d44/f5e 0 2024-04-26T04:42:22.783 INFO:tasks.workunit.client.0.smithi090.stdout:9/396: chown db 1834 1 2024-04-26T04:42:22.783 INFO:tasks.workunit.client.0.smithi090.stdout:1/444: dwrite d0/d4/d7/d4b/f4c [0,4194304] 0 2024-04-26T04:42:22.784 INFO:tasks.workunit.client.1.smithi104.stdout:1/846: dread d0/d6/d49/f4a [4194304,4194304] 0 2024-04-26T04:42:22.785 INFO:tasks.workunit.client.1.smithi104.stdout:5/801: dwrite d0/d1/f65 [0,4194304] 0 2024-04-26T04:42:22.785 INFO:tasks.workunit.client.0.smithi090.stdout:9/397: truncate db/dd/d44/f34 610870 0 2024-04-26T04:42:22.785 INFO:tasks.workunit.client.0.smithi090.stdout:9/398: chown db/dd/d18/d31/d58/f62 125123811 1 2024-04-26T04:42:22.788 INFO:tasks.workunit.client.1.smithi104.stdout:5/802: truncate d0/d1/d4/d2a/fd2 1233147 0 2024-04-26T04:42:22.790 INFO:tasks.workunit.client.0.smithi090.stdout:1/445: truncate d0/d4/f48 1061334 0 2024-04-26T04:42:22.791 INFO:tasks.workunit.client.1.smithi104.stdout:5/803: rename d0/d1/d4/d2a/f50 to d0/d1/d64/d26/d36/f10d 0 2024-04-26T04:42:22.791 INFO:tasks.workunit.client.1.smithi104.stdout:5/804: truncate d0/d1/d64/fde 355634 0 2024-04-26T04:42:22.792 INFO:tasks.workunit.client.0.smithi090.stdout:9/399: mknod db/dd/d44/c8a 0 2024-04-26T04:42:22.792 INFO:tasks.workunit.client.0.smithi090.stdout:9/400: write db/f87 [269432,77695] 0 2024-04-26T04:42:22.792 INFO:tasks.workunit.client.0.smithi090.stdout:9/401: chown db/dd/d18/d31/d58/c5b 1451432 1 2024-04-26T04:42:22.792 INFO:tasks.workunit.client.1.smithi104.stdout:5/805: symlink d0/d1/d4/d2a/d2b/d5c/l10e 0 2024-04-26T04:42:22.792 INFO:tasks.workunit.client.1.smithi104.stdout:5/806: write d0/f58 [5083249,16342] 0 2024-04-26T04:42:22.794 INFO:tasks.workunit.client.0.smithi090.stdout:9/402: symlink db/dd/d18/d31/l8b 0 2024-04-26T04:42:22.794 INFO:tasks.workunit.client.0.smithi090.stdout:9/403: stat db/dd/d18/d31/d6a/c77 0 2024-04-26T04:42:22.796 INFO:tasks.workunit.client.1.smithi104.stdout:9/673: dwrite d11/d15/d32/d4c/d7c/fb4 [0,4194304] 0 2024-04-26T04:42:22.797 INFO:tasks.workunit.client.0.smithi090.stdout:1/446: rename d0/d4/d7/d4b/d53/f6e to d0/d4/d7/d17/d22/f89 0 2024-04-26T04:42:22.797 INFO:tasks.workunit.client.0.smithi090.stdout:1/447: chown d0/d4/l41 0 1 2024-04-26T04:42:22.799 INFO:tasks.workunit.client.0.smithi090.stdout:9/404: symlink db/dd/l8c 0 2024-04-26T04:42:22.799 INFO:tasks.workunit.client.1.smithi104.stdout:1/847: write d0/d6/d19/d21/d66/d29/d3f/fa0 [686041,38732] 0 2024-04-26T04:42:22.800 INFO:tasks.workunit.client.0.smithi090.stdout:2/528: dwrite db/d11/d14/d27/d28/fa5 [0,4194304] 0 2024-04-26T04:42:22.801 INFO:tasks.workunit.client.1.smithi104.stdout:8/703: dwrite d0/f4e [0,4194304] 0 2024-04-26T04:42:22.802 INFO:tasks.workunit.client.0.smithi090.stdout:4/474: dwrite d3/db/d12/d20/d2d/d58/d6a/f71 [0,4194304] 0 2024-04-26T04:42:22.802 INFO:tasks.workunit.client.0.smithi090.stdout:7/527: dwrite d0/d1/d4/f10 [0,4194304] 0 2024-04-26T04:42:22.802 INFO:tasks.workunit.client.0.smithi090.stdout:7/528: readlink d0/db/d53/db7/d4f/l67 0 2024-04-26T04:42:22.802 INFO:tasks.workunit.client.0.smithi090.stdout:3/437: dread d1/d17/d36/f67 [0,4194304] 0 2024-04-26T04:42:22.803 INFO:tasks.workunit.client.1.smithi104.stdout:9/674: rename d11/d1d/d71/d94/db8 to d11/d15/d32/d4c/d54/ddc 0 2024-04-26T04:42:22.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/848: symlink d0/d6/d2e/d32/d4d/l112 0 2024-04-26T04:42:22.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/849: write d0/d6/d19/d21/d66/d29/d3f/fa0 [3786974,60265] 0 2024-04-26T04:42:22.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/850: write d0/d6/d19/d21/d28/f36 [4327732,53097] 0 2024-04-26T04:42:22.805 INFO:tasks.workunit.client.1.smithi104.stdout:1/851: write d0/d6/d49/f87 [4783917,107455] 0 2024-04-26T04:42:22.807 INFO:tasks.workunit.client.0.smithi090.stdout:2/529: rename db/d11/d14/d25/d2d/d48/d81/la0 to db/d11/d14/d25/d41/lb5 0 2024-04-26T04:42:22.809 INFO:tasks.workunit.client.1.smithi104.stdout:1/852: getdents d0/d6/d2e/d32 0 2024-04-26T04:42:22.812 INFO:tasks.workunit.client.1.smithi104.stdout:8/704: write d0/f2f [4982760,41434] 0 2024-04-26T04:42:22.812 INFO:tasks.workunit.client.1.smithi104.stdout:8/705: write d0/d1c/d27/f72 [1683598,112072] 0 2024-04-26T04:42:22.812 INFO:tasks.workunit.client.1.smithi104.stdout:1/853: rename d0/d6/led to d0/d6/d9e/d10c/da7/l113 0 2024-04-26T04:42:22.823 INFO:tasks.workunit.client.0.smithi090.stdout:9/405: truncate db/dd/d18/d31/d58/f6c 463428 0 2024-04-26T04:42:22.823 INFO:tasks.workunit.client.0.smithi090.stdout:9/406: fdatasync db/f13 0 2024-04-26T04:42:22.824 INFO:tasks.workunit.client.1.smithi104.stdout:8/706: dread d0/d1c/d27/d47/d67/d76/d7f/fac [0,4194304] 0 2024-04-26T04:42:22.824 INFO:tasks.workunit.client.1.smithi104.stdout:2/958: sync 2024-04-26T04:42:22.825 INFO:tasks.workunit.client.1.smithi104.stdout:0/676: sync 2024-04-26T04:42:22.826 INFO:tasks.workunit.client.1.smithi104.stdout:0/677: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fd5 x:0 0 0 2024-04-26T04:42:22.826 INFO:tasks.workunit.client.1.smithi104.stdout:0/678: read d0/d2/d21/d27/d1e/f65 [65376,76585] 0 2024-04-26T04:42:22.827 INFO:tasks.workunit.client.1.smithi104.stdout:8/707: mknod d0/d9a/cdb 0 2024-04-26T04:42:22.827 INFO:tasks.workunit.client.1.smithi104.stdout:8/708: write d0/f9 [4952474,101758] 0 2024-04-26T04:42:22.828 INFO:tasks.workunit.client.0.smithi090.stdout:7/529: creat d0/db/df/d4d/d7b/db3/fc4 x:0 0 0 2024-04-26T04:42:22.829 INFO:tasks.workunit.client.1.smithi104.stdout:0/679: mknod d0/d2/d10/d3d/dc0/cd6 0 2024-04-26T04:42:22.830 INFO:tasks.workunit.client.0.smithi090.stdout:3/438: creat d1/d17/d36/d33/d43/f92 x:0 0 0 2024-04-26T04:42:22.832 INFO:tasks.workunit.client.1.smithi104.stdout:8/709: mknod d0/d50/cdc 0 2024-04-26T04:42:22.833 INFO:tasks.workunit.client.0.smithi090.stdout:4/475: rename d3/db/d12/f28 to d3/db/d12/d20/d2d/d2e/f98 0 2024-04-26T04:42:22.833 INFO:tasks.workunit.client.1.smithi104.stdout:0/680: mknod d0/d2/d21/d27/d1e/d57/d6c/dd1/d7c/cd7 0 2024-04-26T04:42:22.834 INFO:tasks.workunit.client.0.smithi090.stdout:9/407: rmdir db/d5f 39 2024-04-26T04:42:22.835 INFO:tasks.workunit.client.1.smithi104.stdout:0/681: read d0/d2/d21/d27/f83 [6975,11349] 0 2024-04-26T04:42:22.837 INFO:tasks.workunit.client.0.smithi090.stdout:9/408: creat db/dd/f8d x:0 0 0 2024-04-26T04:42:22.837 INFO:tasks.workunit.client.0.smithi090.stdout:9/409: creat db/dd/d44/f8e x:0 0 0 2024-04-26T04:42:22.837 INFO:tasks.workunit.client.1.smithi104.stdout:8/710: rename d0/d1c/d27/d47/l14 to d0/d1c/d27/d47/ldd 0 2024-04-26T04:42:22.839 INFO:tasks.workunit.client.1.smithi104.stdout:0/682: rename d0/d2/d21/d27/d1e/d57/d6c/f56 to d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fd8 0 2024-04-26T04:42:22.840 INFO:tasks.workunit.client.1.smithi104.stdout:8/711: mknod d0/d1c/d27/d47/d67/d76/daf/cde 0 2024-04-26T04:42:22.842 INFO:tasks.workunit.client.1.smithi104.stdout:0/683: mknod d0/d2/d21/d27/d1e/d57/d6c/cd9 0 2024-04-26T04:42:22.845 INFO:tasks.workunit.client.1.smithi104.stdout:0/684: rename d0/d2/d21/d49/d5e/d9c/d53/fa6 to d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/fda 0 2024-04-26T04:42:22.846 INFO:tasks.workunit.client.0.smithi090.stdout:7/530: write d0/db/d53/db7/f3a [3489384,58512] 0 2024-04-26T04:42:22.846 INFO:tasks.workunit.client.0.smithi090.stdout:7/531: fsync d0/d1/d4/d98/d4a/fbe 0 2024-04-26T04:42:22.847 INFO:tasks.workunit.client.1.smithi104.stdout:9/675: dwrite f4 [0,4194304] 0 2024-04-26T04:42:22.847 INFO:tasks.workunit.client.1.smithi104.stdout:0/685: truncate d0/d2/d21/d27/d1e/d57/d6c/f6a 51306 0 2024-04-26T04:42:22.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/532: creat d0/d1/d4/fc5 x:0 0 0 2024-04-26T04:42:22.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/533: readlink d0/db/df/l11 0 2024-04-26T04:42:22.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/534: dread - d0/db/d53/d5e/fa9 zero size 2024-04-26T04:42:22.848 INFO:tasks.workunit.client.1.smithi104.stdout:0/686: mkdir d0/d4/ddb 0 2024-04-26T04:42:22.850 INFO:tasks.workunit.client.1.smithi104.stdout:0/687: write d0/d2/d10/f64 [4314436,85326] 0 2024-04-26T04:42:22.852 INFO:tasks.workunit.client.1.smithi104.stdout:0/688: creat d0/d2/d10/d3d/dc0/fdc x:0 0 0 2024-04-26T04:42:22.853 INFO:tasks.workunit.client.1.smithi104.stdout:0/689: getdents d0/d2/d21/d49/d5e/d9c 0 2024-04-26T04:42:22.853 INFO:tasks.workunit.client.1.smithi104.stdout:0/690: chown d0/c93 6810 1 2024-04-26T04:42:22.860 INFO:tasks.workunit.client.0.smithi090.stdout:7/535: write d0/db/df/d4d/fc1 [199337,48003] 0 2024-04-26T04:42:22.861 INFO:tasks.workunit.client.0.smithi090.stdout:7/536: chown d0/d1/d4/d98/d4a/lbf 1866807 1 2024-04-26T04:42:22.865 INFO:tasks.workunit.client.0.smithi090.stdout:1/448: dwrite d0/d4/d43/f54 [0,4194304] 0 2024-04-26T04:42:22.867 INFO:tasks.workunit.client.0.smithi090.stdout:1/449: mknod d0/d4/d7/d4b/d53/c8a 0 2024-04-26T04:42:22.868 INFO:tasks.workunit.client.0.smithi090.stdout:1/450: symlink d0/d4/d7/d17/l8b 0 2024-04-26T04:42:22.869 INFO:tasks.workunit.client.1.smithi104.stdout:0/691: write d0/d2/d21/d27/d1e/d57/d6c/dd1/f36 [2673534,65712] 0 2024-04-26T04:42:22.869 INFO:tasks.workunit.client.0.smithi090.stdout:1/451: mknod d0/c8c 0 2024-04-26T04:42:22.869 INFO:tasks.workunit.client.0.smithi090.stdout:1/452: fsync d0/d4/d7/d17/d22/f52 0 2024-04-26T04:42:22.870 INFO:tasks.workunit.client.0.smithi090.stdout:1/453: symlink d0/d4/d7/d49/l8d 0 2024-04-26T04:42:22.871 INFO:tasks.workunit.client.0.smithi090.stdout:2/530: dwrite db/d11/d14/d27/d2f/d70/d71/f83 [0,4194304] 0 2024-04-26T04:42:22.874 INFO:tasks.workunit.client.1.smithi104.stdout:2/959: dwrite d6/de/d1d/f8f [0,4194304] 0 2024-04-26T04:42:22.874 INFO:tasks.workunit.client.1.smithi104.stdout:8/712: dwrite d0/d33/d35/d6d/f9c [0,4194304] 0 2024-04-26T04:42:22.874 INFO:tasks.workunit.client.1.smithi104.stdout:8/713: read - d0/d33/d51/f92 zero size 2024-04-26T04:42:22.875 INFO:tasks.workunit.client.1.smithi104.stdout:9/676: dwrite d11/d15/d1b/f72 [0,4194304] 0 2024-04-26T04:42:22.875 INFO:tasks.workunit.client.1.smithi104.stdout:9/677: dread - d11/d90/f99 zero size 2024-04-26T04:42:22.875 INFO:tasks.workunit.client.1.smithi104.stdout:9/678: dread - d11/d15/d32/dd6/fd9 zero size 2024-04-26T04:42:22.875 INFO:tasks.workunit.client.1.smithi104.stdout:9/679: dread - d11/d53/fb6 zero size 2024-04-26T04:42:22.875 INFO:tasks.workunit.client.0.smithi090.stdout:3/439: dwrite d1/d17/d36/d33/d43/f92 [0,4194304] 0 2024-04-26T04:42:22.877 INFO:tasks.workunit.client.1.smithi104.stdout:1/854: dwrite d0/d6/d9e/d10c/d35/fd1 [0,4194304] 0 2024-04-26T04:42:22.877 INFO:tasks.workunit.client.1.smithi104.stdout:1/855: dread - d0/d6/d19/f111 zero size 2024-04-26T04:42:22.879 INFO:tasks.workunit.client.0.smithi090.stdout:3/440: symlink d1/d17/d36/d2a/d3a/d5d/l93 0 2024-04-26T04:42:22.888 INFO:tasks.workunit.client.1.smithi104.stdout:8/714: symlink d0/d1c/d2d/d7a/ldf 0 2024-04-26T04:42:22.890 INFO:tasks.workunit.client.0.smithi090.stdout:3/441: symlink d1/d17/d52/l94 0 2024-04-26T04:42:22.890 INFO:tasks.workunit.client.1.smithi104.stdout:9/680: unlink d11/d15/d32/d35/d7b/lba 0 2024-04-26T04:42:22.891 INFO:tasks.workunit.client.1.smithi104.stdout:2/960: dread d6/f8b [0,4194304] 0 2024-04-26T04:42:22.891 INFO:tasks.workunit.client.1.smithi104.stdout:1/856: symlink d0/d6/d19/d21/d66/d29/df7/l114 0 2024-04-26T04:42:22.892 INFO:tasks.workunit.client.1.smithi104.stdout:1/857: readlink d0/d6/d19/d21/d59/lbb 0 2024-04-26T04:42:22.892 INFO:tasks.workunit.client.1.smithi104.stdout:8/715: chown d0/l94 16 1 2024-04-26T04:42:22.894 INFO:tasks.workunit.client.1.smithi104.stdout:9/681: link d11/d1d/d34/c39 d11/d90/cdd 0 2024-04-26T04:42:22.895 INFO:tasks.workunit.client.0.smithi090.stdout:0/560: sync 2024-04-26T04:42:22.895 INFO:tasks.workunit.client.0.smithi090.stdout:5/512: sync 2024-04-26T04:42:22.895 INFO:tasks.workunit.client.0.smithi090.stdout:8/520: sync 2024-04-26T04:42:22.895 INFO:tasks.workunit.client.0.smithi090.stdout:6/563: sync 2024-04-26T04:42:22.896 INFO:tasks.workunit.client.1.smithi104.stdout:1/858: mkdir d0/d6/d19/d21/d66/d29/df7/d115 0 2024-04-26T04:42:22.896 INFO:tasks.workunit.client.0.smithi090.stdout:8/521: read dc/d1f/d49/d39/f7c [167450,71930] 0 2024-04-26T04:42:22.897 INFO:tasks.workunit.client.0.smithi090.stdout:0/561: stat d0/d4b/d5d/d5c/d8a/l83 0 2024-04-26T04:42:22.898 INFO:tasks.workunit.client.1.smithi104.stdout:9/682: creat d11/d15/d32/d35/d7b/fde x:0 0 0 2024-04-26T04:42:22.898 INFO:tasks.workunit.client.1.smithi104.stdout:9/683: read d11/d15/d1b/f37 [1471603,34327] 0 2024-04-26T04:42:22.898 INFO:tasks.workunit.client.1.smithi104.stdout:9/684: truncate d11/d15/d32/d4c/d54/f61 2012142 0 2024-04-26T04:42:22.899 INFO:tasks.workunit.client.1.smithi104.stdout:2/961: read d6/de/d85/fa6 [2746678,30859] 0 2024-04-26T04:42:22.899 INFO:tasks.workunit.client.1.smithi104.stdout:2/962: truncate d6/d8/d5a/fb0 9368388 0 2024-04-26T04:42:22.899 INFO:tasks.workunit.client.1.smithi104.stdout:1/859: rename d0/d6/d9e/d10c/d35/c65 to d0/d6/d19/d21/d66/d20/c116 0 2024-04-26T04:42:22.900 INFO:tasks.workunit.client.0.smithi090.stdout:5/513: creat d4/d1a/d1f/d53/d59/d5e/fa5 x:0 0 0 2024-04-26T04:42:22.901 INFO:tasks.workunit.client.1.smithi104.stdout:2/963: symlink d6/d1b/d68/d105/l13c 0 2024-04-26T04:42:22.901 INFO:tasks.workunit.client.1.smithi104.stdout:1/860: mkdir d0/d6/d19/d21/d66/d20/d2a/dbc/d117 0 2024-04-26T04:42:22.901 INFO:tasks.workunit.client.1.smithi104.stdout:2/964: readlink d6/de/d1d/la9 0 2024-04-26T04:42:22.901 INFO:tasks.workunit.client.1.smithi104.stdout:1/861: readlink d0/d6/d9e/d10c/d35/l5f 0 2024-04-26T04:42:22.903 INFO:tasks.workunit.client.1.smithi104.stdout:1/862: link d0/d6/d49/lff d0/d6/d2e/d32/l118 0 2024-04-26T04:42:22.905 INFO:tasks.workunit.client.0.smithi090.stdout:1/454: dwrite d0/d4/d43/f47 [0,4194304] 0 2024-04-26T04:42:22.906 INFO:tasks.workunit.client.0.smithi090.stdout:8/522: link dc/d1f/d49/d4f/d58/d95/c77 dc/d1f/d49/d39/d99/cb6 0 2024-04-26T04:42:22.907 INFO:tasks.workunit.client.0.smithi090.stdout:7/537: dwrite d0/d1/d4/d3e/f90 [4194304,4194304] 0 2024-04-26T04:42:22.911 INFO:tasks.workunit.client.0.smithi090.stdout:0/562: link d0/d4b/d5d/d5c/d8a/d45/f96 d0/d4b/d5d/d48/fca 0 2024-04-26T04:42:22.911 INFO:tasks.workunit.client.0.smithi090.stdout:1/455: dread d0/d4/d7/d17/f29 [0,4194304] 0 2024-04-26T04:42:22.914 INFO:tasks.workunit.client.0.smithi090.stdout:5/514: rename d4/d1a/d2a/f7c to d4/d1a/d45/fa6 0 2024-04-26T04:42:22.914 INFO:tasks.workunit.client.0.smithi090.stdout:5/515: truncate d4/d1a/df/d4a/f7a 72238 0 2024-04-26T04:42:22.916 INFO:tasks.workunit.client.1.smithi104.stdout:0/692: dwrite d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fd5 [0,4194304] 0 2024-04-26T04:42:22.918 INFO:tasks.workunit.client.0.smithi090.stdout:6/564: mkdir d9/dcb 0 2024-04-26T04:42:22.920 INFO:tasks.workunit.client.0.smithi090.stdout:8/523: mkdir dc/d1f/d49/d4f/db7 0 2024-04-26T04:42:22.921 INFO:tasks.workunit.client.0.smithi090.stdout:7/538: mknod d0/db/df/d4d/d7b/cc6 0 2024-04-26T04:42:22.922 INFO:tasks.workunit.client.0.smithi090.stdout:1/456: dread d0/d4/d7/d49/f5e [0,4194304] 0 2024-04-26T04:42:22.923 INFO:tasks.workunit.client.0.smithi090.stdout:5/516: creat d4/d1a/df/fa7 x:0 0 0 2024-04-26T04:42:22.923 INFO:tasks.workunit.client.0.smithi090.stdout:5/517: chown d4/d1a/d45/f4f 9 1 2024-04-26T04:42:22.923 INFO:tasks.workunit.client.0.smithi090.stdout:8/524: dread dc/d5a/f62 [0,4194304] 0 2024-04-26T04:42:22.923 INFO:tasks.workunit.client.0.smithi090.stdout:8/525: chown dc/d1f/d49/d39/d99/cb6 723164 1 2024-04-26T04:42:22.924 INFO:tasks.workunit.client.0.smithi090.stdout:0/563: rename d0/d4b/d5d/d5c/d8a/d45/fb4 to d0/d4b/db3/fcb 0 2024-04-26T04:42:22.925 INFO:tasks.workunit.client.0.smithi090.stdout:0/564: chown d0/de/f90 1415 1 2024-04-26T04:42:22.925 INFO:tasks.workunit.client.0.smithi090.stdout:0/565: dread - d0/d4b/db3/fcb zero size 2024-04-26T04:42:22.925 INFO:tasks.workunit.client.0.smithi090.stdout:6/565: unlink d9/d21/fc2 0 2024-04-26T04:42:22.926 INFO:tasks.workunit.client.0.smithi090.stdout:1/457: symlink d0/d4/d7/d4b/d53/l8e 0 2024-04-26T04:42:22.927 INFO:tasks.workunit.client.1.smithi104.stdout:0/693: rename d0/d2/d21/d27/f6d to d0/d2/d21/fdd 0 2024-04-26T04:42:22.928 INFO:tasks.workunit.client.0.smithi090.stdout:0/566: mkdir d0/d4b/db3/dcc 0 2024-04-26T04:42:22.929 INFO:tasks.workunit.client.0.smithi090.stdout:1/458: link d0/d4/d7/d17/f29 d0/d4/d62/d78/f8f 0 2024-04-26T04:42:22.929 INFO:tasks.workunit.client.0.smithi090.stdout:1/459: dread - d0/d55/f67 zero size 2024-04-26T04:42:22.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/567: mkdir d0/d4b/d5d/d5c/dcd 0 2024-04-26T04:42:22.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/568: chown d0/d4b/d5d/d5c/d8a/d45/cb0 208 1 2024-04-26T04:42:22.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/569: write d0/d4b/fbd [579000,9061] 0 2024-04-26T04:42:22.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/570: truncate d0/de/f89 328045 0 2024-04-26T04:42:22.931 INFO:tasks.workunit.client.0.smithi090.stdout:0/571: write d0/d4b/fbd [250709,51177] 0 2024-04-26T04:42:22.931 INFO:tasks.workunit.client.0.smithi090.stdout:1/460: truncate d0/d4/d7/d17/f2a 376858 0 2024-04-26T04:42:22.932 INFO:tasks.workunit.client.0.smithi090.stdout:0/572: link d0/d4b/d5d/d5c/d8a/l83 d0/d4b/d5d/d48/d7d/lce 0 2024-04-26T04:42:22.933 INFO:tasks.workunit.client.0.smithi090.stdout:1/461: getdents d0/d4/d7/d4b/d53/d7b 0 2024-04-26T04:42:22.935 INFO:tasks.workunit.client.0.smithi090.stdout:0/573: mkdir d0/d4b/d5d/d5c/d8a/d45/dcf 0 2024-04-26T04:42:22.935 INFO:tasks.workunit.client.0.smithi090.stdout:0/574: dread - d0/d4b/d5d/d5c/d8a/d45/d53/f87 zero size 2024-04-26T04:42:22.936 INFO:tasks.workunit.client.0.smithi090.stdout:1/462: creat d0/d4/d7/f90 x:0 0 0 2024-04-26T04:42:22.937 INFO:tasks.workunit.client.0.smithi090.stdout:1/463: truncate d0/d4/d7/d49/f6f 5188410 0 2024-04-26T04:42:22.938 INFO:tasks.workunit.client.0.smithi090.stdout:0/575: unlink d0/d4b/d5d/d5c/d8a/la 0 2024-04-26T04:42:22.938 INFO:tasks.workunit.client.0.smithi090.stdout:7/539: dread d0/db/d15/d29/f14 [0,4194304] 0 2024-04-26T04:42:22.941 INFO:tasks.workunit.client.0.smithi090.stdout:0/576: mknod d0/d4b/d5d/d5c/dc4/cd0 0 2024-04-26T04:42:22.942 INFO:tasks.workunit.client.1.smithi104.stdout:0/694: write d0/d2/d21/d27/d1e/d57/d6c/f66 [1707797,76110] 0 2024-04-26T04:42:22.942 INFO:tasks.workunit.client.1.smithi104.stdout:0/695: mknod d0/d4/d3b/cde 0 2024-04-26T04:42:22.942 INFO:tasks.workunit.client.1.smithi104.stdout:0/696: chown d0/d4/cbb 14 1 2024-04-26T04:42:22.942 INFO:tasks.workunit.client.1.smithi104.stdout:0/697: write d0/d2/d21/d27/fd4 [948839,16122] 0 2024-04-26T04:42:22.943 INFO:tasks.workunit.client.1.smithi104.stdout:0/698: getdents d0/d2/da0 0 2024-04-26T04:42:22.943 INFO:tasks.workunit.client.0.smithi090.stdout:7/540: creat d0/d1/d4/d98/d65/fc7 x:0 0 0 2024-04-26T04:42:22.943 INFO:tasks.workunit.client.0.smithi090.stdout:0/577: truncate d0/d4b/d6d/f9a 935949 0 2024-04-26T04:42:22.944 INFO:tasks.workunit.client.0.smithi090.stdout:0/578: chown d0/d4b/d5d/d5c/dc4/cd0 9 1 2024-04-26T04:42:22.944 INFO:tasks.workunit.client.0.smithi090.stdout:0/579: fdatasync d0/d4b/d5d/d5c/f9c 0 2024-04-26T04:42:22.944 INFO:tasks.workunit.client.0.smithi090.stdout:0/580: write d0/f7a [346571,34304] 0 2024-04-26T04:42:22.944 INFO:tasks.workunit.client.1.smithi104.stdout:0/699: truncate d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/fda 1522165 0 2024-04-26T04:42:22.944 INFO:tasks.workunit.client.1.smithi104.stdout:0/700: write d0/d2/d21/d49/d5e/fa7 [4190992,17249] 0 2024-04-26T04:42:22.945 INFO:tasks.workunit.client.1.smithi104.stdout:0/701: truncate d0/d2/d21/d49/d5e/fb3 34618 0 2024-04-26T04:42:22.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/541: mknod d0/d1/cc8 0 2024-04-26T04:42:22.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/542: stat d0/db/d53/db7 0 2024-04-26T04:42:22.945 INFO:tasks.workunit.client.0.smithi090.stdout:3/442: dwrite d1/d17/f25 [0,4194304] 0 2024-04-26T04:42:22.948 INFO:tasks.workunit.client.1.smithi104.stdout:8/716: dwrite d0/d50/fb4 [0,4194304] 0 2024-04-26T04:42:22.948 INFO:tasks.workunit.client.0.smithi090.stdout:3/443: mkdir d1/d17/d36/d33/d95 0 2024-04-26T04:42:22.952 INFO:tasks.workunit.client.1.smithi104.stdout:8/717: unlink d0/d33/l3c 0 2024-04-26T04:42:22.953 INFO:tasks.workunit.client.1.smithi104.stdout:8/718: mkdir d0/d33/da6/de0 0 2024-04-26T04:42:22.954 INFO:tasks.workunit.client.1.smithi104.stdout:8/719: symlink d0/d1c/d27/d47/db1/le1 0 2024-04-26T04:42:22.955 INFO:tasks.workunit.client.1.smithi104.stdout:8/720: creat d0/d33/da6/de0/fe2 x:0 0 0 2024-04-26T04:42:22.956 INFO:tasks.workunit.client.1.smithi104.stdout:9/685: dwrite d11/d15/d32/d35/f65 [0,4194304] 0 2024-04-26T04:42:22.956 INFO:tasks.workunit.client.1.smithi104.stdout:2/965: dwrite d6/d8/f37 [0,4194304] 0 2024-04-26T04:42:22.959 INFO:tasks.workunit.client.1.smithi104.stdout:8/721: mkdir d0/d9a/de3 0 2024-04-26T04:42:22.959 INFO:tasks.workunit.client.1.smithi104.stdout:8/722: stat d0/d1c/d2d/d7a/lb7 0 2024-04-26T04:42:22.960 INFO:tasks.workunit.client.1.smithi104.stdout:2/966: symlink d6/d1b/d75/l13d 0 2024-04-26T04:42:22.961 INFO:tasks.workunit.client.0.smithi090.stdout:3/444: write d1/d6/f14 [3304545,10754] 0 2024-04-26T04:42:22.961 INFO:tasks.workunit.client.1.smithi104.stdout:1/863: dwrite d0/d6/d19/d21/d66/d20/da3/dde/ff0 [0,4194304] 0 2024-04-26T04:42:22.962 INFO:tasks.workunit.client.1.smithi104.stdout:8/723: link d0/d33/d35/f87 d0/d1c/d27/d47/d67/d76/d7f/fe4 0 2024-04-26T04:42:22.962 INFO:tasks.workunit.client.1.smithi104.stdout:8/724: readlink d0/d1c/l64 0 2024-04-26T04:42:22.964 INFO:tasks.workunit.client.0.smithi090.stdout:3/445: rename d1/d17/d52/c8b to d1/d6/d1b/c96 0 2024-04-26T04:42:22.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/446: truncate d1/f27 199763 0 2024-04-26T04:42:22.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/447: chown d1/d9/fa 7 1 2024-04-26T04:42:22.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/448: chown d1/d17/d61/c73 3 1 2024-04-26T04:42:22.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/449: mkdir d1/d6/d2c/d97 0 2024-04-26T04:42:22.970 INFO:tasks.workunit.client.0.smithi090.stdout:6/566: dwrite d9/d21/d40/d4c/d6f/d9f/dae/fc1 [0,4194304] 0 2024-04-26T04:42:22.970 INFO:tasks.workunit.client.0.smithi090.stdout:8/526: dwrite dc/d34/f84 [0,4194304] 0 2024-04-26T04:42:22.970 INFO:tasks.workunit.client.0.smithi090.stdout:8/527: fsync dc/d1f/d49/d39/d55/d64/f7a 0 2024-04-26T04:42:22.971 INFO:tasks.workunit.client.1.smithi104.stdout:1/864: rename d0/d6/d49/f87 to d0/d6/d19/d21/d66/d20/da3/f119 0 2024-04-26T04:42:22.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:22 smithi090 ceph-mon[26648]: pgmap v253: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 215 MiB/s rd, 273 MiB/s wr, 489 op/s 2024-04-26T04:42:22.973 INFO:tasks.workunit.client.1.smithi104.stdout:8/725: unlink d0/d1c/d27/d47/f38 0 2024-04-26T04:42:22.973 INFO:tasks.workunit.client.1.smithi104.stdout:8/726: getdents d0/d50/dc4/dcf 0 2024-04-26T04:42:22.975 INFO:tasks.workunit.client.0.smithi090.stdout:8/528: getdents dc/d1f/d49 0 2024-04-26T04:42:22.976 INFO:tasks.workunit.client.1.smithi104.stdout:8/727: creat d0/d9a/dce/fe5 x:0 0 0 2024-04-26T04:42:22.976 INFO:tasks.workunit.client.1.smithi104.stdout:8/728: chown d0/f9 1 1 2024-04-26T04:42:22.976 INFO:tasks.workunit.client.1.smithi104.stdout:9/686: dread d11/d15/d32/d35/f57 [0,4194304] 0 2024-04-26T04:42:22.977 INFO:tasks.workunit.client.0.smithi090.stdout:3/450: dread d1/d59/d5e/f62 [0,4194304] 0 2024-04-26T04:42:22.977 INFO:tasks.workunit.client.0.smithi090.stdout:8/529: mkdir dc/d1f/d49/d3b/d3d/db8 0 2024-04-26T04:42:22.978 INFO:tasks.workunit.client.0.smithi090.stdout:8/530: creat dc/d1f/d49/d39/fb9 x:0 0 0 2024-04-26T04:42:22.979 INFO:tasks.workunit.client.0.smithi090.stdout:3/451: symlink d1/d6/d2c/d97/l98 0 2024-04-26T04:42:22.983 INFO:tasks.workunit.client.0.smithi090.stdout:1/464: dwrite d0/f86 [0,4194304] 0 2024-04-26T04:42:22.983 INFO:tasks.workunit.client.0.smithi090.stdout:1/465: chown d0/l3 1689435 1 2024-04-26T04:42:22.984 INFO:tasks.workunit.client.1.smithi104.stdout:3/854: sync 2024-04-26T04:42:22.984 INFO:tasks.workunit.client.1.smithi104.stdout:4/846: sync 2024-04-26T04:42:22.985 INFO:tasks.workunit.client.1.smithi104.stdout:3/855: read d5/d9/d80/f9e [34198,81024] 0 2024-04-26T04:42:22.985 INFO:tasks.workunit.client.1.smithi104.stdout:6/852: sync 2024-04-26T04:42:22.985 INFO:tasks.workunit.client.1.smithi104.stdout:4/847: creat d4/f129 x:0 0 0 2024-04-26T04:42:22.985 INFO:tasks.workunit.client.1.smithi104.stdout:7/734: sync 2024-04-26T04:42:22.985 INFO:tasks.workunit.client.1.smithi104.stdout:4/848: write d4/f37 [1415481,8875] 0 2024-04-26T04:42:22.985 INFO:tasks.workunit.client.1.smithi104.stdout:4/849: fsync d4/d20/d55/d56/da3/fc4 0 2024-04-26T04:42:22.985 INFO:tasks.workunit.client.1.smithi104.stdout:4/850: readlink d4/d7/d1d/d2e/d74/dbd/dee/l11e 0 2024-04-26T04:42:22.987 INFO:tasks.workunit.client.1.smithi104.stdout:2/967: dread d6/de/d1d/dcf/f12e [0,4194304] 0 2024-04-26T04:42:22.989 INFO:tasks.workunit.client.1.smithi104.stdout:3/856: truncate d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/faa 6271427 0 2024-04-26T04:42:22.989 INFO:tasks.workunit.client.1.smithi104.stdout:8/729: dread d0/d1c/d2d/f6a [0,4194304] 0 2024-04-26T04:42:22.991 INFO:tasks.workunit.client.1.smithi104.stdout:6/853: rename d0/d9/d13/d41/l62 to d0/d9/d13/d41/d96/ded/d10d/d71/da5/l114 0 2024-04-26T04:42:22.994 INFO:tasks.workunit.client.0.smithi090.stdout:1/466: link d0/f86 d0/d4/f91 0 2024-04-26T04:42:22.994 INFO:tasks.workunit.client.0.smithi090.stdout:1/467: creat d0/d4/d7/d4b/d53/f92 x:0 0 0 2024-04-26T04:42:22.994 INFO:tasks.workunit.client.0.smithi090.stdout:1/468: dread - d0/d55/f83 zero size 2024-04-26T04:42:22.994 INFO:tasks.workunit.client.1.smithi104.stdout:2/968: mkdir d6/de/d1d/d13e 0 2024-04-26T04:42:22.995 INFO:tasks.workunit.client.1.smithi104.stdout:1/865: dread d0/d6/fc0 [0,4194304] 0 2024-04-26T04:42:22.996 INFO:tasks.workunit.client.1.smithi104.stdout:3/857: creat d5/db2/da3/dd0/ddb/d3a/f12d x:0 0 0 2024-04-26T04:42:22.996 INFO:tasks.workunit.client.1.smithi104.stdout:1/866: write d0/d6/d19/d21/d66/d20/d2a/dbc/fd4 [187784,118334] 0 2024-04-26T04:42:22.997 INFO:tasks.workunit.client.1.smithi104.stdout:9/687: write d11/d15/d32/d4c/d54/f77 [2659961,5279] 0 2024-04-26T04:42:22.998 INFO:tasks.workunit.client.1.smithi104.stdout:6/854: mknod d0/d9/d13/d41/d96/dac/dec/dbd/c115 0 2024-04-26T04:42:22.998 INFO:tasks.workunit.client.1.smithi104.stdout:6/855: chown d0/d9/d13/d41/d82/dc8/dba/daa/dda/cef 4669505 1 2024-04-26T04:42:22.998 INFO:tasks.workunit.client.1.smithi104.stdout:6/856: dread - d0/d9/d13/dcd/f110 zero size 2024-04-26T04:42:22.999 INFO:tasks.workunit.client.0.smithi090.stdout:6/567: dwrite d9/d77/db0/db5/fbe [0,4194304] 0 2024-04-26T04:42:22.999 INFO:tasks.workunit.client.0.smithi090.stdout:6/568: getdents d9/dcb 0 2024-04-26T04:42:22.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/469: mkdir d0/d4/d7/d17/d22/d93 0 2024-04-26T04:42:23.000 INFO:tasks.workunit.client.1.smithi104.stdout:4/851: write d4/d7/d1d/d1e/d15/f1a [5840222,121911] 0 2024-04-26T04:42:23.000 INFO:tasks.workunit.client.1.smithi104.stdout:4/852: readlink d4/d20/l7b 0 2024-04-26T04:42:23.000 INFO:tasks.workunit.client.1.smithi104.stdout:8/730: rename d0/d1c/d27/c99 to d0/d1c/d27/d47/d67/d76/daf/dd4/ce6 0 2024-04-26T04:42:23.001 INFO:tasks.workunit.client.1.smithi104.stdout:3/858: dread d5/db2/da3/dd0/ddb/d32/d48/dac/f89 [0,4194304] 0 2024-04-26T04:42:23.001 INFO:tasks.workunit.client.1.smithi104.stdout:3/859: write d5/db2/da3/dd0/ddb/f111 [963492,21035] 0 2024-04-26T04:42:23.001 INFO:tasks.workunit.client.0.smithi090.stdout:0/581: dwrite d0/d4b/d5d/d5c/d8a/d45/d53/dab/fc8 [0,4194304] 0 2024-04-26T04:42:23.002 INFO:tasks.workunit.client.1.smithi104.stdout:2/969: creat d6/d1b/d93/f13f x:0 0 0 2024-04-26T04:42:23.002 INFO:tasks.workunit.client.1.smithi104.stdout:2/970: creat d6/de/d85/f140 x:0 0 0 2024-04-26T04:42:23.003 INFO:tasks.workunit.client.0.smithi090.stdout:6/569: symlink d9/d21/d40/dc7/lcc 0 2024-04-26T04:42:23.004 INFO:tasks.workunit.client.0.smithi090.stdout:6/570: chown d9/d1c/d31/d79 107923 1 2024-04-26T04:42:23.005 INFO:tasks.workunit.client.1.smithi104.stdout:2/971: dread d6/d1b/d51/f87 [0,4194304] 0 2024-04-26T04:42:23.005 INFO:tasks.workunit.client.1.smithi104.stdout:1/867: creat d0/d6/d19/d21/d58/db4/f11a x:0 0 0 2024-04-26T04:42:23.005 INFO:tasks.workunit.client.0.smithi090.stdout:7/543: dwrite d0/db/df/d1c/d23/f31 [0,4194304] 0 2024-04-26T04:42:23.006 INFO:tasks.workunit.client.0.smithi090.stdout:7/544: chown d0/db/df/d4d/d7b/cb9 1625 1 2024-04-26T04:42:23.007 INFO:tasks.workunit.client.0.smithi090.stdout:6/571: creat d9/d21/d40/d4c/d6f/d9f/fcd x:0 0 0 2024-04-26T04:42:23.008 INFO:tasks.workunit.client.1.smithi104.stdout:9/688: link d11/d53/l6f d11/d1d/d34/ldf 0 2024-04-26T04:42:23.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:22 smithi104 ceph-mon[32369]: pgmap v253: 65 pgs: 65 active+clean; 1.9 GiB data, 5.6 GiB used, 531 GiB / 536 GiB avail; 215 MiB/s rd, 273 MiB/s wr, 489 op/s 2024-04-26T04:42:23.012 INFO:tasks.workunit.client.0.smithi090.stdout:6/572: creat d9/d1c/dc5/fce x:0 0 0 2024-04-26T04:42:23.012 INFO:tasks.workunit.client.0.smithi090.stdout:6/573: fsync d9/f24 0 2024-04-26T04:42:23.014 INFO:tasks.workunit.client.1.smithi104.stdout:4/853: rename d4/d20/d55/d56/f31 to d4/d7/d1d/d2e/f12a 0 2024-04-26T04:42:23.015 INFO:tasks.workunit.client.0.smithi090.stdout:6/574: unlink d9/d1c/l34 0 2024-04-26T04:42:23.015 INFO:tasks.workunit.client.0.smithi090.stdout:6/575: readlink d9/d21/d40/d4c/l83 0 2024-04-26T04:42:23.015 INFO:tasks.workunit.client.0.smithi090.stdout:6/576: chown d9/d77/cbb 1602758 1 2024-04-26T04:42:23.016 INFO:tasks.workunit.client.0.smithi090.stdout:6/577: dread d9/f11 [0,4194304] 0 2024-04-26T04:42:23.016 INFO:tasks.workunit.client.0.smithi090.stdout:6/578: dread - d9/d77/f9d zero size 2024-04-26T04:42:23.019 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.018+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1618068c50 con 0x7f160406c680 2024-04-26T04:42:23.019 INFO:tasks.workunit.client.1.smithi104.stdout:3/860: symlink d5/db2/d46/db7/de6/d112/l12e 0 2024-04-26T04:42:23.020 INFO:tasks.workunit.client.1.smithi104.stdout:3/861: fsync d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/fc6 0 2024-04-26T04:42:23.020 INFO:tasks.workunit.client.0.smithi090.stdout:6/579: rename d9/d1c/d31/d79/f7e to d9/d77/db0/db5/fcf 0 2024-04-26T04:42:23.020 INFO:tasks.workunit.client.0.smithi090.stdout:6/580: fsync d9/d77/db0/db5/fa3 0 2024-04-26T04:42:23.020 INFO:tasks.workunit.client.0.smithi090.stdout:6/581: creat d9/d1c/dc5/fd0 x:0 0 0 2024-04-26T04:42:23.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.020+0000 7f160e7fc700 1 -- 172.21.15.90:0/641884593 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f1618068c50 con 0x7f160406c680 2024-04-26T04:42:23.022 INFO:tasks.workunit.client.0.smithi090.stdout:6/582: truncate d9/d21/d40/d4c/f7b 1444032 0 2024-04-26T04:42:23.022 INFO:tasks.workunit.client.1.smithi104.stdout:2/972: unlink d6/d8/d3e/lfd 0 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.022+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f160406c680 msgr2=0x7f160406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.022+0000 7f161f8f4700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f160406c680 0x7f160406eb30 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1614007900 tx=0x7f1614008040 comp rx=0 tx=0).stop 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.022+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 msgr2=0x7f16181017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.022+0000 7f161f8f4700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f16181017b0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f160800bd70 tx=0x7f160800be50 comp rx=0 tx=0).stop 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.023+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 shutdown_connections 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.023+0000 7f161f8f4700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f160406c680 0x7f160406eb30 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.023+0000 7f161f8f4700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16180ff130 0x7f1618103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.023+0000 7f161f8f4700 1 --2- 172.21.15.90:0/641884593 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16181003d0 0x7f16181017b0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.023+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 >> 172.21.15.90:0/641884593 conn(0x7f16180fa780 msgr2=0x7f1618068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.023+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 shutdown_connections 2024-04-26T04:42:23.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.023+0000 7f161f8f4700 1 -- 172.21.15.90:0/641884593 wait complete. 2024-04-26T04:42:23.029 INFO:tasks.workunit.client.0.smithi090.stdout:7/545: dread d0/db/d15/d29/f14 [4194304,4194304] 0 2024-04-26T04:42:23.029 INFO:tasks.workunit.client.0.smithi090.stdout:7/546: truncate d0/d1/d4/d98/db1/fb5 1468233 0 2024-04-26T04:42:23.029 INFO:tasks.workunit.client.0.smithi090.stdout:7/547: chown d0/d1/d4/d98/f4c 2622 1 2024-04-26T04:42:23.029 INFO:tasks.workunit.client.0.smithi090.stdout:7/548: fsync d0/d1/d4/d98/d65/db2/fba 0 2024-04-26T04:42:23.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/549: creat d0/d1/d4/d98/d65/dbb/fc9 x:0 0 0 2024-04-26T04:42:23.031 INFO:tasks.workunit.client.0.smithi090.stdout:7/550: stat d0/db/d53/db7/f32 0 2024-04-26T04:42:23.040 INFO:tasks.workunit.client.0.smithi090.stdout:7/551: dread d0/db/d53/db7/f32 [0,4194304] 0 2024-04-26T04:42:23.040 INFO:tasks.workunit.client.0.smithi090.stdout:7/552: chown d0/db/d53/db7/d4f/f95 476 1 2024-04-26T04:42:23.040 INFO:tasks.workunit.client.1.smithi104.stdout:1/868: rename d0/d6/d19/d21/d66/d20/d2a/dbc/fd4 to d0/d6/d19/d21/d66/d20/d2a/d109/f11b 0 2024-04-26T04:42:23.041 INFO:tasks.workunit.client.1.smithi104.stdout:4/854: symlink d4/d7/d1d/d1e/d15/l12b 0 2024-04-26T04:42:23.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/553: creat d0/db/d53/db7/d8e/fca x:0 0 0 2024-04-26T04:42:23.042 INFO:tasks.workunit.client.0.smithi090.stdout:7/554: creat d0/d1/d4/d98/d65/db2/fcb x:0 0 0 2024-04-26T04:42:23.044 INFO:tasks.workunit.client.0.smithi090.stdout:7/555: rename d0/db/d53/db7/d8e/c9d to d0/d1/d4/ccc 0 2024-04-26T04:42:23.048 INFO:tasks.workunit.client.1.smithi104.stdout:5/807: sync 2024-04-26T04:42:23.049 INFO:tasks.workunit.client.1.smithi104.stdout:2/973: dread d6/d1b/d75/f7c [0,4194304] 0 2024-04-26T04:42:23.052 INFO:tasks.workunit.client.1.smithi104.stdout:5/808: getdents d0/d1 0 2024-04-26T04:42:23.052 INFO:tasks.workunit.client.1.smithi104.stdout:5/809: readlink d0/d41/l7f 0 2024-04-26T04:42:23.052 INFO:tasks.workunit.client.1.smithi104.stdout:5/810: creat d0/d89/db2/f10f x:0 0 0 2024-04-26T04:42:23.054 INFO:tasks.workunit.client.0.smithi090.stdout:8/531: dwrite dc/d34/d4d/f9e [0,4194304] 0 2024-04-26T04:42:23.054 INFO:tasks.workunit.client.0.smithi090.stdout:8/532: write dc/d1f/d49/f28 [461876,122707] 0 2024-04-26T04:42:23.054 INFO:tasks.workunit.client.0.smithi090.stdout:8/533: creat dc/d1f/d49/d39/fba x:0 0 0 2024-04-26T04:42:23.056 INFO:tasks.workunit.client.1.smithi104.stdout:2/974: rename d6/d1b/d68/d105/f130 to d6/d1b/d51/db8/f141 0 2024-04-26T04:42:23.057 INFO:tasks.workunit.client.0.smithi090.stdout:3/452: dwrite d1/d9/fb [4194304,4194304] 0 2024-04-26T04:42:23.057 INFO:tasks.workunit.client.0.smithi090.stdout:0/582: dwrite d0/d4b/fad [0,4194304] 0 2024-04-26T04:42:23.057 INFO:tasks.workunit.client.0.smithi090.stdout:0/583: chown d0/d4b/d5d/d5c/d8a/d45/d93/fa1 454 1 2024-04-26T04:42:23.057 INFO:tasks.workunit.client.0.smithi090.stdout:6/583: write d9/d21/f5f [1276107,72897] 0 2024-04-26T04:42:23.057 INFO:tasks.workunit.client.0.smithi090.stdout:1/470: dwrite d0/d55/f67 [0,4194304] 0 2024-04-26T04:42:23.058 INFO:tasks.workunit.client.0.smithi090.stdout:1/471: chown d0/l3e 360985934 1 2024-04-26T04:42:23.058 INFO:tasks.workunit.client.0.smithi090.stdout:1/472: creat d0/f94 x:0 0 0 2024-04-26T04:42:23.058 INFO:tasks.workunit.client.1.smithi104.stdout:7/735: dwrite d2/d4/fe3 [0,4194304] 0 2024-04-26T04:42:23.062 INFO:tasks.workunit.client.0.smithi090.stdout:8/534: rename dc/d1f/d49/d39/d55/d64/c5f to dc/d1f/d49/d4f/cbb 0 2024-04-26T04:42:23.063 INFO:tasks.workunit.client.1.smithi104.stdout:5/811: mknod d0/d1/d64/d26/dd7/c110 0 2024-04-26T04:42:23.067 INFO:tasks.workunit.client.1.smithi104.stdout:2/975: dread d6/d8/d3e/fc0 [0,4194304] 0 2024-04-26T04:42:23.068 INFO:tasks.workunit.client.1.smithi104.stdout:2/976: fsync d6/de/d1d/f5f 0 2024-04-26T04:42:23.068 INFO:tasks.workunit.client.1.smithi104.stdout:4/855: dwrite d4/d7/d1d/d1e/d24/d48/f5c [0,4194304] 0 2024-04-26T04:42:23.068 INFO:tasks.workunit.client.1.smithi104.stdout:4/856: chown d4/d118 56 1 2024-04-26T04:42:23.068 INFO:tasks.workunit.client.1.smithi104.stdout:3/862: dwrite d5/d9/fa [4194304,4194304] 0 2024-04-26T04:42:23.069 INFO:tasks.workunit.client.0.smithi090.stdout:0/584: creat d0/d4b/db3/dcc/fd1 x:0 0 0 2024-04-26T04:42:23.069 INFO:tasks.workunit.client.0.smithi090.stdout:0/585: dread - d0/d4b/d5d/d5c/d8a/d45/d53/f87 zero size 2024-04-26T04:42:23.070 INFO:tasks.workunit.client.1.smithi104.stdout:6/857: rmdir d0/d9 39 2024-04-26T04:42:23.070 INFO:tasks.workunit.client.1.smithi104.stdout:4/857: dread d4/f71 [0,4194304] 0 2024-04-26T04:42:23.072 INFO:tasks.workunit.client.1.smithi104.stdout:7/736: mknod d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/dd8/cee 0 2024-04-26T04:42:23.073 INFO:tasks.workunit.client.1.smithi104.stdout:8/731: getdents d0/d1c/d27 0 2024-04-26T04:42:23.075 INFO:tasks.workunit.client.1.smithi104.stdout:5/812: creat d0/d1/d4/d2a/d2b/d37/f111 x:0 0 0 2024-04-26T04:42:23.076 INFO:tasks.workunit.client.0.smithi090.stdout:8/535: write dc/d1f/f36 [325121,122538] 0 2024-04-26T04:42:23.079 INFO:tasks.workunit.client.1.smithi104.stdout:4/858: mknod d4/d7/d1d/d2e/d74/dbd/c12c 0 2024-04-26T04:42:23.081 INFO:tasks.workunit.client.1.smithi104.stdout:8/732: truncate d0/d1c/d27/d47/d67/d76/daf/fb2 1934596 0 2024-04-26T04:42:23.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/473: getdents d0/d4/d7/d4a/d75/d80 0 2024-04-26T04:42:23.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/474: chown d0/d4/d62/d78 79674345 1 2024-04-26T04:42:23.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/475: truncate d0/d4/d7/d17/d22/f74 608341 0 2024-04-26T04:42:23.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/476: write d0/d4/d7/d17/f29 [720868,130037] 0 2024-04-26T04:42:23.083 INFO:tasks.workunit.client.1.smithi104.stdout:5/813: mknod d0/d41/d4f/c112 0 2024-04-26T04:42:23.083 INFO:tasks.workunit.client.1.smithi104.stdout:5/814: fsync d0/d1/d1e/fb1 0 2024-04-26T04:42:23.084 INFO:tasks.workunit.client.0.smithi090.stdout:6/584: rmdir d9/dcb 0 2024-04-26T04:42:23.086 INFO:tasks.workunit.client.1.smithi104.stdout:2/977: getdents d6/d8/df6/dfe 0 2024-04-26T04:42:23.086 INFO:tasks.workunit.client.0.smithi090.stdout:8/536: mknod dc/d1f/d49/d3b/cbc 0 2024-04-26T04:42:23.089 INFO:tasks.workunit.client.0.smithi090.stdout:8/537: getdents dc/d1f/d49/d4f/da6 0 2024-04-26T04:42:23.089 INFO:tasks.workunit.client.0.smithi090.stdout:8/538: truncate dc/f30 4668479 0 2024-04-26T04:42:23.090 INFO:tasks.workunit.client.1.smithi104.stdout:7/737: read d2/d4/f6e [1536525,48233] 0 2024-04-26T04:42:23.091 INFO:tasks.workunit.client.1.smithi104.stdout:4/859: creat d4/d20/d55/d56/da3/dea/d11c/f12d x:0 0 0 2024-04-26T04:42:23.091 INFO:tasks.workunit.client.1.smithi104.stdout:4/860: chown d4/d7/d1d/d8e/cd9 464 1 2024-04-26T04:42:23.092 INFO:tasks.workunit.client.1.smithi104.stdout:8/733: creat d0/d1c/d27/d46/fe7 x:0 0 0 2024-04-26T04:42:23.094 INFO:tasks.workunit.client.1.smithi104.stdout:5/815: mkdir d0/d1/d4/d2a/d2b/d37/d48/dc1/d113 0 2024-04-26T04:42:23.094 INFO:tasks.workunit.client.1.smithi104.stdout:3/863: dread d5/db2/da3/fa4 [0,4194304] 0 2024-04-26T04:42:23.094 INFO:tasks.workunit.client.1.smithi104.stdout:3/864: chown d5/c5b 7 1 2024-04-26T04:42:23.095 INFO:tasks.workunit.client.0.smithi090.stdout:8/539: mknod dc/d1f/d49/d3b/d3d/cbd 0 2024-04-26T04:42:23.095 INFO:tasks.workunit.client.1.smithi104.stdout:7/738: link d2/d4/d1d/l5d d2/d4/d32/lef 0 2024-04-26T04:42:23.095 INFO:tasks.workunit.client.1.smithi104.stdout:7/739: chown d2/d4/d32/d35/d44/c59 1 1 2024-04-26T04:42:23.095 INFO:tasks.workunit.client.1.smithi104.stdout:7/740: fdatasync d2/d4/d32/d3d/d41/fd2 0 2024-04-26T04:42:23.096 INFO:tasks.workunit.client.1.smithi104.stdout:4/861: stat d4/d20/l2d 0 2024-04-26T04:42:23.097 INFO:tasks.workunit.client.0.smithi090.stdout:1/477: dread d0/d55/f67 [0,4194304] 0 2024-04-26T04:42:23.097 INFO:tasks.workunit.client.1.smithi104.stdout:1/869: dwrite d0/d6/d19/d21/d66/d20/d2a/f31 [4194304,4194304] 0 2024-04-26T04:42:23.098 INFO:tasks.workunit.client.1.smithi104.stdout:5/816: creat d0/d1/d4/d2a/d2b/d37/d48/dc1/dfd/f114 x:0 0 0 2024-04-26T04:42:23.099 INFO:tasks.workunit.client.1.smithi104.stdout:5/817: readlink d0/d1/d64/d26/l46 0 2024-04-26T04:42:23.099 INFO:tasks.workunit.client.1.smithi104.stdout:5/818: write d0/d1/d4/d90/d109/fea [165501,31371] 0 2024-04-26T04:42:23.099 INFO:tasks.workunit.client.0.smithi090.stdout:8/540: truncate dc/d1f/d49/d39/f81 220983 0 2024-04-26T04:42:23.100 INFO:tasks.workunit.client.1.smithi104.stdout:7/741: mkdir d2/d4/d1d/d34/db0/df0 0 2024-04-26T04:42:23.100 INFO:tasks.workunit.client.1.smithi104.stdout:7/742: dread - d2/fb8 zero size 2024-04-26T04:42:23.100 INFO:tasks.workunit.client.1.smithi104.stdout:7/743: readlink d2/d4/d32/l52 0 2024-04-26T04:42:23.100 INFO:tasks.workunit.client.1.smithi104.stdout:7/744: truncate d2/d4/f74 256077 0 2024-04-26T04:42:23.100 INFO:tasks.workunit.client.1.smithi104.stdout:7/745: dread - d2/fb8 zero size 2024-04-26T04:42:23.100 INFO:tasks.workunit.client.1.smithi104.stdout:7/746: write d2/d4/d32/d35/dbe/feb [703927,18343] 0 2024-04-26T04:42:23.100 INFO:tasks.workunit.client.1.smithi104.stdout:7/747: fdatasync d2/d4/d32/d35/d44/d5b/fcd 0 2024-04-26T04:42:23.101 INFO:tasks.workunit.client.1.smithi104.stdout:7/748: creat d2/ff1 x:0 0 0 2024-04-26T04:42:23.101 INFO:tasks.workunit.client.0.smithi090.stdout:8/541: mknod dc/d1f/d49/d4f/d58/d95/cbe 0 2024-04-26T04:42:23.101 INFO:tasks.workunit.client.1.smithi104.stdout:4/862: creat d4/d7/d1d/d1e/d24/d59/f12e x:0 0 0 2024-04-26T04:42:23.101 INFO:tasks.workunit.client.1.smithi104.stdout:4/863: write d4/d7/d1d/d1e/d24/f68 [2133939,117717] 0 2024-04-26T04:42:23.102 INFO:tasks.workunit.client.1.smithi104.stdout:8/734: rmdir d0/d33/d51 39 2024-04-26T04:42:23.102 INFO:tasks.workunit.client.1.smithi104.stdout:2/978: write d6/d1b/d51/d6a/f6f [5818214,19992] 0 2024-04-26T04:42:23.103 INFO:tasks.workunit.client.0.smithi090.stdout:7/556: dwrite d0/db/d15/d29/f18 [0,4194304] 0 2024-04-26T04:42:23.103 INFO:tasks.workunit.client.1.smithi104.stdout:6/858: dwrite d0/d9/d13/d41/d96/ded/d10d/d71/f69 [0,4194304] 0 2024-04-26T04:42:23.106 INFO:tasks.workunit.client.0.smithi090.stdout:0/586: dwrite d0/d4b/d5d/d5c/d8a/f94 [0,4194304] 0 2024-04-26T04:42:23.108 INFO:tasks.workunit.client.1.smithi104.stdout:1/870: dread d0/d6/d2e/d32/d4c/db1/dc3/f108 [0,4194304] 0 2024-04-26T04:42:23.108 INFO:tasks.workunit.client.1.smithi104.stdout:1/871: read d0/d6/d2e/d32/d4c/fb0 [588520,22749] 0 2024-04-26T04:42:23.108 INFO:tasks.workunit.client.1.smithi104.stdout:6/859: dread d0/f9c [0,4194304] 0 2024-04-26T04:42:23.110 INFO:tasks.workunit.client.1.smithi104.stdout:7/749: rmdir d2/d4/d1d/d34/db0/de4/de5 0 2024-04-26T04:42:23.110 INFO:tasks.workunit.client.1.smithi104.stdout:3/865: read d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/fb0 [3078170,75378] 0 2024-04-26T04:42:23.110 INFO:tasks.workunit.client.1.smithi104.stdout:3/866: chown d5/d9/ff 3 1 2024-04-26T04:42:23.111 INFO:tasks.workunit.client.1.smithi104.stdout:7/750: getdents d2/d4/d32/d3d/d47/db6 0 2024-04-26T04:42:23.111 INFO:tasks.workunit.client.1.smithi104.stdout:7/751: write d2/d4/d32/d3d/d41/d5f/d6d/faa [5084455,79498] 0 2024-04-26T04:42:23.116 INFO:tasks.workunit.client.1.smithi104.stdout:4/864: dread d4/d20/d55/d56/da3/fc4 [0,4194304] 0 2024-04-26T04:42:23.116 INFO:tasks.workunit.client.1.smithi104.stdout:4/865: stat d4/d7/d1d/d1e/d24/d59/l62 0 2024-04-26T04:42:23.116 INFO:tasks.workunit.client.1.smithi104.stdout:7/752: symlink d2/d4/d5a/da0/lf2 0 2024-04-26T04:42:23.116 INFO:tasks.workunit.client.1.smithi104.stdout:7/753: stat d2/d4/d1d/d34/fc6 0 2024-04-26T04:42:23.117 INFO:tasks.workunit.client.0.smithi090.stdout:7/557: unlink d0/db/l46 0 2024-04-26T04:42:23.117 INFO:tasks.workunit.client.0.smithi090.stdout:7/558: read - d0/d1/d4/d98/f4c zero size 2024-04-26T04:42:23.118 INFO:tasks.workunit.client.1.smithi104.stdout:8/735: rename d0/c41 to d0/d9a/ce8 0 2024-04-26T04:42:23.119 INFO:tasks.workunit.client.0.smithi090.stdout:8/542: link dc/d1f/d49/d4f/f7f dc/d1f/d49/d3b/fbf 0 2024-04-26T04:42:23.119 INFO:tasks.workunit.client.1.smithi104.stdout:7/754: mknod d2/d4/d32/d60/cf3 0 2024-04-26T04:42:23.120 INFO:tasks.workunit.client.0.smithi090.stdout:8/543: dread dc/d5a/d5d/f66 [0,4194304] 0 2024-04-26T04:42:23.120 INFO:tasks.workunit.client.0.smithi090.stdout:8/544: stat dc/d1f/d49/d39/d55 0 2024-04-26T04:42:23.121 INFO:tasks.workunit.client.0.smithi090.stdout:0/587: mkdir d0/d4b/d5d/d48/dd2 0 2024-04-26T04:42:23.122 INFO:tasks.workunit.client.1.smithi104.stdout:6/860: dread d0/d9/d13/d41/d96/ded/d10d/d71/fa3 [0,4194304] 0 2024-04-26T04:42:23.122 INFO:tasks.workunit.client.1.smithi104.stdout:8/736: dread d0/d33/d35/d6d/f9b [0,4194304] 0 2024-04-26T04:42:23.122 INFO:tasks.workunit.client.1.smithi104.stdout:8/737: creat d0/d1c/d27/d47/d67/d76/daf/fe9 x:0 0 0 2024-04-26T04:42:23.122 INFO:tasks.workunit.client.1.smithi104.stdout:8/738: write d0/d1c/d27/d47/d67/d76/d7f/fd8 [79383,5391] 0 2024-04-26T04:42:23.123 INFO:tasks.workunit.client.1.smithi104.stdout:2/979: rename d6/de/d1d/dcf/f12e to d6/de/d1d/dcf/d128/f142 0 2024-04-26T04:42:23.123 INFO:tasks.workunit.client.1.smithi104.stdout:2/980: write d6/de/d8e/f4c [888608,18777] 0 2024-04-26T04:42:23.123 INFO:tasks.workunit.client.1.smithi104.stdout:2/981: dread - d6/d1b/d68/d81/fc6 zero size 2024-04-26T04:42:23.123 INFO:tasks.workunit.client.1.smithi104.stdout:2/982: truncate d6/d1b/d51/d6a/f13b 259318 0 2024-04-26T04:42:23.123 INFO:tasks.workunit.client.1.smithi104.stdout:1/872: write d0/d6/d2e/d32/f62 [2210564,31212] 0 2024-04-26T04:42:23.124 INFO:tasks.workunit.client.0.smithi090.stdout:7/559: unlink d0/db/df/d1c/c1d 0 2024-04-26T04:42:23.125 INFO:tasks.workunit.client.0.smithi090.stdout:8/545: stat dc/d34/l51 0 2024-04-26T04:42:23.125 INFO:tasks.workunit.client.0.smithi090.stdout:6/585: dwrite d9/d1c/dc5/fce [0,4194304] 0 2024-04-26T04:42:23.127 INFO:tasks.workunit.client.1.smithi104.stdout:6/861: creat d0/d9/d25/d10c/f116 x:0 0 0 2024-04-26T04:42:23.127 INFO:tasks.workunit.client.0.smithi090.stdout:0/588: creat d0/d4b/d5d/d5c/d8a/d45/d53/dc5/fd3 x:0 0 0 2024-04-26T04:42:23.129 INFO:tasks.workunit.client.0.smithi090.stdout:8/546: creat dc/d5a/d5d/fc0 x:0 0 0 2024-04-26T04:42:23.130 INFO:tasks.workunit.client.0.smithi090.stdout:1/478: dwrite d0/d4/d7/d49/f6f [0,4194304] 0 2024-04-26T04:42:23.131 INFO:tasks.workunit.client.1.smithi104.stdout:4/866: dread d4/d7/d1d/d1e/d8f/db6/f88 [0,4194304] 0 2024-04-26T04:42:23.132 INFO:tasks.workunit.client.0.smithi090.stdout:3/453: dwrite d1/d6/d1b/f35 [0,4194304] 0 2024-04-26T04:42:23.132 INFO:tasks.workunit.client.0.smithi090.stdout:3/454: creat d1/d5c/f99 x:0 0 0 2024-04-26T04:42:23.133 INFO:tasks.workunit.client.0.smithi090.stdout:6/586: readlink d9/d1c/d4d/d6d/d92/l2e 0 2024-04-26T04:42:23.135 INFO:tasks.workunit.client.1.smithi104.stdout:3/867: rename d5/db2/d72 to d5/d9/d12f 0 2024-04-26T04:42:23.135 INFO:tasks.workunit.client.1.smithi104.stdout:3/868: fsync d5/db2/da3/dd0/ddb/f21 0 2024-04-26T04:42:23.136 INFO:tasks.workunit.client.0.smithi090.stdout:8/547: link dc/d34/l91 dc/d34/d87/lc1 0 2024-04-26T04:42:23.136 INFO:tasks.workunit.client.0.smithi090.stdout:8/548: write dc/d1f/f21 [335134,57837] 0 2024-04-26T04:42:23.136 INFO:tasks.workunit.client.0.smithi090.stdout:8/549: fdatasync dc/d71/fae 0 2024-04-26T04:42:23.137 INFO:tasks.workunit.client.1.smithi104.stdout:1/873: mkdir d0/d6/d19/d21/d66/d29/d11c 0 2024-04-26T04:42:23.137 INFO:tasks.workunit.client.1.smithi104.stdout:1/874: fsync d0/d6/d19/d21/d66/de4/fcd 0 2024-04-26T04:42:23.137 INFO:tasks.workunit.client.1.smithi104.stdout:6/862: readlink d0/d9/d13/d41/d96/ded/d10d/d47/l6d 0 2024-04-26T04:42:23.138 INFO:tasks.workunit.client.0.smithi090.stdout:1/479: symlink d0/d4/d62/d78/l95 0 2024-04-26T04:42:23.141 INFO:tasks.workunit.client.0.smithi090.stdout:0/589: write d0/d4b/d5d/d5c/d8a/d45/fb5 [3544401,39477] 0 2024-04-26T04:42:23.141 INFO:tasks.workunit.client.0.smithi090.stdout:0/590: chown d0/d4b/d5d/d5c/d8a/d45/d53/d8e/c9f 85596286 1 2024-04-26T04:42:23.141 INFO:tasks.workunit.client.0.smithi090.stdout:8/550: write dc/d1f/d49/d4f/d58/d95/fa7 [224093,130801] 0 2024-04-26T04:42:23.142 INFO:tasks.workunit.client.0.smithi090.stdout:8/551: chown dc/d71/fae 6 1 2024-04-26T04:42:23.142 INFO:tasks.workunit.client.0.smithi090.stdout:3/455: symlink d1/d17/d36/d2a/d3a/d5d/d82/l9a 0 2024-04-26T04:42:23.143 INFO:tasks.workunit.client.0.smithi090.stdout:3/456: chown d1/d17/d36/l84 0 1 2024-04-26T04:42:23.143 INFO:tasks.workunit.client.0.smithi090.stdout:3/457: dread - d1/d17/d36/d33/f89 zero size 2024-04-26T04:42:23.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/591: stat d0/d4b/d5d/d5c/d8a/d45/l5a 0 2024-04-26T04:42:23.146 INFO:tasks.workunit.client.0.smithi090.stdout:0/592: chown d0/d4b/fad 1037262 1 2024-04-26T04:42:23.148 INFO:tasks.workunit.client.0.smithi090.stdout:8/552: link dc/c8f dc/d5a/cc2 0 2024-04-26T04:42:23.148 INFO:tasks.workunit.client.0.smithi090.stdout:6/587: dread d9/d1c/d4d/d6d/d66/f75 [0,4194304] 0 2024-04-26T04:42:23.150 INFO:tasks.workunit.client.1.smithi104.stdout:3/869: dread d5/db2/da3/dd0/ddb/d69/fe7 [0,4194304] 0 2024-04-26T04:42:23.150 INFO:tasks.workunit.client.1.smithi104.stdout:3/870: chown d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/dd6 0 1 2024-04-26T04:42:23.150 INFO:tasks.workunit.client.0.smithi090.stdout:0/593: unlink d0/f4f 0 2024-04-26T04:42:23.151 INFO:tasks.workunit.client.0.smithi090.stdout:8/553: rmdir dc/d1f/d49/d39/d99 39 2024-04-26T04:42:23.152 INFO:tasks.workunit.client.0.smithi090.stdout:0/594: creat d0/df/fd4 x:0 0 0 2024-04-26T04:42:23.154 INFO:tasks.workunit.client.1.smithi104.stdout:7/755: dwrite d2/d4/f74 [0,4194304] 0 2024-04-26T04:42:23.154 INFO:tasks.workunit.client.1.smithi104.stdout:2/983: dwrite d6/d8/f46 [0,4194304] 0 2024-04-26T04:42:23.154 INFO:tasks.workunit.client.1.smithi104.stdout:2/984: chown d6/d1b/d51/d6a/f7d 63064 1 2024-04-26T04:42:23.154 INFO:tasks.workunit.client.0.smithi090.stdout:0/595: creat d0/d4b/d5d/d5c/d8a/d45/d93/fd5 x:0 0 0 2024-04-26T04:42:23.156 INFO:tasks.workunit.client.1.smithi104.stdout:2/985: write d6/de/d8e/f4c [488934,35295] 0 2024-04-26T04:42:23.156 INFO:tasks.workunit.client.1.smithi104.stdout:2/986: chown d6/de/c5c 1296226293 1 2024-04-26T04:42:23.157 INFO:tasks.workunit.client.1.smithi104.stdout:4/867: symlink d4/d7/d1d/d1e/d8f/db6/d7d/l12f 0 2024-04-26T04:42:23.158 INFO:tasks.workunit.client.1.smithi104.stdout:1/875: creat d0/d6/d19/d21/d66/de4/f11d x:0 0 0 2024-04-26T04:42:23.159 INFO:tasks.workunit.client.1.smithi104.stdout:1/876: truncate d0/d6/d19/d21/d66/d20/d2a/d56/d8c/fe1 712823 0 2024-04-26T04:42:23.160 INFO:tasks.workunit.client.1.smithi104.stdout:6/863: rmdir d0/dd8 39 2024-04-26T04:42:23.161 INFO:tasks.workunit.client.0.smithi090.stdout:6/588: dread d9/d21/d3b/f46 [0,4194304] 0 2024-04-26T04:42:23.163 INFO:tasks.workunit.client.1.smithi104.stdout:3/871: symlink d5/db2/d46/d95/l130 0 2024-04-26T04:42:23.163 INFO:tasks.workunit.client.0.smithi090.stdout:3/458: dread d1/d17/f1f [0,4194304] 0 2024-04-26T04:42:23.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/476: sync 2024-04-26T04:42:23.164 INFO:tasks.workunit.client.0.smithi090.stdout:2/531: sync 2024-04-26T04:42:23.164 INFO:tasks.workunit.client.0.smithi090.stdout:9/410: sync 2024-04-26T04:42:23.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/477: write d3/d13/f81 [616522,124288] 0 2024-04-26T04:42:23.165 INFO:tasks.workunit.client.1.smithi104.stdout:3/872: read d5/db2/da3/dd0/ddb/d32/d48/f5f [150339,31296] 0 2024-04-26T04:42:23.166 INFO:tasks.workunit.client.0.smithi090.stdout:4/478: rmdir d3/db/d12/d20/d24/d30 39 2024-04-26T04:42:23.166 INFO:tasks.workunit.client.0.smithi090.stdout:4/479: stat d3/db/d12/d20/d2d/d93 0 2024-04-26T04:42:23.167 INFO:tasks.workunit.client.1.smithi104.stdout:7/756: write d2/d4/d32/d35/f4d [969242,78982] 0 2024-04-26T04:42:23.167 INFO:tasks.workunit.client.0.smithi090.stdout:6/589: dread d9/f68 [0,4194304] 0 2024-04-26T04:42:23.168 INFO:tasks.workunit.client.0.smithi090.stdout:1/480: dwrite d0/d4/d7/d4b/d53/f6a [0,4194304] 0 2024-04-26T04:42:23.169 INFO:tasks.workunit.client.0.smithi090.stdout:9/411: rename db/dd/d18/c53 to db/dd/d18/d38/d39/c8f 0 2024-04-26T04:42:23.171 INFO:tasks.workunit.client.1.smithi104.stdout:8/739: getdents d0 0 2024-04-26T04:42:23.171 INFO:tasks.workunit.client.1.smithi104.stdout:8/740: write d0/d1c/d27/d47/d67/d76/fbf [421005,118655] 0 2024-04-26T04:42:23.171 INFO:tasks.workunit.client.1.smithi104.stdout:8/741: dread - d0/d1c/d27/f86 zero size 2024-04-26T04:42:23.173 INFO:tasks.workunit.client.0.smithi090.stdout:6/590: link d9/d1c/d31/f3a d9/d1c/d31/fd1 0 2024-04-26T04:42:23.173 INFO:tasks.workunit.client.0.smithi090.stdout:6/591: chown d9/d21/f70 99107 1 2024-04-26T04:42:23.176 INFO:tasks.workunit.client.1.smithi104.stdout:2/987: mknod d6/d1b/d68/d81/dce/c143 0 2024-04-26T04:42:23.179 INFO:tasks.workunit.client.0.smithi090.stdout:2/532: rename db/d11/d14/d25/d65/d95/c9f to db/d11/d14/d1b/d20/d4c/d98/db4/cb6 0 2024-04-26T04:42:23.179 INFO:tasks.workunit.client.0.smithi090.stdout:2/533: readlink db/d11/d14/d25/d2d/l64 0 2024-04-26T04:42:23.179 INFO:tasks.workunit.client.0.smithi090.stdout:6/592: chown d9/d1c/d4d/d6d/d92/c4f 257457342 1 2024-04-26T04:42:23.179 INFO:tasks.workunit.client.1.smithi104.stdout:4/868: unlink d4/d7/d1d/d1e/d8f/db6/f88 0 2024-04-26T04:42:23.181 INFO:tasks.workunit.client.1.smithi104.stdout:1/877: symlink d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/l11e 0 2024-04-26T04:42:23.181 INFO:tasks.workunit.client.0.smithi090.stdout:6/593: readlink d9/d1c/d4d/d6d/d92/l27 0 2024-04-26T04:42:23.182 INFO:tasks.workunit.client.1.smithi104.stdout:7/757: write d2/d4/fe3 [834700,92515] 0 2024-04-26T04:42:23.184 INFO:tasks.workunit.client.0.smithi090.stdout:6/594: symlink d9/d1c/d4d/d58/dab/ld2 0 2024-04-26T04:42:23.184 INFO:tasks.workunit.client.0.smithi090.stdout:6/595: chown d9/d77/dc0 4132 1 2024-04-26T04:42:23.185 INFO:tasks.workunit.client.1.smithi104.stdout:8/742: rmdir d0 39 2024-04-26T04:42:23.185 INFO:tasks.workunit.client.1.smithi104.stdout:8/743: creat d0/d1c/d27/d47/d67/d76/fea x:0 0 0 2024-04-26T04:42:23.186 INFO:tasks.workunit.client.1.smithi104.stdout:8/744: creat d0/d1c/d27/d47/d67/d76/daf/feb x:0 0 0 2024-04-26T04:42:23.186 INFO:tasks.workunit.client.1.smithi104.stdout:8/745: dread - d0/d1c/d27/d46/fe7 zero size 2024-04-26T04:42:23.186 INFO:tasks.workunit.client.1.smithi104.stdout:8/746: read - d0/d33/d35/fc8 zero size 2024-04-26T04:42:23.186 INFO:tasks.workunit.client.0.smithi090.stdout:6/596: rmdir d9/d77/db0 39 2024-04-26T04:42:23.186 INFO:tasks.workunit.client.0.smithi090.stdout:6/597: chown d9/d1c/da2 126 1 2024-04-26T04:42:23.188 INFO:tasks.workunit.client.0.smithi090.stdout:4/480: dread d3/db/d12/d20/d2d/d58/f63 [0,4194304] 0 2024-04-26T04:42:23.188 INFO:tasks.workunit.client.0.smithi090.stdout:4/481: chown d3/db/d12/d20/d24/d43/d72/d7a 97998 1 2024-04-26T04:42:23.189 INFO:tasks.workunit.client.1.smithi104.stdout:2/988: rename d6/de/cba to d6/d1b/d68/d81/dfc/c144 0 2024-04-26T04:42:23.191 INFO:tasks.workunit.client.0.smithi090.stdout:0/596: dread d0/d4b/fad [0,4194304] 0 2024-04-26T04:42:23.192 INFO:tasks.workunit.client.1.smithi104.stdout:4/869: mknod d4/c130 0 2024-04-26T04:42:23.192 INFO:tasks.workunit.client.0.smithi090.stdout:9/412: write db/dd/d18/d38/d39/f7e [2073164,93406] 0 2024-04-26T04:42:23.195 INFO:tasks.workunit.client.1.smithi104.stdout:1/878: mkdir d0/d6/d2e/d32/d11f 0 2024-04-26T04:42:23.197 INFO:tasks.workunit.client.0.smithi090.stdout:0/597: write d0/d4b/fad [696355,5173] 0 2024-04-26T04:42:23.197 INFO:tasks.workunit.client.0.smithi090.stdout:0/598: fsync d0/d4b/d5d/d5c/d8a/f3e 0 2024-04-26T04:42:23.197 INFO:tasks.workunit.client.0.smithi090.stdout:0/599: readlink d0/de/lbf 0 2024-04-26T04:42:23.197 INFO:tasks.workunit.client.0.smithi090.stdout:0/600: write d0/d4b/d5d/d5c/d8a/d45/d53/fba [169875,95223] 0 2024-04-26T04:42:23.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/534: dread db/d11/d14/d27/d2f/d70/d71/f83 [0,4194304] 0 2024-04-26T04:42:23.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/535: chown db/d11/d14/d27/d2f/d70 15 1 2024-04-26T04:42:23.201 INFO:tasks.workunit.client.0.smithi090.stdout:2/536: getdents db/d11/d14/d27/d2f 0 2024-04-26T04:42:23.202 INFO:tasks.workunit.client.1.smithi104.stdout:7/758: creat d2/d4/d1d/d34/ff4 x:0 0 0 2024-04-26T04:42:23.203 INFO:tasks.workunit.client.0.smithi090.stdout:2/537: unlink db/ce 0 2024-04-26T04:42:23.205 INFO:tasks.workunit.client.0.smithi090.stdout:2/538: link db/d11/d14/d63/l7e db/d11/d14/d27/d2f/d70/d71/lb7 0 2024-04-26T04:42:23.206 INFO:tasks.workunit.client.1.smithi104.stdout:8/747: symlink d0/d1c/d27/d47/d67/d76/d7f/lec 0 2024-04-26T04:42:23.207 INFO:tasks.workunit.client.0.smithi090.stdout:2/539: unlink l5 0 2024-04-26T04:42:23.207 INFO:tasks.workunit.client.0.smithi090.stdout:2/540: truncate db/d11/d14/d27/d28/fa5 5055603 0 2024-04-26T04:42:23.207 INFO:tasks.workunit.client.0.smithi090.stdout:2/541: readlink db/d11/d14/d1b/d20/la4 0 2024-04-26T04:42:23.207 INFO:tasks.workunit.client.1.smithi104.stdout:2/989: creat d6/d1b/d51/db8/f145 x:0 0 0 2024-04-26T04:42:23.208 INFO:tasks.workunit.client.1.smithi104.stdout:2/990: fsync d6/d1b/f31 0 2024-04-26T04:42:23.208 INFO:tasks.workunit.client.1.smithi104.stdout:4/870: getdents d4/db/d6b/df0 0 2024-04-26T04:42:23.208 INFO:tasks.workunit.client.0.smithi090.stdout:9/413: write db/dd/d44/f2c [3608967,92335] 0 2024-04-26T04:42:23.208 INFO:tasks.workunit.client.0.smithi090.stdout:2/542: creat db/d11/d14/d1b/d20/fb8 x:0 0 0 2024-04-26T04:42:23.209 INFO:tasks.workunit.client.0.smithi090.stdout:9/414: creat db/dd/d18/d38/f90 x:0 0 0 2024-04-26T04:42:23.210 INFO:tasks.workunit.client.0.smithi090.stdout:2/543: mkdir db/d11/d14/d25/d86/db9 0 2024-04-26T04:42:23.210 INFO:tasks.workunit.client.0.smithi090.stdout:2/544: read db/dc/f1f [679667,22008] 0 2024-04-26T04:42:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 -- 172.21.15.90:0/3827140846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc100350 msgr2=0x7f8dbc1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 --2- 172.21.15.90:0/3827140846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc100350 0x7f8dbc1007a0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f8dac009910 tx=0x7f8dac009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 -- 172.21.15.90:0/3827140846 shutdown_connections 2024-04-26T04:42:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 --2- 172.21.15.90:0/3827140846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc100350 0x7f8dbc1007a0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 --2- 172.21.15.90:0/3827140846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0ff150 0x7f8dbc0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 -- 172.21.15.90:0/3827140846 >> 172.21.15.90:0/3827140846 conn(0x7f8dbc0fa700 msgr2=0x7f8dbc0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:23.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 -- 172.21.15.90:0/3827140846 shutdown_connections 2024-04-26T04:42:23.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.210+0000 7f8dc4126700 1 -- 172.21.15.90:0/3827140846 wait complete. 2024-04-26T04:42:23.212 INFO:tasks.workunit.client.1.smithi104.stdout:1/879: rename d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/dec/lee to d0/d6/d2e/d32/d11f/l120 0 2024-04-26T04:42:23.212 INFO:tasks.workunit.client.1.smithi104.stdout:1/880: chown d0/d6/d19/d21/d66/d29/df7/l114 1018631 1 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc4126700 1 Processor -- start 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc4126700 1 -- start start 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc4126700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ff150 0x7f8dbc197cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc4126700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc100350 0x7f8dbc1981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc4126700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc1987f0 con 0x7f8dbc0ff150 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc4126700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc198960 con 0x7f8dbc100350 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc1ec2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ff150 0x7f8dbc197cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc1ec2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ff150 0x7f8dbc197cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52738/0 (socket says 172.21.15.90:52738) 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.211+0000 7f8dc1ec2700 1 -- 172.21.15.90:0/2158609233 learned_addr learned my addr 172.21.15.90:0/2158609233 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.212+0000 7f8dc16c1700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc100350 0x7f8dbc1981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.212+0000 7f8dc1ec2700 1 -- 172.21.15.90:0/2158609233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc100350 msgr2=0x7f8dbc1981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.213+0000 7f8dc1ec2700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc100350 0x7f8dbc1981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.213+0000 7f8dc1ec2700 1 -- 172.21.15.90:0/2158609233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dac0095f0 con 0x7f8dbc0ff150 2024-04-26T04:42:23.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.213+0000 7f8dc1ec2700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ff150 0x7f8dbc197cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8db800d900 tx=0x7f8db800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:23.215 INFO:tasks.workunit.client.1.smithi104.stdout:7/759: getdents d2/d4/d32/d35/d44 0 2024-04-26T04:42:23.216 INFO:tasks.workunit.client.1.smithi104.stdout:8/748: mknod d0/d1c/d27/d47/d67/d76/daf/dd4/ced 0 2024-04-26T04:42:23.216 INFO:tasks.workunit.client.0.smithi090.stdout:0/601: dread d0/de/f78 [0,4194304] 0 2024-04-26T04:42:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.214+0000 7f8db2ffd700 1 -- 172.21.15.90:0/2158609233 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db80041f0 con 0x7f8dbc0ff150 2024-04-26T04:42:23.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.214+0000 7f8db2ffd700 1 -- 172.21.15.90:0/2158609233 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8db8004350 con 0x7f8dbc0ff150 2024-04-26T04:42:23.217 INFO:tasks.workunit.client.0.smithi090.stdout:0/602: write d0/d4b/d5d/d5c/d8a/d45/d53/fba [5341,67714] 0 2024-04-26T04:42:23.217 INFO:tasks.workunit.client.0.smithi090.stdout:0/603: write d0/d4b/d5d/d5c/d8a/f17 [1039450,74284] 0 2024-04-26T04:42:23.217 INFO:tasks.workunit.client.0.smithi090.stdout:0/604: chown d0/d4b/d5d/d5c/d8a/d64/c75 13359 1 2024-04-26T04:42:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.214+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc19d3f0 con 0x7f8dbc0ff150 2024-04-26T04:42:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.214+0000 7f8db2ffd700 1 -- 172.21.15.90:0/2158609233 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db8003d20 con 0x7f8dbc0ff150 2024-04-26T04:42:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.214+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc19d940 con 0x7f8dbc0ff150 2024-04-26T04:42:23.217 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.215+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc107ca0 con 0x7f8dbc0ff150 2024-04-26T04:42:23.218 INFO:tasks.workunit.client.1.smithi104.stdout:4/871: unlink d4/d7/d1d/d1e/d15/c54 0 2024-04-26T04:42:23.218 INFO:tasks.workunit.client.1.smithi104.stdout:0/702: sync 2024-04-26T04:42:23.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.217+0000 7f8db2ffd700 1 -- 172.21.15.90:0/2158609233 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f8db8022020 con 0x7f8dbc0ff150 2024-04-26T04:42:23.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.217+0000 7f8db2ffd700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8da806c680 0x7f8da806eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:23.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.217+0000 7f8db2ffd700 1 -- 172.21.15.90:0/2158609233 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f8db808b260 con 0x7f8dbc0ff150 2024-04-26T04:42:23.219 INFO:tasks.workunit.client.0.smithi090.stdout:8/554: dwrite dc/d1f/d49/d4f/d58/f6f [0,4194304] 0 2024-04-26T04:42:23.219 INFO:tasks.workunit.client.0.smithi090.stdout:1/481: dwrite d0/d4/d7/d49/f6f [4194304,4194304] 0 2024-04-26T04:42:23.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.218+0000 7f8dc16c1700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8da806c680 0x7f8da806eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:23.220 INFO:tasks.workunit.client.0.smithi090.stdout:8/555: write dc/d1f/f35 [1240097,118717] 0 2024-04-26T04:42:23.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.219+0000 7f8dc16c1700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8da806c680 0x7f8da806eb30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1991c0 tx=0x7f8dac00b540 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:23.221 INFO:tasks.workunit.client.0.smithi090.stdout:9/415: dread db/f1b [0,4194304] 0 2024-04-26T04:42:23.221 INFO:tasks.workunit.client.1.smithi104.stdout:1/881: rename d0/d6/d19/d21/d66/d20/d2a/d56/de5 to d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/dec/d121 0 2024-04-26T04:42:23.221 INFO:tasks.workunit.client.1.smithi104.stdout:1/882: fsync d0/d6/d2e/f84 0 2024-04-26T04:42:23.221 INFO:tasks.workunit.client.1.smithi104.stdout:1/883: truncate d0/d6/d9e/d10c/da5/fd8 35182 0 2024-04-26T04:42:23.222 INFO:tasks.workunit.client.1.smithi104.stdout:1/884: write d0/d6/d9e/d10c/fd7 [1372626,66873] 0 2024-04-26T04:42:23.222 INFO:tasks.workunit.client.1.smithi104.stdout:3/873: dwrite d5/db2/da3/dd0/ddb/d69/f3e [0,4194304] 0 2024-04-26T04:42:23.222 INFO:tasks.workunit.client.1.smithi104.stdout:7/760: getdents d2/d4/d32/d35 0 2024-04-26T04:42:23.222 INFO:tasks.workunit.client.1.smithi104.stdout:7/761: chown d2/d4/d32/d3d/d41/d7e/fd7 30 1 2024-04-26T04:42:23.222 INFO:tasks.workunit.client.1.smithi104.stdout:7/762: dread - d2/d4/d32/d3d/d41/d5f/d6d/fe6 zero size 2024-04-26T04:42:23.222 INFO:tasks.workunit.client.0.smithi090.stdout:1/482: creat d0/d4/d7/d4a/f96 x:0 0 0 2024-04-26T04:42:23.222 INFO:tasks.workunit.client.0.smithi090.stdout:1/483: chown d0/d4/d7/d17/d22/f68 889 1 2024-04-26T04:42:23.224 INFO:tasks.workunit.client.1.smithi104.stdout:8/749: link d0/d33/d51/c8a d0/d33/da6/de0/cee 0 2024-04-26T04:42:23.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.224+0000 7f8db2ffd700 1 -- 172.21.15.90:0/2158609233 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f8db8055a00 con 0x7f8dbc0ff150 2024-04-26T04:42:23.226 INFO:tasks.workunit.client.0.smithi090.stdout:6/598: dwrite d9/d21/d3b/f51 [0,4194304] 0 2024-04-26T04:42:23.226 INFO:tasks.workunit.client.0.smithi090.stdout:8/556: rename dc/d1f/d49/d4f/da6/caa to dc/d1f/d49/d4f/cc3 0 2024-04-26T04:42:23.226 INFO:tasks.workunit.client.0.smithi090.stdout:2/545: dread db/d11/d14/d25/d2d/f6a [0,4194304] 0 2024-04-26T04:42:23.230 INFO:tasks.workunit.client.0.smithi090.stdout:3/459: dwrite d1/f27 [0,4194304] 0 2024-04-26T04:42:23.231 INFO:tasks.workunit.client.1.smithi104.stdout:4/872: dread d4/d7/d1d/d1e/d24/d59/fbc [0,4194304] 0 2024-04-26T04:42:23.231 INFO:tasks.workunit.client.1.smithi104.stdout:3/874: creat d5/db2/da3/dd0/ddb/d32/d47/d11a/f131 x:0 0 0 2024-04-26T04:42:23.231 INFO:tasks.workunit.client.1.smithi104.stdout:3/875: fsync d5/d9/d80/dbc/fcb 0 2024-04-26T04:42:23.231 INFO:tasks.workunit.client.1.smithi104.stdout:4/873: getdents d4/d7/d1d/d1e/d15/d113 0 2024-04-26T04:42:23.231 INFO:tasks.workunit.client.1.smithi104.stdout:4/874: fsync d4/d20/d55/d56/da3/faf 0 2024-04-26T04:42:23.232 INFO:tasks.workunit.client.1.smithi104.stdout:7/763: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d89/df5 0 2024-04-26T04:42:23.234 INFO:tasks.workunit.client.0.smithi090.stdout:9/416: unlink db/dd/d18/d38/d39/f3b 0 2024-04-26T04:42:23.234 INFO:tasks.workunit.client.0.smithi090.stdout:1/484: mkdir d0/d4/d7/d4a/d75/d80/d58/d97 0 2024-04-26T04:42:23.234 INFO:tasks.workunit.client.0.smithi090.stdout:9/417: fdatasync db/dd/d18/d38/f42 0 2024-04-26T04:42:23.234 INFO:tasks.workunit.client.0.smithi090.stdout:1/485: truncate d0/d55/f83 311472 0 2024-04-26T04:42:23.236 INFO:tasks.workunit.client.0.smithi090.stdout:8/557: unlink dc/d1f/d49/l68 0 2024-04-26T04:42:23.236 INFO:tasks.workunit.client.0.smithi090.stdout:8/558: fdatasync dc/d34/d4d/f9e 0 2024-04-26T04:42:23.236 INFO:tasks.workunit.client.0.smithi090.stdout:8/559: chown dc/c29 134 1 2024-04-26T04:42:23.237 INFO:tasks.workunit.client.1.smithi104.stdout:8/750: unlink d0/d1c/l64 0 2024-04-26T04:42:23.238 INFO:tasks.workunit.client.0.smithi090.stdout:1/486: mknod d0/d4/c98 0 2024-04-26T04:42:23.241 INFO:tasks.workunit.client.1.smithi104.stdout:7/764: truncate d2/d4/f6e 1054314 0 2024-04-26T04:42:23.246 INFO:tasks.workunit.client.0.smithi090.stdout:6/599: dread d9/d1c/f47 [0,4194304] 0 2024-04-26T04:42:23.246 INFO:tasks.workunit.client.0.smithi090.stdout:6/600: dread - d9/d21/f70 zero size 2024-04-26T04:42:23.246 INFO:tasks.workunit.client.1.smithi104.stdout:8/751: mkdir d0/d33/d35/d6d/def 0 2024-04-26T04:42:23.246 INFO:tasks.workunit.client.1.smithi104.stdout:7/765: mknod d2/d4/d1d/d34/cf6 0 2024-04-26T04:42:23.248 INFO:tasks.workunit.client.0.smithi090.stdout:8/560: dread dc/d1f/d49/d3b/d3d/f65 [0,4194304] 0 2024-04-26T04:42:23.248 INFO:tasks.workunit.client.1.smithi104.stdout:8/752: truncate d0/d1c/d27/d47/d67/d76/d7f/fac 2805938 0 2024-04-26T04:42:23.249 INFO:tasks.workunit.client.1.smithi104.stdout:7/766: creat d2/d4/d32/ff7 x:0 0 0 2024-04-26T04:42:23.250 INFO:tasks.workunit.client.0.smithi090.stdout:8/561: truncate dc/f1e 170248 0 2024-04-26T04:42:23.251 INFO:tasks.workunit.client.0.smithi090.stdout:8/562: rmdir dc/d1f/d49 39 2024-04-26T04:42:23.251 INFO:tasks.workunit.client.0.smithi090.stdout:8/563: creat dc/d34/d4d/fc4 x:0 0 0 2024-04-26T04:42:23.252 INFO:tasks.workunit.client.0.smithi090.stdout:8/564: truncate dc/d1f/d49/d3b/d3d/f9c 831210 0 2024-04-26T04:42:23.254 INFO:tasks.workunit.client.1.smithi104.stdout:1/885: dread d0/d6/d9e/d10c/d35/f40 [0,4194304] 0 2024-04-26T04:42:23.256 INFO:tasks.workunit.client.0.smithi090.stdout:3/460: dread d1/d59/f64 [0,4194304] 0 2024-04-26T04:42:23.256 INFO:tasks.workunit.client.0.smithi090.stdout:3/461: chown d1/d6/d2c 20 1 2024-04-26T04:42:23.256 INFO:tasks.workunit.client.0.smithi090.stdout:3/462: creat d1/d6/d2c/f9b x:0 0 0 2024-04-26T04:42:23.257 INFO:tasks.workunit.client.0.smithi090.stdout:8/565: rename dc/d34/f40 to dc/d34/d87/fc5 0 2024-04-26T04:42:23.258 INFO:tasks.workunit.client.0.smithi090.stdout:3/463: symlink d1/d17/l9c 0 2024-04-26T04:42:23.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/464: write d1/d5c/f99 [1040926,83651] 0 2024-04-26T04:42:23.259 INFO:tasks.workunit.client.0.smithi090.stdout:3/465: stat d1/d17/d36 0 2024-04-26T04:42:23.265 INFO:tasks.workunit.client.1.smithi104.stdout:3/876: write d5/d9/d86/fb5 [677629,55750] 0 2024-04-26T04:42:23.266 INFO:tasks.workunit.client.0.smithi090.stdout:3/466: mkdir d1/d17/d61/d9d 0 2024-04-26T04:42:23.269 INFO:tasks.workunit.client.0.smithi090.stdout:3/467: creat d1/d17/d36/d33/d68/d71/f9e x:0 0 0 2024-04-26T04:42:23.271 INFO:tasks.workunit.client.1.smithi104.stdout:3/877: symlink d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/d118/l132 0 2024-04-26T04:42:23.272 INFO:tasks.workunit.client.1.smithi104.stdout:3/878: creat d5/d9/d86/d115/f133 x:0 0 0 2024-04-26T04:42:23.279 INFO:tasks.workunit.client.1.smithi104.stdout:7/767: write d2/d4/d1d/d34/f40 [4088712,82593] 0 2024-04-26T04:42:23.281 INFO:tasks.workunit.client.1.smithi104.stdout:0/703: dwrite d0/d2/d10/d84/f90 [0,4194304] 0 2024-04-26T04:42:23.281 INFO:tasks.workunit.client.1.smithi104.stdout:0/704: fdatasync d0/d2/d21/d49/f99 0 2024-04-26T04:42:23.281 INFO:tasks.workunit.client.1.smithi104.stdout:2/991: dwrite d6/d8/f12d [0,4194304] 0 2024-04-26T04:42:23.281 INFO:tasks.workunit.client.1.smithi104.stdout:2/992: truncate d6/d1b/d93/fe5 378467 0 2024-04-26T04:42:23.281 INFO:tasks.workunit.client.1.smithi104.stdout:7/768: symlink d2/d4/d32/d35/d44/lf8 0 2024-04-26T04:42:23.282 INFO:tasks.workunit.client.0.smithi090.stdout:0/605: dwrite d0/d4b/d5d/d5c/d8a/d45/d53/faf [0,4194304] 0 2024-04-26T04:42:23.282 INFO:tasks.workunit.client.0.smithi090.stdout:0/606: creat d0/d4b/d5d/d5c/d8a/d45/d53/dc5/fd6 x:0 0 0 2024-04-26T04:42:23.282 INFO:tasks.workunit.client.0.smithi090.stdout:0/607: truncate d0/d4b/d5d/d5c/d8a/d7c/fc9 764829 0 2024-04-26T04:42:23.285 INFO:tasks.workunit.client.1.smithi104.stdout:7/769: symlink d2/d4/d1d/d34/lf9 0 2024-04-26T04:42:23.286 INFO:tasks.workunit.client.0.smithi090.stdout:0/608: rename d0/d4b/d5d/d5c/dcd to d0/d4b/d5d/d5c/d8a/d45/d53/dc2/dd7 0 2024-04-26T04:42:23.287 INFO:tasks.workunit.client.0.smithi090.stdout:0/609: mknod d0/d4b/d5d/d5c/d8a/d45/d53/dc2/cd8 0 2024-04-26T04:42:23.287 INFO:tasks.workunit.client.1.smithi104.stdout:2/993: unlink d6/d1b/d75/cbe 0 2024-04-26T04:42:23.287 INFO:tasks.workunit.client.1.smithi104.stdout:2/994: fsync d6/d8/d3e/f7a 0 2024-04-26T04:42:23.289 INFO:tasks.workunit.client.0.smithi090.stdout:0/610: dread d0/f7a [0,4194304] 0 2024-04-26T04:42:23.289 INFO:tasks.workunit.client.0.smithi090.stdout:0/611: readlink d0/df/l63 0 2024-04-26T04:42:23.292 INFO:tasks.workunit.client.1.smithi104.stdout:3/879: dread d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/faa [0,4194304] 0 2024-04-26T04:42:23.292 INFO:tasks.workunit.client.0.smithi090.stdout:0/612: creat d0/d4b/d5d/fd9 x:0 0 0 2024-04-26T04:42:23.293 INFO:tasks.workunit.client.1.smithi104.stdout:0/705: dread d0/d2/d10/f64 [4194304,4194304] 0 2024-04-26T04:42:23.293 INFO:tasks.workunit.client.0.smithi090.stdout:0/613: rmdir d0/d4b/d5d/d5c/dc4 39 2024-04-26T04:42:23.294 INFO:tasks.workunit.client.0.smithi090.stdout:0/614: write d0/d4b/d5d/d5c/d8a/d45/d53/dc5/fd6 [168361,79903] 0 2024-04-26T04:42:23.294 INFO:tasks.workunit.client.0.smithi090.stdout:0/615: stat d0/d4b/d5d/d5c/d8a/d45/f96 0 2024-04-26T04:42:23.294 INFO:tasks.workunit.client.1.smithi104.stdout:3/880: mknod d5/db2/da3/dd0/ddb/d32/d47/d11a/c134 0 2024-04-26T04:42:23.295 INFO:tasks.workunit.client.0.smithi090.stdout:0/616: creat d0/d4b/d5d/d5c/d8a/d45/d53/dc2/dd7/fda x:0 0 0 2024-04-26T04:42:23.295 INFO:tasks.workunit.client.0.smithi090.stdout:0/617: fdatasync d0/d4b/d5d/d5c/d8a/f44 0 2024-04-26T04:42:23.297 INFO:tasks.workunit.client.1.smithi104.stdout:3/881: rename d5/db2/da3/dd0/ddb/d32/dd9/f110 to d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/f135 0 2024-04-26T04:42:23.297 INFO:tasks.workunit.client.1.smithi104.stdout:3/882: chown d5/d9/d80/dbc/fcb 1 1 2024-04-26T04:42:23.297 INFO:tasks.workunit.client.1.smithi104.stdout:3/883: fsync d5/db2/fed 0 2024-04-26T04:42:23.297 INFO:tasks.workunit.client.1.smithi104.stdout:3/884: chown f3 70904 1 2024-04-26T04:42:23.303 INFO:tasks.workunit.client.0.smithi090.stdout:0/618: dread d0/d4b/fbd [0,4194304] 0 2024-04-26T04:42:23.303 INFO:tasks.workunit.client.0.smithi090.stdout:0/619: write d0/d4b/d5d/d5c/d8a/f44 [749914,79973] 0 2024-04-26T04:42:23.307 INFO:tasks.workunit.client.1.smithi104.stdout:7/770: dread d2/d4/d32/d35/d44/fb3 [0,4194304] 0 2024-04-26T04:42:23.307 INFO:tasks.workunit.client.0.smithi090.stdout:1/487: dwrite d0/d4/d7/d69/f71 [0,4194304] 0 2024-04-26T04:42:23.308 INFO:tasks.workunit.client.0.smithi090.stdout:0/620: rmdir d0/d4b/d5d/d48/dd2 0 2024-04-26T04:42:23.308 INFO:tasks.workunit.client.0.smithi090.stdout:9/418: dwrite db/f84 [0,4194304] 0 2024-04-26T04:42:23.309 INFO:tasks.workunit.client.0.smithi090.stdout:2/546: dwrite db/d11/d14/d1b/f6c [0,4194304] 0 2024-04-26T04:42:23.311 INFO:tasks.workunit.client.1.smithi104.stdout:0/706: write d0/d4/f29 [161425,16174] 0 2024-04-26T04:42:23.311 INFO:tasks.workunit.client.1.smithi104.stdout:0/707: creat d0/d2/d21/fdf x:0 0 0 2024-04-26T04:42:23.312 INFO:tasks.workunit.client.0.smithi090.stdout:9/419: rename db/dd/d18/d31/f52 to db/dd/d18/d38/d39/d3f/f91 0 2024-04-26T04:42:23.312 INFO:tasks.workunit.client.0.smithi090.stdout:9/420: write db/f1b [747015,68155] 0 2024-04-26T04:42:23.314 INFO:tasks.workunit.client.1.smithi104.stdout:0/708: symlink d0/d2/d21/d27/d1e/d57/le0 0 2024-04-26T04:42:23.316 INFO:tasks.workunit.client.0.smithi090.stdout:2/547: creat db/d11/d14/d25/d41/fba x:0 0 0 2024-04-26T04:42:23.316 INFO:tasks.workunit.client.0.smithi090.stdout:2/548: write db/d11/d7f/fa1 [563487,57962] 0 2024-04-26T04:42:23.317 INFO:tasks.workunit.client.0.smithi090.stdout:8/566: dwrite dc/d34/f84 [0,4194304] 0 2024-04-26T04:42:23.317 INFO:tasks.workunit.client.0.smithi090.stdout:4/482: dwrite d3/d13/f1f [0,4194304] 0 2024-04-26T04:42:23.317 INFO:tasks.workunit.client.0.smithi090.stdout:4/483: chown d3/db/d12/d20/d2d/d58/d6a 12 1 2024-04-26T04:42:23.317 INFO:tasks.workunit.client.0.smithi090.stdout:8/567: creat dc/d5a/fc6 x:0 0 0 2024-04-26T04:42:23.317 INFO:tasks.workunit.client.0.smithi090.stdout:9/421: truncate db/f41 422519 0 2024-04-26T04:42:23.318 INFO:tasks.workunit.client.1.smithi104.stdout:7/771: dread d2/d4/fe3 [0,4194304] 0 2024-04-26T04:42:23.318 INFO:tasks.workunit.client.0.smithi090.stdout:2/549: mknod db/d11/cbb 0 2024-04-26T04:42:23.319 INFO:tasks.workunit.client.0.smithi090.stdout:3/468: dwrite d1/d59/f64 [0,4194304] 0 2024-04-26T04:42:23.321 INFO:tasks.workunit.client.1.smithi104.stdout:2/995: dwrite d6/d1b/d68/d81/fc6 [0,4194304] 0 2024-04-26T04:42:23.322 INFO:tasks.workunit.client.0.smithi090.stdout:1/488: write d0/d4/d7/d69/f71 [1189333,93089] 0 2024-04-26T04:42:23.322 INFO:tasks.workunit.client.0.smithi090.stdout:1/489: readlink d0/d4/d7/d4b/d53/l65 0 2024-04-26T04:42:23.322 INFO:tasks.workunit.client.0.smithi090.stdout:1/490: chown d0/d4/d7/d17/f3c 32 1 2024-04-26T04:42:23.323 INFO:tasks.workunit.client.1.smithi104.stdout:1/886: dwrite d0/d6/d19/d21/d66/d20/f10d [0,4194304] 0 2024-04-26T04:42:23.324 INFO:tasks.workunit.client.0.smithi090.stdout:0/621: write d0/d4b/d5d/d5c/f5e [8354787,111373] 0 2024-04-26T04:42:23.324 INFO:tasks.workunit.client.0.smithi090.stdout:0/622: creat d0/d4b/d5d/d5c/d8a/d7c/fdb x:0 0 0 2024-04-26T04:42:23.324 INFO:tasks.workunit.client.0.smithi090.stdout:6/601: dwrite d9/d1c/d4d/d6d/d92/f28 [0,4194304] 0 2024-04-26T04:42:23.324 INFO:tasks.workunit.client.0.smithi090.stdout:6/602: write d9/d21/d3b/f46 [1611438,13803] 0 2024-04-26T04:42:23.324 INFO:tasks.workunit.client.0.smithi090.stdout:6/603: dread - d9/d77/f9d zero size 2024-04-26T04:42:23.325 INFO:tasks.workunit.client.0.smithi090.stdout:9/422: creat db/dd/d18/d31/d6a/f92 x:0 0 0 2024-04-26T04:42:23.325 INFO:tasks.workunit.client.0.smithi090.stdout:9/423: write db/dd/f8d [605575,12812] 0 2024-04-26T04:42:23.329 INFO:tasks.workunit.client.0.smithi090.stdout:3/469: rmdir d1/d17/d36/d33/d95 0 2024-04-26T04:42:23.329 INFO:tasks.workunit.client.1.smithi104.stdout:0/709: dread d0/d2/d21/d27/d1e/d57/d6c/f66 [0,4194304] 0 2024-04-26T04:42:23.334 INFO:tasks.workunit.client.1.smithi104.stdout:7/772: write d2/d4/d32/d3d/d47/f8a [2234346,95852] 0 2024-04-26T04:42:23.335 INFO:tasks.workunit.client.0.smithi090.stdout:2/550: dread db/d11/d14/d1b/f51 [0,4194304] 0 2024-04-26T04:42:23.336 INFO:tasks.workunit.client.0.smithi090.stdout:1/491: unlink d0/d4/f87 0 2024-04-26T04:42:23.340 INFO:tasks.workunit.client.1.smithi104.stdout:1/887: symlink d0/d6/d2e/d32/d4c/db1/dc3/l122 0 2024-04-26T04:42:23.342 INFO:tasks.workunit.client.1.smithi104.stdout:3/885: dwrite d5/d9/d80/f100 [0,4194304] 0 2024-04-26T04:42:23.342 INFO:tasks.workunit.client.1.smithi104.stdout:2/996: dread d6/d8/d3e/f7a [0,4194304] 0 2024-04-26T04:42:23.342 INFO:tasks.workunit.client.1.smithi104.stdout:3/886: fdatasync d5/d9/d80/da2/fad 0 2024-04-26T04:42:23.342 INFO:tasks.workunit.client.1.smithi104.stdout:2/997: readlink d6/de/le3 0 2024-04-26T04:42:23.342 INFO:tasks.workunit.client.1.smithi104.stdout:2/998: truncate d6/d1b/d75/fc9 339531 0 2024-04-26T04:42:23.364 INFO:tasks.workunit.client.0.smithi090.stdout:0/623: creat d0/fdc x:0 0 0 2024-04-26T04:42:23.366 INFO:tasks.workunit.client.0.smithi090.stdout:1/492: symlink d0/d4/d7/l99 0 2024-04-26T04:42:23.368 INFO:tasks.workunit.client.1.smithi104.stdout:2/999: getdents d6/d1b/d51/d6a 0 2024-04-26T04:42:23.369 INFO:tasks.workunit.client.0.smithi090.stdout:0/624: write d0/d4b/d6d/fc6 [526504,103474] 0 2024-04-26T04:42:23.371 INFO:tasks.workunit.client.1.smithi104.stdout:0/710: dread d0/d4/d3b/f77 [0,4194304] 0 2024-04-26T04:42:23.372 INFO:tasks.workunit.client.1.smithi104.stdout:0/711: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fe1 x:0 0 0 2024-04-26T04:42:23.374 INFO:tasks.workunit.client.0.smithi090.stdout:3/470: dwrite d1/d17/f1f [0,4194304] 0 2024-04-26T04:42:23.375 INFO:tasks.workunit.client.1.smithi104.stdout:0/712: link d0/f2f d0/d2/d21/d27/d1e/d57/d6c/dd1/fe2 0 2024-04-26T04:42:23.376 INFO:tasks.workunit.client.0.smithi090.stdout:2/551: dwrite db/d11/d14/d27/d2f/d42/d5c/fb3 [0,4194304] 0 2024-04-26T04:42:23.376 INFO:tasks.workunit.client.0.smithi090.stdout:2/552: read - db/d11/d14/d25/d65/d95/faf zero size 2024-04-26T04:42:23.376 INFO:tasks.workunit.client.0.smithi090.stdout:2/553: write db/f84 [716818,110075] 0 2024-04-26T04:42:23.377 INFO:tasks.workunit.client.1.smithi104.stdout:0/713: mknod d0/d2/d10/d3d/ce3 0 2024-04-26T04:42:23.377 INFO:tasks.workunit.client.0.smithi090.stdout:8/568: dwrite dc/d1f/d49/d4f/f9f [0,4194304] 0 2024-04-26T04:42:23.378 INFO:tasks.workunit.client.0.smithi090.stdout:4/484: dwrite d3/db/d12/d20/d45/f92 [0,4194304] 0 2024-04-26T04:42:23.378 INFO:tasks.workunit.client.0.smithi090.stdout:4/485: stat d3/d13/d1d/d50/d51/f97 0 2024-04-26T04:42:23.378 INFO:tasks.workunit.client.1.smithi104.stdout:7/773: dwrite d2/d4/d32/f69 [0,4194304] 0 2024-04-26T04:42:23.379 INFO:tasks.workunit.client.1.smithi104.stdout:7/774: fsync d2/d4/d32/f69 0 2024-04-26T04:42:23.379 INFO:tasks.workunit.client.0.smithi090.stdout:8/569: write dc/d1f/f67 [319772,50385] 0 2024-04-26T04:42:23.380 INFO:tasks.workunit.client.0.smithi090.stdout:6/604: rename d9/ca to d9/d1c/d4d/d6d/d92/cd3 0 2024-04-26T04:42:23.380 INFO:tasks.workunit.client.0.smithi090.stdout:6/605: fsync d9/d21/d40/d4c/d6f/d9f/fcd 0 2024-04-26T04:42:23.381 INFO:tasks.workunit.client.1.smithi104.stdout:0/714: symlink d0/d2/d21/d27/d1e/d57/d6c/dd1/d7c/le4 0 2024-04-26T04:42:23.381 INFO:tasks.workunit.client.1.smithi104.stdout:0/715: fdatasync d0/d2/d21/d27/d1e/f65 0 2024-04-26T04:42:23.383 INFO:tasks.workunit.client.0.smithi090.stdout:1/493: mknod d0/c9a 0 2024-04-26T04:42:23.383 INFO:tasks.workunit.client.1.smithi104.stdout:3/887: dread d5/f24 [0,4194304] 0 2024-04-26T04:42:23.383 INFO:tasks.workunit.client.1.smithi104.stdout:3/888: write d5/d9/d86/f113 [30987,297] 0 2024-04-26T04:42:23.385 INFO:tasks.workunit.client.1.smithi104.stdout:0/716: creat d0/d2/fe5 x:0 0 0 2024-04-26T04:42:23.388 INFO:tasks.workunit.client.0.smithi090.stdout:3/471: creat d1/d59/f9f x:0 0 0 2024-04-26T04:42:23.388 INFO:tasks.workunit.client.0.smithi090.stdout:3/472: write d1/d6/f91 [842235,53032] 0 2024-04-26T04:42:23.389 INFO:tasks.workunit.client.1.smithi104.stdout:3/889: mkdir d5/db2/d46/d136 0 2024-04-26T04:42:23.390 INFO:tasks.workunit.client.1.smithi104.stdout:0/717: symlink d0/d2/d21/d49/d5e/d9c/le6 0 2024-04-26T04:42:23.391 INFO:tasks.workunit.client.0.smithi090.stdout:0/625: dwrite d0/d4b/d5d/d5c/d8a/d45/fb5 [0,4194304] 0 2024-04-26T04:42:23.391 INFO:tasks.workunit.client.0.smithi090.stdout:3/473: read d1/d59/d5e/f62 [1879831,19019] 0 2024-04-26T04:42:23.393 INFO:tasks.workunit.client.0.smithi090.stdout:2/554: getdents db/d11/d14/d1b 0 2024-04-26T04:42:23.393 INFO:tasks.workunit.client.0.smithi090.stdout:2/555: chown db/d11/d14/d27/d2f/d70/d71/lb1 429288 1 2024-04-26T04:42:23.394 INFO:tasks.workunit.client.1.smithi104.stdout:3/890: creat d5/db2/f137 x:0 0 0 2024-04-26T04:42:23.394 INFO:tasks.workunit.client.1.smithi104.stdout:3/891: stat d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/c4e 0 2024-04-26T04:42:23.400 INFO:tasks.workunit.client.0.smithi090.stdout:3/474: dread d1/d59/f64 [4194304,4194304] 0 2024-04-26T04:42:23.400 INFO:tasks.workunit.client.0.smithi090.stdout:3/475: write d1/d5c/f6e [211892,75684] 0 2024-04-26T04:42:23.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/606: creat d9/d1c/d31/d69/fd4 x:0 0 0 2024-04-26T04:42:23.404 INFO:tasks.workunit.client.0.smithi090.stdout:1/494: dread - d0/d4/d7/d4a/f77 zero size 2024-04-26T04:42:23.405 INFO:tasks.workunit.client.1.smithi104.stdout:7/775: dwrite d2/fb8 [0,4194304] 0 2024-04-26T04:42:23.407 INFO:tasks.workunit.client.1.smithi104.stdout:7/776: chown d2/d4/d32/d35/d44/d5b/f7c 1289 1 2024-04-26T04:42:23.408 INFO:tasks.workunit.client.1.smithi104.stdout:7/777: mkdir d2/d4/d32/d3d/d47/db1/dfa 0 2024-04-26T04:42:23.408 INFO:tasks.workunit.client.0.smithi090.stdout:0/626: rename d0/d4b/d5d/d5c/d8a/d45/d53/dab/lbb to d0/d4b/d5d/d5c/d8a/d45/ldd 0 2024-04-26T04:42:23.408 INFO:tasks.workunit.client.0.smithi090.stdout:0/627: truncate d0/fdc 376434 0 2024-04-26T04:42:23.412 INFO:tasks.workunit.client.0.smithi090.stdout:2/556: truncate db/d59/f62 3783156 0 2024-04-26T04:42:23.413 INFO:tasks.workunit.client.0.smithi090.stdout:2/557: truncate db/d11/d14/d27/d2f/f8d 80122 0 2024-04-26T04:42:23.413 INFO:tasks.workunit.client.1.smithi104.stdout:3/892: dread d5/db2/d60/f84 [0,4194304] 0 2024-04-26T04:42:23.414 INFO:tasks.workunit.client.0.smithi090.stdout:4/486: mkdir d3/db/d12/d99 0 2024-04-26T04:42:23.415 INFO:tasks.workunit.client.1.smithi104.stdout:3/893: mkdir d5/d9/d12f/d138 0 2024-04-26T04:42:23.416 INFO:tasks.workunit.client.0.smithi090.stdout:8/570: symlink dc/d1f/d49/d3b/lc7 0 2024-04-26T04:42:23.418 INFO:tasks.workunit.client.1.smithi104.stdout:3/894: mkdir d5/db2/d60/d139 0 2024-04-26T04:42:23.418 INFO:tasks.workunit.client.0.smithi090.stdout:6/607: creat d9/d1c/fd5 x:0 0 0 2024-04-26T04:42:23.419 INFO:tasks.workunit.client.0.smithi090.stdout:5/518: sync 2024-04-26T04:42:23.419 INFO:tasks.workunit.client.0.smithi090.stdout:2/558: write db/d11/d14/d25/d41/f93 [455094,77828] 0 2024-04-26T04:42:23.420 INFO:tasks.workunit.client.0.smithi090.stdout:1/495: link d0/d4/d7/d4a/f96 d0/d4/d7/d4b/f9b 0 2024-04-26T04:42:23.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/496: chown d0/d4/d7/d49/f5e 534552 1 2024-04-26T04:42:23.422 INFO:tasks.workunit.client.1.smithi104.stdout:0/718: dwrite d0/d2/d21/d27/d1e/f65 [0,4194304] 0 2024-04-26T04:42:23.425 INFO:tasks.workunit.client.0.smithi090.stdout:8/571: link dc/d1f/d49/d39/d55/d64/fb0 dc/d5a/fc8 0 2024-04-26T04:42:23.425 INFO:tasks.workunit.client.0.smithi090.stdout:8/572: chown dc 6935 1 2024-04-26T04:42:23.431 INFO:tasks.workunit.client.0.smithi090.stdout:2/559: mkdir db/d11/d14/d27/d2f/dbc 0 2024-04-26T04:42:23.434 INFO:tasks.workunit.client.0.smithi090.stdout:4/487: rename d3/db/d12/d20/d2d/d2e/f98 to d3/db/d12/f9a 0 2024-04-26T04:42:23.434 INFO:tasks.workunit.client.1.smithi104.stdout:9/689: sync 2024-04-26T04:42:23.434 INFO:tasks.workunit.client.1.smithi104.stdout:9/690: dread - d11/d15/d32/dd6/fd9 zero size 2024-04-26T04:42:23.435 INFO:tasks.workunit.client.1.smithi104.stdout:9/691: link d11/d90/c97 d11/d15/d32/ce0 0 2024-04-26T04:42:23.437 INFO:tasks.workunit.client.0.smithi090.stdout:4/488: creat d3/db/d12/d20/d24/d43/d72/f9b x:0 0 0 2024-04-26T04:42:23.439 INFO:tasks.workunit.client.1.smithi104.stdout:0/719: dread d0/d2/d21/d49/f55 [0,4194304] 0 2024-04-26T04:42:23.441 INFO:tasks.workunit.client.1.smithi104.stdout:0/720: link d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fbc d0/d2/d10/d84/fe7 0 2024-04-26T04:42:23.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/489: mkdir d3/db/d12/d20/d2d/d58/d6a/d9c 0 2024-04-26T04:42:23.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/490: write d3/f9 [1875091,93858] 0 2024-04-26T04:42:23.448 INFO:tasks.workunit.client.1.smithi104.stdout:6/864: sync 2024-04-26T04:42:23.449 INFO:tasks.workunit.client.1.smithi104.stdout:5/819: sync 2024-04-26T04:42:23.455 INFO:tasks.workunit.client.0.smithi090.stdout:1/497: write d0/d4/d7/d17/d22/d31/f34 [3400963,121592] 0 2024-04-26T04:42:23.455 INFO:tasks.workunit.client.0.smithi090.stdout:1/498: creat d0/d4/d7/f9c x:0 0 0 2024-04-26T04:42:23.458 INFO:tasks.workunit.client.0.smithi090.stdout:1/499: creat d0/d4/d7/d4a/f9d x:0 0 0 2024-04-26T04:42:23.460 INFO:tasks.workunit.client.0.smithi090.stdout:1/500: creat d0/d4/d7/d4a/d75/d80/f9e x:0 0 0 2024-04-26T04:42:23.460 INFO:tasks.workunit.client.0.smithi090.stdout:1/501: write d0/d4/d7/d17/d22/f89 [382084,3099] 0 2024-04-26T04:42:23.460 INFO:tasks.workunit.client.0.smithi090.stdout:1/502: chown d0/d4/d43/f54 559 1 2024-04-26T04:42:23.462 INFO:tasks.workunit.client.1.smithi104.stdout:6/865: dread d0/d9/d13/dcd/dd9/f10a [0,4194304] 0 2024-04-26T04:42:23.462 INFO:tasks.workunit.client.1.smithi104.stdout:6/866: fsync d0/d9/d13/d41/d96/dac/f22 0 2024-04-26T04:42:23.462 INFO:tasks.workunit.client.1.smithi104.stdout:6/867: write d0/dd8/de0/d102/f105 [433726,64828] 0 2024-04-26T04:42:23.463 INFO:tasks.workunit.client.0.smithi090.stdout:1/503: write d0/d4/d7/d4a/f51 [806995,44808] 0 2024-04-26T04:42:23.463 INFO:tasks.workunit.client.1.smithi104.stdout:7/778: dwrite d2/d4/d32/d3d/d47/fbc [0,4194304] 0 2024-04-26T04:42:23.464 INFO:tasks.workunit.client.1.smithi104.stdout:6/868: link d0/d9/da8/ld5 d0/d9/d13/d41/l117 0 2024-04-26T04:42:23.464 INFO:tasks.workunit.client.1.smithi104.stdout:6/869: read - d0/d9/d13/d41/d96/dac/db3/fe5 zero size 2024-04-26T04:42:23.468 INFO:tasks.workunit.client.1.smithi104.stdout:7/779: creat d2/d4/d32/d3d/d47/db1/dfa/ffb x:0 0 0 2024-04-26T04:42:23.470 INFO:tasks.workunit.client.0.smithi090.stdout:3/476: dwrite d1/d9/fb [0,4194304] 0 2024-04-26T04:42:23.470 INFO:tasks.workunit.client.0.smithi090.stdout:3/477: chown d1/d17/d36/d33 55 1 2024-04-26T04:42:23.471 INFO:tasks.workunit.client.0.smithi090.stdout:3/478: write d1/d17/d36/f6a [1710160,36756] 0 2024-04-26T04:42:23.471 INFO:tasks.workunit.client.1.smithi104.stdout:3/895: dwrite d5/d9/d80/dbc/fcb [0,4194304] 0 2024-04-26T04:42:23.471 INFO:tasks.workunit.client.1.smithi104.stdout:6/870: rename d0/d9/d13/f91 to d0/d9/d13/d41/d96/dac/dc4/f118 0 2024-04-26T04:42:23.471 INFO:tasks.workunit.client.1.smithi104.stdout:6/871: dread - d0/dd8/de0/d102/fe8 zero size 2024-04-26T04:42:23.472 INFO:tasks.workunit.client.0.smithi090.stdout:3/479: mkdir d1/d6/d1b/d58/da0 0 2024-04-26T04:42:23.472 INFO:tasks.workunit.client.0.smithi090.stdout:3/480: creat d1/d59/fa1 x:0 0 0 2024-04-26T04:42:23.472 INFO:tasks.workunit.client.0.smithi090.stdout:6/608: dwrite d9/d1c/d31/fd1 [0,4194304] 0 2024-04-26T04:42:23.472 INFO:tasks.workunit.client.0.smithi090.stdout:6/609: readlink d9/d21/d40/l38 0 2024-04-26T04:42:23.473 INFO:tasks.workunit.client.1.smithi104.stdout:7/780: mknod d2/d4/d32/d35/d44/d5b/db2/cfc 0 2024-04-26T04:42:23.474 INFO:tasks.workunit.client.0.smithi090.stdout:3/481: truncate d1/d17/f2f 3096626 0 2024-04-26T04:42:23.475 INFO:tasks.workunit.client.1.smithi104.stdout:3/896: mkdir d5/d13a 0 2024-04-26T04:42:23.476 INFO:tasks.workunit.client.0.smithi090.stdout:5/519: dwrite d4/d1a/df/d36/f54 [4194304,4194304] 0 2024-04-26T04:42:23.477 INFO:tasks.workunit.client.1.smithi104.stdout:6/872: creat d0/d9/dc6/f119 x:0 0 0 2024-04-26T04:42:23.480 INFO:tasks.workunit.client.0.smithi090.stdout:6/610: unlink d9/d1c/d4d/d6d/d66/c5a 0 2024-04-26T04:42:23.481 INFO:tasks.workunit.client.0.smithi090.stdout:6/611: write d9/d21/d40/d4c/f59 [341333,6542] 0 2024-04-26T04:42:23.481 INFO:tasks.workunit.client.0.smithi090.stdout:6/612: write d9/d77/f9d [438167,43004] 0 2024-04-26T04:42:23.484 INFO:tasks.workunit.client.1.smithi104.stdout:0/721: dwrite d0/d2/d21/d27/d1e/fc8 [4194304,4194304] 0 2024-04-26T04:42:23.484 INFO:tasks.workunit.client.0.smithi090.stdout:2/560: dwrite db/d11/d14/d1b/d20/f24 [0,4194304] 0 2024-04-26T04:42:23.485 INFO:tasks.workunit.client.0.smithi090.stdout:3/482: rename d1/d17/d36/d2a/d3a/l45 to d1/d17/d36/d33/la2 0 2024-04-26T04:42:23.486 INFO:tasks.workunit.client.1.smithi104.stdout:5/820: dwrite d0/d1/d4/d2a/fd2 [0,4194304] 0 2024-04-26T04:42:23.486 INFO:tasks.workunit.client.0.smithi090.stdout:4/491: dwrite d3/d13/d6c/f90 [0,4194304] 0 2024-04-26T04:42:23.488 INFO:tasks.workunit.client.0.smithi090.stdout:5/520: truncate d4/d1a/d45/f74 2983697 0 2024-04-26T04:42:23.488 INFO:tasks.workunit.client.0.smithi090.stdout:5/521: fsync d4/d1a/d45/f3c 0 2024-04-26T04:42:23.498 INFO:tasks.workunit.client.0.smithi090.stdout:5/522: truncate d4/d1a/df/d36/f5c 836967 0 2024-04-26T04:42:23.498 INFO:tasks.workunit.client.0.smithi090.stdout:5/523: chown d4/d1a/d1f/d53/d59/d5e/f6a 34009272 1 2024-04-26T04:42:23.498 INFO:tasks.workunit.client.0.smithi090.stdout:5/524: dread - d4/d1a/d1f/d53/d59/d5e/f6a zero size 2024-04-26T04:42:23.498 INFO:tasks.workunit.client.0.smithi090.stdout:5/525: chown d4/d1a/d2a/d79/d90/d56/f8b 178197 1 2024-04-26T04:42:23.498 INFO:tasks.workunit.client.0.smithi090.stdout:5/526: write d4/d1a/df/f28 [5218827,79515] 0 2024-04-26T04:42:23.505 INFO:tasks.workunit.client.0.smithi090.stdout:2/561: write db/f10 [2844296,41677] 0 2024-04-26T04:42:23.505 INFO:tasks.workunit.client.0.smithi090.stdout:2/562: chown db/d11/d14/d27/d2f/d42/d5c/f80 1732071 1 2024-04-26T04:42:23.506 INFO:tasks.workunit.client.1.smithi104.stdout:3/897: dwrite d5/db2/da3/dd0/ddb/d32/d48/f5f [0,4194304] 0 2024-04-26T04:42:23.506 INFO:tasks.workunit.client.1.smithi104.stdout:3/898: write d5/db2/f34 [1732807,74882] 0 2024-04-26T04:42:23.507 INFO:tasks.workunit.client.0.smithi090.stdout:2/563: mknod db/d11/d14/d1b/d20/d4c/cbd 0 2024-04-26T04:42:23.507 INFO:tasks.workunit.client.1.smithi104.stdout:6/873: dwrite d0/d9/d13/d41/d82/dc8/fb6 [0,4194304] 0 2024-04-26T04:42:23.508 INFO:tasks.workunit.client.0.smithi090.stdout:2/564: mknod db/d11/d14/d25/d65/d95/cbe 0 2024-04-26T04:42:23.509 INFO:tasks.workunit.client.0.smithi090.stdout:2/565: stat db/d59/d66/d77 0 2024-04-26T04:42:23.511 INFO:tasks.workunit.client.0.smithi090.stdout:3/483: dwrite d1/d17/d36/d2a/d3a/f53 [0,4194304] 0 2024-04-26T04:42:23.511 INFO:tasks.workunit.client.0.smithi090.stdout:3/484: truncate d1/d59/d5e/f62 3322261 0 2024-04-26T04:42:23.530 INFO:tasks.workunit.client.0.smithi090.stdout:3/485: dread d1/d17/d36/f40 [0,4194304] 0 2024-04-26T04:42:23.530 INFO:tasks.workunit.client.0.smithi090.stdout:3/486: mknod d1/d17/d52/d7d/ca3 0 2024-04-26T04:42:23.531 INFO:tasks.workunit.client.0.smithi090.stdout:3/487: write d1/d17/d36/d2a/d3a/f4b [4974520,35167] 0 2024-04-26T04:42:23.531 INFO:tasks.workunit.client.0.smithi090.stdout:3/488: fdatasync d1/d17/d36/d33/d43/f92 0 2024-04-26T04:42:23.533 INFO:tasks.workunit.client.0.smithi090.stdout:3/489: mkdir d1/d6/d1b/d58/da4 0 2024-04-26T04:42:23.533 INFO:tasks.workunit.client.0.smithi090.stdout:5/527: dwrite d4/d1a/df/d72/f77 [0,4194304] 0 2024-04-26T04:42:23.533 INFO:tasks.workunit.client.0.smithi090.stdout:5/528: fdatasync d4/f1e 0 2024-04-26T04:42:23.534 INFO:tasks.workunit.client.0.smithi090.stdout:3/490: creat d1/d17/d61/d9d/fa5 x:0 0 0 2024-04-26T04:42:23.535 INFO:tasks.workunit.client.1.smithi104.stdout:3/899: dwrite d5/d9/f12 [4194304,4194304] 0 2024-04-26T04:42:23.535 INFO:tasks.workunit.client.1.smithi104.stdout:3/900: chown d5/db2/da3/dd0/ddb/d12c 784118 1 2024-04-26T04:42:23.537 INFO:tasks.workunit.client.0.smithi090.stdout:3/491: symlink d1/d6/d2c/d97/la6 0 2024-04-26T04:42:23.537 INFO:tasks.workunit.client.0.smithi090.stdout:3/492: chown d1/d17/d36/d2a/d3a 0 1 2024-04-26T04:42:23.538 INFO:tasks.workunit.client.1.smithi104.stdout:0/722: symlink d0/d2/d21/d27/d1e/le8 0 2024-04-26T04:42:23.539 INFO:tasks.workunit.client.0.smithi090.stdout:3/493: mkdir d1/d17/d61/d9d/da7 0 2024-04-26T04:42:23.539 INFO:tasks.workunit.client.0.smithi090.stdout:6/613: dwrite d9/d77/f9d [0,4194304] 0 2024-04-26T04:42:23.541 INFO:tasks.workunit.client.0.smithi090.stdout:4/492: dwrite d3/db/d12/d20/d45/f96 [4194304,4194304] 0 2024-04-26T04:42:23.541 INFO:tasks.workunit.client.0.smithi090.stdout:4/493: fsync d3/db/d12/d20/d2d/d2e/f3a 0 2024-04-26T04:42:23.541 INFO:tasks.workunit.client.0.smithi090.stdout:4/494: chown d3/db/d12/d20/d2d/d58 6574340 1 2024-04-26T04:42:23.542 INFO:tasks.workunit.client.0.smithi090.stdout:6/614: symlink d9/d21/d40/d4c/d6f/d9f/dae/ld6 0 2024-04-26T04:42:23.543 INFO:tasks.workunit.client.1.smithi104.stdout:5/821: creat d0/d1/d4/d2a/d6b/d73/f115 x:0 0 0 2024-04-26T04:42:23.549 INFO:tasks.workunit.client.1.smithi104.stdout:6/874: creat d0/d9/d13/f11a x:0 0 0 2024-04-26T04:42:23.553 INFO:tasks.workunit.client.1.smithi104.stdout:7/781: rmdir d2/d4/d32/d35/d44/d5b 39 2024-04-26T04:42:23.553 INFO:tasks.workunit.client.1.smithi104.stdout:7/782: chown d2/c1a 328 1 2024-04-26T04:42:23.554 INFO:tasks.workunit.client.0.smithi090.stdout:4/495: mkdir d3/db/d12/d20/d45/d53/d9d 0 2024-04-26T04:42:23.554 INFO:tasks.workunit.client.0.smithi090.stdout:6/615: mknod d9/d77/db0/db5/dca/cd7 0 2024-04-26T04:42:23.554 INFO:tasks.workunit.client.0.smithi090.stdout:6/616: readlink d9/d77/d9a/lb9 0 2024-04-26T04:42:23.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.553+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dbc0694e0 con 0x7f8da806c680 2024-04-26T04:42:23.556 INFO:tasks.workunit.client.1.smithi104.stdout:3/901: symlink d5/db2/da3/dd0/ddb/d3a/l13b 0 2024-04-26T04:42:23.556 INFO:tasks.workunit.client.0.smithi090.stdout:4/496: mknod d3/db/d12/d20/d2d/d58/d6a/d74/d85/c9e 0 2024-04-26T04:42:23.557 INFO:tasks.workunit.client.1.smithi104.stdout:0/723: mknod d0/d2/d21/ce9 0 2024-04-26T04:42:23.557 INFO:tasks.workunit.client.1.smithi104.stdout:0/724: write d0/d2/d21/fdf [234035,101346] 0 2024-04-26T04:42:23.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/497: creat d3/d13/f9f x:0 0 0 2024-04-26T04:42:23.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/498: chown d3/f7d 4 1 2024-04-26T04:42:23.559 INFO:tasks.workunit.client.0.smithi090.stdout:4/499: truncate f1 6370591 0 2024-04-26T04:42:23.561 INFO:tasks.workunit.client.1.smithi104.stdout:5/822: unlink d0/l16 0 2024-04-26T04:42:23.564 INFO:tasks.workunit.client.1.smithi104.stdout:6/875: creat d0/d9/d13/d41/d96/dac/f11b x:0 0 0 2024-04-26T04:42:23.565 INFO:tasks.workunit.client.1.smithi104.stdout:8/753: sync 2024-04-26T04:42:23.565 INFO:tasks.workunit.client.1.smithi104.stdout:4/875: sync 2024-04-26T04:42:23.565 INFO:tasks.workunit.client.1.smithi104.stdout:4/876: truncate d4/d7/fe8 318266 0 2024-04-26T04:42:23.565 INFO:tasks.workunit.client.1.smithi104.stdout:5/823: rename d0/d1/d4/d2a/d6b/d73/da4 to d0/d1/d4/d2a/d6b/d116 0 2024-04-26T04:42:23.565 INFO:tasks.workunit.client.1.smithi104.stdout:6/876: mknod d0/d9/d13/dcd/dd9/df5/c11c 0 2024-04-26T04:42:23.568 INFO:tasks.workunit.client.1.smithi104.stdout:7/783: dread d2/d4/d1d/f90 [0,4194304] 0 2024-04-26T04:42:23.568 INFO:tasks.workunit.client.0.smithi090.stdout:4/500: dread d3/db/d12/d20/d24/f32 [0,4194304] 0 2024-04-26T04:42:23.571 INFO:tasks.workunit.client.1.smithi104.stdout:4/877: rename d4/db/d6b/df0/f102 to d4/d20/d55/d56/da3/f131 0 2024-04-26T04:42:23.572 INFO:tasks.workunit.client.1.smithi104.stdout:5/824: link d0/d1/d4/d2a/d2b/d37/ff7 d0/d1/d1e/d98/f117 0 2024-04-26T04:42:23.572 INFO:tasks.workunit.client.1.smithi104.stdout:5/825: chown d0/d1/d4/d2a/d6b/d91/dbd/fc5 33557 1 2024-04-26T04:42:23.574 INFO:tasks.workunit.client.1.smithi104.stdout:3/902: dread d5/d9/d80/dbc/fde [0,4194304] 0 2024-04-26T04:42:23.575 INFO:tasks.workunit.client.0.smithi090.stdout:5/529: dwrite d4/d1a/df/f28 [0,4194304] 0 2024-04-26T04:42:23.575 INFO:tasks.workunit.client.0.smithi090.stdout:4/501: dread d3/f9 [0,4194304] 0 2024-04-26T04:42:23.575 INFO:tasks.workunit.client.0.smithi090.stdout:5/530: creat d4/d1a/d2a/d79/d90/fa8 x:0 0 0 2024-04-26T04:42:23.575 INFO:tasks.workunit.client.0.smithi090.stdout:5/531: chown d4/d1a/d2a/d79/d90/d49/f9a 401567895 1 2024-04-26T04:42:23.576 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.573+0000 7f8db2ffd700 1 -- 172.21.15.90:0/2158609233 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3456 (secure 0 0 0) 0x7f8dbc0694e0 con 0x7f8da806c680 2024-04-26T04:42:23.576 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:42:23.576 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (6m) 2m ago 7m 25.9M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:42:23.576 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (8m) 2m ago 8m 7407k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:42:23.576 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (7m) 2m ago 7m 7335k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:42:23.577 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (8m) 2m ago 8m 6660k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:42:23.577 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (7m) 2m ago 7m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:42:23.577 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (6m) 2m ago 7m 83.8M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:42:23.577 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (2m) 2m ago 2m 15.6M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:42:23.577 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (2m) 2m ago 2m 11.2M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:42:23.577 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (2m) 2m ago 2m 12.2M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:42:23.578 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (2m) 2m ago 2m 18.4M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:42:23.578 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:9283,8765,8443 running (9m) 2m ago 9m 487M - 18.2.0 dc2bc1663786 a008eb0a2d7e 2024-04-26T04:42:23.578 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (6m) 2m ago 6m 437M - 18.2.0 dc2bc1663786 b809d29d97b5 2024-04-26T04:42:23.578 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (9m) 2m ago 9m 49.1M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:42:23.578 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (6m) 2m ago 6m 35.3M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:42:23.578 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (8m) 2m ago 8m 14.3M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:42:23.579 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (7m) 2m ago 7m 15.5M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:42:23.579 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (6m) 2m ago 6m 48.1M 3946M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:42:23.579 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (5m) 2m ago 5m 47.6M 3946M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:42:23.579 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (5m) 2m ago 5m 47.3M 3946M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:42:23.579 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (4m) 2m ago 4m 45.4M 4970M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:42:23.579 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (4m) 2m ago 4m 44.8M 4970M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:42:23.579 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (3m) 2m ago 3m 44.3M 4970M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:42:23.580 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (6m) 2m ago 7m 60.0M - 2.43.0 a07b618ecd1d 8382d6553d6c 2024-04-26T04:42:23.580 INFO:tasks.workunit.client.1.smithi104.stdout:8/754: write d0/f4d [1089482,52977] 0 2024-04-26T04:42:23.580 INFO:tasks.workunit.client.1.smithi104.stdout:7/784: creat d2/d4/d1d/d34/db0/de4/ffd x:0 0 0 2024-04-26T04:42:23.580 INFO:tasks.workunit.client.1.smithi104.stdout:8/755: write d0/d1c/fd0 [2596777,14719] 0 2024-04-26T04:42:23.580 INFO:tasks.workunit.client.1.smithi104.stdout:6/877: truncate d0/d9/d13/d41/d96/dac/dc4/f118 3911947 0 2024-04-26T04:42:23.581 INFO:tasks.workunit.client.1.smithi104.stdout:4/878: mknod d4/d7/c132 0 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8da806c680 msgr2=0x7f8da806eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8da806c680 0x7f8da806eb30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc1991c0 tx=0x7f8dac00b540 comp rx=0 tx=0).stop 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ff150 msgr2=0x7f8dbc197cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ff150 0x7f8dbc197cb0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f8db800d900 tx=0x7f8db800dc10 comp rx=0 tx=0).stop 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 shutdown_connections 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f8da806c680 0x7f8da806eb30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ff150 0x7f8dbc197cb0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 --2- 172.21.15.90:0/2158609233 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc100350 0x7f8dbc1981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 >> 172.21.15.90:0/2158609233 conn(0x7f8dbc0fa700 msgr2=0x7f8dbc068dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:23.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 shutdown_connections 2024-04-26T04:42:23.582 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.576+0000 7f8dc4126700 1 -- 172.21.15.90:0/2158609233 wait complete. 2024-04-26T04:42:23.582 INFO:tasks.workunit.client.0.smithi090.stdout:3/494: dwrite d1/d59/f64 [0,4194304] 0 2024-04-26T04:42:23.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/502: unlink d3/db/d12/d20/d2d/d58/d6a/d74/f86 0 2024-04-26T04:42:23.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/503: fsync d3/db/d12/ff 0 2024-04-26T04:42:23.582 INFO:tasks.workunit.client.0.smithi090.stdout:5/532: mknod d4/d1a/d45/ca9 0 2024-04-26T04:42:23.585 INFO:tasks.workunit.client.1.smithi104.stdout:1/888: sync 2024-04-26T04:42:23.586 INFO:tasks.workunit.client.0.smithi090.stdout:4/504: rmdir d3/db/d12/d20/d2d/d58/d6a/d74/d85 39 2024-04-26T04:42:23.590 INFO:tasks.workunit.client.0.smithi090.stdout:6/617: dwrite d9/f15 [4194304,4194304] 0 2024-04-26T04:42:23.590 INFO:tasks.workunit.client.1.smithi104.stdout:8/756: link d0/d33/d51/f92 d0/d1c/d27/d47/db1/ff0 0 2024-04-26T04:42:23.591 INFO:tasks.workunit.client.0.smithi090.stdout:4/505: creat d3/db/d12/fa0 x:0 0 0 2024-04-26T04:42:23.593 INFO:tasks.workunit.client.0.smithi090.stdout:7/560: sync 2024-04-26T04:42:23.593 INFO:tasks.workunit.client.0.smithi090.stdout:7/561: creat d0/d1/d4/d98/d65/db2/fcd x:0 0 0 2024-04-26T04:42:23.594 INFO:tasks.workunit.client.1.smithi104.stdout:1/889: truncate d0/d6/fc0 3441981 0 2024-04-26T04:42:23.594 INFO:tasks.workunit.client.1.smithi104.stdout:6/878: symlink d0/d9/d13/d41/d96/ded/d10d/d71/d3f/l11d 0 2024-04-26T04:42:23.594 INFO:tasks.workunit.client.1.smithi104.stdout:6/879: write d0/d9/f49 [2018595,56823] 0 2024-04-26T04:42:23.596 INFO:tasks.workunit.client.1.smithi104.stdout:1/890: symlink d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/dec/l123 0 2024-04-26T04:42:23.598 INFO:tasks.workunit.client.0.smithi090.stdout:6/618: truncate d9/d1c/f57 3922281 0 2024-04-26T04:42:23.600 INFO:tasks.workunit.client.1.smithi104.stdout:6/880: mknod d0/d9/d13/d41/d82/dc8/dba/daa/dda/c11e 0 2024-04-26T04:42:23.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/562: symlink d0/d1/da3/lce 0 2024-04-26T04:42:23.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/563: chown d0/db/d15/d29/l96 308730 1 2024-04-26T04:42:23.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/564: truncate d0/d1/f71 163617 0 2024-04-26T04:42:23.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/565: creat d0/d1/d4/d3e/fcf x:0 0 0 2024-04-26T04:42:23.603 INFO:tasks.workunit.client.0.smithi090.stdout:6/619: mkdir d9/d77/db0/db5/dca/dd8 0 2024-04-26T04:42:23.605 INFO:tasks.workunit.client.1.smithi104.stdout:6/881: mknod d0/d9/d13/d41/d96/ded/d10d/d47/c11f 0 2024-04-26T04:42:23.605 INFO:tasks.workunit.client.0.smithi090.stdout:7/566: creat d0/db/d53/db7/d4f/fd0 x:0 0 0 2024-04-26T04:42:23.606 INFO:tasks.workunit.client.1.smithi104.stdout:6/882: rmdir d0/d9/d13/d41/d82/dc8/d80 39 2024-04-26T04:42:23.610 INFO:tasks.workunit.client.1.smithi104.stdout:6/883: stat d0/d9/d13/d41/l117 0 2024-04-26T04:42:23.610 INFO:tasks.workunit.client.1.smithi104.stdout:6/884: write d0/d9/d13/d41/d96/dac/dec/f9b [4686309,103577] 0 2024-04-26T04:42:23.611 INFO:tasks.workunit.client.0.smithi090.stdout:4/506: dread d3/db/d12/d20/d24/d30/f5a [0,4194304] 0 2024-04-26T04:42:23.614 INFO:tasks.workunit.client.1.smithi104.stdout:6/885: dread d0/d9/d13/d41/fbf [0,4194304] 0 2024-04-26T04:42:23.615 INFO:tasks.workunit.client.1.smithi104.stdout:6/886: read d0/d9/d13/d41/d96/dac/f4e [141149,118591] 0 2024-04-26T04:42:23.617 INFO:tasks.workunit.client.1.smithi104.stdout:6/887: mkdir d0/d9/d13/dcd/dd9/d120 0 2024-04-26T04:42:23.617 INFO:tasks.workunit.client.1.smithi104.stdout:6/888: write d0/d9/d13/f11a [531265,110465] 0 2024-04-26T04:42:23.617 INFO:tasks.workunit.client.1.smithi104.stdout:6/889: dread - d0/d9/d13/d41/d82/dc8/dba/daa/fdc zero size 2024-04-26T04:42:23.617 INFO:tasks.workunit.client.1.smithi104.stdout:6/890: chown d0/d9/dc6/f9e 139257249 1 2024-04-26T04:42:23.617 INFO:tasks.workunit.client.0.smithi090.stdout:6/620: truncate d9/d1c/d4d/d6d/d92/f28 3555130 0 2024-04-26T04:42:23.618 INFO:tasks.workunit.client.1.smithi104.stdout:6/891: creat d0/d9/d13/d41/d82/f121 x:0 0 0 2024-04-26T04:42:23.618 INFO:tasks.workunit.client.1.smithi104.stdout:8/757: dread d0/d33/d35/f3e [0,4194304] 0 2024-04-26T04:42:23.619 INFO:tasks.workunit.client.1.smithi104.stdout:9/692: sync 2024-04-26T04:42:23.620 INFO:tasks.workunit.client.0.smithi090.stdout:4/507: link d3/l33 d3/la1 0 2024-04-26T04:42:23.620 INFO:tasks.workunit.client.0.smithi090.stdout:4/508: dread - d3/d13/d64/f6b zero size 2024-04-26T04:42:23.620 INFO:tasks.workunit.client.0.smithi090.stdout:4/509: fsync d3/db/d12/d20/d24/d43/d72/f8a 0 2024-04-26T04:42:23.620 INFO:tasks.workunit.client.0.smithi090.stdout:6/621: chown d9/d1c/d4d/d6d/d92/f29 985460434 1 2024-04-26T04:42:23.620 INFO:tasks.workunit.client.1.smithi104.stdout:8/758: rename d0/d1c/f7e to d0/d1c/d59/ff1 0 2024-04-26T04:42:23.621 INFO:tasks.workunit.client.0.smithi090.stdout:4/510: rename d3/db/d12/d20/d24/d30 to d3/db/d12/d20/d2d/d58/d6a/d74/da2 0 2024-04-26T04:42:23.623 INFO:tasks.workunit.client.0.smithi090.stdout:4/511: rmdir d3/db/d12/d20/d45/d53/d9d 0 2024-04-26T04:42:23.623 INFO:tasks.workunit.client.1.smithi104.stdout:9/693: rename d11/f1a to d11/d1d/d34/fe1 0 2024-04-26T04:42:23.623 INFO:tasks.workunit.client.1.smithi104.stdout:6/892: dread d0/d9/d13/d41/d82/dc8/f84 [0,4194304] 0 2024-04-26T04:42:23.625 INFO:tasks.workunit.client.1.smithi104.stdout:8/759: mknod d0/d1c/d27/d47/d67/d76/d7f/cf2 0 2024-04-26T04:42:23.626 INFO:tasks.workunit.client.1.smithi104.stdout:9/694: symlink d11/d15/d32/le2 0 2024-04-26T04:42:23.626 INFO:tasks.workunit.client.1.smithi104.stdout:6/893: mknod d0/d9/d13/d41/d96/dac/c122 0 2024-04-26T04:42:23.628 INFO:tasks.workunit.client.1.smithi104.stdout:8/760: rename d0/d50/dc4/ld3 to d0/d33/d51/lf3 0 2024-04-26T04:42:23.630 INFO:tasks.workunit.client.1.smithi104.stdout:6/894: creat d0/d9/d13/d41/d82/dc8/d80/f123 x:0 0 0 2024-04-26T04:42:23.631 INFO:tasks.workunit.client.1.smithi104.stdout:8/761: mknod d0/d33/d51/cf4 0 2024-04-26T04:42:23.632 INFO:tasks.workunit.client.1.smithi104.stdout:6/895: creat d0/d9/d13/d41/d96/ded/d10d/f124 x:0 0 0 2024-04-26T04:42:23.633 INFO:tasks.workunit.client.1.smithi104.stdout:8/762: creat d0/d33/d35/d6d/da4/ff5 x:0 0 0 2024-04-26T04:42:23.634 INFO:tasks.workunit.client.1.smithi104.stdout:0/725: dwrite d0/d2/d21/fdd [4194304,4194304] 0 2024-04-26T04:42:23.634 INFO:tasks.workunit.client.1.smithi104.stdout:0/726: read d0/d2/d21/f52 [2559672,39148] 0 2024-04-26T04:42:23.636 INFO:tasks.workunit.client.1.smithi104.stdout:6/896: unlink d0/d9/d13/d41/d96/ded/d10d/d71/f74 0 2024-04-26T04:42:23.636 INFO:tasks.workunit.client.1.smithi104.stdout:6/897: creat d0/d9/d13/d41/d82/dc8/d80/f125 x:0 0 0 2024-04-26T04:42:23.636 INFO:tasks.workunit.client.0.smithi090.stdout:3/495: dwrite d1/d17/d36/f24 [0,4194304] 0 2024-04-26T04:42:23.638 INFO:tasks.workunit.client.1.smithi104.stdout:0/727: truncate d0/fa 1729599 0 2024-04-26T04:42:23.639 INFO:tasks.workunit.client.0.smithi090.stdout:4/512: dread d3/d13/d6c/f90 [4194304,4194304] 0 2024-04-26T04:42:23.639 INFO:tasks.workunit.client.0.smithi090.stdout:4/513: chown d3/d13/f21 3 1 2024-04-26T04:42:23.641 INFO:tasks.workunit.client.1.smithi104.stdout:5/826: dwrite d0/d1/d4/d2a/d6b/d91/dc7/fcd [4194304,4194304] 0 2024-04-26T04:42:23.641 INFO:tasks.workunit.client.1.smithi104.stdout:5/827: dread - d0/d7b/ff0 zero size 2024-04-26T04:42:23.641 INFO:tasks.workunit.client.1.smithi104.stdout:5/828: fsync d0/d41/f74 0 2024-04-26T04:42:23.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/514: mkdir d3/db/d12/d20/d2d/d58/d6a/da3 0 2024-04-26T04:42:23.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/515: fdatasync d3/f9 0 2024-04-26T04:42:23.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/516: fsync d3/d13/d6c/d7e/f94 0 2024-04-26T04:42:23.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/517: write d3/db/d12/f4e [2318,7980] 0 2024-04-26T04:42:23.642 INFO:tasks.workunit.client.0.smithi090.stdout:4/518: fsync d3/d13/d64/f6b 0 2024-04-26T04:42:23.644 INFO:tasks.workunit.client.1.smithi104.stdout:5/829: mkdir d0/d1/d118 0 2024-04-26T04:42:23.645 INFO:tasks.workunit.client.1.smithi104.stdout:5/830: readlink d0/d1/d4/d2a/d2b/d37/l69 0 2024-04-26T04:42:23.647 INFO:tasks.workunit.client.1.smithi104.stdout:5/831: creat d0/d1/d4/d2a/d6b/f119 x:0 0 0 2024-04-26T04:42:23.647 INFO:tasks.workunit.client.1.smithi104.stdout:9/695: dread d11/d15/d32/d35/f80 [0,4194304] 0 2024-04-26T04:42:23.647 INFO:tasks.workunit.client.1.smithi104.stdout:9/696: write d11/d53/fb6 [926079,101883] 0 2024-04-26T04:42:23.647 INFO:tasks.workunit.client.1.smithi104.stdout:9/697: write d11/d1d/d34/d69/f78 [1339563,35425] 0 2024-04-26T04:42:23.647 INFO:tasks.workunit.client.1.smithi104.stdout:9/698: fdatasync d11/d1d/f6b 0 2024-04-26T04:42:23.648 INFO:tasks.workunit.client.1.smithi104.stdout:0/728: dread d0/d2/d21/d49/d5e/d9c/d53/d5d/f88 [0,4194304] 0 2024-04-26T04:42:23.648 INFO:tasks.workunit.client.1.smithi104.stdout:5/832: unlink d0/d1/d4/d2a/d2b/fa1 0 2024-04-26T04:42:23.650 INFO:tasks.workunit.client.1.smithi104.stdout:0/729: link d0/d2/d21/d27/c1f d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/cea 0 2024-04-26T04:42:23.651 INFO:tasks.workunit.client.1.smithi104.stdout:5/833: link d0/d1/d4/d2a/d2b/d37/d48/dc1/fd8 d0/d1/d4/d2a/d2b/d5c/f11a 0 2024-04-26T04:42:23.651 INFO:tasks.workunit.client.1.smithi104.stdout:5/834: fdatasync d0/d1/d1e/f30 0 2024-04-26T04:42:23.652 INFO:tasks.workunit.client.1.smithi104.stdout:0/730: mknod d0/d2/d21/d49/d5e/d9c/d53/d5d/ceb 0 2024-04-26T04:42:23.653 INFO:tasks.workunit.client.1.smithi104.stdout:0/731: getdents d0/d2/d21/d27/d1e/d57/d6c/d85/d9a 0 2024-04-26T04:42:23.656 INFO:tasks.workunit.client.1.smithi104.stdout:0/732: link d0/d2/d21/d27/d1e/le8 d0/d2/d21/d27/d1e/d57/d6c/dd1/lec 0 2024-04-26T04:42:23.657 INFO:tasks.workunit.client.0.smithi090.stdout:4/519: read d3/db/d12/f10 [765763,117551] 0 2024-04-26T04:42:23.657 INFO:tasks.workunit.client.1.smithi104.stdout:4/879: dwrite d4/d20/d55/d56/da3/fc4 [0,4194304] 0 2024-04-26T04:42:23.662 INFO:tasks.workunit.client.0.smithi090.stdout:5/533: dwrite d4/d1a/df/f6d [0,4194304] 0 2024-04-26T04:42:23.662 INFO:tasks.workunit.client.1.smithi104.stdout:3/903: dwrite d5/db2/f34 [0,4194304] 0 2024-04-26T04:42:23.662 INFO:tasks.workunit.client.1.smithi104.stdout:3/904: chown d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/fb0 481 1 2024-04-26T04:42:23.662 INFO:tasks.workunit.client.1.smithi104.stdout:7/785: dwrite d2/d4/d5a/fec [0,4194304] 0 2024-04-26T04:42:23.662 INFO:tasks.workunit.client.1.smithi104.stdout:7/786: fdatasync d2/d4/d5a/f7b 0 2024-04-26T04:42:23.664 INFO:tasks.workunit.client.1.smithi104.stdout:3/905: mkdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/d13c 0 2024-04-26T04:42:23.665 INFO:tasks.workunit.client.1.smithi104.stdout:5/835: write d0/d1/d64/f19 [352777,67085] 0 2024-04-26T04:42:23.665 INFO:tasks.workunit.client.1.smithi104.stdout:8/763: dwrite d0/d1c/d27/f86 [0,4194304] 0 2024-04-26T04:42:23.666 INFO:tasks.workunit.client.1.smithi104.stdout:8/764: chown d0/d1c/d59/d85/fc3 5588526 1 2024-04-26T04:42:23.670 INFO:tasks.workunit.client.1.smithi104.stdout:5/836: creat d0/d41/d4f/df2/d108/f11b x:0 0 0 2024-04-26T04:42:23.671 INFO:tasks.workunit.client.0.smithi090.stdout:5/534: getdents d4 0 2024-04-26T04:42:23.671 INFO:tasks.workunit.client.0.smithi090.stdout:5/535: chown d4/d1a/d2a/d79/d90/d49/f7d 252 1 2024-04-26T04:42:23.671 INFO:tasks.workunit.client.0.smithi090.stdout:5/536: write d4/d1a/df/d36/f54 [8559201,8661] 0 2024-04-26T04:42:23.673 INFO:tasks.workunit.client.0.smithi090.stdout:6/622: dwrite d9/d1c/d31/f3a [0,4194304] 0 2024-04-26T04:42:23.673 INFO:tasks.workunit.client.0.smithi090.stdout:7/567: dwrite d0/d1/f38 [8388608,4194304] 0 2024-04-26T04:42:23.674 INFO:tasks.workunit.client.1.smithi104.stdout:8/765: creat d0/d33/d35/d6d/def/ff6 x:0 0 0 2024-04-26T04:42:23.676 INFO:tasks.workunit.client.1.smithi104.stdout:7/787: rmdir d2/d4/d32/d35/dbe 39 2024-04-26T04:42:23.676 INFO:tasks.workunit.client.1.smithi104.stdout:7/788: chown d2/d4/d32/d60/f82 1701864 1 2024-04-26T04:42:23.677 INFO:tasks.workunit.client.0.smithi090.stdout:7/568: mkdir d0/db/df/d4d/daf/dd1 0 2024-04-26T04:42:23.677 INFO:tasks.workunit.client.1.smithi104.stdout:1/891: dwrite d0/d6/d49/f4a [0,4194304] 0 2024-04-26T04:42:23.679 INFO:tasks.workunit.client.0.smithi090.stdout:3/496: dwrite d1/d17/f4f [0,4194304] 0 2024-04-26T04:42:23.690 INFO:tasks.workunit.client.1.smithi104.stdout:8/766: mkdir d0/d33/d35/d6d/def/df7 0 2024-04-26T04:42:23.690 INFO:tasks.workunit.client.1.smithi104.stdout:8/767: truncate d0/d1c/f91 855003 0 2024-04-26T04:42:23.690 INFO:tasks.workunit.client.1.smithi104.stdout:8/768: write d0/d1c/d27/d47/db1/fc9 [762645,65148] 0 2024-04-26T04:42:23.691 INFO:tasks.workunit.client.0.smithi090.stdout:6/623: write d9/d21/d40/d4c/d6f/d9f/dae/fc1 [1541586,113628] 0 2024-04-26T04:42:23.692 INFO:tasks.workunit.client.0.smithi090.stdout:6/624: mkdir d9/d1c/dc5/dd9 0 2024-04-26T04:42:23.694 INFO:tasks.workunit.client.1.smithi104.stdout:5/837: rename d0/d1/d1e/c86 to d0/c11c 0 2024-04-26T04:42:23.694 INFO:tasks.workunit.client.1.smithi104.stdout:5/838: chown d0/d1/d1e/l94 114 1 2024-04-26T04:42:23.696 INFO:tasks.workunit.client.1.smithi104.stdout:5/839: mkdir d0/d1/d4/d2a/d6b/d73/d11d 0 2024-04-26T04:42:23.696 INFO:tasks.workunit.client.1.smithi104.stdout:5/840: fdatasync d0/d1/d1e/fd6 0 2024-04-26T04:42:23.696 INFO:tasks.workunit.client.1.smithi104.stdout:5/841: chown d0/d1/d4/d2a/d2b/d37/d48 155904362 1 2024-04-26T04:42:23.697 INFO:tasks.workunit.client.1.smithi104.stdout:5/842: dread d0/d1/d64/fde [0,4194304] 0 2024-04-26T04:42:23.697 INFO:tasks.workunit.client.0.smithi090.stdout:3/497: dread d1/d17/d52/f7c [0,4194304] 0 2024-04-26T04:42:23.697 INFO:tasks.workunit.client.0.smithi090.stdout:3/498: symlink d1/d17/d36/d33/d68/d71/la8 0 2024-04-26T04:42:23.698 INFO:tasks.workunit.client.1.smithi104.stdout:5/843: creat d0/d1/d3e/f11e x:0 0 0 2024-04-26T04:42:23.699 INFO:tasks.workunit.client.1.smithi104.stdout:5/844: creat d0/d1/d1e/df5/f11f x:0 0 0 2024-04-26T04:42:23.699 INFO:tasks.workunit.client.0.smithi090.stdout:3/499: dread d1/d6/f79 [0,4194304] 0 2024-04-26T04:42:23.700 INFO:tasks.workunit.client.0.smithi090.stdout:3/500: truncate d1/d17/d36/f32 4131894 0 2024-04-26T04:42:23.700 INFO:tasks.workunit.client.1.smithi104.stdout:5/845: unlink d0/d1/d4/d2a/d2b/d37/l69 0 2024-04-26T04:42:23.702 INFO:tasks.workunit.client.1.smithi104.stdout:5/846: creat d0/d1/d25/f120 x:0 0 0 2024-04-26T04:42:23.702 INFO:tasks.workunit.client.0.smithi090.stdout:3/501: dread d1/d17/d36/f67 [0,4194304] 0 2024-04-26T04:42:23.706 INFO:tasks.workunit.client.0.smithi090.stdout:3/502: dread d1/d9/f12 [4194304,4194304] 0 2024-04-26T04:42:23.707 INFO:tasks.workunit.client.0.smithi090.stdout:3/503: creat d1/d17/d36/fa9 x:0 0 0 2024-04-26T04:42:23.713 INFO:tasks.workunit.client.1.smithi104.stdout:9/699: dwrite d11/d15/d1b/f60 [4194304,4194304] 0 2024-04-26T04:42:23.713 INFO:tasks.workunit.client.1.smithi104.stdout:8/769: write d0/d1c/f6e [828820,83064] 0 2024-04-26T04:42:23.716 INFO:tasks.workunit.client.1.smithi104.stdout:5/847: dread d0/d1/d64/f92 [0,4194304] 0 2024-04-26T04:42:23.716 INFO:tasks.workunit.client.1.smithi104.stdout:9/700: creat d11/d15/d98/fe3 x:0 0 0 2024-04-26T04:42:23.720 INFO:tasks.workunit.client.1.smithi104.stdout:0/733: dwrite d0/d2/d21/d27/d1e/d57/d6c/dd1/f36 [0,4194304] 0 2024-04-26T04:42:23.720 INFO:tasks.workunit.client.1.smithi104.stdout:5/848: mkdir d0/d1/d4/d2a/d2b/d37/d48/dc1/d113/d121 0 2024-04-26T04:42:23.721 INFO:tasks.workunit.client.0.smithi090.stdout:3/504: read d1/d17/d36/d2a/d3a/f66 [3038453,1251] 0 2024-04-26T04:42:23.721 INFO:tasks.workunit.client.1.smithi104.stdout:3/906: dwrite d5/db2/da3/dd0/ddb/d32/d47/d11a/f131 [0,4194304] 0 2024-04-26T04:42:23.721 INFO:tasks.workunit.client.1.smithi104.stdout:3/907: truncate d5/db2/da3/dd0/ddb/f109 64257 0 2024-04-26T04:42:23.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/537: dwrite d4/d1a/f78 [0,4194304] 0 2024-04-26T04:42:23.722 INFO:tasks.workunit.client.1.smithi104.stdout:1/892: dwrite d0/d6/d2e/f86 [0,4194304] 0 2024-04-26T04:42:23.722 INFO:tasks.workunit.client.0.smithi090.stdout:4/520: dwrite d3/f7d [0,4194304] 0 2024-04-26T04:42:23.723 INFO:tasks.workunit.client.0.smithi090.stdout:7/569: dwrite d0/d1/f38 [0,4194304] 0 2024-04-26T04:42:23.723 INFO:tasks.workunit.client.0.smithi090.stdout:7/570: truncate d0/db/d53/db7/d4f/fd0 531350 0 2024-04-26T04:42:23.723 INFO:tasks.workunit.client.1.smithi104.stdout:7/789: dwrite d2/d4/d32/d3d/f50 [0,4194304] 0 2024-04-26T04:42:23.723 INFO:tasks.workunit.client.1.smithi104.stdout:7/790: chown d2/d4/d1d 129141 1 2024-04-26T04:42:23.723 INFO:tasks.workunit.client.1.smithi104.stdout:9/701: mknod d11/d1d/d34/db3/ce4 0 2024-04-26T04:42:23.727 INFO:tasks.workunit.client.1.smithi104.stdout:0/734: getdents d0/d2/d21/d27/d1e/d57/d6c/dd1/d7c 0 2024-04-26T04:42:23.728 INFO:tasks.workunit.client.0.smithi090.stdout:6/625: rename d9/d1c/d4d to d9/d77/db0/db5/dda 0 2024-04-26T04:42:23.729 INFO:tasks.workunit.client.1.smithi104.stdout:5/849: truncate d0/f6 321043 0 2024-04-26T04:42:23.729 INFO:tasks.workunit.client.1.smithi104.stdout:5/850: truncate d0/d1/d3e/f11e 928644 0 2024-04-26T04:42:23.737 INFO:tasks.workunit.client.0.smithi090.stdout:7/571: mkdir d0/db/d15/dbc/dd2 0 2024-04-26T04:42:23.737 INFO:tasks.workunit.client.0.smithi090.stdout:7/572: chown d0/db/df/d1c/cc3 700 1 2024-04-26T04:42:23.737 INFO:tasks.workunit.client.0.smithi090.stdout:3/505: write d1/d17/d36/d33/d43/f92 [4131255,41067] 0 2024-04-26T04:42:23.737 INFO:tasks.workunit.client.0.smithi090.stdout:4/521: write f0 [6086521,10386] 0 2024-04-26T04:42:23.740 INFO:tasks.workunit.client.0.smithi090.stdout:3/506: truncate d1/d17/d36/d33/f80 2522342 0 2024-04-26T04:42:23.741 INFO:tasks.workunit.client.0.smithi090.stdout:4/522: link d3/db/d12/d20/d24/f52 d3/db/d12/d20/d2d/d58/d6a/d74/fa4 0 2024-04-26T04:42:23.758 INFO:tasks.workunit.client.0.smithi090.stdout:6/626: dread d9/d21/d40/f4b [0,4194304] 0 2024-04-26T04:42:23.759 INFO:tasks.workunit.client.1.smithi104.stdout:3/908: rename d5/db2/da3/dd0/ddb/f40 to d5/db2/d46/d95/f13d 0 2024-04-26T04:42:23.759 INFO:tasks.workunit.client.1.smithi104.stdout:3/909: fsync d5/db2/da3/dd0/ddb/d69/fe7 0 2024-04-26T04:42:23.759 INFO:tasks.workunit.client.0.smithi090.stdout:4/523: write d3/db/d12/d20/d2d/d58/d6a/f8b [3200316,55067] 0 2024-04-26T04:42:23.759 INFO:tasks.workunit.client.0.smithi090.stdout:4/524: chown d3/db/d12/d20/d2d/d58/d6a/d9c 0 1 2024-04-26T04:42:23.759 INFO:tasks.workunit.client.0.smithi090.stdout:4/525: fdatasync d3/db/f23 0 2024-04-26T04:42:23.762 INFO:tasks.workunit.client.1.smithi104.stdout:9/702: symlink d11/d15/d1b/d6d/d9c/le5 0 2024-04-26T04:42:23.763 INFO:tasks.workunit.client.1.smithi104.stdout:8/770: rmdir d0/d33/d35/d6d 39 2024-04-26T04:42:23.764 INFO:tasks.workunit.client.1.smithi104.stdout:0/735: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/ded 0 2024-04-26T04:42:23.766 INFO:tasks.workunit.client.1.smithi104.stdout:5/851: mknod d0/d41/d4f/c122 0 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 -- 172.21.15.90:0/2786818015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 msgr2=0x7f94381018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 --2- 172.21.15.90:0/2786818015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f94381018a0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f942c009910 tx=0x7f942c009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 -- 172.21.15.90:0/2786818015 shutdown_connections 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 --2- 172.21.15.90:0/2786818015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f94381018a0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 --2- 172.21.15.90:0/2786818015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9438100250 0x7f9438100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 -- 172.21.15.90:0/2786818015 >> 172.21.15.90:0/2786818015 conn(0x7f94380fb7e0 msgr2=0x7f94380fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 -- 172.21.15.90:0/2786818015 shutdown_connections 2024-04-26T04:42:23.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.765+0000 7f943d854700 1 -- 172.21.15.90:0/2786818015 wait complete. 2024-04-26T04:42:23.768 INFO:tasks.workunit.client.1.smithi104.stdout:5/852: write d0/d1/d3e/f79 [561241,50869] 0 2024-04-26T04:42:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.766+0000 7f943d854700 1 Processor -- start 2024-04-26T04:42:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.766+0000 7f943d854700 1 -- start start 2024-04-26T04:42:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.766+0000 7f943d854700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9438100250 0x7f9438074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.766+0000 7f943d854700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f9438073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.766+0000 7f943d854700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9438073680 con 0x7f9438101450 2024-04-26T04:42:23.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.766+0000 7f943d854700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94380737c0 con 0x7f9438100250 2024-04-26T04:42:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f94367fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f9438073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f94367fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f9438073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52752/0 (socket says 172.21.15.90:52752) 2024-04-26T04:42:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f94367fc700 1 -- 172.21.15.90:0/3364988170 learned_addr learned my addr 172.21.15.90:0/3364988170 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f9436ffd700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9438100250 0x7f9438074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f94367fc700 1 -- 172.21.15.90:0/3364988170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9438100250 msgr2=0x7f9438074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:23.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f94367fc700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9438100250 0x7f9438074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:23.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f94367fc700 1 -- 172.21.15.90:0/3364988170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f942c0095f0 con 0x7f9438101450 2024-04-26T04:42:23.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f94367fc700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f9438073140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f942c000c00 tx=0x7f942c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:23.774 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f943c852700 1 -- 172.21.15.90:0/3364988170 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f942c01d070 con 0x7f9438101450 2024-04-26T04:42:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f943c852700 1 -- 172.21.15.90:0/3364988170 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f942c005dc0 con 0x7f9438101450 2024-04-26T04:42:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.767+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9438073a40 con 0x7f9438101450 2024-04-26T04:42:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.768+0000 7f943c852700 1 -- 172.21.15.90:0/3364988170 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f942c0226a0 con 0x7f9438101450 2024-04-26T04:42:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.768+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9438073f30 con 0x7f9438101450 2024-04-26T04:42:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.769+0000 7f943c852700 1 -- 172.21.15.90:0/3364988170 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f942c0038f0 con 0x7f9438101450 2024-04-26T04:42:23.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.769+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9438108da0 con 0x7f9438101450 2024-04-26T04:42:23.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.769+0000 7f943c852700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f942006c680 0x7f942006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:23.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.770+0000 7f943c852700 1 -- 172.21.15.90:0/3364988170 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f942c08cd70 con 0x7f9438101450 2024-04-26T04:42:23.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.770+0000 7f9436ffd700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f942006c680 0x7f942006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:23.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.770+0000 7f9436ffd700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f942006c680 0x7f942006eb30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f94381012b0 tx=0x7f9428009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:23.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:23.774+0000 7f943c852700 1 -- 172.21.15.90:0/3364988170 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f942c05ae80 con 0x7f9438101450 2024-04-26T04:42:23.778 INFO:tasks.workunit.client.0.smithi090.stdout:7/573: dwrite d0/d1/d4/d98/db1/fb5 [0,4194304] 0 2024-04-26T04:42:23.778 INFO:tasks.workunit.client.0.smithi090.stdout:3/507: dwrite d1/d17/d36/f32 [0,4194304] 0 2024-04-26T04:42:23.778 INFO:tasks.workunit.client.1.smithi104.stdout:3/910: dread d5/db2/da3/dd0/ddb/d32/d48/f5f [0,4194304] 0 2024-04-26T04:42:23.778 INFO:tasks.workunit.client.1.smithi104.stdout:4/880: dwrite d4/d7/d1d/d1e/d24/d4a/fac [0,4194304] 0 2024-04-26T04:42:23.778 INFO:tasks.workunit.client.1.smithi104.stdout:4/881: readlink d4/d7/d1d/d1e/d24/d48/l49 0 2024-04-26T04:42:23.778 INFO:tasks.workunit.client.1.smithi104.stdout:4/882: creat d4/d7/d1d/d1e/d24/d59/f133 x:0 0 0 2024-04-26T04:42:23.779 INFO:tasks.workunit.client.1.smithi104.stdout:7/791: rename d2/d4/d32/d3d/d41/fd2 to d2/d4/d32/d3d/ffe 0 2024-04-26T04:42:23.779 INFO:tasks.workunit.client.1.smithi104.stdout:7/792: getdents d2/d4/d32/d3d/d41/d5f/d6c/d89/df5 0 2024-04-26T04:42:23.779 INFO:tasks.workunit.client.1.smithi104.stdout:9/703: mkdir d11/d1d/d34/db3/de6 0 2024-04-26T04:42:23.781 INFO:tasks.workunit.client.1.smithi104.stdout:8/771: creat d0/d1c/d2d/d7a/ff8 x:0 0 0 2024-04-26T04:42:23.782 INFO:tasks.workunit.client.1.smithi104.stdout:0/736: mknod d0/d2/d10/cee 0 2024-04-26T04:42:23.784 INFO:tasks.workunit.client.1.smithi104.stdout:3/911: mknod d5/db2/da3/dd0/ddb/d32/d47/d11a/c13e 0 2024-04-26T04:42:23.789 INFO:tasks.workunit.client.1.smithi104.stdout:4/883: mknod d4/d7/d1d/d1e/d15/d113/c134 0 2024-04-26T04:42:23.789 INFO:tasks.workunit.client.1.smithi104.stdout:7/793: write d2/d4/d1d/d34/d7f/faf [1076925,118424] 0 2024-04-26T04:42:23.789 INFO:tasks.workunit.client.1.smithi104.stdout:7/794: fsync d2/d4/d1d/d34/fc6 0 2024-04-26T04:42:23.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/526: truncate d3/db/d12/d20/d2d/d58/d6a/f8b 1883890 0 2024-04-26T04:42:23.790 INFO:tasks.workunit.client.1.smithi104.stdout:7/795: write d2/d4/d32/d3d/d41/d5f/d6c/fb9 [93476,62935] 0 2024-04-26T04:42:23.790 INFO:tasks.workunit.client.1.smithi104.stdout:7/796: fdatasync d2/d4/d32/d3d/fb4 0 2024-04-26T04:42:23.791 INFO:tasks.workunit.client.1.smithi104.stdout:5/853: rename d0/d1/d64/d26/cdb to d0/d1/d25/c123 0 2024-04-26T04:42:23.792 INFO:tasks.workunit.client.0.smithi090.stdout:4/527: dread d3/db/d12/d20/f41 [0,4194304] 0 2024-04-26T04:42:23.792 INFO:tasks.workunit.client.0.smithi090.stdout:4/528: getdents d3/db/d12/d20/d2d/d93 0 2024-04-26T04:42:23.794 INFO:tasks.workunit.client.0.smithi090.stdout:4/529: mkdir d3/d3e/da5 0 2024-04-26T04:42:23.795 INFO:tasks.workunit.client.0.smithi090.stdout:4/530: dread - d3/db/d12/d20/d2d/d58/d6a/f8d zero size 2024-04-26T04:42:23.799 INFO:tasks.workunit.client.1.smithi104.stdout:3/912: write d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/faa [5293908,11111] 0 2024-04-26T04:42:23.799 INFO:tasks.workunit.client.1.smithi104.stdout:5/854: mknod d0/d1/c124 0 2024-04-26T04:42:23.799 INFO:tasks.workunit.client.1.smithi104.stdout:9/704: creat d11/d1d/d34/db3/de6/fe7 x:0 0 0 2024-04-26T04:42:23.799 INFO:tasks.workunit.client.1.smithi104.stdout:0/737: symlink d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/lef 0 2024-04-26T04:42:23.799 INFO:tasks.workunit.client.1.smithi104.stdout:0/738: stat d0/d2/d10/fcd 0 2024-04-26T04:42:23.800 INFO:tasks.workunit.client.0.smithi090.stdout:4/531: mknod d3/db/d12/d20/d24/d43/ca6 0 2024-04-26T04:42:23.800 INFO:tasks.workunit.client.0.smithi090.stdout:4/532: write d3/db/d12/d20/d2d/d58/f63 [4212049,38670] 0 2024-04-26T04:42:23.800 INFO:tasks.workunit.client.0.smithi090.stdout:4/533: mknod d3/db/d12/d20/d2d/d93/ca7 0 2024-04-26T04:42:23.800 INFO:tasks.workunit.client.1.smithi104.stdout:0/739: symlink d0/d2/d21/d49/d5e/d9c/db2/lf0 0 2024-04-26T04:42:23.801 INFO:tasks.workunit.client.0.smithi090.stdout:4/534: truncate f1 660282 0 2024-04-26T04:42:23.801 INFO:tasks.workunit.client.0.smithi090.stdout:4/535: write d3/db/d12/d20/d24/f52 [580344,15674] 0 2024-04-26T04:42:23.801 INFO:tasks.workunit.client.0.smithi090.stdout:4/536: stat f0 0 2024-04-26T04:42:23.803 INFO:tasks.workunit.client.0.smithi090.stdout:4/537: getdents d3/db/d12/d20/d2d/d58/d6a 0 2024-04-26T04:42:23.803 INFO:tasks.workunit.client.1.smithi104.stdout:0/740: mkdir d0/d2/d10/d3d/dc0/df1 0 2024-04-26T04:42:23.804 INFO:tasks.workunit.client.0.smithi090.stdout:4/538: symlink d3/db/d12/d20/d2d/la8 0 2024-04-26T04:42:23.805 INFO:tasks.workunit.client.1.smithi104.stdout:5/855: rename d0/d89 to d0/d1/d4/d2a/d6b/d73/d11d/d125 0 2024-04-26T04:42:23.812 INFO:tasks.workunit.client.1.smithi104.stdout:9/705: write d11/d15/f59 [3356577,9285] 0 2024-04-26T04:42:23.812 INFO:tasks.workunit.client.1.smithi104.stdout:9/706: chown d11/d1d/d34/c45 342995518 1 2024-04-26T04:42:23.815 INFO:tasks.workunit.client.1.smithi104.stdout:9/707: symlink d11/d15/d32/d4c/d54/ddc/le8 0 2024-04-26T04:42:23.815 INFO:tasks.workunit.client.1.smithi104.stdout:9/708: chown d11/c8c 72 1 2024-04-26T04:42:23.815 INFO:tasks.workunit.client.1.smithi104.stdout:9/709: read d11/d1d/d34/fe1 [173107,92804] 0 2024-04-26T04:42:23.815 INFO:tasks.workunit.client.1.smithi104.stdout:9/710: stat d11/d15/d1b/d6d/d9c/dbe/cca 0 2024-04-26T04:42:23.815 INFO:tasks.workunit.client.1.smithi104.stdout:4/884: dwrite d4/ffb [0,4194304] 0 2024-04-26T04:42:23.815 INFO:tasks.workunit.client.1.smithi104.stdout:4/885: chown d4/ffb 0 1 2024-04-26T04:42:23.816 INFO:tasks.workunit.client.1.smithi104.stdout:9/711: creat d11/d53/d84/fe9 x:0 0 0 2024-04-26T04:42:23.817 INFO:tasks.workunit.client.1.smithi104.stdout:5/856: dread d0/d1/d64/f31 [0,4194304] 0 2024-04-26T04:42:23.817 INFO:tasks.workunit.client.1.smithi104.stdout:5/857: chown d0/d1/d4/d2a/d6b/d73/d95/fba 1406629 1 2024-04-26T04:42:23.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/424: sync 2024-04-26T04:42:23.818 INFO:tasks.workunit.client.0.smithi090.stdout:8/573: sync 2024-04-26T04:42:23.818 INFO:tasks.workunit.client.0.smithi090.stdout:0/628: sync 2024-04-26T04:42:23.818 INFO:tasks.workunit.client.0.smithi090.stdout:8/574: readlink dc/d1f/l2a 0 2024-04-26T04:42:23.819 INFO:tasks.workunit.client.1.smithi104.stdout:4/886: mknod d4/db/d6b/d7f/c135 0 2024-04-26T04:42:23.819 INFO:tasks.workunit.client.1.smithi104.stdout:4/887: creat d4/d7/d1d/d8e/f136 x:0 0 0 2024-04-26T04:42:23.819 INFO:tasks.workunit.client.1.smithi104.stdout:4/888: chown d4/d7/ff 121936109 1 2024-04-26T04:42:23.820 INFO:tasks.workunit.client.1.smithi104.stdout:9/712: truncate d11/fc6 518181 0 2024-04-26T04:42:23.820 INFO:tasks.workunit.client.0.smithi090.stdout:0/629: creat d0/d4b/d5d/d5c/d8a/d45/fde x:0 0 0 2024-04-26T04:42:23.821 INFO:tasks.workunit.client.0.smithi090.stdout:0/630: fsync d0/d4b/d5d/d5c/d8a/d45/fb5 0 2024-04-26T04:42:23.821 INFO:tasks.workunit.client.1.smithi104.stdout:5/858: mkdir d0/d1/d4/d2a/d6b/d116/d126 0 2024-04-26T04:42:23.822 INFO:tasks.workunit.client.0.smithi090.stdout:0/631: write d0/d4b/d5d/f27 [1036440,97248] 0 2024-04-26T04:42:23.823 INFO:tasks.workunit.client.0.smithi090.stdout:9/425: link db/dd/d44/f34 db/dd/d18/d38/d39/d3f/f93 0 2024-04-26T04:42:23.824 INFO:tasks.workunit.client.0.smithi090.stdout:9/426: truncate db/dd/d18/d31/d58/f6c 481538 0 2024-04-26T04:42:23.824 INFO:tasks.workunit.client.1.smithi104.stdout:9/713: mknod d11/d15/d98/cea 0 2024-04-26T04:42:23.824 INFO:tasks.workunit.client.1.smithi104.stdout:9/714: creat d11/d15/d32/d4c/d7c/feb x:0 0 0 2024-04-26T04:42:23.825 INFO:tasks.workunit.client.0.smithi090.stdout:9/427: truncate f2 4076804 0 2024-04-26T04:42:23.826 INFO:tasks.workunit.client.0.smithi090.stdout:9/428: truncate db/dd/d18/d38/f42 1028497 0 2024-04-26T04:42:23.826 INFO:tasks.workunit.client.0.smithi090.stdout:3/508: dwrite d1/d17/d36/f24 [4194304,4194304] 0 2024-04-26T04:42:23.826 INFO:tasks.workunit.client.1.smithi104.stdout:9/715: mknod d11/d15/d1b/d6d/dd5/cec 0 2024-04-26T04:42:23.826 INFO:tasks.workunit.client.0.smithi090.stdout:0/632: getdents d0/d4b 0 2024-04-26T04:42:23.834 INFO:tasks.workunit.client.0.smithi090.stdout:9/429: creat db/dd/d18/d57/f94 x:0 0 0 2024-04-26T04:42:23.839 INFO:tasks.workunit.client.1.smithi104.stdout:5/859: write d0/d1/d4/d2a/d6b/d91/dc7/fcd [763857,7018] 0 2024-04-26T04:42:23.839 INFO:tasks.workunit.client.1.smithi104.stdout:0/741: dwrite d0/d2/d21/d49/d5e/d9c/d53/f9d [0,4194304] 0 2024-04-26T04:42:23.840 INFO:tasks.workunit.client.0.smithi090.stdout:4/539: dread d3/db/d12/d20/d2d/d58/d6a/d74/fa4 [0,4194304] 0 2024-04-26T04:42:23.841 INFO:tasks.workunit.client.1.smithi104.stdout:5/860: creat d0/d1/f127 x:0 0 0 2024-04-26T04:42:23.841 INFO:tasks.workunit.client.1.smithi104.stdout:5/861: truncate d0/d7b/ff0 597923 0 2024-04-26T04:42:23.842 INFO:tasks.workunit.client.1.smithi104.stdout:5/862: dread d0/d1/d4/d90/d109/dfe/fc0 [0,4194304] 0 2024-04-26T04:42:23.843 INFO:tasks.workunit.client.0.smithi090.stdout:4/540: mkdir d3/db/d12/d20/d24/da9 0 2024-04-26T04:42:23.843 INFO:tasks.workunit.client.0.smithi090.stdout:4/541: fdatasync d3/db/d12/fa0 0 2024-04-26T04:42:23.843 INFO:tasks.workunit.client.0.smithi090.stdout:4/542: write d3/db/f19 [1961040,26347] 0 2024-04-26T04:42:23.844 INFO:tasks.workunit.client.0.smithi090.stdout:4/543: creat d3/db/d12/d20/d24/da9/faa x:0 0 0 2024-04-26T04:42:23.844 INFO:tasks.workunit.client.0.smithi090.stdout:4/544: fdatasync d3/db/d12/d3b/f78 0 2024-04-26T04:42:23.844 INFO:tasks.workunit.client.1.smithi104.stdout:0/742: creat d0/d4/d3b/ff2 x:0 0 0 2024-04-26T04:42:23.845 INFO:tasks.workunit.client.0.smithi090.stdout:4/545: symlink d3/db/d12/d20/d2d/d58/lab 0 2024-04-26T04:42:23.845 INFO:tasks.workunit.client.0.smithi090.stdout:4/546: readlink d3/db/d12/d20/d24/d43/l46 0 2024-04-26T04:42:23.846 INFO:tasks.workunit.client.0.smithi090.stdout:4/547: creat d3/db/fac x:0 0 0 2024-04-26T04:42:23.847 INFO:tasks.workunit.client.0.smithi090.stdout:4/548: mknod d3/db/d12/d20/d2d/d58/cad 0 2024-04-26T04:42:23.847 INFO:tasks.workunit.client.1.smithi104.stdout:5/863: mkdir d0/d1/de6/d128 0 2024-04-26T04:42:23.848 INFO:tasks.workunit.client.1.smithi104.stdout:5/864: write d0/d1/d64/f31 [3058601,126333] 0 2024-04-26T04:42:23.851 INFO:tasks.workunit.client.0.smithi090.stdout:4/549: rename d3/d3e to d3/d13/d64/dae 0 2024-04-26T04:42:23.856 INFO:tasks.workunit.client.0.smithi090.stdout:0/633: rmdir d0/d4b/d5d 39 2024-04-26T04:42:23.856 INFO:tasks.workunit.client.0.smithi090.stdout:0/634: chown d0/de 1131 1 2024-04-26T04:42:23.856 INFO:tasks.workunit.client.0.smithi090.stdout:0/635: chown d0/df/l2a 380685 1 2024-04-26T04:42:23.859 INFO:tasks.workunit.client.0.smithi090.stdout:0/636: creat d0/d4b/d5d/d5c/d8a/d45/d93/fdf x:0 0 0 2024-04-26T04:42:23.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/637: creat d0/d4b/d5d/d5c/d8a/d64/fe0 x:0 0 0 2024-04-26T04:42:23.860 INFO:tasks.workunit.client.0.smithi090.stdout:0/638: dread - d0/d4b/d5d/d48/f56 zero size 2024-04-26T04:42:23.860 INFO:tasks.workunit.client.0.smithi090.stdout:4/550: dread d3/db/d12/d20/d2d/d58/f63 [0,4194304] 0 2024-04-26T04:42:23.860 INFO:tasks.workunit.client.0.smithi090.stdout:4/551: write d3/db/d12/d20/d24/da9/faa [683081,109445] 0 2024-04-26T04:42:23.861 INFO:tasks.workunit.client.0.smithi090.stdout:0/639: symlink d0/d4b/d5d/d48/d7d/le1 0 2024-04-26T04:42:23.861 INFO:tasks.workunit.client.0.smithi090.stdout:4/552: mknod d3/d13/caf 0 2024-04-26T04:42:23.868 INFO:tasks.workunit.client.1.smithi104.stdout:9/716: dwrite d11/d15/d1b/d6d/fc0 [0,4194304] 0 2024-04-26T04:42:23.868 INFO:tasks.workunit.client.1.smithi104.stdout:4/889: dwrite d4/d7/d1d/d1e/d24/d4a/f82 [0,4194304] 0 2024-04-26T04:42:23.869 INFO:tasks.workunit.client.0.smithi090.stdout:8/575: dwrite dc/d1f/d49/d3b/f4a [0,4194304] 0 2024-04-26T04:42:23.870 INFO:tasks.workunit.client.1.smithi104.stdout:0/743: dwrite d0/f91 [0,4194304] 0 2024-04-26T04:42:23.870 INFO:tasks.workunit.client.1.smithi104.stdout:0/744: creat d0/d2/ff3 x:0 0 0 2024-04-26T04:42:23.873 INFO:tasks.workunit.client.0.smithi090.stdout:8/576: mknod dc/d1f/d49/d4f/da6/db1/cc9 0 2024-04-26T04:42:23.876 INFO:tasks.workunit.client.1.smithi104.stdout:4/890: getdents d4/db/d6b 0 2024-04-26T04:42:23.881 INFO:tasks.workunit.client.1.smithi104.stdout:0/745: mkdir d0/d2/d21/d27/d1e/d57/d6c/d85/df4 0 2024-04-26T04:42:23.881 INFO:tasks.workunit.client.1.smithi104.stdout:0/746: chown d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1 2 1 2024-04-26T04:42:23.888 INFO:tasks.workunit.client.0.smithi090.stdout:8/577: read dc/d1f/d49/d3b/f85 [1625887,85347] 0 2024-04-26T04:42:23.888 INFO:tasks.workunit.client.0.smithi090.stdout:8/578: chown dc/d5a/l76 902143157 1 2024-04-26T04:42:23.896 INFO:tasks.workunit.client.0.smithi090.stdout:0/640: dwrite d0/d4b/d5d/d5c/d8a/d45/f96 [0,4194304] 0 2024-04-26T04:42:23.897 INFO:tasks.workunit.client.0.smithi090.stdout:0/641: fdatasync d0/d4b/d5d/d5c/d8a/d7c/f95 0 2024-04-26T04:42:23.908 INFO:tasks.workunit.client.0.smithi090.stdout:8/579: rename dc/d1f/d49 to dc/dca 0 2024-04-26T04:42:23.909 INFO:tasks.workunit.client.0.smithi090.stdout:8/580: mknod dc/dca/d3b/d3d/ccb 0 2024-04-26T04:42:23.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/642: dwrite d0/d4b/fad [0,4194304] 0 2024-04-26T04:42:23.916 INFO:tasks.workunit.client.0.smithi090.stdout:0/643: readlink d0/d4b/d5d/d5c/d8a/d45/d53/l54 0 2024-04-26T04:42:23.926 INFO:tasks.workunit.client.0.smithi090.stdout:8/581: dwrite dc/dca/d3b/fbf [0,4194304] 0 2024-04-26T04:42:23.926 INFO:tasks.workunit.client.0.smithi090.stdout:8/582: write dc/dca/d4f/d58/d95/f60 [4437558,12984] 0 2024-04-26T04:42:23.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:23 smithi090 ceph-mon[26648]: from='client.24363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:23 smithi104 ceph-mon[32369]: from='client.24363 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:24.016 INFO:tasks.workunit.client.1.smithi104.stdout:7/797: dwrite d2/d4/d5a/f7b [4194304,4194304] 0 2024-04-26T04:42:24.016 INFO:tasks.workunit.client.1.smithi104.stdout:7/798: chown d2/d4/d32/d3d/d47/db1/dfa 234325489 1 2024-04-26T04:42:24.016 INFO:tasks.workunit.client.1.smithi104.stdout:7/799: fdatasync d2/d4/f11 0 2024-04-26T04:42:24.022 INFO:tasks.workunit.client.1.smithi104.stdout:3/913: dwrite d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/f98 [0,4194304] 0 2024-04-26T04:42:24.025 INFO:tasks.workunit.client.1.smithi104.stdout:3/914: mkdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/d13c/d13f 0 2024-04-26T04:42:24.026 INFO:tasks.workunit.client.1.smithi104.stdout:9/717: dwrite d11/d53/d84/fe9 [0,4194304] 0 2024-04-26T04:42:24.026 INFO:tasks.workunit.client.1.smithi104.stdout:9/718: write d11/d15/f59 [350305,94153] 0 2024-04-26T04:42:24.040 INFO:tasks.workunit.client.0.smithi090.stdout:1/504: sync 2024-04-26T04:42:24.040 INFO:tasks.workunit.client.0.smithi090.stdout:2/566: sync 2024-04-26T04:42:24.041 INFO:tasks.workunit.client.0.smithi090.stdout:1/505: stat d0/d4/d7/d17/f2a 0 2024-04-26T04:42:24.041 INFO:tasks.workunit.client.0.smithi090.stdout:1/506: fdatasync d0/d4/d7/d69/f71 0 2024-04-26T04:42:24.041 INFO:tasks.workunit.client.1.smithi104.stdout:5/865: dwrite d0/d1/d4/d2a/d2b/d37/f8e [0,4194304] 0 2024-04-26T04:42:24.041 INFO:tasks.workunit.client.1.smithi104.stdout:7/800: dwrite d2/fe8 [0,4194304] 0 2024-04-26T04:42:24.041 INFO:tasks.workunit.client.1.smithi104.stdout:7/801: chown d2/d4/f26 15 1 2024-04-26T04:42:24.041 INFO:tasks.workunit.client.1.smithi104.stdout:4/891: dwrite d4/d7/d1d/d1e/d24/f68 [0,4194304] 0 2024-04-26T04:42:24.042 INFO:tasks.workunit.client.0.smithi090.stdout:8/583: dwrite dc/dca/d4f/f9d [0,4194304] 0 2024-04-26T04:42:24.042 INFO:tasks.workunit.client.0.smithi090.stdout:8/584: chown dc/d5a/d5d 22 1 2024-04-26T04:42:24.042 INFO:tasks.workunit.client.0.smithi090.stdout:8/585: chown dc/dca/d3b/d3d/d97/dad 205514 1 2024-04-26T04:42:24.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/507: creat d0/d4/d43/f9f x:0 0 0 2024-04-26T04:42:24.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/508: write d0/d4/f15 [2007781,3292] 0 2024-04-26T04:42:24.043 INFO:tasks.workunit.client.0.smithi090.stdout:1/509: write d0/d4/d7/d17/d22/f5c [1922868,126762] 0 2024-04-26T04:42:24.044 INFO:tasks.workunit.client.0.smithi090.stdout:8/586: getdents dc 0 2024-04-26T04:42:24.044 INFO:tasks.workunit.client.0.smithi090.stdout:8/587: truncate dc/dca/d39/fba 341580 0 2024-04-26T04:42:24.044 INFO:tasks.workunit.client.0.smithi090.stdout:8/588: fdatasync dc/dca/d3b/d3d/f65 0 2024-04-26T04:42:24.048 INFO:tasks.workunit.client.0.smithi090.stdout:8/589: dread dc/dca/d39/f7c [0,4194304] 0 2024-04-26T04:42:24.048 INFO:tasks.workunit.client.0.smithi090.stdout:8/590: write dc/dca/f20 [988521,41083] 0 2024-04-26T04:42:24.050 INFO:tasks.workunit.client.0.smithi090.stdout:2/567: write db/d59/f62 [530198,83859] 0 2024-04-26T04:42:24.050 INFO:tasks.workunit.client.1.smithi104.stdout:4/892: rmdir d4/d7/d1d/d1e/d24/d4a 39 2024-04-26T04:42:24.051 INFO:tasks.workunit.client.0.smithi090.stdout:1/510: dread d0/d4/f27 [0,4194304] 0 2024-04-26T04:42:24.051 INFO:tasks.workunit.client.1.smithi104.stdout:7/802: dread d2/d4/d5a/fec [0,4194304] 0 2024-04-26T04:42:24.051 INFO:tasks.workunit.client.1.smithi104.stdout:5/866: getdents d0/d41/d4f/df2/d108 0 2024-04-26T04:42:24.052 INFO:tasks.workunit.client.1.smithi104.stdout:6/898: sync 2024-04-26T04:42:24.052 INFO:tasks.workunit.client.1.smithi104.stdout:1/893: sync 2024-04-26T04:42:24.052 INFO:tasks.workunit.client.1.smithi104.stdout:4/893: write d4/d7/d1d/d1e/d24/d4a/f50 [396342,118645] 0 2024-04-26T04:42:24.054 INFO:tasks.workunit.client.1.smithi104.stdout:6/899: stat d0/d9/l4d 0 2024-04-26T04:42:24.055 INFO:tasks.workunit.client.0.smithi090.stdout:2/568: dread db/dc/f19 [0,4194304] 0 2024-04-26T04:42:24.056 INFO:tasks.workunit.client.0.smithi090.stdout:8/591: dread dc/dca/d39/f52 [0,4194304] 0 2024-04-26T04:42:24.056 INFO:tasks.workunit.client.0.smithi090.stdout:2/569: getdents db/dc/d6f 0 2024-04-26T04:42:24.056 INFO:tasks.workunit.client.1.smithi104.stdout:5/867: read d0/d1/d4/d2a/d6b/d91/fc4 [3219070,31446] 0 2024-04-26T04:42:24.056 INFO:tasks.workunit.client.1.smithi104.stdout:5/868: fsync d0/d1/d1e/d98/dd4/ff9 0 2024-04-26T04:42:24.056 INFO:tasks.workunit.client.1.smithi104.stdout:4/894: write d4/d7/d1d/d2e/d74/f93 [596,15829] 0 2024-04-26T04:42:24.056 INFO:tasks.workunit.client.1.smithi104.stdout:4/895: truncate d4/d7/d1d/d1e/d24/d59/db8/fe3 822747 0 2024-04-26T04:42:24.058 INFO:tasks.workunit.client.1.smithi104.stdout:4/896: rename d4/d20/d55/d56/la0 to d4/db/d6b/l137 0 2024-04-26T04:42:24.058 INFO:tasks.workunit.client.1.smithi104.stdout:4/897: chown d4/d7/d1d/d1e/d8f/db6/d7d/cc7 5 1 2024-04-26T04:42:24.058 INFO:tasks.workunit.client.1.smithi104.stdout:4/898: chown d4/d7/d1d/d1e/d8f/db6/d7d/cc7 1479 1 2024-04-26T04:42:24.059 INFO:tasks.workunit.client.0.smithi090.stdout:1/511: read d0/d4/f20 [2567464,15773] 0 2024-04-26T04:42:24.059 INFO:tasks.workunit.client.0.smithi090.stdout:1/512: truncate d0/d4/f15 3089283 0 2024-04-26T04:42:24.061 INFO:tasks.workunit.client.1.smithi104.stdout:4/899: mkdir d4/d7/d1d/d2e/d74/dbd/d138 0 2024-04-26T04:42:24.063 INFO:tasks.workunit.client.0.smithi090.stdout:1/513: rename d0/d4/f48 to d0/d4/d7/fa0 0 2024-04-26T04:42:24.063 INFO:tasks.workunit.client.0.smithi090.stdout:1/514: chown d0/d4/d7/d17/f79 1 1 2024-04-26T04:42:24.065 INFO:tasks.workunit.client.0.smithi090.stdout:1/515: symlink d0/d4/d7/d49/la1 0 2024-04-26T04:42:24.069 INFO:tasks.workunit.client.0.smithi090.stdout:1/516: unlink d0/d4/cd 0 2024-04-26T04:42:24.069 INFO:tasks.workunit.client.0.smithi090.stdout:1/517: creat d0/d4/d7/d4a/d75/d80/fa2 x:0 0 0 2024-04-26T04:42:24.069 INFO:tasks.workunit.client.0.smithi090.stdout:8/592: dread dc/d1f/f21 [0,4194304] 0 2024-04-26T04:42:24.071 INFO:tasks.workunit.client.0.smithi090.stdout:1/518: symlink d0/d4/d7/d4a/d75/d80/d58/la3 0 2024-04-26T04:42:24.072 INFO:tasks.workunit.client.1.smithi104.stdout:6/900: dread d0/d9/dc6/f7c [0,4194304] 0 2024-04-26T04:42:24.073 INFO:tasks.workunit.client.0.smithi090.stdout:8/593: creat dc/dca/d3b/d3d/fcc x:0 0 0 2024-04-26T04:42:24.074 INFO:tasks.workunit.client.1.smithi104.stdout:6/901: rename d0/d9/d13/d41/d96/ded/d10d/d47/da4/dfc to d0/d9/d13/d41/d96/ded/d10d/d126 0 2024-04-26T04:42:24.074 INFO:tasks.workunit.client.1.smithi104.stdout:6/902: write d0/d9/d13/d41/d96/ded/d10d/f124 [598687,71915] 0 2024-04-26T04:42:24.074 INFO:tasks.workunit.client.1.smithi104.stdout:6/903: write d0/d9/d13/d41/d96/fb0 [4633438,76998] 0 2024-04-26T04:42:24.075 INFO:tasks.workunit.client.0.smithi090.stdout:1/519: rmdir d0/d4/d62/d78 39 2024-04-26T04:42:24.078 INFO:tasks.workunit.client.1.smithi104.stdout:6/904: symlink d0/d9/d13/d41/d82/dc8/l127 0 2024-04-26T04:42:24.078 INFO:tasks.workunit.client.1.smithi104.stdout:6/905: write d0/d9/d13/d41/d82/f121 [137291,63158] 0 2024-04-26T04:42:24.078 INFO:tasks.workunit.client.0.smithi090.stdout:1/520: truncate d0/f2 3233621 0 2024-04-26T04:42:24.079 INFO:tasks.workunit.client.0.smithi090.stdout:1/521: truncate d0/d4/d7/d4a/f51 2255016 0 2024-04-26T04:42:24.080 INFO:tasks.workunit.client.0.smithi090.stdout:8/594: rename dc/d34/d87 to dc/d34/dcd 0 2024-04-26T04:42:24.080 INFO:tasks.workunit.client.1.smithi104.stdout:6/906: rename d0/d9/d13/d41/d96/ded/d10d/lcb to d0/d9/l128 0 2024-04-26T04:42:24.081 INFO:tasks.workunit.client.0.smithi090.stdout:8/595: mkdir dc/d5a/d5d/d73/dce 0 2024-04-26T04:42:24.081 INFO:tasks.workunit.client.0.smithi090.stdout:8/596: chown dc/dca/d3b/d3d/cbd 63372 1 2024-04-26T04:42:24.081 INFO:tasks.workunit.client.0.smithi090.stdout:8/597: readlink dc/d34/l51 0 2024-04-26T04:42:24.082 INFO:tasks.workunit.client.1.smithi104.stdout:6/907: creat d0/d9/d13/d41/d96/ded/d10d/d71/f129 x:0 0 0 2024-04-26T04:42:24.082 INFO:tasks.workunit.client.1.smithi104.stdout:4/900: read d4/d7/d1d/d1e/d24/d4a/fac [1186791,21986] 0 2024-04-26T04:42:24.083 INFO:tasks.workunit.client.0.smithi090.stdout:8/598: link dc/d34/f84 dc/d1f/fcf 0 2024-04-26T04:42:24.083 INFO:tasks.workunit.client.0.smithi090.stdout:8/599: fsync dc/dca/d3b/d3d/d97/fa2 0 2024-04-26T04:42:24.083 INFO:tasks.workunit.client.0.smithi090.stdout:8/600: readlink dc/dca/d39/d55/d64/l45 0 2024-04-26T04:42:24.083 INFO:tasks.workunit.client.0.smithi090.stdout:8/601: chown dc/d5a 1 1 2024-04-26T04:42:24.083 INFO:tasks.workunit.client.0.smithi090.stdout:8/602: write dc/d5a/d5d/f69 [1032905,20825] 0 2024-04-26T04:42:24.084 INFO:tasks.workunit.client.0.smithi090.stdout:8/603: mkdir dc/dca/d4f/da6/dd0 0 2024-04-26T04:42:24.091 INFO:tasks.workunit.client.1.smithi104.stdout:1/894: dwrite d0/d6/d2e/d32/d4c/db1/dc3/f108 [0,4194304] 0 2024-04-26T04:42:24.096 INFO:tasks.workunit.client.1.smithi104.stdout:5/869: dwrite d0/f22 [0,4194304] 0 2024-04-26T04:42:24.096 INFO:tasks.workunit.client.1.smithi104.stdout:7/803: dwrite d2/d4/d32/d35/dbe/feb [0,4194304] 0 2024-04-26T04:42:24.097 INFO:tasks.workunit.client.0.smithi090.stdout:2/570: dwrite db/d11/d14/d63/fac [0,4194304] 0 2024-04-26T04:42:24.101 INFO:tasks.workunit.client.1.smithi104.stdout:5/870: dread d0/d1/d4/d90/d109/dfe/fc0 [0,4194304] 0 2024-04-26T04:42:24.101 INFO:tasks.workunit.client.1.smithi104.stdout:5/871: write d0/d1/d1e/fb1 [757684,28984] 0 2024-04-26T04:42:24.101 INFO:tasks.workunit.client.1.smithi104.stdout:5/872: fsync d0/d41/f74 0 2024-04-26T04:42:24.101 INFO:tasks.workunit.client.1.smithi104.stdout:5/873: chown d0/d1/d4/d2a/d2b/d37/d48/dc1/dfd 15 1 2024-04-26T04:42:24.102 INFO:tasks.workunit.client.1.smithi104.stdout:1/895: getdents d0/d6/d9e 0 2024-04-26T04:42:24.102 INFO:tasks.workunit.client.1.smithi104.stdout:1/896: getdents d0/d6/d19/d21/d66/d20/d107 0 2024-04-26T04:42:24.105 INFO:tasks.workunit.client.1.smithi104.stdout:1/897: mknod d0/d6/d19/d21/d66/d29/df7/c124 0 2024-04-26T04:42:24.106 INFO:tasks.workunit.client.1.smithi104.stdout:1/898: mkdir d0/d6/d2e/d32/d4c/db1/d125 0 2024-04-26T04:42:24.110 INFO:tasks.workunit.client.1.smithi104.stdout:1/899: creat d0/d6/d19/d21/d66/de4/d81/f126 x:0 0 0 2024-04-26T04:42:24.116 INFO:tasks.workunit.client.1.smithi104.stdout:1/900: mkdir d0/d6/d19/d21/d58/db4/d127 0 2024-04-26T04:42:24.117 INFO:tasks.workunit.client.1.smithi104.stdout:1/901: symlink d0/d6/d19/d21/d66/l128 0 2024-04-26T04:42:24.117 INFO:tasks.workunit.client.1.smithi104.stdout:1/902: creat d0/d6/d49/f129 x:0 0 0 2024-04-26T04:42:24.128 INFO:tasks.workunit.client.0.smithi090.stdout:8/604: dwrite dc/dca/d4f/d58/d95/f60 [0,4194304] 0 2024-04-26T04:42:24.131 INFO:tasks.workunit.client.1.smithi104.stdout:7/804: dwrite d2/d4/d32/d35/d44/d5b/db2/fe1 [0,4194304] 0 2024-04-26T04:42:24.131 INFO:tasks.workunit.client.1.smithi104.stdout:4/901: dwrite d4/db/f10e [4194304,4194304] 0 2024-04-26T04:42:24.131 INFO:tasks.workunit.client.1.smithi104.stdout:7/805: fsync d2/fe8 0 2024-04-26T04:42:24.132 INFO:tasks.workunit.client.1.smithi104.stdout:6/908: dwrite d0/d9/d13/d41/d82/dc8/d80/f75 [0,4194304] 0 2024-04-26T04:42:24.133 INFO:tasks.workunit.client.0.smithi090.stdout:5/538: sync 2024-04-26T04:42:24.137 INFO:tasks.workunit.client.1.smithi104.stdout:7/806: stat d2/d4/l17 0 2024-04-26T04:42:24.140 INFO:tasks.workunit.client.1.smithi104.stdout:4/902: unlink d4/db/f28 0 2024-04-26T04:42:24.140 INFO:tasks.workunit.client.1.smithi104.stdout:4/903: chown d4/d7/d1d/d2e/d74/dbd 2050830 1 2024-04-26T04:42:24.145 INFO:tasks.workunit.client.1.smithi104.stdout:6/909: rename d0/d9/d13/d41/d82/dc8/dba/daa/fdc to d0/d9/d13/d41/d96/dac/f12a 0 2024-04-26T04:42:24.145 INFO:tasks.workunit.client.1.smithi104.stdout:5/874: dwrite d0/d1/d4/d2a/d6b/d91/fb0 [0,4194304] 0 2024-04-26T04:42:24.145 INFO:tasks.workunit.client.1.smithi104.stdout:5/875: write d0/d1/d1e/df5/f11f [815216,105247] 0 2024-04-26T04:42:24.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.144+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f943819df20 con 0x7f9438101450 2024-04-26T04:42:24.146 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.144+0000 7f943c852700 1 -- 172.21.15.90:0/3364988170 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7f942c003680 con 0x7f9438101450 2024-04-26T04:42:24.146 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:42:24.147 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:42:24.147 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:42:24.147 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:24.147 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:42:24.147 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:42:24.147 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:42:24.148 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.146+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f942006c680 msgr2=0x7f942006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.146+0000 7f943d854700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f942006c680 0x7f942006eb30 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f94381012b0 tx=0x7f9428009260 comp rx=0 tx=0).stop 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.146+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 msgr2=0x7f9438073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.146+0000 7f943d854700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f9438073140 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f942c000c00 tx=0x7f942c00f690 comp rx=0 tx=0).stop 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.147+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 shutdown_connections 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.147+0000 7f943d854700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f942006c680 0x7f942006eb30 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.147+0000 7f943d854700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9438100250 0x7f9438074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.147+0000 7f943d854700 1 --2- 172.21.15.90:0/3364988170 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9438101450 0x7f9438073140 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.147+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 >> 172.21.15.90:0/3364988170 conn(0x7f94380fb7e0 msgr2=0x7f9438104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.147+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 shutdown_connections 2024-04-26T04:42:24.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.147+0000 7f943d854700 1 -- 172.21.15.90:0/3364988170 wait complete. 2024-04-26T04:42:24.150 INFO:tasks.workunit.client.1.smithi104.stdout:7/807: mkdir d2/d4/d32/d60/dff 0 2024-04-26T04:42:24.151 INFO:tasks.workunit.client.1.smithi104.stdout:6/910: mknod d0/d9/d13/d41/d96/c12b 0 2024-04-26T04:42:24.151 INFO:tasks.workunit.client.1.smithi104.stdout:5/876: write d0/d1/d4/d2a/d2b/d37/f57 [354089,125418] 0 2024-04-26T04:42:24.151 INFO:tasks.workunit.client.1.smithi104.stdout:5/877: write d0/d1/d4/d2a/d2b/d37/d48/dc1/dfd/f114 [853842,31966] 0 2024-04-26T04:42:24.151 INFO:tasks.workunit.client.1.smithi104.stdout:4/904: write d4/d7/d1d/d1e/d24/d4a/f82 [1115244,29001] 0 2024-04-26T04:42:24.151 INFO:tasks.workunit.client.1.smithi104.stdout:7/808: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/dd8/d100 0 2024-04-26T04:42:24.151 INFO:tasks.workunit.client.1.smithi104.stdout:6/911: symlink d0/d9/d13/d41/d96/dac/dc4/l12c 0 2024-04-26T04:42:24.153 INFO:tasks.workunit.client.1.smithi104.stdout:4/905: symlink d4/d7/d1d/d2e/d74/dbd/dd3/l139 0 2024-04-26T04:42:24.153 INFO:tasks.workunit.client.1.smithi104.stdout:4/906: fsync d4/d7/d1d/d1e/f61 0 2024-04-26T04:42:24.153 INFO:tasks.workunit.client.1.smithi104.stdout:7/809: truncate d2/d4/d32/d3d/fb4 1452566 0 2024-04-26T04:42:24.153 INFO:tasks.workunit.client.1.smithi104.stdout:7/810: chown d2/d4/d32/d3d/d41/d5f/d6c/d94 6 1 2024-04-26T04:42:24.154 INFO:tasks.workunit.client.1.smithi104.stdout:6/912: link d0/d9/f49 d0/d9/d13/d41/d96/dac/db3/f12d 0 2024-04-26T04:42:24.155 INFO:tasks.workunit.client.1.smithi104.stdout:4/907: dread d4/d7/d1d/d1e/d24/d59/f100 [0,4194304] 0 2024-04-26T04:42:24.155 INFO:tasks.workunit.client.1.smithi104.stdout:4/908: truncate d4/f71 1099457 0 2024-04-26T04:42:24.155 INFO:tasks.workunit.client.1.smithi104.stdout:7/811: rename d2/d4/d1d/d34/d7f/cdb to d2/d4/d32/d3d/d41/dbd/c101 0 2024-04-26T04:42:24.157 INFO:tasks.workunit.client.1.smithi104.stdout:1/903: dwrite d0/d6/d19/d21/d66/de4/d81/f8a [0,4194304] 0 2024-04-26T04:42:24.157 INFO:tasks.workunit.client.1.smithi104.stdout:1/904: write d0/d6/d19/d21/d66/de4/d7e/ff9 [29394,39454] 0 2024-04-26T04:42:24.157 INFO:tasks.workunit.client.1.smithi104.stdout:1/905: write d0/d6/d9e/d10c/fd7 [1437563,11371] 0 2024-04-26T04:42:24.157 INFO:tasks.workunit.client.1.smithi104.stdout:6/913: truncate d0/d9/d13/d41/d82/dc8/fb6 4490572 0 2024-04-26T04:42:24.157 INFO:tasks.workunit.client.1.smithi104.stdout:6/914: write d0/d9/d13/d41/d96/ded/d10d/d71/f4b [587550,8434] 0 2024-04-26T04:42:24.157 INFO:tasks.workunit.client.1.smithi104.stdout:6/915: readlink d0/d9/d13/l1c 0 2024-04-26T04:42:24.158 INFO:tasks.workunit.client.0.smithi090.stdout:8/605: dwrite dc/d34/dcd/fa4 [0,4194304] 0 2024-04-26T04:42:24.159 INFO:tasks.workunit.client.0.smithi090.stdout:8/606: symlink dc/ld1 0 2024-04-26T04:42:24.161 INFO:tasks.workunit.client.1.smithi104.stdout:5/878: dread d0/d1/d4/d2a/d2b/d37/f8e [0,4194304] 0 2024-04-26T04:42:24.161 INFO:tasks.workunit.client.1.smithi104.stdout:5/879: fsync d0/d1/d4/f8c 0 2024-04-26T04:42:24.161 INFO:tasks.workunit.client.1.smithi104.stdout:5/880: read d0/d1/d4/f24 [57249,33388] 0 2024-04-26T04:42:24.162 INFO:tasks.workunit.client.1.smithi104.stdout:5/881: dread d0/d1/d4/d2a/d6b/d73/d11d/d125/faf [0,4194304] 0 2024-04-26T04:42:24.163 INFO:tasks.workunit.client.1.smithi104.stdout:4/909: rename d4/d7/d1d/d1e/d24/d59/dc1/lca to d4/d7/d1d/d1e/d8f/db6/l13a 0 2024-04-26T04:42:24.163 INFO:tasks.workunit.client.1.smithi104.stdout:4/910: dread - d4/d7/d1d/d2e/d74/dbd/dd3/fd7 zero size 2024-04-26T04:42:24.163 INFO:tasks.workunit.client.1.smithi104.stdout:4/911: chown d4/d20/f36 317655 1 2024-04-26T04:42:24.163 INFO:tasks.workunit.client.1.smithi104.stdout:4/912: readlink d4/d7/d1d/d2e/d74/dbd/lf9 0 2024-04-26T04:42:24.163 INFO:tasks.workunit.client.1.smithi104.stdout:4/913: creat d4/db/d6b/d7f/f13b x:0 0 0 2024-04-26T04:42:24.164 INFO:tasks.workunit.client.1.smithi104.stdout:7/812: mknod d2/d4/d32/d35/d44/d5b/db2/dd3/c102 0 2024-04-26T04:42:24.167 INFO:tasks.workunit.client.0.smithi090.stdout:8/607: dread dc/dca/d39/d55/f6c [0,4194304] 0 2024-04-26T04:42:24.169 INFO:tasks.workunit.client.1.smithi104.stdout:1/906: link d0/d6/d19/d21/d66/d29/c82 d0/d6/d19/d21/d66/d20/c12a 0 2024-04-26T04:42:24.169 INFO:tasks.workunit.client.1.smithi104.stdout:1/907: chown d0/d6/d2e/d32/d11f 1 1 2024-04-26T04:42:24.171 INFO:tasks.workunit.client.1.smithi104.stdout:6/916: link d0/d9/dc6/l33 d0/d9/d13/d41/d96/dac/l12e 0 2024-04-26T04:42:24.171 INFO:tasks.workunit.client.1.smithi104.stdout:6/917: write d0/d9/d13/d41/d82/dc8/f2b [2966496,38513] 0 2024-04-26T04:42:24.171 INFO:tasks.workunit.client.1.smithi104.stdout:6/918: chown d0/d9/d13/d41/d96/ded/d10d/d71/f4b 727053 1 2024-04-26T04:42:24.174 INFO:tasks.workunit.client.1.smithi104.stdout:5/882: mknod d0/d1/d4/d2a/d2b/d37/d48/dc6/c129 0 2024-04-26T04:42:24.178 INFO:tasks.workunit.client.1.smithi104.stdout:7/813: mknod d2/d4/d32/d60/c103 0 2024-04-26T04:42:24.178 INFO:tasks.workunit.client.1.smithi104.stdout:7/814: chown d2/d4/d1d/d34/d7f/faf 217668 1 2024-04-26T04:42:24.179 INFO:tasks.workunit.client.1.smithi104.stdout:1/908: unlink d0/fbf 0 2024-04-26T04:42:24.179 INFO:tasks.workunit.client.1.smithi104.stdout:1/909: read - d0/d6/d2e/d32/fc7 zero size 2024-04-26T04:42:24.182 INFO:tasks.workunit.client.1.smithi104.stdout:5/883: symlink d0/d1/d4/d2a/d2b/d37/d48/dc1/l12a 0 2024-04-26T04:42:24.183 INFO:tasks.workunit.client.1.smithi104.stdout:5/884: truncate d0/d1/d1e/fd6 317298 0 2024-04-26T04:42:24.186 INFO:tasks.workunit.client.1.smithi104.stdout:4/914: rename d4/d7/d1d/d1e/d24/f68 to d4/d7/d1d/d1e/d24/d4a/dbb/f13c 0 2024-04-26T04:42:24.186 INFO:tasks.workunit.client.1.smithi104.stdout:7/815: read d2/d4/d5a/da0/fab [2019062,82165] 0 2024-04-26T04:42:24.189 INFO:tasks.workunit.client.1.smithi104.stdout:8/772: sync 2024-04-26T04:42:24.189 INFO:tasks.workunit.client.1.smithi104.stdout:0/747: sync 2024-04-26T04:42:24.189 INFO:tasks.workunit.client.1.smithi104.stdout:5/885: unlink d0/d1/d64/f6c 0 2024-04-26T04:42:24.189 INFO:tasks.workunit.client.1.smithi104.stdout:0/748: write d0/d2/d21/f52 [4685158,119241] 0 2024-04-26T04:42:24.189 INFO:tasks.workunit.client.1.smithi104.stdout:0/749: fdatasync d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fd8 0 2024-04-26T04:42:24.190 INFO:tasks.workunit.client.1.smithi104.stdout:0/750: readlink d0/d2/d21/d49/d5e/d9c/db2/lf0 0 2024-04-26T04:42:24.192 INFO:tasks.workunit.client.1.smithi104.stdout:8/773: rename d0/d1c/f84 to d0/d50/ff9 0 2024-04-26T04:42:24.193 INFO:tasks.workunit.client.1.smithi104.stdout:5/886: mknod d0/d1/d4/d2a/d6b/d116/db5/c12b 0 2024-04-26T04:42:24.193 INFO:tasks.workunit.client.1.smithi104.stdout:5/887: write d0/d1/d64/d26/d36/da7/db3/fb4 [696323,110621] 0 2024-04-26T04:42:24.193 INFO:tasks.workunit.client.1.smithi104.stdout:5/888: readlink d0/d41/d4f/l9a 0 2024-04-26T04:42:24.193 INFO:tasks.workunit.client.1.smithi104.stdout:0/751: mknod d0/d2/d21/d49/d5e/d9c/db2/cf5 0 2024-04-26T04:42:24.197 INFO:tasks.workunit.client.1.smithi104.stdout:8/774: rename d0/d33/d35/d6d/f9b to d0/d1c/d59/ffa 0 2024-04-26T04:42:24.197 INFO:tasks.workunit.client.1.smithi104.stdout:8/775: fdatasync d0/f4e 0 2024-04-26T04:42:24.198 INFO:tasks.workunit.client.0.smithi090.stdout:8/608: dwrite dc/dca/d39/d55/f6c [4194304,4194304] 0 2024-04-26T04:42:24.199 INFO:tasks.workunit.client.1.smithi104.stdout:0/752: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d6f/df6 0 2024-04-26T04:42:24.201 INFO:tasks.workunit.client.1.smithi104.stdout:0/753: mkdir d0/d2/d21/d49/d5e/d9c/df7 0 2024-04-26T04:42:24.202 INFO:tasks.workunit.client.0.smithi090.stdout:8/609: rename dc/dca to dc/dd2 0 2024-04-26T04:42:24.207 INFO:tasks.workunit.client.0.smithi090.stdout:8/610: symlink dc/d1f/ld3 0 2024-04-26T04:42:24.207 INFO:tasks.workunit.client.0.smithi090.stdout:8/611: chown dc/dd2/l16 1232878109 1 2024-04-26T04:42:24.207 INFO:tasks.workunit.client.1.smithi104.stdout:3/915: sync 2024-04-26T04:42:24.207 INFO:tasks.workunit.client.1.smithi104.stdout:5/889: dread d0/d1/d1e/f8f [4194304,4194304] 0 2024-04-26T04:42:24.208 INFO:tasks.workunit.client.1.smithi104.stdout:5/890: fdatasync d0/d1/d4/d90/d109/da9/fce 0 2024-04-26T04:42:24.209 INFO:tasks.workunit.client.1.smithi104.stdout:7/816: read d2/d4/d32/d3d/fb4 [91519,69396] 0 2024-04-26T04:42:24.209 INFO:tasks.workunit.client.1.smithi104.stdout:7/817: write d2/d4/d32/d35/dbe/feb [5029849,36705] 0 2024-04-26T04:42:24.209 INFO:tasks.workunit.client.1.smithi104.stdout:7/818: creat d2/d4/d5a/f104 x:0 0 0 2024-04-26T04:42:24.209 INFO:tasks.workunit.client.1.smithi104.stdout:7/819: write d2/d4/d32/d3d/d41/d5f/d6d/faa [1280328,62434] 0 2024-04-26T04:42:24.213 INFO:tasks.workunit.client.1.smithi104.stdout:7/820: rmdir d2/d4/d32/d5e/d9f 0 2024-04-26T04:42:24.271 INFO:tasks.workunit.client.1.smithi104.stdout:7/821: dread d2/f56 [0,4194304] 0 2024-04-26T04:42:24.272 INFO:tasks.workunit.client.1.smithi104.stdout:7/822: chown d2/le 67841 1 2024-04-26T04:42:24.276 INFO:tasks.workunit.client.1.smithi104.stdout:4/915: dread d4/db/d6b/fa2 [0,4194304] 0 2024-04-26T04:42:24.290 INFO:tasks.workunit.client.1.smithi104.stdout:0/754: dwrite d0/d2/d21/d49/d5e/d9c/d53/d5d/d6f/f8c [0,4194304] 0 2024-04-26T04:42:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.291+0000 7f0628e24700 1 -- 172.21.15.90:0/2115240672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 msgr2=0x7f0624107b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.291+0000 7f0628e24700 1 --2- 172.21.15.90:0/2115240672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624107b10 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f0618009910 tx=0x7f0618009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.291+0000 7f0628e24700 1 -- 172.21.15.90:0/2115240672 shutdown_connections 2024-04-26T04:42:24.297 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.291+0000 7f0628e24700 1 --2- 172.21.15.90:0/2115240672 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624107b10 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.291+0000 7f0628e24700 1 --2- 172.21.15.90:0/2115240672 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0624069250 0x7f06241051f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.291+0000 7f0628e24700 1 -- 172.21.15.90:0/2115240672 >> 172.21.15.90:0/2115240672 conn(0x7f06240fa780 msgr2=0x7f06240fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 -- 172.21.15.90:0/2115240672 shutdown_connections 2024-04-26T04:42:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 -- 172.21.15.90:0/2115240672 wait complete. 2024-04-26T04:42:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 Processor -- start 2024-04-26T04:42:24.298 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 -- start start 2024-04-26T04:42:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0624069250 0x7f0624197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0624198890 con 0x7f0624105730 2024-04-26T04:42:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.292+0000 7f0628e24700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062419d2a0 con 0x7f0624069250 2024-04-26T04:42:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:24.299 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0621d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624198270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:52774/0 (socket says 172.21.15.90:52774) 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0621d9b700 1 -- 172.21.15.90:0/4193963988 learned_addr learned my addr 172.21.15.90:0/4193963988 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f062259c700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0624069250 0x7f0624197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0621d9b700 1 -- 172.21.15.90:0/4193963988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0624069250 msgr2=0x7f0624197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0621d9b700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0624069250 0x7f0624197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0621d9b700 1 -- 172.21.15.90:0/4193963988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06180095f0 con 0x7f0624105730 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0621d9b700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624198270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f061800f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f06137fe700 1 -- 172.21.15.90:0/4193963988 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f061801d070 con 0x7f0624105730 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f06137fe700 1 -- 172.21.15.90:0/4193963988 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0618005dc0 con 0x7f0624105730 2024-04-26T04:42:24.300 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f06137fe700 1 -- 172.21.15.90:0/4193963988 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f06180226a0 con 0x7f0624105730 2024-04-26T04:42:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062419d440 con 0x7f0624105730 2024-04-26T04:42:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.293+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062419d900 con 0x7f0624105730 2024-04-26T04:42:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.294+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0624191e10 con 0x7f0624105730 2024-04-26T04:42:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.295+0000 7f06137fe700 1 -- 172.21.15.90:0/4193963988 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f06180038f0 con 0x7f0624105730 2024-04-26T04:42:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.295+0000 7f06137fe700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f060c06c680 0x7f060c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.295+0000 7f06137fe700 1 -- 172.21.15.90:0/4193963988 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f061808cd50 con 0x7f0624105730 2024-04-26T04:42:24.301 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.295+0000 7f062259c700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f060c06c680 0x7f060c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.296+0000 7f062259c700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f060c06c680 0x7f060c06eb30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0624199210 tx=0x7f0614009260 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:24.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.300+0000 7f06137fe700 1 -- 172.21.15.90:0/4193963988 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f06180573b0 con 0x7f0624105730 2024-04-26T04:42:24.307 INFO:tasks.workunit.client.1.smithi104.stdout:0/755: dread d0/d2/d10/d3d/f9e [0,4194304] 0 2024-04-26T04:42:24.309 INFO:tasks.workunit.client.1.smithi104.stdout:7/823: dwrite d2/d4/d32/d3d/fb4 [0,4194304] 0 2024-04-26T04:42:24.309 INFO:tasks.workunit.client.1.smithi104.stdout:3/916: dwrite d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/fd3 [0,4194304] 0 2024-04-26T04:42:24.309 INFO:tasks.workunit.client.0.smithi090.stdout:8/612: dwrite dc/d34/dcd/fc5 [0,4194304] 0 2024-04-26T04:42:24.310 INFO:tasks.workunit.client.0.smithi090.stdout:8/613: chown dc/dd2/d3b/d3d/db8 11697544 1 2024-04-26T04:42:24.310 INFO:tasks.workunit.client.1.smithi104.stdout:1/910: dwrite d0/d6/d2e/d32/d4c/db1/dc3/f108 [0,4194304] 0 2024-04-26T04:42:24.310 INFO:tasks.workunit.client.1.smithi104.stdout:5/891: dwrite d0/d1/d4/d90/d109/da9/fce [0,4194304] 0 2024-04-26T04:42:24.310 INFO:tasks.workunit.client.1.smithi104.stdout:1/911: write d0/d6/d19/d21/feb [350109,2932] 0 2024-04-26T04:42:24.310 INFO:tasks.workunit.client.1.smithi104.stdout:4/916: dwrite d4/d7/d1d/d1e/d24/d4a/f50 [0,4194304] 0 2024-04-26T04:42:24.310 INFO:tasks.workunit.client.1.smithi104.stdout:5/892: write d0/d1/d4/fbb [1041483,35749] 0 2024-04-26T04:42:24.311 INFO:tasks.workunit.client.1.smithi104.stdout:0/756: link d0/d2/d21/d27/l48 d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/lf8 0 2024-04-26T04:42:24.311 INFO:tasks.workunit.client.0.smithi090.stdout:8/614: read dc/dd2/d3b/d3d/f5e [938084,91408] 0 2024-04-26T04:42:24.311 INFO:tasks.workunit.client.0.smithi090.stdout:8/615: truncate dc/d1f/fcf 4813257 0 2024-04-26T04:42:24.313 INFO:tasks.workunit.client.0.smithi090.stdout:8/616: symlink dc/d71/ld4 0 2024-04-26T04:42:24.315 INFO:tasks.workunit.client.1.smithi104.stdout:3/917: dread d5/f24 [0,4194304] 0 2024-04-26T04:42:24.315 INFO:tasks.workunit.client.0.smithi090.stdout:8/617: creat dc/dd2/d4f/da6/fd5 x:0 0 0 2024-04-26T04:42:24.318 INFO:tasks.workunit.client.1.smithi104.stdout:1/912: mkdir d0/d6/d19/d12b 0 2024-04-26T04:42:24.320 INFO:tasks.workunit.client.1.smithi104.stdout:4/917: creat d4/d7/d1d/f13d x:0 0 0 2024-04-26T04:42:24.320 INFO:tasks.workunit.client.1.smithi104.stdout:3/918: dread d5/db2/da3/dd0/ddb/f21 [0,4194304] 0 2024-04-26T04:42:24.320 INFO:tasks.workunit.client.1.smithi104.stdout:3/919: dread - d5/db2/da3/dd0/ddb/d32/d48/fc7 zero size 2024-04-26T04:42:24.321 INFO:tasks.workunit.client.1.smithi104.stdout:0/757: symlink d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dba/lf9 0 2024-04-26T04:42:24.324 INFO:tasks.workunit.client.1.smithi104.stdout:1/913: rename d0/d6/d19/d21/d66/de4/d81/f8a to d0/d6/d19/d21/d66/d29/d3f/f12c 0 2024-04-26T04:42:24.324 INFO:tasks.workunit.client.1.smithi104.stdout:1/914: fdatasync d0/d6/d9e/d10c/da5/fa6 0 2024-04-26T04:42:24.324 INFO:tasks.workunit.client.1.smithi104.stdout:1/915: readlink d0/d6/d9e/d10c/d35/l91 0 2024-04-26T04:42:24.325 INFO:tasks.workunit.client.1.smithi104.stdout:5/893: getdents d0/d1/d1e/df5 0 2024-04-26T04:42:24.325 INFO:tasks.workunit.client.1.smithi104.stdout:5/894: stat d0/f88 0 2024-04-26T04:42:24.325 INFO:tasks.workunit.client.1.smithi104.stdout:5/895: chown d0/d1/d4/d2a/d6b/d91/dbd 613 1 2024-04-26T04:42:24.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/918: symlink d4/d7/d1d/d1e/d15/dd8/l13e 0 2024-04-26T04:42:24.326 INFO:tasks.workunit.client.1.smithi104.stdout:4/919: creat d4/d7/d1d/d8e/f13f x:0 0 0 2024-04-26T04:42:24.328 INFO:tasks.workunit.client.1.smithi104.stdout:0/758: link d0/d2/d21/d27/f82 d0/d2/d21/d27/d1e/ffa 0 2024-04-26T04:42:24.329 INFO:tasks.workunit.client.1.smithi104.stdout:9/719: sync 2024-04-26T04:42:24.330 INFO:tasks.workunit.client.1.smithi104.stdout:7/824: dwrite d2/d4/d1d/d34/ff4 [0,4194304] 0 2024-04-26T04:42:24.331 INFO:tasks.workunit.client.1.smithi104.stdout:4/920: link d4/d7/d1d/d1e/d8f/db6/l77 d4/d7/d1d/d1e/d15/dd8/l140 0 2024-04-26T04:42:24.333 INFO:tasks.workunit.client.1.smithi104.stdout:1/916: dread d0/d6/f1a [0,4194304] 0 2024-04-26T04:42:24.340 INFO:tasks.workunit.client.1.smithi104.stdout:9/720: dread d11/d15/d32/d35/f80 [0,4194304] 0 2024-04-26T04:42:24.341 INFO:tasks.workunit.client.1.smithi104.stdout:4/921: mkdir d4/db/d6b/d141 0 2024-04-26T04:42:24.341 INFO:tasks.workunit.client.0.smithi090.stdout:8/618: dwrite dc/dd2/d4f/f9d [0,4194304] 0 2024-04-26T04:42:24.341 INFO:tasks.workunit.client.1.smithi104.stdout:0/759: write d0/d2/d10/d3d/f8d [7919631,118268] 0 2024-04-26T04:42:24.342 INFO:tasks.workunit.client.1.smithi104.stdout:9/721: mkdir d11/d15/d32/dd6/ded 0 2024-04-26T04:42:24.342 INFO:tasks.workunit.client.1.smithi104.stdout:9/722: dread - d11/d1d/d34/db3/de6/fe7 zero size 2024-04-26T04:42:24.342 INFO:tasks.workunit.client.1.smithi104.stdout:9/723: chown d11/c2c 4782152 1 2024-04-26T04:42:24.343 INFO:tasks.workunit.client.1.smithi104.stdout:9/724: fdatasync d11/d15/f3d 0 2024-04-26T04:42:24.343 INFO:tasks.workunit.client.1.smithi104.stdout:9/725: write d11/d15/d32/f56 [809772,78402] 0 2024-04-26T04:42:24.343 INFO:tasks.workunit.client.1.smithi104.stdout:9/726: stat d11/l3e 0 2024-04-26T04:42:24.347 INFO:tasks.workunit.client.1.smithi104.stdout:0/760: rename d0/d2/d21/d49/d5e/fb3 to d0/d2/d21/d49/ffb 0 2024-04-26T04:42:24.347 INFO:tasks.workunit.client.1.smithi104.stdout:0/761: truncate d0/d4/d3b/ff2 512653 0 2024-04-26T04:42:24.347 INFO:tasks.workunit.client.1.smithi104.stdout:0/762: chown d0/d2/da0/lb9 18133801 1 2024-04-26T04:42:24.349 INFO:tasks.workunit.client.1.smithi104.stdout:9/727: creat d11/d1d/d34/db3/fee x:0 0 0 2024-04-26T04:42:24.349 INFO:tasks.workunit.client.1.smithi104.stdout:9/728: chown d11/f20 28804 1 2024-04-26T04:42:24.352 INFO:tasks.workunit.client.0.smithi090.stdout:8/619: dread dc/dd2/d39/d55/d64/f8d [0,4194304] 0 2024-04-26T04:42:24.353 INFO:tasks.workunit.client.1.smithi104.stdout:0/763: getdents d0/d2/d10 0 2024-04-26T04:42:24.353 INFO:tasks.workunit.client.1.smithi104.stdout:0/764: dread - d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fc9 zero size 2024-04-26T04:42:24.357 INFO:tasks.workunit.client.0.smithi090.stdout:6/627: sync 2024-04-26T04:42:24.357 INFO:tasks.workunit.client.0.smithi090.stdout:7/574: sync 2024-04-26T04:42:24.358 INFO:tasks.workunit.client.1.smithi104.stdout:0/765: rmdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d80 39 2024-04-26T04:42:24.358 INFO:tasks.workunit.client.1.smithi104.stdout:0/766: truncate d0/d2/fe5 218453 0 2024-04-26T04:42:24.358 INFO:tasks.workunit.client.0.smithi090.stdout:7/575: mkdir d0/d1/d4/d3e/dd3 0 2024-04-26T04:42:24.359 INFO:tasks.workunit.client.1.smithi104.stdout:0/767: mknod d0/d2/d21/d49/d5e/d9c/db2/cfc 0 2024-04-26T04:42:24.359 INFO:tasks.workunit.client.1.smithi104.stdout:0/768: chown d0/d4/f11 7 1 2024-04-26T04:42:24.360 INFO:tasks.workunit.client.0.smithi090.stdout:7/576: symlink d0/d1/d4/d3e/ld4 0 2024-04-26T04:42:24.361 INFO:tasks.workunit.client.0.smithi090.stdout:7/577: mknod d0/db/df/d1c/d23/cd5 0 2024-04-26T04:42:24.361 INFO:tasks.workunit.client.1.smithi104.stdout:0/769: link d0/d2/d10/f71 d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/ffd 0 2024-04-26T04:42:24.363 INFO:tasks.workunit.client.1.smithi104.stdout:0/770: rmdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d6f/df6 0 2024-04-26T04:42:24.364 INFO:tasks.workunit.client.0.smithi090.stdout:6/628: dread d9/d21/d40/d4c/f59 [0,4194304] 0 2024-04-26T04:42:24.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/629: mkdir d9/d3c/db4/ddb 0 2024-04-26T04:42:24.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/630: fsync d9/d1c/dc5/fd0 0 2024-04-26T04:42:24.366 INFO:tasks.workunit.client.0.smithi090.stdout:6/631: creat d9/d21/d3b/d86/fdc x:0 0 0 2024-04-26T04:42:24.367 INFO:tasks.workunit.client.0.smithi090.stdout:7/578: dread d0/db/d15/f91 [0,4194304] 0 2024-04-26T04:42:24.368 INFO:tasks.workunit.client.0.smithi090.stdout:7/579: mkdir d0/db/df/d4d/d64/db0/dd6 0 2024-04-26T04:42:24.370 INFO:tasks.workunit.client.0.smithi090.stdout:7/580: mknod d0/db/d15/dbc/cd7 0 2024-04-26T04:42:24.371 INFO:tasks.workunit.client.0.smithi090.stdout:7/581: mknod d0/d1/d4/d3e/dd3/cd8 0 2024-04-26T04:42:24.373 INFO:tasks.workunit.client.0.smithi090.stdout:8/620: dwrite dc/dd2/d39/d55/d64/f7a [0,4194304] 0 2024-04-26T04:42:24.373 INFO:tasks.workunit.client.0.smithi090.stdout:8/621: readlink dc/dd2/d39/d55/d64/lb4 0 2024-04-26T04:42:24.373 INFO:tasks.workunit.client.1.smithi104.stdout:1/917: dwrite d0/d6/d19/d21/d66/d29/d3f/fa0 [0,4194304] 0 2024-04-26T04:42:24.374 INFO:tasks.workunit.client.0.smithi090.stdout:7/582: link d0/f8 d0/db/fd9 0 2024-04-26T04:42:24.375 INFO:tasks.workunit.client.0.smithi090.stdout:7/583: stat d0/d1/d4/d98/d65/dbb/dc0 0 2024-04-26T04:42:24.375 INFO:tasks.workunit.client.0.smithi090.stdout:7/584: chown d0/db/df/l12 19 1 2024-04-26T04:42:24.378 INFO:tasks.workunit.client.0.smithi090.stdout:8/622: link dc/dd2/d39/fba dc/dd2/fd6 0 2024-04-26T04:42:24.378 INFO:tasks.workunit.client.0.smithi090.stdout:8/623: write dc/d34/d4d/fc4 [156685,109899] 0 2024-04-26T04:42:24.389 INFO:tasks.workunit.client.1.smithi104.stdout:1/918: write d0/d6/d49/f4a [2559107,100560] 0 2024-04-26T04:42:24.389 INFO:tasks.workunit.client.1.smithi104.stdout:1/919: write d0/d6/d19/f111 [434333,54165] 0 2024-04-26T04:42:24.390 INFO:tasks.workunit.client.1.smithi104.stdout:1/920: chown d0/d6/d19/d21/d66/de4/d7e/ff9 20082 1 2024-04-26T04:42:24.390 INFO:tasks.workunit.client.1.smithi104.stdout:1/921: write d0/d6/d19/f111 [763606,118555] 0 2024-04-26T04:42:24.391 INFO:tasks.workunit.client.1.smithi104.stdout:1/922: chown d0/d6/d19/d21/d66/d20/d2a/lab 36476 1 2024-04-26T04:42:24.392 INFO:tasks.workunit.client.0.smithi090.stdout:8/624: dread dc/dd2/d3b/d3d/d97/fa2 [0,4194304] 0 2024-04-26T04:42:24.395 INFO:tasks.workunit.client.1.smithi104.stdout:0/771: dwrite d0/d2/d21/fdf [0,4194304] 0 2024-04-26T04:42:24.396 INFO:tasks.workunit.client.1.smithi104.stdout:0/772: write d0/d2/d10/d84/fb6 [429149,71609] 0 2024-04-26T04:42:24.396 INFO:tasks.workunit.client.0.smithi090.stdout:6/632: dwrite d9/d77/db0/db5/dda/d6d/fc6 [0,4194304] 0 2024-04-26T04:42:24.396 INFO:tasks.workunit.client.1.smithi104.stdout:1/923: rename d0/d6/d9e/d10c/da7 to d0/d6/d19/d21/d66/d20/d107/d12d 0 2024-04-26T04:42:24.396 INFO:tasks.workunit.client.1.smithi104.stdout:1/924: write d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/fb6 [127667,15865] 0 2024-04-26T04:42:24.398 INFO:tasks.workunit.client.1.smithi104.stdout:0/773: truncate d0/d2/d10/d84/f90 2871142 0 2024-04-26T04:42:24.400 INFO:tasks.workunit.client.0.smithi090.stdout:6/633: rename d9/d21/d3b/f7c to d9/d21/d40/d4c/d6f/d9f/dae/fdd 0 2024-04-26T04:42:24.401 INFO:tasks.workunit.client.1.smithi104.stdout:0/774: rename d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fbc to d0/d2/d21/d27/d1e/d57/d6c/dc7/ffe 0 2024-04-26T04:42:24.401 INFO:tasks.workunit.client.1.smithi104.stdout:0/775: dread - d0/d2/d21/d27/d1e/d57/d6c/dc7/ffe zero size 2024-04-26T04:42:24.402 INFO:tasks.workunit.client.0.smithi090.stdout:6/634: unlink d9/d21/d40/c6e 0 2024-04-26T04:42:24.402 INFO:tasks.workunit.client.1.smithi104.stdout:0/776: mknod d0/d2/d21/d27/d1e/d57/d6c/dd1/dcf/cff 0 2024-04-26T04:42:24.403 INFO:tasks.workunit.client.1.smithi104.stdout:0/777: write d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da3/fd8 [4732845,89767] 0 2024-04-26T04:42:24.403 INFO:tasks.workunit.client.1.smithi104.stdout:0/778: chown d0/d4/l92 8230525 1 2024-04-26T04:42:24.404 INFO:tasks.workunit.client.1.smithi104.stdout:0/779: unlink d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/cc4 0 2024-04-26T04:42:24.404 INFO:tasks.workunit.client.0.smithi090.stdout:6/635: rename d9/d1c/d31/d79/lbf to d9/d1c/da2/lde 0 2024-04-26T04:42:24.405 INFO:tasks.workunit.client.1.smithi104.stdout:0/780: creat d0/d2/d10/d3d/dc0/df1/f100 x:0 0 0 2024-04-26T04:42:24.406 INFO:tasks.workunit.client.0.smithi090.stdout:6/636: unlink d9/d77/db0/db5/dda/d58/dab/ld2 0 2024-04-26T04:42:24.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/637: mknod d9/d77/db0/db5/dda/d6d/d92/cdf 0 2024-04-26T04:42:24.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/638: dread - d9/d77/f7d zero size 2024-04-26T04:42:24.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/639: chown d9/d21/f5f 463981 1 2024-04-26T04:42:24.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/640: chown d9/d77/db0/db5/dda/d6d/d66/cac 242977301 1 2024-04-26T04:42:24.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/641: read d9/d21/d40/d4c/f59 [1450322,14056] 0 2024-04-26T04:42:24.409 INFO:tasks.workunit.client.0.smithi090.stdout:6/642: fdatasync d9/d77/db0/db5/dda/d6d/f93 0 2024-04-26T04:42:24.410 INFO:tasks.workunit.client.0.smithi090.stdout:6/643: symlink d9/d21/d40/d4c/d6f/le0 0 2024-04-26T04:42:24.410 INFO:tasks.workunit.client.0.smithi090.stdout:6/644: write d9/d21/f5f [2499645,128925] 0 2024-04-26T04:42:24.410 INFO:tasks.workunit.client.0.smithi090.stdout:6/645: creat d9/d21/d3b/d86/fe1 x:0 0 0 2024-04-26T04:42:24.411 INFO:tasks.workunit.client.1.smithi104.stdout:1/925: write d0/d6/d2e/f84 [3564944,97139] 0 2024-04-26T04:42:24.411 INFO:tasks.workunit.client.1.smithi104.stdout:1/926: creat d0/d6/d19/d21/f12e x:0 0 0 2024-04-26T04:42:24.412 INFO:tasks.workunit.client.0.smithi090.stdout:8/625: dwrite dc/dd2/d3b/f4a [0,4194304] 0 2024-04-26T04:42:24.412 INFO:tasks.workunit.client.0.smithi090.stdout:8/626: chown dc/dd2/d3b/d3d 733660 1 2024-04-26T04:42:24.412 INFO:tasks.workunit.client.0.smithi090.stdout:8/627: chown dc/dd2/f17 471800 1 2024-04-26T04:42:24.412 INFO:tasks.workunit.client.0.smithi090.stdout:8/628: chown dc/dd2/d4f/cc3 6465 1 2024-04-26T04:42:24.413 INFO:tasks.workunit.client.1.smithi104.stdout:1/927: dread d0/d6/d19/d21/d66/d20/d2a/d109/f11b [0,4194304] 0 2024-04-26T04:42:24.413 INFO:tasks.workunit.client.1.smithi104.stdout:1/928: chown d0/d6/d19/d21/d58/c93 0 1 2024-04-26T04:42:24.413 INFO:tasks.workunit.client.0.smithi090.stdout:6/646: rename d9/d77/db0/db5/dda/d6d/d66/cac to d9/d1c/ce2 0 2024-04-26T04:42:24.414 INFO:tasks.workunit.client.0.smithi090.stdout:6/647: chown d9/d1c/dc5/dd9 13318363 1 2024-04-26T04:42:24.414 INFO:tasks.workunit.client.0.smithi090.stdout:8/629: mknod dc/dd2/cd7 0 2024-04-26T04:42:24.417 INFO:tasks.workunit.client.1.smithi104.stdout:0/781: write d0/f2f [3066006,7326] 0 2024-04-26T04:42:24.417 INFO:tasks.workunit.client.0.smithi090.stdout:8/630: rename dc/d34/dcd to dc/dd2/d4f/da6/dd8 0 2024-04-26T04:42:24.419 INFO:tasks.workunit.client.0.smithi090.stdout:8/631: mkdir dc/dd2/dd9 0 2024-04-26T04:42:24.419 INFO:tasks.workunit.client.0.smithi090.stdout:8/632: dread - dc/dd2/d4f/f96 zero size 2024-04-26T04:42:24.420 INFO:tasks.workunit.client.1.smithi104.stdout:9/729: getdents d11/d15/d32/dd6 0 2024-04-26T04:42:24.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/585: dread d0/db/d53/f16 [0,4194304] 0 2024-04-26T04:42:24.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/586: fdatasync d0/d1/d4/fc5 0 2024-04-26T04:42:24.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/587: chown d0/db/d15/d29/fc 391 1 2024-04-26T04:42:24.422 INFO:tasks.workunit.client.0.smithi090.stdout:8/633: rename dc/d1f/l37 to dc/d5a/lda 0 2024-04-26T04:42:24.422 INFO:tasks.workunit.client.0.smithi090.stdout:8/634: write dc/d1f/f67 [884891,68611] 0 2024-04-26T04:42:24.423 INFO:tasks.workunit.client.1.smithi104.stdout:9/730: mkdir d11/d15/d1b/d6d/d9c/dbe/def 0 2024-04-26T04:42:24.423 INFO:tasks.workunit.client.1.smithi104.stdout:9/731: write d11/f95 [779450,119639] 0 2024-04-26T04:42:24.423 INFO:tasks.workunit.client.1.smithi104.stdout:9/732: write d11/d1d/d71/fcc [926530,85888] 0 2024-04-26T04:42:24.423 INFO:tasks.workunit.client.0.smithi090.stdout:8/635: symlink dc/d71/ldb 0 2024-04-26T04:42:24.425 INFO:tasks.workunit.client.0.smithi090.stdout:8/636: mkdir dc/dd2/d3b/d3d/db8/ddc 0 2024-04-26T04:42:24.430 INFO:tasks.workunit.client.0.smithi090.stdout:8/637: mkdir dc/dd2/d4f/da6/db1/ddd 0 2024-04-26T04:42:24.434 INFO:tasks.workunit.client.0.smithi090.stdout:3/509: sync 2024-04-26T04:42:24.435 INFO:tasks.workunit.client.1.smithi104.stdout:0/782: dread d0/d2/d21/d27/d1e/d57/d6c/f66 [0,4194304] 0 2024-04-26T04:42:24.435 INFO:tasks.workunit.client.1.smithi104.stdout:0/783: write d0/d2/d21/d49/d5e/d9c/d53/f9d [4609960,100300] 0 2024-04-26T04:42:24.435 INFO:tasks.workunit.client.1.smithi104.stdout:0/784: chown d0/d2/d21/d49/d5e/d9c/db2/cfc 5824635 1 2024-04-26T04:42:24.436 INFO:tasks.workunit.client.1.smithi104.stdout:7/825: dwrite d2/f75 [0,4194304] 0 2024-04-26T04:42:24.437 INFO:tasks.workunit.client.0.smithi090.stdout:3/510: mkdir d1/d17/d36/d33/daa 0 2024-04-26T04:42:24.438 INFO:tasks.workunit.client.1.smithi104.stdout:1/929: dwrite d0/d6/d19/d21/d59/f94 [0,4194304] 0 2024-04-26T04:42:24.440 INFO:tasks.workunit.client.1.smithi104.stdout:0/785: creat d0/d2/d21/d27/d1e/d57/d6c/dc7/f101 x:0 0 0 2024-04-26T04:42:24.440 INFO:tasks.workunit.client.1.smithi104.stdout:0/786: write d0/d2/d21/d27/d1e/f65 [4541697,109121] 0 2024-04-26T04:42:24.444 INFO:tasks.workunit.client.1.smithi104.stdout:9/733: write f4 [2285682,109817] 0 2024-04-26T04:42:24.444 INFO:tasks.workunit.client.1.smithi104.stdout:9/734: truncate d11/d15/da3/fd1 758813 0 2024-04-26T04:42:24.452 INFO:tasks.workunit.client.1.smithi104.stdout:1/930: truncate d0/d6/d19/d21/d28/f85 3306307 0 2024-04-26T04:42:24.455 INFO:tasks.workunit.client.1.smithi104.stdout:0/787: link d0/d2/d10/cee d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dba/c102 0 2024-04-26T04:42:24.456 INFO:tasks.workunit.client.1.smithi104.stdout:1/931: mknod d0/d6/d19/d21/d66/d20/d2a/d56/d67/d104/c12f 0 2024-04-26T04:42:24.458 INFO:tasks.workunit.client.1.smithi104.stdout:0/788: mknod d0/d2/d21/c103 0 2024-04-26T04:42:24.458 INFO:tasks.workunit.client.1.smithi104.stdout:0/789: chown d0/d2/d21/d49/d5e/fa7 154948319 1 2024-04-26T04:42:24.458 INFO:tasks.workunit.client.1.smithi104.stdout:1/932: creat d0/d6/d19/d21/d66/de4/d81/d110/f130 x:0 0 0 2024-04-26T04:42:24.464 INFO:tasks.workunit.client.1.smithi104.stdout:1/933: dread d0/d6/d19/d21/d66/de4/d7e/dc6/fce [0,4194304] 0 2024-04-26T04:42:24.464 INFO:tasks.workunit.client.1.smithi104.stdout:1/934: truncate d0/d6/d49/f129 884444 0 2024-04-26T04:42:24.465 INFO:tasks.workunit.client.1.smithi104.stdout:0/790: dread d0/d2/d21/d27/faa [0,4194304] 0 2024-04-26T04:42:24.466 INFO:tasks.workunit.client.1.smithi104.stdout:0/791: fdatasync d0/d2/d21/d49/d5e/d9c/f4b 0 2024-04-26T04:42:24.466 INFO:tasks.workunit.client.1.smithi104.stdout:0/792: creat d0/d2/f104 x:0 0 0 2024-04-26T04:42:24.466 INFO:tasks.workunit.client.1.smithi104.stdout:1/935: symlink d0/d6/d9e/l131 0 2024-04-26T04:42:24.467 INFO:tasks.workunit.client.1.smithi104.stdout:0/793: symlink d0/d2/d21/d49/d5e/l105 0 2024-04-26T04:42:24.468 INFO:tasks.workunit.client.1.smithi104.stdout:1/936: mkdir d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/dec/d132 0 2024-04-26T04:42:24.468 INFO:tasks.workunit.client.1.smithi104.stdout:1/937: creat d0/d6/d19/d21/d66/de4/d81/d110/f133 x:0 0 0 2024-04-26T04:42:24.470 INFO:tasks.workunit.client.1.smithi104.stdout:1/938: mkdir d0/d6/d2e/d32/d4c/db1/d125/d134 0 2024-04-26T04:42:24.471 INFO:tasks.workunit.client.1.smithi104.stdout:1/939: rename d0/d6/d19/d21/d66/d20/da3/dde/l100 to d0/d6/d2e/d32/d4c/db1/l135 0 2024-04-26T04:42:24.471 INFO:tasks.workunit.client.1.smithi104.stdout:1/940: chown d0/d6/d2e/f86 128729173 1 2024-04-26T04:42:24.472 INFO:tasks.workunit.client.1.smithi104.stdout:1/941: readlink d0/d6/d19/d21/d66/de4/l6c 0 2024-04-26T04:42:24.472 INFO:tasks.workunit.client.1.smithi104.stdout:0/794: dread d0/d2/d21/d49/f99 [0,4194304] 0 2024-04-26T04:42:24.472 INFO:tasks.workunit.client.1.smithi104.stdout:0/795: chown d0/d2/d21/d49/d5e/d9c/f46 14884 1 2024-04-26T04:42:24.472 INFO:tasks.workunit.client.0.smithi090.stdout:7/588: dwrite d0/db/d15/d29/f18 [0,4194304] 0 2024-04-26T04:42:24.473 INFO:tasks.workunit.client.1.smithi104.stdout:0/796: mknod d0/d2/d10/c106 0 2024-04-26T04:42:24.473 INFO:tasks.workunit.client.1.smithi104.stdout:0/797: read d0/d4/f11 [5639894,78401] 0 2024-04-26T04:42:24.476 INFO:tasks.workunit.client.0.smithi090.stdout:7/589: mknod d0/db/d15/dbc/dd2/cda 0 2024-04-26T04:42:24.476 INFO:tasks.workunit.client.0.smithi090.stdout:7/590: fsync d0/d1/d4/d98/d4a/f8c 0 2024-04-26T04:42:24.478 INFO:tasks.workunit.client.1.smithi104.stdout:0/798: truncate d0/d2/d21/d49/d5e/f78 3486527 0 2024-04-26T04:42:24.478 INFO:tasks.workunit.client.1.smithi104.stdout:0/799: symlink d0/d2/d21/d49/d5e/d9c/d53/d5d/l107 0 2024-04-26T04:42:24.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/591: mknod d0/db/d53/d5e/cdb 0 2024-04-26T04:42:24.479 INFO:tasks.workunit.client.0.smithi090.stdout:7/592: stat d0/db/d15/d29 0 2024-04-26T04:42:24.483 INFO:tasks.workunit.client.0.smithi090.stdout:3/511: dwrite d1/d6/f91 [0,4194304] 0 2024-04-26T04:42:24.486 INFO:tasks.workunit.client.1.smithi104.stdout:1/942: dread d0/d6/d49/f4a [0,4194304] 0 2024-04-26T04:42:24.486 INFO:tasks.workunit.client.1.smithi104.stdout:1/943: truncate d0/d6/d9e/d10c/d35/fd1 4268129 0 2024-04-26T04:42:24.486 INFO:tasks.workunit.client.1.smithi104.stdout:1/944: readlink d0/d6/d2e/d32/d4c/db1/dc3/ldb 0 2024-04-26T04:42:24.488 INFO:tasks.workunit.client.1.smithi104.stdout:1/945: mkdir d0/d6/d19/d21/d66/d20/d2a/d136 0 2024-04-26T04:42:24.491 INFO:tasks.workunit.client.0.smithi090.stdout:6/648: dwrite d9/d1c/f57 [0,4194304] 0 2024-04-26T04:42:24.492 INFO:tasks.workunit.client.0.smithi090.stdout:8/638: dwrite dc/dd2/d39/d55/f70 [0,4194304] 0 2024-04-26T04:42:24.492 INFO:tasks.workunit.client.1.smithi104.stdout:7/826: dwrite d2/d4/d1d/d34/d7f/faf [4194304,4194304] 0 2024-04-26T04:42:24.492 INFO:tasks.workunit.client.1.smithi104.stdout:1/946: link d0/d6/d2e/d32/d4d/l3e d0/d6/d19/d21/d66/d20/da3/l137 0 2024-04-26T04:42:24.492 INFO:tasks.workunit.client.1.smithi104.stdout:0/800: write d0/d2/d21/d49/f55 [2066396,23990] 0 2024-04-26T04:42:24.492 INFO:tasks.workunit.client.1.smithi104.stdout:0/801: write d0/d2/d21/d49/ffb [1066338,31898] 0 2024-04-26T04:42:24.494 INFO:tasks.workunit.client.0.smithi090.stdout:6/649: mknod d9/d77/db0/db5/dda/d6d/d66/ce3 0 2024-04-26T04:42:24.496 INFO:tasks.workunit.client.0.smithi090.stdout:3/512: write d1/d17/f25 [742744,112269] 0 2024-04-26T04:42:24.497 INFO:tasks.workunit.client.0.smithi090.stdout:3/513: chown d1/d17/d36/d33/d43/l81 1201910 1 2024-04-26T04:42:24.497 INFO:tasks.workunit.client.0.smithi090.stdout:3/514: chown d1/d6/d1b/d58/da0 115519899 1 2024-04-26T04:42:24.497 INFO:tasks.workunit.client.1.smithi104.stdout:0/802: mknod d0/d4/c108 0 2024-04-26T04:42:24.500 INFO:tasks.workunit.client.1.smithi104.stdout:9/735: dwrite d11/f95 [0,4194304] 0 2024-04-26T04:42:24.500 INFO:tasks.workunit.client.1.smithi104.stdout:9/736: truncate d11/d1d/d34/d69/fa7 203745 0 2024-04-26T04:42:24.501 INFO:tasks.workunit.client.1.smithi104.stdout:9/737: write d11/d15/d1b/f60 [8765368,64151] 0 2024-04-26T04:42:24.501 INFO:tasks.workunit.client.1.smithi104.stdout:9/738: fdatasync d11/d15/d32/d4c/d54/f75 0 2024-04-26T04:42:24.501 INFO:tasks.workunit.client.0.smithi090.stdout:6/650: link d9/d21/d40/d4c/d6f/d9f/dae/fdd d9/d1c/dc5/dd9/fe4 0 2024-04-26T04:42:24.504 INFO:tasks.workunit.client.0.smithi090.stdout:8/639: dread dc/dd2/d4f/f9d [0,4194304] 0 2024-04-26T04:42:24.505 INFO:tasks.workunit.client.0.smithi090.stdout:3/515: mkdir d1/d17/d36/dab 0 2024-04-26T04:42:24.505 INFO:tasks.workunit.client.0.smithi090.stdout:3/516: chown d1/d17/d36/d33/d42/d70/l7a 27029 1 2024-04-26T04:42:24.505 INFO:tasks.workunit.client.1.smithi104.stdout:1/947: write d0/d6/d2e/d32/d4c/fef [2034477,95727] 0 2024-04-26T04:42:24.508 INFO:tasks.workunit.client.0.smithi090.stdout:6/651: unlink d9/d21/d40/d4c/d6f/fb2 0 2024-04-26T04:42:24.509 INFO:tasks.workunit.client.0.smithi090.stdout:8/640: mknod dc/d34/d4d/cde 0 2024-04-26T04:42:24.510 INFO:tasks.workunit.client.1.smithi104.stdout:1/948: rename d0/d6/d19/d21/l37 to d0/l138 0 2024-04-26T04:42:24.511 INFO:tasks.workunit.client.1.smithi104.stdout:8/776: sync 2024-04-26T04:42:24.511 INFO:tasks.workunit.client.1.smithi104.stdout:6/919: sync 2024-04-26T04:42:24.511 INFO:tasks.workunit.client.1.smithi104.stdout:8/777: chown d0/d1c/d27/d47/d67/d76/d7f/fd8 205855 1 2024-04-26T04:42:24.511 INFO:tasks.workunit.client.1.smithi104.stdout:9/739: read d11/f95 [2639519,113146] 0 2024-04-26T04:42:24.514 INFO:tasks.workunit.client.0.smithi090.stdout:6/652: dread d9/d21/f48 [0,4194304] 0 2024-04-26T04:42:24.514 INFO:tasks.workunit.client.0.smithi090.stdout:6/653: fsync d9/d1c/dc5/fd0 0 2024-04-26T04:42:24.516 INFO:tasks.workunit.client.0.smithi090.stdout:3/517: link d1/d17/c23 d1/d17/d36/d33/d68/d76/cac 0 2024-04-26T04:42:24.519 INFO:tasks.workunit.client.0.smithi090.stdout:7/593: dwrite d0/db/df/d4d/f5a [0,4194304] 0 2024-04-26T04:42:24.524 INFO:tasks.workunit.client.1.smithi104.stdout:7/827: dwrite d2/d4/d32/d60/f82 [0,4194304] 0 2024-04-26T04:42:24.525 INFO:tasks.workunit.client.1.smithi104.stdout:1/949: rmdir d0/d6/d2e 39 2024-04-26T04:42:24.525 INFO:tasks.workunit.client.1.smithi104.stdout:1/950: chown d0/d6/d19/d21/f12e 55 1 2024-04-26T04:42:24.526 INFO:tasks.workunit.client.0.smithi090.stdout:6/654: creat d9/d77/db0/db5/dca/fe5 x:0 0 0 2024-04-26T04:42:24.527 INFO:tasks.workunit.client.1.smithi104.stdout:0/803: dwrite d0/d2/ff3 [0,4194304] 0 2024-04-26T04:42:24.527 INFO:tasks.workunit.client.1.smithi104.stdout:0/804: chown d0/d2/d21/d27/d1e 1959083411 1 2024-04-26T04:42:24.527 INFO:tasks.workunit.client.0.smithi090.stdout:3/518: dread d1/d5c/f99 [0,4194304] 0 2024-04-26T04:42:24.529 INFO:tasks.workunit.client.1.smithi104.stdout:7/828: mknod d2/d4/d32/d35/dbe/dda/c105 0 2024-04-26T04:42:24.529 INFO:tasks.workunit.client.1.smithi104.stdout:1/951: mkdir d0/d6/d19/d21/d66/d20/d2a/d109/d139 0 2024-04-26T04:42:24.529 INFO:tasks.workunit.client.1.smithi104.stdout:7/829: write d2/d4/d32/ff7 [798381,54250] 0 2024-04-26T04:42:24.532 INFO:tasks.workunit.client.0.smithi090.stdout:3/519: mknod d1/d17/d36/d33/d42/d70/cad 0 2024-04-26T04:42:24.533 INFO:tasks.workunit.client.0.smithi090.stdout:3/520: chown d1 388920 1 2024-04-26T04:42:24.534 INFO:tasks.workunit.client.1.smithi104.stdout:1/952: creat d0/d6/d9e/d10c/d35/f13a x:0 0 0 2024-04-26T04:42:24.535 INFO:tasks.workunit.client.0.smithi090.stdout:3/521: truncate d1/d17/f2f 2762108 0 2024-04-26T04:42:24.537 INFO:tasks.workunit.client.1.smithi104.stdout:1/953: dread d0/d6/d19/d21/d66/de4/d7e/dc6/fce [0,4194304] 0 2024-04-26T04:42:24.537 INFO:tasks.workunit.client.0.smithi090.stdout:3/522: mkdir d1/d17/d36/d33/d68/dae 0 2024-04-26T04:42:24.537 INFO:tasks.workunit.client.0.smithi090.stdout:3/523: creat d1/d17/d36/d33/d42/faf x:0 0 0 2024-04-26T04:42:24.541 INFO:tasks.workunit.client.0.smithi090.stdout:3/524: rename d1/d6 to d1/d17/d36/d33/d43/db0 0 2024-04-26T04:42:24.542 INFO:tasks.workunit.client.1.smithi104.stdout:1/954: rename d0/d6/d2e/d32/d4c/db1/d125/d134 to d0/d6/d19/d21/d66/d20/d107/d12d/d13b 0 2024-04-26T04:42:24.542 INFO:tasks.workunit.client.1.smithi104.stdout:1/955: write d0/d6/d19/fae [54345,57010] 0 2024-04-26T04:42:24.542 INFO:tasks.workunit.client.1.smithi104.stdout:1/956: chown d0/d6/d19/d21/d66/de4/d7e/dc6 27580499 1 2024-04-26T04:42:24.542 INFO:tasks.workunit.client.1.smithi104.stdout:1/957: stat d0/d6/d19/d21/d58/db4 0 2024-04-26T04:42:24.543 INFO:tasks.workunit.client.1.smithi104.stdout:1/958: readlink d0/d6/d19/d21/d66/d20/d2a/lca 0 2024-04-26T04:42:24.544 INFO:tasks.workunit.client.0.smithi090.stdout:3/525: write d1/d17/d36/d33/d43/db0/d2c/f5f [193884,104006] 0 2024-04-26T04:42:24.544 INFO:tasks.workunit.client.0.smithi090.stdout:6/655: dread d9/d1c/d31/fd1 [0,4194304] 0 2024-04-26T04:42:24.544 INFO:tasks.workunit.client.0.smithi090.stdout:6/656: readlink d9/d77/db0/db5/dda/d6d/d92/l27 0 2024-04-26T04:42:24.544 INFO:tasks.workunit.client.0.smithi090.stdout:6/657: write d9/d77/db0/db5/fa3 [435882,63296] 0 2024-04-26T04:42:24.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/658: write d9/d21/f5f [1502147,116815] 0 2024-04-26T04:42:24.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/659: write d9/d77/f9d [5238925,109654] 0 2024-04-26T04:42:24.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/660: fdatasync d9/d3c/fb3 0 2024-04-26T04:42:24.545 INFO:tasks.workunit.client.0.smithi090.stdout:3/526: mkdir d1/d17/d36/d33/d43/db1 0 2024-04-26T04:42:24.545 INFO:tasks.workunit.client.0.smithi090.stdout:3/527: fdatasync d1/d9/f38 0 2024-04-26T04:42:24.546 INFO:tasks.workunit.client.1.smithi104.stdout:1/959: creat d0/d6/d9e/d10c/da5/f13c x:0 0 0 2024-04-26T04:42:24.546 INFO:tasks.workunit.client.1.smithi104.stdout:1/960: rename d0/d6/d19/d21/d66/d20/c51 to d0/d6/d19/d21/d66/d20/d107/d12d/db8/c13d 0 2024-04-26T04:42:24.546 INFO:tasks.workunit.client.1.smithi104.stdout:1/961: stat d0/d6/d19/d21/d66/d20/d2a/lca 0 2024-04-26T04:42:24.547 INFO:tasks.workunit.client.1.smithi104.stdout:1/962: rmdir d0/d6/d2e/d32/d4c/db1/dc3 39 2024-04-26T04:42:24.547 INFO:tasks.workunit.client.1.smithi104.stdout:1/963: dread - d0/d6/d2e/d32/fc7 zero size 2024-04-26T04:42:24.548 INFO:tasks.workunit.client.1.smithi104.stdout:1/964: mknod d0/d6/d2e/d32/d4c/db1/dc3/c13e 0 2024-04-26T04:42:24.548 INFO:tasks.workunit.client.1.smithi104.stdout:1/965: dread - d0/d6/d9e/d10c/da5/f13c zero size 2024-04-26T04:42:24.548 INFO:tasks.workunit.client.1.smithi104.stdout:1/966: chown d0/d6/d19/d21/d66/d29/la4 12782 1 2024-04-26T04:42:24.549 INFO:tasks.workunit.client.1.smithi104.stdout:1/967: unlink d0/d6/l89 0 2024-04-26T04:42:24.551 INFO:tasks.workunit.client.1.smithi104.stdout:1/968: unlink d0/d6/d2e/d32/fac 0 2024-04-26T04:42:24.552 INFO:tasks.workunit.client.1.smithi104.stdout:1/969: link d0/d6/d19/d21/d66/d20/d107/d12d/db8/f10e d0/d6/d19/d21/d28/f13f 0 2024-04-26T04:42:24.566 INFO:tasks.workunit.client.0.smithi090.stdout:3/528: dread d1/d17/d52/f7c [0,4194304] 0 2024-04-26T04:42:24.568 INFO:tasks.workunit.client.0.smithi090.stdout:3/529: creat d1/d17/d36/d33/d43/db0/d2c/d97/fb2 x:0 0 0 2024-04-26T04:42:24.578 INFO:tasks.workunit.client.0.smithi090.stdout:8/641: dwrite dc/dd2/d4f/d58/d95/f83 [0,4194304] 0 2024-04-26T04:42:24.578 INFO:tasks.workunit.client.1.smithi104.stdout:6/920: dwrite d0/d9/f87 [4194304,4194304] 0 2024-04-26T04:42:24.578 INFO:tasks.workunit.client.1.smithi104.stdout:0/805: dwrite d0/d2/fe5 [0,4194304] 0 2024-04-26T04:42:24.578 INFO:tasks.workunit.client.1.smithi104.stdout:0/806: write d0/d2/d21/d27/d1e/d57/d6c/d85/fa8 [564867,55996] 0 2024-04-26T04:42:24.578 INFO:tasks.workunit.client.1.smithi104.stdout:9/740: dwrite d11/d15/d32/d4c/d54/f7a [0,4194304] 0 2024-04-26T04:42:24.579 INFO:tasks.workunit.client.1.smithi104.stdout:8/778: dwrite d0/d1c/d27/f86 [0,4194304] 0 2024-04-26T04:42:24.579 INFO:tasks.workunit.client.1.smithi104.stdout:8/779: write d0/f2f [5629771,123168] 0 2024-04-26T04:42:24.579 INFO:tasks.workunit.client.1.smithi104.stdout:1/970: dwrite d0/d6/d19/d21/d66/d20/d2a/f6f [0,4194304] 0 2024-04-26T04:42:24.579 INFO:tasks.workunit.client.1.smithi104.stdout:1/971: chown d0/d6/d2e/fb3 389 1 2024-04-26T04:42:24.579 INFO:tasks.workunit.client.1.smithi104.stdout:1/972: creat d0/d6/d19/d21/d66/d29/d3f/f140 x:0 0 0 2024-04-26T04:42:24.580 INFO:tasks.workunit.client.0.smithi090.stdout:7/594: dwrite d0/d1/d4/d98/d4a/fbe [0,4194304] 0 2024-04-26T04:42:24.582 INFO:tasks.workunit.client.1.smithi104.stdout:6/921: creat d0/d9/d13/d41/d96/dac/dec/dbd/f12f x:0 0 0 2024-04-26T04:42:24.584 INFO:tasks.workunit.client.1.smithi104.stdout:0/807: symlink d0/d2/d21/d27/d1e/d57/d6c/dd1/dcf/l109 0 2024-04-26T04:42:24.585 INFO:tasks.workunit.client.0.smithi090.stdout:3/530: dwrite d1/d17/d36/d33/d43/db0/d2c/f9b [0,4194304] 0 2024-04-26T04:42:24.587 INFO:tasks.workunit.client.1.smithi104.stdout:8/780: dread d0/f5 [0,4194304] 0 2024-04-26T04:42:24.587 INFO:tasks.workunit.client.1.smithi104.stdout:8/781: creat d0/d1c/d27/d47/d67/d76/daf/ffb x:0 0 0 2024-04-26T04:42:24.588 INFO:tasks.workunit.client.1.smithi104.stdout:8/782: write d0/d1c/d27/d47/d67/d76/daf/fb8 [2033333,122039] 0 2024-04-26T04:42:24.595 INFO:tasks.workunit.client.0.smithi090.stdout:7/595: mknod d0/db/df/d4d/d7b/cdc 0 2024-04-26T04:42:24.598 INFO:tasks.workunit.client.0.smithi090.stdout:8/642: rmdir dc/d5a/d5d 39 2024-04-26T04:42:24.600 INFO:tasks.workunit.client.0.smithi090.stdout:7/596: rmdir d0/d1/d4/d3e 39 2024-04-26T04:42:24.601 INFO:tasks.workunit.client.0.smithi090.stdout:7/597: rmdir d0/d1/d4/d3e 39 2024-04-26T04:42:24.603 INFO:tasks.workunit.client.0.smithi090.stdout:7/598: creat d0/db/df/d1c/d23/d8d/fdd x:0 0 0 2024-04-26T04:42:24.603 INFO:tasks.workunit.client.0.smithi090.stdout:7/599: truncate d0/d1/d4/fc5 756066 0 2024-04-26T04:42:24.603 INFO:tasks.workunit.client.0.smithi090.stdout:7/600: fsync d0/d1/da3/fa4 0 2024-04-26T04:42:24.603 INFO:tasks.workunit.client.0.smithi090.stdout:7/601: creat d0/d1/d4/d98/d4a/fde x:0 0 0 2024-04-26T04:42:24.605 INFO:tasks.workunit.client.1.smithi104.stdout:0/808: mknod d0/d2/d21/d27/d1e/d57/c10a 0 2024-04-26T04:42:24.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/602: creat d0/d1/d4/d98/d65/dbb/dc0/fdf x:0 0 0 2024-04-26T04:42:24.606 INFO:tasks.workunit.client.0.smithi090.stdout:7/603: chown d0/db/d15/d29/c3f 864975627 1 2024-04-26T04:42:24.607 INFO:tasks.workunit.client.0.smithi090.stdout:7/604: symlink d0/db/df/d4d/daf/le0 0 2024-04-26T04:42:24.607 INFO:tasks.workunit.client.1.smithi104.stdout:1/973: dwrite d0/d6/d19/d21/d66/f83 [0,4194304] 0 2024-04-26T04:42:24.609 INFO:tasks.workunit.client.0.smithi090.stdout:7/605: rename d0/db/df/l35 to d0/d1/d4/d98/d4a/le1 0 2024-04-26T04:42:24.610 INFO:tasks.workunit.client.1.smithi104.stdout:8/783: rename d0/f4e to d0/d1c/d27/d47/d67/d76/d7f/ffc 0 2024-04-26T04:42:24.610 INFO:tasks.workunit.client.1.smithi104.stdout:6/922: getdents d0/dd8/de0/d102 0 2024-04-26T04:42:24.610 INFO:tasks.workunit.client.0.smithi090.stdout:7/606: mkdir d0/d1/d4/d3e/dd3/de2 0 2024-04-26T04:42:24.612 INFO:tasks.workunit.client.0.smithi090.stdout:3/531: dwrite d1/d17/d36/f67 [0,4194304] 0 2024-04-26T04:42:24.612 INFO:tasks.workunit.client.0.smithi090.stdout:3/532: write d1/d59/fa1 [739558,34463] 0 2024-04-26T04:42:24.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/974: mkdir d0/d6/d19/d21/d66/de4/d7e/dc6/d141 0 2024-04-26T04:42:24.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/975: fdatasync d0/d6/d9e/d10c/d35/fd1 0 2024-04-26T04:42:24.612 INFO:tasks.workunit.client.1.smithi104.stdout:1/976: write d0/d6/d19/d21/d66/d20/d2a/f2c [4772051,120857] 0 2024-04-26T04:42:24.613 INFO:tasks.workunit.client.0.smithi090.stdout:3/533: creat d1/d17/d36/d33/d68/dae/fb3 x:0 0 0 2024-04-26T04:42:24.613 INFO:tasks.workunit.client.0.smithi090.stdout:3/534: write d1/d17/d36/fa9 [356654,92314] 0 2024-04-26T04:42:24.614 INFO:tasks.workunit.client.1.smithi104.stdout:8/784: link d0/d33/d35/d6d/f95 d0/d33/d35/ffd 0 2024-04-26T04:42:24.615 INFO:tasks.workunit.client.0.smithi090.stdout:8/643: dwrite dc/dd2/f28 [0,4194304] 0 2024-04-26T04:42:24.615 INFO:tasks.workunit.client.0.smithi090.stdout:7/607: dread d0/db/df/d4d/d64/d66/faa [0,4194304] 0 2024-04-26T04:42:24.615 INFO:tasks.workunit.client.0.smithi090.stdout:7/608: dread - d0/d1/d4/d98/d4a/f8c zero size 2024-04-26T04:42:24.616 INFO:tasks.workunit.client.0.smithi090.stdout:3/535: creat d1/d17/d36/d33/d43/db0/d1b/d58/da4/fb4 x:0 0 0 2024-04-26T04:42:24.616 INFO:tasks.workunit.client.0.smithi090.stdout:3/536: stat d1/d17/d36/d33/d43/db0/d1b/c22 0 2024-04-26T04:42:24.618 INFO:tasks.workunit.client.0.smithi090.stdout:8/644: creat dc/dd2/d4f/db7/fdf x:0 0 0 2024-04-26T04:42:24.621 INFO:tasks.workunit.client.0.smithi090.stdout:7/609: rename d0/db/df/l12 to d0/db/df/d4d/daf/dd1/le3 0 2024-04-26T04:42:24.621 INFO:tasks.workunit.client.0.smithi090.stdout:7/610: write d0/db/df/d4d/d64/d66/faa [1684243,59358] 0 2024-04-26T04:42:24.621 INFO:tasks.workunit.client.0.smithi090.stdout:7/611: chown d0/db/df/d1c/d23/f31 1 1 2024-04-26T04:42:24.621 INFO:tasks.workunit.client.0.smithi090.stdout:7/612: write d0/d1/f71 [786642,42330] 0 2024-04-26T04:42:24.621 INFO:tasks.workunit.client.0.smithi090.stdout:7/613: dread - d0/db/df/d1c/d23/d8d/fdd zero size 2024-04-26T04:42:24.622 INFO:tasks.workunit.client.1.smithi104.stdout:1/977: rmdir d0/d6/d49 39 2024-04-26T04:42:24.622 INFO:tasks.workunit.client.1.smithi104.stdout:1/978: truncate d0/d6/d2e/d32/d4c/fb0 797460 0 2024-04-26T04:42:24.623 INFO:tasks.workunit.client.0.smithi090.stdout:3/537: unlink d1/d17/d36/d33/d68/c8a 0 2024-04-26T04:42:24.624 INFO:tasks.workunit.client.1.smithi104.stdout:1/979: mkdir d0/d6/d19/d21/d58/db4/d127/d142 0 2024-04-26T04:42:24.624 INFO:tasks.workunit.client.1.smithi104.stdout:1/980: chown d0/d6/d2e/f86 1 1 2024-04-26T04:42:24.624 INFO:tasks.workunit.client.0.smithi090.stdout:8/645: symlink dc/dd2/d3b/d3d/db8/le0 0 2024-04-26T04:42:24.625 INFO:tasks.workunit.client.1.smithi104.stdout:4/922: sync 2024-04-26T04:42:24.625 INFO:tasks.workunit.client.1.smithi104.stdout:5/896: sync 2024-04-26T04:42:24.625 INFO:tasks.workunit.client.1.smithi104.stdout:5/897: readlink d0/d1/d4/d2a/d2b/d5c/l10e 0 2024-04-26T04:42:24.625 INFO:tasks.workunit.client.1.smithi104.stdout:3/920: sync 2024-04-26T04:42:24.626 INFO:tasks.workunit.client.0.smithi090.stdout:3/538: mkdir d1/d17/d36/d33/d43/db0/d1b/d58/db5 0 2024-04-26T04:42:24.626 INFO:tasks.workunit.client.1.smithi104.stdout:5/898: link d0/l2d d0/d1/d1e/d98/dd4/l12c 0 2024-04-26T04:42:24.626 INFO:tasks.workunit.client.1.smithi104.stdout:5/899: fdatasync d0/d1/de6/ff6 0 2024-04-26T04:42:24.627 INFO:tasks.workunit.client.1.smithi104.stdout:4/923: read d4/d7/d1d/d1e/d24/d59/db8/fe3 [575781,32234] 0 2024-04-26T04:42:24.627 INFO:tasks.workunit.client.1.smithi104.stdout:3/921: unlink d5/db2/d46/d95/f108 0 2024-04-26T04:42:24.627 INFO:tasks.workunit.client.1.smithi104.stdout:3/922: fsync d5/d9/d80/da2/fe3 0 2024-04-26T04:42:24.627 INFO:tasks.workunit.client.1.smithi104.stdout:3/923: truncate d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/fec 572701 0 2024-04-26T04:42:24.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/539: unlink d1/d17/d36/d33/d43/db0/c4c 0 2024-04-26T04:42:24.628 INFO:tasks.workunit.client.1.smithi104.stdout:5/900: symlink d0/d1/d1e/d98/dd4/l12d 0 2024-04-26T04:42:24.630 INFO:tasks.workunit.client.1.smithi104.stdout:4/924: creat d4/d7/d1d/d1e/d24/d4a/f142 x:0 0 0 2024-04-26T04:42:24.631 INFO:tasks.workunit.client.0.smithi090.stdout:3/540: rename d1/d17/d36/d33/d43/db0/c2e to d1/d17/d36/d2a/d3a/d5d/d82/cb6 0 2024-04-26T04:42:24.632 INFO:tasks.workunit.client.0.smithi090.stdout:3/541: write d1/d17/d36/d33/d42/faf [61243,108266] 0 2024-04-26T04:42:24.632 INFO:tasks.workunit.client.0.smithi090.stdout:3/542: chown d1/d17/d36/d33/d43/c51 0 1 2024-04-26T04:42:24.632 INFO:tasks.workunit.client.1.smithi104.stdout:3/924: truncate d5/d9/f12 7528808 0 2024-04-26T04:42:24.632 INFO:tasks.workunit.client.1.smithi104.stdout:3/925: fdatasync d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/f122 0 2024-04-26T04:42:24.633 INFO:tasks.workunit.client.0.smithi090.stdout:3/543: unlink d1/d17/d36/d33/d42/c8c 0 2024-04-26T04:42:24.634 INFO:tasks.workunit.client.1.smithi104.stdout:0/809: dwrite d0/d2/d21/d49/d5e/d9c/f86 [4194304,4194304] 0 2024-04-26T04:42:24.635 INFO:tasks.workunit.client.1.smithi104.stdout:4/925: creat d4/d7/d1d/d1e/d15/df3/f143 x:0 0 0 2024-04-26T04:42:24.635 INFO:tasks.workunit.client.1.smithi104.stdout:4/926: readlink d4/d20/d55/d56/da3/dea/dfd/d11a/l121 0 2024-04-26T04:42:24.637 INFO:tasks.workunit.client.1.smithi104.stdout:1/981: dread d0/d6/d2e/f86 [0,4194304] 0 2024-04-26T04:42:24.637 INFO:tasks.workunit.client.1.smithi104.stdout:1/982: chown d0/d6/d19/d21/d66/l10 10247 1 2024-04-26T04:42:24.637 INFO:tasks.workunit.client.1.smithi104.stdout:1/983: stat d0/d6/d19/d21/d66/d20/d2a/d56/d8c/fe1 0 2024-04-26T04:42:24.637 INFO:tasks.workunit.client.1.smithi104.stdout:1/984: chown d0/d6/d19/d21/d66/d20/d2a/d56/fd5 114713780 1 2024-04-26T04:42:24.637 INFO:tasks.workunit.client.1.smithi104.stdout:1/985: dread - d0/d6/d19/d21/d66/de4/d81/d110/f130 zero size 2024-04-26T04:42:24.639 INFO:tasks.workunit.client.0.smithi090.stdout:7/614: dread d0/d1/d4/f10 [0,4194304] 0 2024-04-26T04:42:24.639 INFO:tasks.workunit.client.1.smithi104.stdout:3/926: creat d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/d13c/f140 x:0 0 0 2024-04-26T04:42:24.639 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.638+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0624061190 con 0x7f060c06c680 2024-04-26T04:42:24.640 INFO:tasks.workunit.client.1.smithi104.stdout:0/810: stat d0/c93 0 2024-04-26T04:42:24.641 INFO:tasks.workunit.client.1.smithi104.stdout:5/901: write d0/d1/d4/d90/d109/da9/fce [2014183,14269] 0 2024-04-26T04:42:24.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.639+0000 7f06137fe700 1 -- 172.21.15.90:0/4193963988 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+439 (secure 0 0 0) 0x7f0624061190 con 0x7f060c06c680 2024-04-26T04:42:24.641 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:42:24.641 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:42:24.641 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:42:24.641 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:42:24.641 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-26T04:42:24.642 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "0/2 daemons upgraded", 2024-04-26T04:42:24.642 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Pulling quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 image on host smithi104", 2024-04-26T04:42:24.642 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:42:24.642 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:42:24.642 INFO:tasks.workunit.client.1.smithi104.stdout:4/927: rename d4/d20/d55/d56/da3/dea/f103 to d4/d7/d1d/d1e/d15/dd8/f144 0 2024-04-26T04:42:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f060c06c680 msgr2=0x7f060c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f060c06c680 0x7f060c06eb30 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0624199210 tx=0x7f0614009260 comp rx=0 tx=0).stop 2024-04-26T04:42:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 msgr2=0x7f0624198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624198270 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f0618000c00 tx=0x7f061800f690 comp rx=0 tx=0).stop 2024-04-26T04:42:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 shutdown_connections 2024-04-26T04:42:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f060c06c680 0x7f060c06eb30 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0624069250 0x7f0624197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 --2- 172.21.15.90:0/4193963988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0624105730 0x7f0624198270 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 >> 172.21.15.90:0/4193963988 conn(0x7f06240fa780 msgr2=0x7f06240fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 shutdown_connections 2024-04-26T04:42:24.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:24.642+0000 7f0628e24700 1 -- 172.21.15.90:0/4193963988 wait complete. 2024-04-26T04:42:24.644 INFO:tasks.workunit.client.1.smithi104.stdout:5/902: creat d0/d1/d4/d2a/d6b/d73/d11d/d125/db2/f12e x:0 0 0 2024-04-26T04:42:24.644 INFO:tasks.workunit.client.1.smithi104.stdout:5/903: write d0/d1/d4/d2a/d6b/d91/dbd/ffa [24956,98510] 0 2024-04-26T04:42:24.644 INFO:tasks.workunit.client.1.smithi104.stdout:5/904: write d0/d1/d64/d26/d36/da7/db3/fb4 [1614888,116289] 0 2024-04-26T04:42:24.645 INFO:tasks.workunit.client.1.smithi104.stdout:6/923: dwrite d0/d9/f3b [0,4194304] 0 2024-04-26T04:42:24.645 INFO:tasks.workunit.client.1.smithi104.stdout:6/924: chown d0/d9/l4d 1040395338 1 2024-04-26T04:42:24.645 INFO:tasks.workunit.client.0.smithi090.stdout:7/615: dread d0/d1/d4/fc5 [0,4194304] 0 2024-04-26T04:42:24.646 INFO:tasks.workunit.client.0.smithi090.stdout:7/616: creat d0/d1/da3/fe4 x:0 0 0 2024-04-26T04:42:24.646 INFO:tasks.workunit.client.1.smithi104.stdout:1/986: dread d0/d6/d19/d21/d66/d29/d3f/f5b [0,4194304] 0 2024-04-26T04:42:24.646 INFO:tasks.workunit.client.1.smithi104.stdout:3/927: rename d5/db2/c38 to d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/d124/c141 0 2024-04-26T04:42:24.648 INFO:tasks.workunit.client.0.smithi090.stdout:8/646: dwrite dc/dd2/d39/d55/d64/f7a [4194304,4194304] 0 2024-04-26T04:42:24.648 INFO:tasks.workunit.client.0.smithi090.stdout:8/647: fsync dc/f5c 0 2024-04-26T04:42:24.649 INFO:tasks.workunit.client.1.smithi104.stdout:1/987: symlink d0/d6/d2e/d32/d4c/l143 0 2024-04-26T04:42:24.649 INFO:tasks.workunit.client.1.smithi104.stdout:1/988: write d0/d6/d9e/d10c/da5/fd8 [377667,103333] 0 2024-04-26T04:42:24.651 INFO:tasks.workunit.client.0.smithi090.stdout:7/617: dread d0/db/d53/d5e/f8b [0,4194304] 0 2024-04-26T04:42:24.651 INFO:tasks.workunit.client.1.smithi104.stdout:3/928: rename d5/d9/d80/da2/fad to d5/d9/d80/f142 0 2024-04-26T04:42:24.651 INFO:tasks.workunit.client.1.smithi104.stdout:4/928: read d4/d7/d1d/f6f [2700915,79810] 0 2024-04-26T04:42:24.651 INFO:tasks.workunit.client.1.smithi104.stdout:4/929: dread - d4/d7/d1d/d1e/d24/d59/db3/fb5 zero size 2024-04-26T04:42:24.652 INFO:tasks.workunit.client.1.smithi104.stdout:1/989: symlink d0/d6/d19/d21/d66/de4/d81/d110/l144 0 2024-04-26T04:42:24.653 INFO:tasks.workunit.client.0.smithi090.stdout:8/648: stat dc/dd2/d39/d55/d64/l59 0 2024-04-26T04:42:24.653 INFO:tasks.workunit.client.0.smithi090.stdout:8/649: fsync dc/dd2/f17 0 2024-04-26T04:42:24.653 INFO:tasks.workunit.client.0.smithi090.stdout:8/650: write dc/dd2/d3b/d3d/f54 [4623511,114205] 0 2024-04-26T04:42:24.653 INFO:tasks.workunit.client.0.smithi090.stdout:8/651: write dc/dd2/d3b/d3d/d97/fa2 [4949471,76746] 0 2024-04-26T04:42:24.653 INFO:tasks.workunit.client.0.smithi090.stdout:7/618: mknod d0/db/d15/ce5 0 2024-04-26T04:42:24.655 INFO:tasks.workunit.client.0.smithi090.stdout:8/652: link dc/ce dc/dd2/d3b/d3d/db8/ddc/ce1 0 2024-04-26T04:42:24.657 INFO:tasks.workunit.client.0.smithi090.stdout:7/619: unlink d0/d1/d4/d98/db1/fb5 0 2024-04-26T04:42:24.659 INFO:tasks.workunit.client.1.smithi104.stdout:0/811: rename d0/d2/d21/d27/d1e/d57/d6c/dd1/lad to d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/l10b 0 2024-04-26T04:42:24.662 INFO:tasks.workunit.client.0.smithi090.stdout:8/653: unlink dc/dd2/d39/d55/d64/f89 0 2024-04-26T04:42:24.662 INFO:tasks.workunit.client.1.smithi104.stdout:3/929: unlink d5/db2/d60/ca8 0 2024-04-26T04:42:24.663 INFO:tasks.workunit.client.0.smithi090.stdout:8/654: mknod dc/d5a/d5d/d73/ce2 0 2024-04-26T04:42:24.663 INFO:tasks.workunit.client.1.smithi104.stdout:0/812: symlink d0/d2/da0/l10c 0 2024-04-26T04:42:24.664 INFO:tasks.workunit.client.0.smithi090.stdout:7/620: mkdir d0/db/de6 0 2024-04-26T04:42:24.666 INFO:tasks.workunit.client.1.smithi104.stdout:3/930: mkdir d5/d143 0 2024-04-26T04:42:24.666 INFO:tasks.workunit.client.1.smithi104.stdout:1/990: getdents d0/d6/d9e/d10c/da5 0 2024-04-26T04:42:24.666 INFO:tasks.workunit.client.1.smithi104.stdout:1/991: read d0/d6/d2e/f84 [2951596,12222] 0 2024-04-26T04:42:24.666 INFO:tasks.workunit.client.1.smithi104.stdout:4/930: write d4/d7/d1d/f45 [2098695,107734] 0 2024-04-26T04:42:24.667 INFO:tasks.workunit.client.0.smithi090.stdout:8/655: link dc/dd2/d4f/d58/d95/cbe dc/dd2/d3b/d3d/d97/dad/ce3 0 2024-04-26T04:42:24.668 INFO:tasks.workunit.client.1.smithi104.stdout:3/931: symlink d5/db2/da3/dd0/ddb/d32/d47/d11a/l144 0 2024-04-26T04:42:24.668 INFO:tasks.workunit.client.1.smithi104.stdout:1/992: symlink d0/d6/l145 0 2024-04-26T04:42:24.670 INFO:tasks.workunit.client.0.smithi090.stdout:8/656: symlink dc/dd2/le4 0 2024-04-26T04:42:24.673 INFO:tasks.workunit.client.0.smithi090.stdout:8/657: symlink dc/dd2/d4f/da6/db1/le5 0 2024-04-26T04:42:24.673 INFO:tasks.workunit.client.1.smithi104.stdout:5/905: dwrite d0/d1/d3e/f11e [0,4194304] 0 2024-04-26T04:42:24.674 INFO:tasks.workunit.client.1.smithi104.stdout:6/925: dwrite d0/d9/dc6/fce [0,4194304] 0 2024-04-26T04:42:24.674 INFO:tasks.workunit.client.1.smithi104.stdout:5/906: write d0/d1/d1e/f2e [706543,84358] 0 2024-04-26T04:42:24.675 INFO:tasks.workunit.client.1.smithi104.stdout:3/932: mknod d5/db2/d46/db7/de6/c145 0 2024-04-26T04:42:24.675 INFO:tasks.workunit.client.1.smithi104.stdout:3/933: chown d5/db2/da3/dd0/ddb/d32/d48/c105 2 1 2024-04-26T04:42:24.675 INFO:tasks.workunit.client.1.smithi104.stdout:3/934: write d5/db2/da3/dd0/ddb/d32/d48/dac/f89 [1009515,99772] 0 2024-04-26T04:42:24.687 INFO:tasks.workunit.client.1.smithi104.stdout:1/993: mknod d0/d6/d19/d21/d66/d20/d2a/c146 0 2024-04-26T04:42:24.697 INFO:tasks.workunit.client.1.smithi104.stdout:1/994: rename d0/d6/d19/d21/d66/d20/d2a/lca to d0/d6/d2e/d32/d4c/l147 0 2024-04-26T04:42:24.697 INFO:tasks.workunit.client.1.smithi104.stdout:1/995: mknod d0/d6/d2e/d32/d4c/db1/dc3/c148 0 2024-04-26T04:42:24.698 INFO:tasks.workunit.client.1.smithi104.stdout:4/931: write d4/d20/d55/d56/da3/dbf/fc5 [1447214,124881] 0 2024-04-26T04:42:24.699 INFO:tasks.workunit.client.1.smithi104.stdout:1/996: rename d0/d6/d2e/d32/d4c/db1/dc3/ccf to d0/d6/d9e/d10c/d10f/c149 0 2024-04-26T04:42:24.700 INFO:tasks.workunit.client.1.smithi104.stdout:4/932: creat d4/d7/d1d/d2e/d74/dbd/dd3/f145 x:0 0 0 2024-04-26T04:42:24.700 INFO:tasks.workunit.client.1.smithi104.stdout:4/933: write d4/d7/d1d/f6f [5229303,116963] 0 2024-04-26T04:42:24.700 INFO:tasks.workunit.client.1.smithi104.stdout:4/934: chown d4/d7/d1d/d8e/f124 3564 1 2024-04-26T04:42:24.700 INFO:tasks.workunit.client.1.smithi104.stdout:4/935: write d4/d7/d1d/d1e/d24/d59/db8/fe3 [1850837,55969] 0 2024-04-26T04:42:24.700 INFO:tasks.workunit.client.1.smithi104.stdout:4/936: truncate d4/d7/d1d/f13d 633252 0 2024-04-26T04:42:24.700 INFO:tasks.workunit.client.1.smithi104.stdout:1/997: creat d0/d6/d19/d21/d66/d20/d2a/d56/d67/d104/f14a x:0 0 0 2024-04-26T04:42:24.702 INFO:tasks.workunit.client.0.smithi090.stdout:8/658: dwrite dc/dd2/d3b/d3d/f9c [0,4194304] 0 2024-04-26T04:42:24.702 INFO:tasks.workunit.client.1.smithi104.stdout:0/813: dwrite d0/d2/d21/d27/d1e/d57/d6c/d85/faf [0,4194304] 0 2024-04-26T04:42:24.702 INFO:tasks.workunit.client.1.smithi104.stdout:3/935: dwrite d5/db2/f3b [0,4194304] 0 2024-04-26T04:42:24.703 INFO:tasks.workunit.client.1.smithi104.stdout:1/998: symlink d0/d6/d19/d21/d66/d20/d2a/d109/l14b 0 2024-04-26T04:42:24.706 INFO:tasks.workunit.client.0.smithi090.stdout:8/659: symlink dc/dd2/d4f/d58/d95/le6 0 2024-04-26T04:42:24.707 INFO:tasks.workunit.client.1.smithi104.stdout:1/999: link d0/d6/d19/d21/d66/d20/d2a/f6f d0/d6/d19/d21/d66/d20/d2a/d56/d8c/d9a/dec/f14c 0 2024-04-26T04:42:24.708 INFO:tasks.workunit.client.0.smithi090.stdout:4/553: sync 2024-04-26T04:42:24.708 INFO:tasks.workunit.client.0.smithi090.stdout:0/644: sync 2024-04-26T04:42:24.708 INFO:tasks.workunit.client.0.smithi090.stdout:1/522: sync 2024-04-26T04:42:24.708 INFO:tasks.workunit.client.0.smithi090.stdout:5/539: sync 2024-04-26T04:42:24.709 INFO:tasks.workunit.client.0.smithi090.stdout:0/645: chown d0/d4b/d5d/d48/f80 339 1 2024-04-26T04:42:24.709 INFO:tasks.workunit.client.0.smithi090.stdout:9/430: sync 2024-04-26T04:42:24.709 INFO:tasks.workunit.client.0.smithi090.stdout:2/571: sync 2024-04-26T04:42:24.709 INFO:tasks.workunit.client.0.smithi090.stdout:1/523: creat d0/d4/fa4 x:0 0 0 2024-04-26T04:42:24.709 INFO:tasks.workunit.client.0.smithi090.stdout:8/660: link dc/d5a/lda dc/dd2/d39/d55/d64/le7 0 2024-04-26T04:42:24.711 INFO:tasks.workunit.client.0.smithi090.stdout:4/554: truncate d3/db/f23 122646 0 2024-04-26T04:42:24.713 INFO:tasks.workunit.client.0.smithi090.stdout:0/646: creat d0/d4b/db3/dcc/fe2 x:0 0 0 2024-04-26T04:42:24.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/540: dread d4/d1a/d3f/f52 [0,4194304] 0 2024-04-26T04:42:24.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/431: unlink db/dd/d18/l66 0 2024-04-26T04:42:24.722 INFO:tasks.workunit.client.0.smithi090.stdout:2/572: creat db/d11/d14/d25/d2d/d48/fbf x:0 0 0 2024-04-26T04:42:24.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/661: truncate dc/dd2/d4f/d58/f6f 2825723 0 2024-04-26T04:42:24.723 INFO:tasks.workunit.client.0.smithi090.stdout:4/555: creat d3/db/d12/d20/d2d/d58/d6a/d74/d85/fb0 x:0 0 0 2024-04-26T04:42:24.723 INFO:tasks.workunit.client.0.smithi090.stdout:4/556: chown d3/db/d12/d20/d2d/d2e/f3a 114 1 2024-04-26T04:42:24.725 INFO:tasks.workunit.client.0.smithi090.stdout:0/647: mknod d0/d4b/db3/ce3 0 2024-04-26T04:42:24.725 INFO:tasks.workunit.client.0.smithi090.stdout:0/648: chown d0/d4b/d5d/d5c/d8a/d45/d53/l9e 10639 1 2024-04-26T04:42:24.726 INFO:tasks.workunit.client.0.smithi090.stdout:9/432: unlink db/d16/f64 0 2024-04-26T04:42:24.728 INFO:tasks.workunit.client.0.smithi090.stdout:2/573: link db/d11/d14/d25/d41/f93 db/d11/d14/fc0 0 2024-04-26T04:42:24.728 INFO:tasks.workunit.client.0.smithi090.stdout:2/574: write db/f76 [3878349,27199] 0 2024-04-26T04:42:24.729 INFO:tasks.workunit.client.0.smithi090.stdout:2/575: fdatasync db/d11/d14/d27/d2f/d42/d5c/fb3 0 2024-04-26T04:42:24.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/662: link dc/d34/l51 dc/dd2/d4f/le8 0 2024-04-26T04:42:24.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/663: chown dc/dd2/d4f/da6/dd0 0 1 2024-04-26T04:42:24.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/557: symlink d3/db/d60/lb1 0 2024-04-26T04:42:24.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/558: chown d3/d13/d1d 1717466 1 2024-04-26T04:42:24.733 INFO:tasks.workunit.client.0.smithi090.stdout:8/664: mknod dc/dd2/d39/d99/ce9 0 2024-04-26T04:42:24.737 INFO:tasks.workunit.client.0.smithi090.stdout:5/541: dread d4/d1a/d2a/d31/f34 [0,4194304] 0 2024-04-26T04:42:24.737 INFO:tasks.workunit.client.0.smithi090.stdout:8/665: mkdir dc/dd2/d4f/da6/dd8/dea 0 2024-04-26T04:42:24.737 INFO:tasks.workunit.client.0.smithi090.stdout:0/649: symlink d0/df/le4 0 2024-04-26T04:42:24.738 INFO:tasks.workunit.client.0.smithi090.stdout:8/666: dread - dc/dd2/d4f/db7/fdf zero size 2024-04-26T04:42:24.738 INFO:tasks.workunit.client.0.smithi090.stdout:0/650: chown d0/df/l2a 13431028 1 2024-04-26T04:42:24.740 INFO:tasks.workunit.client.0.smithi090.stdout:5/542: symlink d4/d1a/d2a/d79/laa 0 2024-04-26T04:42:24.740 INFO:tasks.workunit.client.0.smithi090.stdout:1/524: dwrite d0/f86 [0,4194304] 0 2024-04-26T04:42:24.741 INFO:tasks.workunit.client.0.smithi090.stdout:0/651: truncate d0/de/f36 2877412 0 2024-04-26T04:42:24.741 INFO:tasks.workunit.client.0.smithi090.stdout:0/652: fsync d0/d4b/d5d/d5c/d8a/f43 0 2024-04-26T04:42:24.742 INFO:tasks.workunit.client.0.smithi090.stdout:1/525: creat d0/d4/d7/d17/d22/d31/fa5 x:0 0 0 2024-04-26T04:42:24.742 INFO:tasks.workunit.client.0.smithi090.stdout:1/526: chown d0/d4/d43/f47 3182031 1 2024-04-26T04:42:24.742 INFO:tasks.workunit.client.0.smithi090.stdout:1/527: write d0/d4/f20 [3077665,60367] 0 2024-04-26T04:42:24.743 INFO:tasks.workunit.client.0.smithi090.stdout:0/653: readlink d0/d4b/l73 0 2024-04-26T04:42:24.743 INFO:tasks.workunit.client.0.smithi090.stdout:0/654: write d0/d4b/d5d/d48/d7d/fc3 [1048530,104752] 0 2024-04-26T04:42:24.743 INFO:tasks.workunit.client.0.smithi090.stdout:0/655: truncate d0/df/fd4 1020561 0 2024-04-26T04:42:24.743 INFO:tasks.workunit.client.0.smithi090.stdout:0/656: write d0/d4b/d5d/d5c/d8a/f43 [364936,98234] 0 2024-04-26T04:42:24.745 INFO:tasks.workunit.client.0.smithi090.stdout:0/657: truncate d0/d4b/d5d/d5c/d8a/d45/f59 3100082 0 2024-04-26T04:42:24.745 INFO:tasks.workunit.client.0.smithi090.stdout:0/658: chown d0/df/c2e 34319933 1 2024-04-26T04:42:24.745 INFO:tasks.workunit.client.0.smithi090.stdout:0/659: chown d0/df/f77 3300648 1 2024-04-26T04:42:24.746 INFO:tasks.workunit.client.0.smithi090.stdout:1/528: mknod d0/d4/d7/d4b/d53/ca6 0 2024-04-26T04:42:24.747 INFO:tasks.workunit.client.1.smithi104.stdout:3/936: dwrite d5/db2/da3/dd0/ddb/d3a/f57 [0,4194304] 0 2024-04-26T04:42:24.747 INFO:tasks.workunit.client.1.smithi104.stdout:4/937: dwrite d4/d20/d55/d56/da3/dbf/fc5 [0,4194304] 0 2024-04-26T04:42:24.748 INFO:tasks.workunit.client.1.smithi104.stdout:3/937: dread d5/d9/f5d [0,4194304] 0 2024-04-26T04:42:24.749 INFO:tasks.workunit.client.0.smithi090.stdout:0/660: mknod d0/d4b/d5d/d5c/d8a/d45/d93/ce5 0 2024-04-26T04:42:24.749 INFO:tasks.workunit.client.0.smithi090.stdout:9/433: dwrite db/dd/d18/d57/f4d [0,4194304] 0 2024-04-26T04:42:24.750 INFO:tasks.workunit.client.1.smithi104.stdout:4/938: mkdir d4/d7/d1d/d1e/d15/d146 0 2024-04-26T04:42:24.750 INFO:tasks.workunit.client.1.smithi104.stdout:4/939: write d4/d7/d1d/d8e/f13f [413844,52638] 0 2024-04-26T04:42:24.750 INFO:tasks.workunit.client.1.smithi104.stdout:0/814: dwrite d0/d2/d21/d27/f82 [4194304,4194304] 0 2024-04-26T04:42:24.750 INFO:tasks.workunit.client.1.smithi104.stdout:4/940: write d4/d7/d1d/d8e/f10a [5029223,123443] 0 2024-04-26T04:42:24.752 INFO:tasks.workunit.client.0.smithi090.stdout:9/434: mkdir db/dd/d18/d95 0 2024-04-26T04:42:24.753 INFO:tasks.workunit.client.1.smithi104.stdout:3/938: truncate d5/f3f 3745788 0 2024-04-26T04:42:24.755 INFO:tasks.workunit.client.1.smithi104.stdout:0/815: link d0/d2/d21/d27/d1e/fc8 d0/f10d 0 2024-04-26T04:42:24.755 INFO:tasks.workunit.client.0.smithi090.stdout:9/435: creat db/dd/d18/d31/d6a/d70/f96 x:0 0 0 2024-04-26T04:42:24.755 INFO:tasks.workunit.client.0.smithi090.stdout:9/436: write db/f84 [4360605,72052] 0 2024-04-26T04:42:24.761 INFO:tasks.workunit.client.0.smithi090.stdout:9/437: rename db/dd/d18/d31/d58/f65 to db/dd/d18/d31/d58/f97 0 2024-04-26T04:42:24.762 INFO:tasks.workunit.client.0.smithi090.stdout:9/438: fdatasync db/dd/f60 0 2024-04-26T04:42:24.764 INFO:tasks.workunit.client.0.smithi090.stdout:4/559: dwrite d3/db/d12/d20/f2a [0,4194304] 0 2024-04-26T04:42:24.768 INFO:tasks.workunit.client.0.smithi090.stdout:2/576: dwrite db/d11/d14/d1b/d20/f24 [0,4194304] 0 2024-04-26T04:42:24.769 INFO:tasks.workunit.client.0.smithi090.stdout:2/577: dread - db/d11/d14/d27/f7b zero size 2024-04-26T04:42:24.770 INFO:tasks.workunit.client.1.smithi104.stdout:0/816: write d0/f91 [1800948,109279] 0 2024-04-26T04:42:24.773 INFO:tasks.workunit.client.1.smithi104.stdout:0/817: symlink d0/d2/d21/d49/d5e/d9c/d53/l10e 0 2024-04-26T04:42:24.773 INFO:tasks.workunit.client.1.smithi104.stdout:0/818: stat d0/d4/f29 0 2024-04-26T04:42:24.773 INFO:tasks.workunit.client.1.smithi104.stdout:0/819: dread - d0/d2/d21/d27/d1e/d57/d6c/dc7/f101 zero size 2024-04-26T04:42:24.774 INFO:tasks.workunit.client.1.smithi104.stdout:0/820: chown d0/d4/d3b/cab 12 1 2024-04-26T04:42:24.775 INFO:tasks.workunit.client.1.smithi104.stdout:0/821: symlink d0/d2/d10/d3d/dc0/l10f 0 2024-04-26T04:42:24.777 INFO:tasks.workunit.client.0.smithi090.stdout:1/529: dread d0/d4/f20 [0,4194304] 0 2024-04-26T04:42:24.777 INFO:tasks.workunit.client.0.smithi090.stdout:1/530: write d0/d4/f91 [4942154,35859] 0 2024-04-26T04:42:24.777 INFO:tasks.workunit.client.0.smithi090.stdout:1/531: write d0/d4/d43/f47 [4830889,8588] 0 2024-04-26T04:42:24.781 INFO:tasks.workunit.client.1.smithi104.stdout:0/822: dread d0/d2/d21/d27/d1e/d57/d6c/dd1/f54 [0,4194304] 0 2024-04-26T04:42:24.781 INFO:tasks.workunit.client.1.smithi104.stdout:0/823: stat d0/d2/d21/ca4 0 2024-04-26T04:42:24.782 INFO:tasks.workunit.client.0.smithi090.stdout:4/560: rmdir d3/db/d12/d20/d24/d43/d72 39 2024-04-26T04:42:24.785 INFO:tasks.workunit.client.0.smithi090.stdout:1/532: truncate d0/f12 2023967 0 2024-04-26T04:42:24.786 INFO:tasks.workunit.client.0.smithi090.stdout:1/533: mknod d0/d4/ca7 0 2024-04-26T04:42:24.786 INFO:tasks.workunit.client.0.smithi090.stdout:1/534: write d0/d4/d7/d17/d22/d31/f6d [770581,113200] 0 2024-04-26T04:42:24.789 INFO:tasks.workunit.client.0.smithi090.stdout:1/535: unlink d0/d4/d7/d4b/f9b 0 2024-04-26T04:42:24.793 INFO:tasks.workunit.client.1.smithi104.stdout:0/824: dread d0/d2/d21/d49/d5e/d9c/d53/f9d [0,4194304] 0 2024-04-26T04:42:24.794 INFO:tasks.workunit.client.1.smithi104.stdout:4/941: dwrite d4/d7/d1d/d2e/d74/dbd/dd3/f10d [0,4194304] 0 2024-04-26T04:42:24.794 INFO:tasks.workunit.client.1.smithi104.stdout:4/942: write d4/d20/d55/d56/ff5 [666440,107029] 0 2024-04-26T04:42:24.794 INFO:tasks.workunit.client.1.smithi104.stdout:3/939: dwrite d5/d9/d80/f94 [0,4194304] 0 2024-04-26T04:42:24.794 INFO:tasks.workunit.client.0.smithi090.stdout:0/661: dwrite d0/d4b/d5d/d5c/d8a/d45/d93/fa1 [0,4194304] 0 2024-04-26T04:42:24.795 INFO:tasks.workunit.client.0.smithi090.stdout:4/561: dread d3/db/d12/d20/d2d/f6f [0,4194304] 0 2024-04-26T04:42:24.795 INFO:tasks.workunit.client.1.smithi104.stdout:0/825: unlink d0/d2/c13 0 2024-04-26T04:42:24.797 INFO:tasks.workunit.client.1.smithi104.stdout:0/826: write d0/d2/d10/d3d/f72 [1018237,60066] 0 2024-04-26T04:42:24.797 INFO:tasks.workunit.client.1.smithi104.stdout:0/827: stat d0/d2/d21/d27/f30 0 2024-04-26T04:42:24.800 INFO:tasks.workunit.client.0.smithi090.stdout:1/536: dread d0/f2 [0,4194304] 0 2024-04-26T04:42:24.800 INFO:tasks.workunit.client.0.smithi090.stdout:9/439: dwrite db/dd/f8d [0,4194304] 0 2024-04-26T04:42:24.806 INFO:tasks.workunit.client.0.smithi090.stdout:9/440: symlink db/l98 0 2024-04-26T04:42:24.806 INFO:tasks.workunit.client.0.smithi090.stdout:9/441: readlink db/dd/d18/d31/d58/l78 0 2024-04-26T04:42:24.806 INFO:tasks.workunit.client.0.smithi090.stdout:9/442: fsync db/dd/d18/d38/d39/f7e 0 2024-04-26T04:42:24.818 INFO:tasks.workunit.client.1.smithi104.stdout:4/943: write d4/d7/d1d/d1e/d24/d4a/f50 [3559654,106132] 0 2024-04-26T04:42:24.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/443: dread db/dd/d18/f36 [0,4194304] 0 2024-04-26T04:42:24.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/444: write db/dd/d18/d31/d6a/f92 [424090,130583] 0 2024-04-26T04:42:24.819 INFO:tasks.workunit.client.0.smithi090.stdout:9/445: mkdir db/dd/d18/d31/d99 0 2024-04-26T04:42:24.821 INFO:tasks.workunit.client.0.smithi090.stdout:9/446: link db/f3d db/d16/f9a 0 2024-04-26T04:42:24.822 INFO:tasks.workunit.client.0.smithi090.stdout:9/447: rename db/d5f/f6f to db/dd/d18/d38/f9b 0 2024-04-26T04:42:24.822 INFO:tasks.workunit.client.0.smithi090.stdout:9/448: chown db/dd/d18/d38/d39/d3f/f76 2 1 2024-04-26T04:42:24.824 INFO:tasks.workunit.client.0.smithi090.stdout:9/449: rename db/d16/l2d to db/dd/d18/d38/d39/d3f/l9c 0 2024-04-26T04:42:24.824 INFO:tasks.workunit.client.0.smithi090.stdout:9/450: stat db/d16/f9a 0 2024-04-26T04:42:24.830 INFO:tasks.workunit.client.1.smithi104.stdout:4/944: dread d4/db/fe [0,4194304] 0 2024-04-26T04:42:24.831 INFO:tasks.workunit.client.1.smithi104.stdout:4/945: chown d4/d118 48 1 2024-04-26T04:42:24.832 INFO:tasks.workunit.client.1.smithi104.stdout:4/946: symlink d4/d7/d1d/d2e/d74/l147 0 2024-04-26T04:42:24.833 INFO:tasks.workunit.client.0.smithi090.stdout:0/662: dwrite d0/d4b/d5d/d5c/d8a/d45/d93/fd5 [0,4194304] 0 2024-04-26T04:42:24.833 INFO:tasks.workunit.client.0.smithi090.stdout:0/663: truncate d0/d4b/d5d/d5c/d8a/d45/d93/fdf 896727 0 2024-04-26T04:42:24.834 INFO:tasks.workunit.client.1.smithi104.stdout:3/940: dwrite d5/db2/da3/dd0/ddb/d32/d48/fdf [0,4194304] 0 2024-04-26T04:42:24.834 INFO:tasks.workunit.client.0.smithi090.stdout:1/537: dwrite d0/d4/d7/f9c [0,4194304] 0 2024-04-26T04:42:24.838 INFO:tasks.workunit.client.0.smithi090.stdout:0/664: creat d0/d4b/d5d/d5c/d8a/d45/d53/dc2/dd7/fe6 x:0 0 0 2024-04-26T04:42:24.841 INFO:tasks.workunit.client.0.smithi090.stdout:1/538: fsync d0/d4/d7/d4a/d75/d80/fa2 0 2024-04-26T04:42:24.841 INFO:tasks.workunit.client.0.smithi090.stdout:1/539: chown d0/d4/d7/d4a/f96 23536215 1 2024-04-26T04:42:24.843 INFO:tasks.workunit.client.1.smithi104.stdout:3/941: rmdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d 39 2024-04-26T04:42:24.843 INFO:tasks.workunit.client.1.smithi104.stdout:3/942: readlink d5/db2/d46/l63 0 2024-04-26T04:42:24.843 INFO:tasks.workunit.client.0.smithi090.stdout:0/665: unlink d0/d4b/d5d/d5c/d8a/d45/d93/fdf 0 2024-04-26T04:42:24.843 INFO:tasks.workunit.client.0.smithi090.stdout:0/666: write d0/d4b/d5d/d5c/d8a/d45/d93/fb7 [370767,31426] 0 2024-04-26T04:42:24.846 INFO:tasks.workunit.client.1.smithi104.stdout:3/943: truncate d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/da9/fb0 827976 0 2024-04-26T04:42:24.848 INFO:tasks.workunit.client.1.smithi104.stdout:3/944: creat d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/dd6/f146 x:0 0 0 2024-04-26T04:42:24.849 INFO:tasks.workunit.client.0.smithi090.stdout:9/451: dwrite db/dd/d18/d38/f79 [0,4194304] 0 2024-04-26T04:42:24.850 INFO:tasks.workunit.client.1.smithi104.stdout:4/947: dwrite d4/d7/d1d/d1e/d15/df3/f143 [0,4194304] 0 2024-04-26T04:42:24.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/452: mkdir db/dd/d18/d31/d6a/d70/d9d 0 2024-04-26T04:42:24.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/453: stat db/dd/d18/d31/f3c 0 2024-04-26T04:42:24.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/454: truncate db/d16/f33 32896 0 2024-04-26T04:42:24.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/455: stat db/l98 0 2024-04-26T04:42:24.855 INFO:tasks.workunit.client.0.smithi090.stdout:9/456: mknod db/dd/d44/d82/d85/c9e 0 2024-04-26T04:42:24.860 INFO:tasks.workunit.client.0.smithi090.stdout:9/457: rename db/dd/d18/d31/d6a/l72 to db/dd/d18/d57/l9f 0 2024-04-26T04:42:24.862 INFO:tasks.workunit.client.1.smithi104.stdout:3/945: dread d5/db2/da3/fa4 [0,4194304] 0 2024-04-26T04:42:24.862 INFO:tasks.workunit.client.1.smithi104.stdout:3/946: chown d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/ccd 3555768 1 2024-04-26T04:42:24.862 INFO:tasks.workunit.client.1.smithi104.stdout:3/947: write d5/d9/d80/dbc/f125 [522913,60110] 0 2024-04-26T04:42:24.862 INFO:tasks.workunit.client.1.smithi104.stdout:3/948: chown d5/d9/d80/da2 125584136 1 2024-04-26T04:42:24.864 INFO:tasks.workunit.client.0.smithi090.stdout:9/458: link db/f47 db/dd/d18/d31/d6a/fa0 0 2024-04-26T04:42:24.864 INFO:tasks.workunit.client.0.smithi090.stdout:1/540: dwrite d0/d4/d7/f2c [0,4194304] 0 2024-04-26T04:42:24.864 INFO:tasks.workunit.client.0.smithi090.stdout:1/541: write d0/d4/d7/d17/f21 [4197935,1348] 0 2024-04-26T04:42:24.865 INFO:tasks.workunit.client.1.smithi104.stdout:4/948: dread d4/d20/d55/d56/f94 [0,4194304] 0 2024-04-26T04:42:24.865 INFO:tasks.workunit.client.1.smithi104.stdout:4/949: dread - d4/db/d6b/d7f/f13b zero size 2024-04-26T04:42:24.865 INFO:tasks.workunit.client.1.smithi104.stdout:4/950: write d4/d7/d1d/f45 [3371109,4433] 0 2024-04-26T04:42:24.867 INFO:tasks.workunit.client.1.smithi104.stdout:4/951: mknod d4/d7/d1d/d1e/d15/d113/c148 0 2024-04-26T04:42:24.868 INFO:tasks.workunit.client.0.smithi090.stdout:1/542: unlink d0/d4/d7/d17/d22/f5c 0 2024-04-26T04:42:24.868 INFO:tasks.workunit.client.0.smithi090.stdout:1/543: read d0/d4/d7/d17/d22/d31/f39 [1860976,45338] 0 2024-04-26T04:42:24.869 INFO:tasks.workunit.client.1.smithi104.stdout:4/952: mknod d4/d20/d55/c149 0 2024-04-26T04:42:24.869 INFO:tasks.workunit.client.1.smithi104.stdout:4/953: chown d4/d7/ff 308337097 1 2024-04-26T04:42:24.869 INFO:tasks.workunit.client.1.smithi104.stdout:3/949: dread d5/db2/da3/dd0/ddb/d32/f5a [0,4194304] 0 2024-04-26T04:42:24.870 INFO:tasks.workunit.client.1.smithi104.stdout:4/954: creat d4/d7/d1d/d1e/d24/d4a/dbb/f14a x:0 0 0 2024-04-26T04:42:24.870 INFO:tasks.workunit.client.1.smithi104.stdout:4/955: stat d4/d7/d1d/d1e/d24/d4a/d9e 0 2024-04-26T04:42:24.871 INFO:tasks.workunit.client.1.smithi104.stdout:3/950: mkdir d5/db2/d46/d136/d147 0 2024-04-26T04:42:24.871 INFO:tasks.workunit.client.1.smithi104.stdout:3/951: creat d5/db2/f148 x:0 0 0 2024-04-26T04:42:24.872 INFO:tasks.workunit.client.1.smithi104.stdout:4/956: link d4/d7/d1d/d1e/d24/d59/fae d4/d7/d1d/d1e/d105/f14b 0 2024-04-26T04:42:24.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/667: dwrite d0/d4b/f8c [0,4194304] 0 2024-04-26T04:42:24.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/668: stat d0/d4b/d5d/d5c/d8a/d45/d53/fa3 0 2024-04-26T04:42:24.873 INFO:tasks.workunit.client.0.smithi090.stdout:0/669: chown d0/d4b/d5d/d5c/d8a/d45/d53/dc2 3314 1 2024-04-26T04:42:24.874 INFO:tasks.workunit.client.0.smithi090.stdout:0/670: write d0/d4b/d5d/d5c/d8a/f3e [255728,55003] 0 2024-04-26T04:42:24.874 INFO:tasks.workunit.client.1.smithi104.stdout:4/957: mknod d4/d7/d1d/d1e/d24/d4a/c14c 0 2024-04-26T04:42:24.874 INFO:tasks.workunit.client.1.smithi104.stdout:4/958: fdatasync d4/d7/d1d/f45 0 2024-04-26T04:42:24.877 INFO:tasks.workunit.client.0.smithi090.stdout:1/544: dread d0/d4/d7/f9c [0,4194304] 0 2024-04-26T04:42:24.879 INFO:tasks.workunit.client.1.smithi104.stdout:3/952: dread d5/db2/da3/dd0/ddb/d3a/f57 [0,4194304] 0 2024-04-26T04:42:24.880 INFO:tasks.workunit.client.1.smithi104.stdout:3/953: mkdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/d149 0 2024-04-26T04:42:24.882 INFO:tasks.workunit.client.0.smithi090.stdout:0/671: rename d0/d4b/d5d/d48/f80 to d0/d4b/d5d/fe7 0 2024-04-26T04:42:24.882 INFO:tasks.workunit.client.0.smithi090.stdout:0/672: fdatasync d0/d4b/d5d/d5c/fa8 0 2024-04-26T04:42:24.882 INFO:tasks.workunit.client.1.smithi104.stdout:3/954: rename d5/fbb to d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/d124/f14a 0 2024-04-26T04:42:24.883 INFO:tasks.workunit.client.1.smithi104.stdout:3/955: stat d5/f43 0 2024-04-26T04:42:24.883 INFO:tasks.workunit.client.0.smithi090.stdout:1/545: truncate d0/d4/d7/d17/d22/f73 393774 0 2024-04-26T04:42:24.884 INFO:tasks.workunit.client.1.smithi104.stdout:4/959: dread d4/db/d6b/fa2 [0,4194304] 0 2024-04-26T04:42:24.884 INFO:tasks.workunit.client.1.smithi104.stdout:4/960: stat d4/d7/d1d/d1e/d15/dd8/ce0 0 2024-04-26T04:42:24.884 INFO:tasks.workunit.client.1.smithi104.stdout:3/956: mknod d5/db2/da3/dd0/ddb/d32/dd9/c14b 0 2024-04-26T04:42:24.894 INFO:tasks.workunit.client.0.smithi090.stdout:0/673: mknod d0/d4b/db3/ce8 0 2024-04-26T04:42:24.895 INFO:tasks.workunit.client.0.smithi090.stdout:0/674: dread - d0/d4b/db3/dcc/fe2 zero size 2024-04-26T04:42:24.895 INFO:tasks.workunit.client.1.smithi104.stdout:3/957: dread d5/d9/f26 [4194304,4194304] 0 2024-04-26T04:42:24.895 INFO:tasks.workunit.client.1.smithi104.stdout:3/958: fsync d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/f74 0 2024-04-26T04:42:24.896 INFO:tasks.workunit.client.1.smithi104.stdout:3/959: creat d5/d9/d80/dbc/dd4/f14c x:0 0 0 2024-04-26T04:42:24.897 INFO:tasks.workunit.client.0.smithi090.stdout:0/675: creat d0/fe9 x:0 0 0 2024-04-26T04:42:24.901 INFO:tasks.workunit.client.1.smithi104.stdout:4/961: dwrite d4/d7/d1d/d1e/d24/d59/fae [0,4194304] 0 2024-04-26T04:42:24.901 INFO:tasks.workunit.client.1.smithi104.stdout:4/962: fdatasync d4/d7/d1d/d1e/d15/df3/f143 0 2024-04-26T04:42:24.902 INFO:tasks.workunit.client.1.smithi104.stdout:3/960: link d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/l3d d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/l14d 0 2024-04-26T04:42:24.902 INFO:tasks.workunit.client.1.smithi104.stdout:3/961: write d5/db2/da3/dd0/ddb/d3a/f123 [1240500,109604] 0 2024-04-26T04:42:24.902 INFO:tasks.workunit.client.1.smithi104.stdout:3/962: chown d5/db2/d46/db7/de6/d112 29647106 1 2024-04-26T04:42:24.907 INFO:tasks.workunit.client.0.smithi090.stdout:0/676: unlink d0/d4b/d5d/d5c/d8a/d7c/ca6 0 2024-04-26T04:42:24.907 INFO:tasks.workunit.client.0.smithi090.stdout:0/677: unlink d0/d4b/d5d/d5c/d8a/d45/d93/fb7 0 2024-04-26T04:42:24.907 INFO:tasks.workunit.client.1.smithi104.stdout:8/785: sync 2024-04-26T04:42:24.907 INFO:tasks.workunit.client.1.smithi104.stdout:3/963: symlink d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/d13c/l14e 0 2024-04-26T04:42:24.907 INFO:tasks.workunit.client.1.smithi104.stdout:7/830: sync 2024-04-26T04:42:24.907 INFO:tasks.workunit.client.1.smithi104.stdout:9/741: sync 2024-04-26T04:42:24.907 INFO:tasks.workunit.client.1.smithi104.stdout:3/964: creat d5/db2/da3/f14f x:0 0 0 2024-04-26T04:42:24.908 INFO:tasks.workunit.client.1.smithi104.stdout:7/831: link d2/d4/d32/f69 d2/d4/d32/d3d/f106 0 2024-04-26T04:42:24.908 INFO:tasks.workunit.client.1.smithi104.stdout:7/832: dread d2/d4/d1d/f3e [0,4194304] 0 2024-04-26T04:42:24.908 INFO:tasks.workunit.client.1.smithi104.stdout:8/786: rename d0/d1c/d27/d47/d67/d76/d7f/fe4 to d0/d50/ffe 0 2024-04-26T04:42:24.908 INFO:tasks.workunit.client.0.smithi090.stdout:0/678: rename d0/d4b/d5d/d48/d7d/le1 to d0/de/lea 0 2024-04-26T04:42:24.908 INFO:tasks.workunit.client.0.smithi090.stdout:0/679: fsync d0/de/f6e 0 2024-04-26T04:42:24.909 INFO:tasks.workunit.client.0.smithi090.stdout:0/680: chown d0/d4b/d5d/d48/f56 97400704 1 2024-04-26T04:42:24.909 INFO:tasks.workunit.client.0.smithi090.stdout:0/681: fsync d0/d4b/d5d/d48/f56 0 2024-04-26T04:42:24.909 INFO:tasks.workunit.client.0.smithi090.stdout:0/682: fsync d0/d4b/d5d/d5c/d8a/d7c/fc9 0 2024-04-26T04:42:24.909 INFO:tasks.workunit.client.1.smithi104.stdout:7/833: mknod d2/d4/d32/d5e/c107 0 2024-04-26T04:42:24.910 INFO:tasks.workunit.client.1.smithi104.stdout:8/787: mknod d0/d33/d35/d6d/def/cff 0 2024-04-26T04:42:24.911 INFO:tasks.workunit.client.1.smithi104.stdout:3/965: write d5/f54 [514109,23928] 0 2024-04-26T04:42:24.911 INFO:tasks.workunit.client.1.smithi104.stdout:3/966: write d5/db2/d60/faf [1008933,32950] 0 2024-04-26T04:42:24.911 INFO:tasks.workunit.client.1.smithi104.stdout:3/967: stat d5/d9/d80/da2/fe3 0 2024-04-26T04:42:24.911 INFO:tasks.workunit.client.1.smithi104.stdout:3/968: chown d5/db2/da3/dd0/ddb/d69/fe7 19 1 2024-04-26T04:42:24.911 INFO:tasks.workunit.client.1.smithi104.stdout:7/834: mknod d2/d4/d1d/d34/db0/c108 0 2024-04-26T04:42:24.912 INFO:tasks.workunit.client.0.smithi090.stdout:0/683: creat d0/d4b/feb x:0 0 0 2024-04-26T04:42:24.912 INFO:tasks.workunit.client.0.smithi090.stdout:0/684: fdatasync d0/d4b/d5d/d5c/d8a/d45/d53/dc2/dd7/fda 0 2024-04-26T04:42:24.913 INFO:tasks.workunit.client.1.smithi104.stdout:3/969: creat d5/db2/da3/dd0/ddb/d12c/f150 x:0 0 0 2024-04-26T04:42:24.913 INFO:tasks.workunit.client.1.smithi104.stdout:3/970: write d5/db2/da3/fa4 [9022397,53282] 0 2024-04-26T04:42:24.913 INFO:tasks.workunit.client.1.smithi104.stdout:3/971: fdatasync d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/f8b 0 2024-04-26T04:42:24.914 INFO:tasks.workunit.client.1.smithi104.stdout:7/835: symlink d2/d4/d1d/d34/l109 0 2024-04-26T04:42:24.916 INFO:tasks.workunit.client.1.smithi104.stdout:7/836: creat d2/d4/d32/d3d/d47/db6/f10a x:0 0 0 2024-04-26T04:42:24.917 INFO:tasks.workunit.client.1.smithi104.stdout:3/972: getdents d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/dd6 0 2024-04-26T04:42:24.918 INFO:tasks.workunit.client.1.smithi104.stdout:7/837: truncate d2/d4/d32/d3d/d41/d5f/f8d 481309 0 2024-04-26T04:42:24.919 INFO:tasks.workunit.client.1.smithi104.stdout:7/838: dread d2/d4/d1d/f3e [0,4194304] 0 2024-04-26T04:42:24.919 INFO:tasks.workunit.client.1.smithi104.stdout:3/973: truncate d5/d9/f12 1280388 0 2024-04-26T04:42:24.919 INFO:tasks.workunit.client.1.smithi104.stdout:7/839: write d2/d4/d1d/d34/db0/de4/ffd [799921,92335] 0 2024-04-26T04:42:24.920 INFO:tasks.workunit.client.1.smithi104.stdout:3/974: mknod d5/db2/da3/dd0/ddb/d32/dd9/c151 0 2024-04-26T04:42:24.921 INFO:tasks.workunit.client.1.smithi104.stdout:7/840: mkdir d2/d4/d32/d3d/d41/dc0/d10b 0 2024-04-26T04:42:24.922 INFO:tasks.workunit.client.1.smithi104.stdout:7/841: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d94/d10c 0 2024-04-26T04:42:24.922 INFO:tasks.workunit.client.1.smithi104.stdout:7/842: write d2/d4/d32/d35/d37/f86 [468873,48861] 0 2024-04-26T04:42:24.927 INFO:tasks.workunit.client.0.smithi090.stdout:9/459: dwrite f2 [0,4194304] 0 2024-04-26T04:42:24.928 INFO:tasks.workunit.client.1.smithi104.stdout:9/742: dwrite d11/d15/d32/d4c/d54/f7a [0,4194304] 0 2024-04-26T04:42:24.928 INFO:tasks.workunit.client.1.smithi104.stdout:9/743: fdatasync d11/d15/d32/d35/f80 0 2024-04-26T04:42:24.928 INFO:tasks.workunit.client.1.smithi104.stdout:9/744: creat d11/d53/d84/ff0 x:0 0 0 2024-04-26T04:42:24.928 INFO:tasks.workunit.client.0.smithi090.stdout:9/460: symlink db/dd/d44/d82/d85/la1 0 2024-04-26T04:42:24.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/461: creat db/fa2 x:0 0 0 2024-04-26T04:42:24.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/462: fdatasync db/dd/d44/f8e 0 2024-04-26T04:42:24.930 INFO:tasks.workunit.client.0.smithi090.stdout:9/463: mkdir db/da3 0 2024-04-26T04:42:24.931 INFO:tasks.workunit.client.1.smithi104.stdout:9/745: getdents d11/d15/d32/d35/d7b 0 2024-04-26T04:42:24.932 INFO:tasks.workunit.client.0.smithi090.stdout:9/464: rename db/dd/d18/d38/d39/d3f/c89 to db/dd/d44/ca4 0 2024-04-26T04:42:24.933 INFO:tasks.workunit.client.0.smithi090.stdout:1/546: dwrite d0/d4/d7/d49/f6f [4194304,4194304] 0 2024-04-26T04:42:24.933 INFO:tasks.workunit.client.0.smithi090.stdout:1/547: read d0/d4/d7/d17/f18 [1556378,97662] 0 2024-04-26T04:42:24.933 INFO:tasks.workunit.client.0.smithi090.stdout:9/465: dread db/dd/d18/d38/f9b [0,4194304] 0 2024-04-26T04:42:24.933 INFO:tasks.workunit.client.1.smithi104.stdout:9/746: mkdir d11/d1d/d34/db3/df1 0 2024-04-26T04:42:24.934 INFO:tasks.workunit.client.1.smithi104.stdout:7/843: dread d2/d4/d32/d35/d44/fb3 [0,4194304] 0 2024-04-26T04:42:24.943 INFO:tasks.workunit.client.1.smithi104.stdout:8/788: dwrite d0/d50/ffe [0,4194304] 0 2024-04-26T04:42:24.943 INFO:tasks.workunit.client.1.smithi104.stdout:8/789: chown d0/d1c/d2d/f32 40 1 2024-04-26T04:42:24.943 INFO:tasks.workunit.client.1.smithi104.stdout:8/790: readlink d0/d1c/d27/d47/db1/le1 0 2024-04-26T04:42:24.945 INFO:tasks.workunit.client.1.smithi104.stdout:3/975: dwrite d5/db2/f91 [0,4194304] 0 2024-04-26T04:42:24.946 INFO:tasks.workunit.client.1.smithi104.stdout:4/963: dwrite d4/db/f4b [0,4194304] 0 2024-04-26T04:42:24.952 INFO:tasks.workunit.client.1.smithi104.stdout:3/976: unlink d5/db2/da3/dd0/ddb/l11 0 2024-04-26T04:42:24.952 INFO:tasks.workunit.client.1.smithi104.stdout:3/977: fsync d5/db2/da3/dd0/ddb/f41 0 2024-04-26T04:42:24.952 INFO:tasks.workunit.client.0.smithi090.stdout:9/466: getdents db 0 2024-04-26T04:42:24.956 INFO:tasks.workunit.client.1.smithi104.stdout:4/964: mknod d4/d7/d1d/d2e/d74/dbd/c14d 0 2024-04-26T04:42:24.958 INFO:tasks.workunit.client.1.smithi104.stdout:3/978: rmdir d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49/d59/d149 0 2024-04-26T04:42:24.961 INFO:tasks.workunit.client.1.smithi104.stdout:4/965: unlink d4/d7/f63 0 2024-04-26T04:42:24.961 INFO:tasks.workunit.client.1.smithi104.stdout:3/979: rmdir d5/db2/da3/dd0/ddb/d32/dd9 39 2024-04-26T04:42:24.962 INFO:tasks.workunit.client.1.smithi104.stdout:4/966: symlink d4/d7/d1d/d2e/d74/dbd/l14e 0 2024-04-26T04:42:24.963 INFO:tasks.workunit.client.0.smithi090.stdout:9/467: dread db/dd/d18/d31/f4e [0,4194304] 0 2024-04-26T04:42:24.963 INFO:tasks.workunit.client.1.smithi104.stdout:4/967: symlink d4/db/d6b/d141/l14f 0 2024-04-26T04:42:24.964 INFO:tasks.workunit.client.1.smithi104.stdout:4/968: creat d4/db/d6b/d7f/f150 x:0 0 0 2024-04-26T04:42:24.964 INFO:tasks.workunit.client.1.smithi104.stdout:4/969: fdatasync d4/d7/d1d/d1e/d24/d4a/fec 0 2024-04-26T04:42:24.964 INFO:tasks.workunit.client.1.smithi104.stdout:4/970: stat d4/d20/d55/d56/ca9 0 2024-04-26T04:42:24.964 INFO:tasks.workunit.client.1.smithi104.stdout:4/971: write d4/d7/d1d/d2e/d74/f93 [947174,91158] 0 2024-04-26T04:42:24.964 INFO:tasks.workunit.client.1.smithi104.stdout:4/972: dread - d4/d7/d1d/d1e/d24/d59/db3/fb5 zero size 2024-04-26T04:42:24.964 INFO:tasks.workunit.client.1.smithi104.stdout:4/973: fdatasync d4/d7/d1d/d1e/d24/d48/fce 0 2024-04-26T04:42:24.964 INFO:tasks.workunit.client.0.smithi090.stdout:9/468: symlink db/dd/d18/d31/d58/la5 0 2024-04-26T04:42:24.968 INFO:tasks.workunit.client.1.smithi104.stdout:3/980: write d5/db2/da3/dd0/ddb/f28 [7902338,79421] 0 2024-04-26T04:42:24.969 INFO:tasks.workunit.client.1.smithi104.stdout:3/981: write d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/fc6 [412805,19711] 0 2024-04-26T04:42:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:24 smithi090 ceph-mon[26648]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:24 smithi090 ceph-mon[26648]: pgmap v254: 65 pgs: 65 active+clean; 2.4 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 276 MiB/s rd, 349 MiB/s wr, 498 op/s 2024-04-26T04:42:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:24 smithi090 ceph-mon[26648]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:24 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3364988170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:42:24.973 INFO:tasks.workunit.client.0.smithi090.stdout:1/548: dwrite d0/d4/d7/d17/d22/f73 [0,4194304] 0 2024-04-26T04:42:24.974 INFO:tasks.workunit.client.1.smithi104.stdout:8/791: dwrite d0/d1c/fd0 [4194304,4194304] 0 2024-04-26T04:42:24.974 INFO:tasks.workunit.client.1.smithi104.stdout:7/844: dwrite d2/d4/f6e [0,4194304] 0 2024-04-26T04:42:24.986 INFO:tasks.workunit.client.1.smithi104.stdout:7/845: write d2/d4/d1d/d34/ff4 [65102,23777] 0 2024-04-26T04:42:24.990 INFO:tasks.workunit.client.1.smithi104.stdout:4/974: dwrite d4/db/f10e [0,4194304] 0 2024-04-26T04:42:24.993 INFO:tasks.workunit.client.1.smithi104.stdout:6/926: sync 2024-04-26T04:42:24.993 INFO:tasks.workunit.client.1.smithi104.stdout:6/927: truncate d0/d9/d13/d41/d82/dc8/d80/fee 970880 0 2024-04-26T04:42:24.993 INFO:tasks.workunit.client.1.smithi104.stdout:8/792: dread d0/d1c/d27/d46/f78 [0,4194304] 0 2024-04-26T04:42:24.993 INFO:tasks.workunit.client.1.smithi104.stdout:7/846: dread d2/d4/d1d/fa7 [0,4194304] 0 2024-04-26T04:42:24.994 INFO:tasks.workunit.client.0.smithi090.stdout:1/549: dwrite d0/d4/d7/d49/f6f [0,4194304] 0 2024-04-26T04:42:24.994 INFO:tasks.workunit.client.1.smithi104.stdout:6/928: mknod d0/d9/d13/dcd/dd9/df5/c130 0 2024-04-26T04:42:24.994 INFO:tasks.workunit.client.1.smithi104.stdout:6/929: chown d0/d9/d13/d41/d82/dc8/d80/f123 965 1 2024-04-26T04:42:24.995 INFO:tasks.workunit.client.0.smithi090.stdout:1/550: truncate d0/d4/f14 865235 0 2024-04-26T04:42:24.996 INFO:tasks.workunit.client.1.smithi104.stdout:8/793: creat d0/d1c/d27/d47/f100 x:0 0 0 2024-04-26T04:42:24.998 INFO:tasks.workunit.client.1.smithi104.stdout:7/847: creat d2/d4/d32/d35/dbe/dda/f10d x:0 0 0 2024-04-26T04:42:25.002 INFO:tasks.workunit.client.1.smithi104.stdout:6/930: rename d0/d9/d13/f3d to d0/dd8/de0/f131 0 2024-04-26T04:42:25.003 INFO:tasks.workunit.client.1.smithi104.stdout:7/848: dread d2/d4/d32/d35/d44/f91 [0,4194304] 0 2024-04-26T04:42:25.004 INFO:tasks.workunit.client.1.smithi104.stdout:8/794: creat d0/d1c/d27/d47/dd5/f101 x:0 0 0 2024-04-26T04:42:25.005 INFO:tasks.workunit.client.1.smithi104.stdout:6/931: creat d0/f132 x:0 0 0 2024-04-26T04:42:25.006 INFO:tasks.workunit.client.1.smithi104.stdout:7/849: symlink d2/d4/d32/d3d/d41/d7e/l10e 0 2024-04-26T04:42:25.008 INFO:tasks.workunit.client.1.smithi104.stdout:8/795: symlink d0/d1c/d2d/d7a/l102 0 2024-04-26T04:42:25.008 INFO:tasks.workunit.client.1.smithi104.stdout:8/796: dread - d0/d1c/d27/d47/dd5/f101 zero size 2024-04-26T04:42:25.009 INFO:tasks.workunit.client.1.smithi104.stdout:7/850: rename d2/d4/d1d/d34/l109 to d2/d4/d1d/d34/d7f/l10f 0 2024-04-26T04:42:25.010 INFO:tasks.workunit.client.1.smithi104.stdout:0/828: sync 2024-04-26T04:42:25.011 INFO:tasks.workunit.client.1.smithi104.stdout:7/851: mknod d2/d4/d32/d3d/d41/dc0/d10b/c110 0 2024-04-26T04:42:25.011 INFO:tasks.workunit.client.1.smithi104.stdout:7/852: dread - d2/d4/d32/d3d/d47/db1/dfa/ffb zero size 2024-04-26T04:42:25.011 INFO:tasks.workunit.client.1.smithi104.stdout:7/853: chown d2/d4/d32/d35/d44/lf8 7 1 2024-04-26T04:42:25.011 INFO:tasks.workunit.client.1.smithi104.stdout:7/854: write d2/d4/d32/d3d/d41/d5f/d6c/f98 [1026858,50725] 0 2024-04-26T04:42:25.012 INFO:tasks.workunit.client.1.smithi104.stdout:7/855: write d2/d4/d5a/da0/fab [313932,121090] 0 2024-04-26T04:42:25.012 INFO:tasks.workunit.client.1.smithi104.stdout:5/907: sync 2024-04-26T04:42:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:24 smithi104 ceph-mon[32369]: from='client.14604 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:24 smithi104 ceph-mon[32369]: pgmap v254: 65 pgs: 65 active+clean; 2.4 GiB data, 7.2 GiB used, 529 GiB / 536 GiB avail; 276 MiB/s rd, 349 MiB/s wr, 498 op/s 2024-04-26T04:42:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:24 smithi104 ceph-mon[32369]: from='client.14608 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:24 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3364988170' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:42:25.013 INFO:tasks.workunit.client.1.smithi104.stdout:8/797: symlink d0/d9a/l103 0 2024-04-26T04:42:25.013 INFO:tasks.workunit.client.1.smithi104.stdout:0/829: mkdir d0/d2/d10/d110 0 2024-04-26T04:42:25.013 INFO:tasks.workunit.client.1.smithi104.stdout:0/830: readlink d0/d2/d21/d49/d5e/d9c/db2/lf0 0 2024-04-26T04:42:25.013 INFO:tasks.workunit.client.1.smithi104.stdout:0/831: chown d0/d2/d21/d27/d1e/d57/d6c/f6a 67779 1 2024-04-26T04:42:25.013 INFO:tasks.workunit.client.1.smithi104.stdout:5/908: rmdir d0/d1/d4/d2a/d6b/d73/d95 39 2024-04-26T04:42:25.014 INFO:tasks.workunit.client.1.smithi104.stdout:4/975: dwrite d4/d7/d1d/d1e/d24/d59/db8/fe3 [0,4194304] 0 2024-04-26T04:42:25.014 INFO:tasks.workunit.client.1.smithi104.stdout:4/976: creat d4/d20/d55/d56/da3/dea/d11c/f151 x:0 0 0 2024-04-26T04:42:25.015 INFO:tasks.workunit.client.1.smithi104.stdout:5/909: rename d0/d7b/l103 to d0/d1/d4/d2a/d2b/d5c/l12f 0 2024-04-26T04:42:25.018 INFO:tasks.workunit.client.0.smithi090.stdout:5/543: sync 2024-04-26T04:42:25.018 INFO:tasks.workunit.client.0.smithi090.stdout:5/544: getdents d4/d1a/df/d36/d61/d9b 0 2024-04-26T04:42:25.018 INFO:tasks.workunit.client.0.smithi090.stdout:6/661: sync 2024-04-26T04:42:25.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/621: sync 2024-04-26T04:42:25.018 INFO:tasks.workunit.client.0.smithi090.stdout:8/667: sync 2024-04-26T04:42:25.018 INFO:tasks.workunit.client.0.smithi090.stdout:8/668: truncate dc/dd2/d3b/d3d/fcc 729904 0 2024-04-26T04:42:25.018 INFO:tasks.workunit.client.0.smithi090.stdout:8/669: truncate dc/f1e 1075674 0 2024-04-26T04:42:25.019 INFO:tasks.workunit.client.0.smithi090.stdout:3/544: sync 2024-04-26T04:42:25.019 INFO:tasks.workunit.client.1.smithi104.stdout:4/977: rename d4/c64 to d4/d7/d1d/d2e/d74/dbd/dee/c152 0 2024-04-26T04:42:25.019 INFO:tasks.workunit.client.1.smithi104.stdout:4/978: truncate d4/d7/d1d/d1e/fad 1741482 0 2024-04-26T04:42:25.020 INFO:tasks.workunit.client.1.smithi104.stdout:6/932: dread d0/d9/d13/d41/d82/dc8/f60 [4194304,4194304] 0 2024-04-26T04:42:25.022 INFO:tasks.workunit.client.1.smithi104.stdout:5/910: symlink d0/d1/de6/l130 0 2024-04-26T04:42:25.022 INFO:tasks.workunit.client.1.smithi104.stdout:7/856: dread d2/d4/f13 [0,4194304] 0 2024-04-26T04:42:25.022 INFO:tasks.workunit.client.1.smithi104.stdout:7/857: dread - d2/d4/d1d/d34/fc6 zero size 2024-04-26T04:42:25.023 INFO:tasks.workunit.client.0.smithi090.stdout:6/662: rename d9/d77/db0/db5/dda/d6d/c5b to d9/d21/d40/d4c/d6f/d9f/dae/ce6 0 2024-04-26T04:42:25.023 INFO:tasks.workunit.client.0.smithi090.stdout:8/670: symlink dc/dd2/d4f/d58/leb 0 2024-04-26T04:42:25.024 INFO:tasks.workunit.client.1.smithi104.stdout:5/911: mknod d0/d1/d4/d2a/d6b/d91/dc7/c131 0 2024-04-26T04:42:25.025 INFO:tasks.workunit.client.0.smithi090.stdout:3/545: rename d1/d17/d36/d33/d43/db0/d2c/f9b to d1/d17/d36/dab/fb7 0 2024-04-26T04:42:25.026 INFO:tasks.workunit.client.1.smithi104.stdout:7/858: mkdir d2/d4/d32/d3d/d47/db6/dce/d111 0 2024-04-26T04:42:25.026 INFO:tasks.workunit.client.0.smithi090.stdout:6/663: mkdir d9/d1c/dc5/de7 0 2024-04-26T04:42:25.028 INFO:tasks.workunit.client.1.smithi104.stdout:5/912: unlink d0/d1/d1e/d98/dd4/fe9 0 2024-04-26T04:42:25.028 INFO:tasks.workunit.client.0.smithi090.stdout:3/546: truncate d1/d9/fb 8276374 0 2024-04-26T04:42:25.029 INFO:tasks.workunit.client.0.smithi090.stdout:6/664: creat d9/d1c/dc5/dd9/fe8 x:0 0 0 2024-04-26T04:42:25.031 INFO:tasks.workunit.client.0.smithi090.stdout:3/547: rename d1/d59/f9f to d1/d17/d36/d33/d43/db0/d2c/fb8 0 2024-04-26T04:42:25.032 INFO:tasks.workunit.client.0.smithi090.stdout:6/665: unlink d9/d1c/da2/lde 0 2024-04-26T04:42:25.033 INFO:tasks.workunit.client.0.smithi090.stdout:3/548: rmdir d1/d17/d36/d33/d68/dae 39 2024-04-26T04:42:25.035 INFO:tasks.workunit.client.0.smithi090.stdout:3/549: symlink d1/d17/d36/d2a/d3a/lb9 0 2024-04-26T04:42:25.035 INFO:tasks.workunit.client.0.smithi090.stdout:7/622: dread d0/d1/f38 [4194304,4194304] 0 2024-04-26T04:42:25.037 INFO:tasks.workunit.client.0.smithi090.stdout:3/550: creat d1/d17/d36/d33/d68/dae/fba x:0 0 0 2024-04-26T04:42:25.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/623: rmdir d0/db/df/d4d/daf/dbd 0 2024-04-26T04:42:25.041 INFO:tasks.workunit.client.0.smithi090.stdout:3/551: truncate d1/d17/d36/d2a/d3a/f66 3824939 0 2024-04-26T04:42:25.041 INFO:tasks.workunit.client.1.smithi104.stdout:8/798: dwrite d0/d1c/d27/d47/d67/d76/daf/fb8 [0,4194304] 0 2024-04-26T04:42:25.042 INFO:tasks.workunit.client.1.smithi104.stdout:5/913: write d0/d1/d64/d26/fb8 [1303185,11908] 0 2024-04-26T04:42:25.043 INFO:tasks.workunit.client.1.smithi104.stdout:8/799: mknod d0/d33/da6/c104 0 2024-04-26T04:42:25.046 INFO:tasks.workunit.client.0.smithi090.stdout:5/545: write d4/d1a/df/f6d [3643507,18899] 0 2024-04-26T04:42:25.047 INFO:tasks.workunit.client.1.smithi104.stdout:0/832: dwrite d0/d2/d21/d49/d5e/d9c/d53/f60 [0,4194304] 0 2024-04-26T04:42:25.047 INFO:tasks.workunit.client.1.smithi104.stdout:5/914: mkdir d0/d41/d4f/df2/d108/d132 0 2024-04-26T04:42:25.047 INFO:tasks.workunit.client.1.smithi104.stdout:5/915: dread - d0/d1/d4/d2a/d2b/d37/ff7 zero size 2024-04-26T04:42:25.048 INFO:tasks.workunit.client.0.smithi090.stdout:5/546: rmdir d4/d1a/d1f 39 2024-04-26T04:42:25.049 INFO:tasks.workunit.client.1.smithi104.stdout:0/833: mkdir d0/d2/d21/d27/d1e/d111 0 2024-04-26T04:42:25.049 INFO:tasks.workunit.client.0.smithi090.stdout:5/547: mkdir d4/dab 0 2024-04-26T04:42:25.049 INFO:tasks.workunit.client.0.smithi090.stdout:8/671: dread dc/dd2/f17 [0,4194304] 0 2024-04-26T04:42:25.052 INFO:tasks.workunit.client.1.smithi104.stdout:0/834: rename d0/d2/d21/d49/d5e/d9c/f86 to d0/d4/ddb/f112 0 2024-04-26T04:42:25.053 INFO:tasks.workunit.client.0.smithi090.stdout:5/548: rename d4/d1a/df/d72/c98 to d4/d1a/df/d36/d61/d9b/cac 0 2024-04-26T04:42:25.054 INFO:tasks.workunit.client.0.smithi090.stdout:5/549: getdents d4/d1a/df/d36 0 2024-04-26T04:42:25.057 INFO:tasks.workunit.client.1.smithi104.stdout:0/835: rmdir d0/d2/d21/d49/d5e/d9c/dbe 39 2024-04-26T04:42:25.058 INFO:tasks.workunit.client.0.smithi090.stdout:5/550: symlink d4/d1a/d1f/d53/d59/d5e/da0/lad 0 2024-04-26T04:42:25.058 INFO:tasks.workunit.client.0.smithi090.stdout:5/551: truncate d4/d1a/d2a/d79/d90/d56/f75 696557 0 2024-04-26T04:42:25.058 INFO:tasks.workunit.client.0.smithi090.stdout:7/624: write d0/db/df/d4d/f5a [3215720,89076] 0 2024-04-26T04:42:25.059 INFO:tasks.workunit.client.0.smithi090.stdout:5/552: symlink d4/d1a/d1f/d53/d59/d5e/da0/lae 0 2024-04-26T04:42:25.059 INFO:tasks.workunit.client.0.smithi090.stdout:5/553: dread - d4/d1a/df/d4a/fa4 zero size 2024-04-26T04:42:25.060 INFO:tasks.workunit.client.0.smithi090.stdout:7/625: mknod d0/db/d15/dbc/dd2/ce7 0 2024-04-26T04:42:25.061 INFO:tasks.workunit.client.0.smithi090.stdout:7/626: read d0/db/df/d4d/d64/d66/f7c [22512,103375] 0 2024-04-26T04:42:25.061 INFO:tasks.workunit.client.0.smithi090.stdout:7/627: stat d0/d1/d4/f10 0 2024-04-26T04:42:25.062 INFO:tasks.workunit.client.0.smithi090.stdout:5/554: symlink d4/d1a/df/d4a/laf 0 2024-04-26T04:42:25.062 INFO:tasks.workunit.client.0.smithi090.stdout:5/555: write d4/d1a/d2a/d79/d90/f96 [573938,31437] 0 2024-04-26T04:42:25.062 INFO:tasks.workunit.client.0.smithi090.stdout:5/556: stat d4/d1a/d3f 0 2024-04-26T04:42:25.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/557: rename d4/d1a/df/c15 to d4/d1a/d2a/d79/d90/d56/d80/cb0 0 2024-04-26T04:42:25.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/558: stat d4/d1a/d2a/d79/d90/d94 0 2024-04-26T04:42:25.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/559: write d4/d1a/d2a/d79/d90/d56/f8b [936731,10172] 0 2024-04-26T04:42:25.070 INFO:tasks.workunit.client.0.smithi090.stdout:7/628: getdents d0/db/d53 0 2024-04-26T04:42:25.070 INFO:tasks.workunit.client.0.smithi090.stdout:7/629: readlink d0/d1/d4/d98/d65/l94 0 2024-04-26T04:42:25.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/836: dread d0/f42 [0,4194304] 0 2024-04-26T04:42:25.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/837: write d0/d2/d21/d49/d5e/d9c/f4b [2522098,31070] 0 2024-04-26T04:42:25.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/838: read d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/f8a [67158,94328] 0 2024-04-26T04:42:25.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/839: read d0/f2f [2635034,65072] 0 2024-04-26T04:42:25.075 INFO:tasks.workunit.client.1.smithi104.stdout:0/840: getdents d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.1.smithi104.stdout:0/841: mkdir d0/d2/d21/d27/d1e/d57/d6c/d113 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.1.smithi104.stdout:0/842: symlink d0/d2/d21/l114 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/560: truncate d4/f46 3070173 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/561: chown d4/d1a/c39 103699 1 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.0.smithi090.stdout:7/630: symlink d0/db/de6/le8 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.0.smithi090.stdout:7/631: write d0/d1/d4/d98/d65/db2/fba [1024776,100694] 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/562: mkdir d4/d1a/d2a/d31/db1 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/563: mkdir d4/d1a/d2a/d79/d90/d56/d80/db2 0 2024-04-26T04:42:25.076 INFO:tasks.workunit.client.0.smithi090.stdout:7/632: getdents d0/d1/d4/d98/d65 0 2024-04-26T04:42:25.077 INFO:tasks.workunit.client.1.smithi104.stdout:0/843: write d0/d2/d21/d27/faa [539915,39102] 0 2024-04-26T04:42:25.077 INFO:tasks.workunit.client.0.smithi090.stdout:7/633: creat d0/db/d15/fe9 x:0 0 0 2024-04-26T04:42:25.078 INFO:tasks.workunit.client.0.smithi090.stdout:7/634: chown d0/db/df/d1c/d23/cd5 22016 1 2024-04-26T04:42:25.078 INFO:tasks.workunit.client.0.smithi090.stdout:7/635: rmdir d0/d1/d4/d98/d4a 39 2024-04-26T04:42:25.078 INFO:tasks.workunit.client.1.smithi104.stdout:0/844: symlink d0/d2/d21/d27/d1e/d111/l115 0 2024-04-26T04:42:25.079 INFO:tasks.workunit.client.1.smithi104.stdout:0/845: rename d0/d2/da0/lb9 to d0/d2/d10/d3d/db7/l116 0 2024-04-26T04:42:25.090 INFO:tasks.workunit.client.1.smithi104.stdout:4/979: dwrite d4/d7/d1d/d8e/ff7 [4194304,4194304] 0 2024-04-26T04:42:25.090 INFO:tasks.workunit.client.1.smithi104.stdout:4/980: fsync d4/d7/f2a 0 2024-04-26T04:42:25.090 INFO:tasks.workunit.client.1.smithi104.stdout:4/981: fdatasync d4/d7/d1d/d1e/d15/dd8/f127 0 2024-04-26T04:42:25.091 INFO:tasks.workunit.client.0.smithi090.stdout:7/636: read d0/db/d53/db7/f32 [3929872,73266] 0 2024-04-26T04:42:25.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/666: dwrite d9/d77/db0/db5/dca/fe5 [0,4194304] 0 2024-04-26T04:42:25.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/667: read d9/d77/db0/db5/fa3 [466192,25510] 0 2024-04-26T04:42:25.095 INFO:tasks.workunit.client.0.smithi090.stdout:6/668: mkdir d9/d77/d9a/de9 0 2024-04-26T04:42:25.095 INFO:tasks.workunit.client.1.smithi104.stdout:8/800: getdents d0/d33/da6 0 2024-04-26T04:42:25.095 INFO:tasks.workunit.client.1.smithi104.stdout:8/801: chown d0/d1c/d59/c6c 0 1 2024-04-26T04:42:25.096 INFO:tasks.workunit.client.0.smithi090.stdout:7/637: dread d0/db/df/f97 [0,4194304] 0 2024-04-26T04:42:25.097 INFO:tasks.workunit.client.0.smithi090.stdout:7/638: truncate d0/db/d53/d5e/f8b 494505 0 2024-04-26T04:42:25.097 INFO:tasks.workunit.client.0.smithi090.stdout:7/639: readlink d0/db/d15/d29/l96 0 2024-04-26T04:42:25.097 INFO:tasks.workunit.client.0.smithi090.stdout:6/669: mkdir d9/d1c/d31/d69/db7/dea 0 2024-04-26T04:42:25.099 INFO:tasks.workunit.client.0.smithi090.stdout:6/670: unlink d9/d77/c78 0 2024-04-26T04:42:25.104 INFO:tasks.workunit.client.1.smithi104.stdout:5/916: dwrite d0/d1/d4/d2a/d2b/d37/ff7 [0,4194304] 0 2024-04-26T04:42:25.104 INFO:tasks.workunit.client.1.smithi104.stdout:5/917: truncate d0/d1/d64/d26/d36/da7/db3/fb4 2028178 0 2024-04-26T04:42:25.104 INFO:tasks.workunit.client.1.smithi104.stdout:5/918: readlink d0/d1/d4/d2a/d6b/d73/ldf 0 2024-04-26T04:42:25.108 INFO:tasks.workunit.client.1.smithi104.stdout:5/919: readlink d0/d1/d64/d26/d36/l5d 0 2024-04-26T04:42:25.108 INFO:tasks.workunit.client.1.smithi104.stdout:6/933: dwrite d0/d9/d13/d41/d82/dc8/f2b [4194304,4194304] 0 2024-04-26T04:42:25.109 INFO:tasks.workunit.client.1.smithi104.stdout:6/934: write d0/f9d [121272,122209] 0 2024-04-26T04:42:25.109 INFO:tasks.workunit.client.1.smithi104.stdout:6/935: chown d0/d9/d13/dcd/dd9 41 1 2024-04-26T04:42:25.109 INFO:tasks.workunit.client.0.smithi090.stdout:3/552: dwrite d1/d17/d36/f40 [0,4194304] 0 2024-04-26T04:42:25.112 INFO:tasks.workunit.client.0.smithi090.stdout:3/553: rename d1/d17/d36/d33/d43/db1 to d1/d17/d36/d33/d68/dae/dbb 0 2024-04-26T04:42:25.112 INFO:tasks.workunit.client.0.smithi090.stdout:7/640: write d0/db/d53/f56 [3100707,86722] 0 2024-04-26T04:42:25.117 INFO:tasks.workunit.client.1.smithi104.stdout:7/859: dwrite d2/d4/d32/d35/d44/f91 [0,4194304] 0 2024-04-26T04:42:25.117 INFO:tasks.workunit.client.0.smithi090.stdout:3/554: link d1/d17/d36/d2a/d3a/d5d/c7b d1/d17/d61/cbc 0 2024-04-26T04:42:25.118 INFO:tasks.workunit.client.0.smithi090.stdout:8/672: dwrite dc/dd2/d39/d55/f70 [0,4194304] 0 2024-04-26T04:42:25.118 INFO:tasks.workunit.client.0.smithi090.stdout:5/564: dwrite d4/d1a/d2a/d79/d90/d56/f75 [0,4194304] 0 2024-04-26T04:42:25.118 INFO:tasks.workunit.client.0.smithi090.stdout:5/565: read d4/d1a/d1f/f2b [6538207,38812] 0 2024-04-26T04:42:25.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/641: unlink d0/db/d15/d29/l48 0 2024-04-26T04:42:25.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/642: truncate d0/db/df/d4d/d64/d66/f7c 1024795 0 2024-04-26T04:42:25.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/643: chown d0/db/df/d4d/d64/f92 1778257 1 2024-04-26T04:42:25.122 INFO:tasks.workunit.client.0.smithi090.stdout:7/644: fsync d0/d1/d4/d98/d4a/f8c 0 2024-04-26T04:42:25.122 INFO:tasks.workunit.client.1.smithi104.stdout:5/920: dread d0/d41/d4f/fa0 [0,4194304] 0 2024-04-26T04:42:25.124 INFO:tasks.workunit.client.1.smithi104.stdout:5/921: creat d0/d41/d4f/df2/d108/d132/f133 x:0 0 0 2024-04-26T04:42:25.124 INFO:tasks.workunit.client.1.smithi104.stdout:5/922: creat d0/d1/f134 x:0 0 0 2024-04-26T04:42:25.124 INFO:tasks.workunit.client.1.smithi104.stdout:5/923: truncate d0/d1/d4/d90/d109/dfe/fc0 430780 0 2024-04-26T04:42:25.124 INFO:tasks.workunit.client.1.smithi104.stdout:5/924: fdatasync d0/d1/d4/d2a/d2b/d37/ff7 0 2024-04-26T04:42:25.124 INFO:tasks.workunit.client.0.smithi090.stdout:3/555: unlink d1/d17/d36/d33/d43/db0/d1b/d58/da4/fb4 0 2024-04-26T04:42:25.128 INFO:tasks.workunit.client.0.smithi090.stdout:5/566: rename d4/d1a/df/d36/d61 to d4/d1a/d2a/d79/d90/db3 0 2024-04-26T04:42:25.129 INFO:tasks.workunit.client.1.smithi104.stdout:0/846: dwrite d0/d2/d21/d27/f30 [0,4194304] 0 2024-04-26T04:42:25.131 INFO:tasks.workunit.client.0.smithi090.stdout:7/645: rename d0/d1/d4/d98/d4a/f8c to d0/d1/d4/d3e/dd3/de2/fea 0 2024-04-26T04:42:25.131 INFO:tasks.workunit.client.0.smithi090.stdout:7/646: rename d0/d1/d4/d3e to d0/d1/d4/d3e/dd3/de2/deb 22 2024-04-26T04:42:25.135 INFO:tasks.workunit.client.1.smithi104.stdout:7/860: read d2/fb8 [1775385,93631] 0 2024-04-26T04:42:25.136 INFO:tasks.workunit.client.0.smithi090.stdout:5/567: link d4/d1a/df/d4a/f89 d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 0 2024-04-26T04:42:25.136 INFO:tasks.workunit.client.0.smithi090.stdout:5/568: fsync d4/fe 0 2024-04-26T04:42:25.136 INFO:tasks.workunit.client.0.smithi090.stdout:8/673: write dc/dd2/d4f/da6/dd8/fc5 [2306239,57130] 0 2024-04-26T04:42:25.136 INFO:tasks.workunit.client.0.smithi090.stdout:8/674: stat dc/dd2/d3b 0 2024-04-26T04:42:25.136 INFO:tasks.workunit.client.0.smithi090.stdout:8/675: creat dc/dd2/d4f/db7/fec x:0 0 0 2024-04-26T04:42:25.136 INFO:tasks.workunit.client.0.smithi090.stdout:7/647: rename d0/d1/c69 to d0/db/df/d4d/cec 0 2024-04-26T04:42:25.139 INFO:tasks.workunit.client.0.smithi090.stdout:5/569: mkdir d4/d1a/d2a/db5 0 2024-04-26T04:42:25.140 INFO:tasks.workunit.client.1.smithi104.stdout:5/925: creat d0/d1/d4/d2a/d6b/dfb/f135 x:0 0 0 2024-04-26T04:42:25.142 INFO:tasks.workunit.client.1.smithi104.stdout:4/982: dwrite d4/d7/d1d/d1e/f61 [0,4194304] 0 2024-04-26T04:42:25.142 INFO:tasks.workunit.client.1.smithi104.stdout:4/983: write d4/db/d6b/fc9 [3623215,18541] 0 2024-04-26T04:42:25.142 INFO:tasks.workunit.client.1.smithi104.stdout:4/984: chown d4/d7/l91 115 1 2024-04-26T04:42:25.144 INFO:tasks.workunit.client.1.smithi104.stdout:0/847: symlink d0/d2/d21/d27/d1e/l117 0 2024-04-26T04:42:25.144 INFO:tasks.workunit.client.1.smithi104.stdout:0/848: creat d0/d2/d21/f118 x:0 0 0 2024-04-26T04:42:25.145 INFO:tasks.workunit.client.0.smithi090.stdout:8/676: symlink dc/dd2/d3b/d3d/db8/led 0 2024-04-26T04:42:25.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/570: mkdir d4/d1a/d45/db6 0 2024-04-26T04:42:25.146 INFO:tasks.workunit.client.0.smithi090.stdout:8/677: rmdir dc/dd2/d3b/d3d/db8/ddc 39 2024-04-26T04:42:25.146 INFO:tasks.workunit.client.0.smithi090.stdout:5/571: write d4/d1a/d3f/f52 [95220,61376] 0 2024-04-26T04:42:25.147 INFO:tasks.workunit.client.1.smithi104.stdout:5/926: truncate d0/d1/d3e/f51 854105 0 2024-04-26T04:42:25.148 INFO:tasks.workunit.client.0.smithi090.stdout:5/572: rmdir d4/d1a/d45/db6 0 2024-04-26T04:42:25.149 INFO:tasks.workunit.client.0.smithi090.stdout:3/556: dread d1/d9/f12 [0,4194304] 0 2024-04-26T04:42:25.153 INFO:tasks.workunit.client.1.smithi104.stdout:0/849: rename d0/d2/d21/d49/d5e/l105 to d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/l119 0 2024-04-26T04:42:25.155 INFO:tasks.workunit.client.1.smithi104.stdout:5/927: link d0/d1/d4/d90/d109/dfe/l106 d0/d41/d4f/df2/d108/d132/l136 0 2024-04-26T04:42:25.155 INFO:tasks.workunit.client.1.smithi104.stdout:5/928: creat d0/d1/d64/f137 x:0 0 0 2024-04-26T04:42:25.156 INFO:tasks.workunit.client.1.smithi104.stdout:5/929: truncate d0/d1/d4/d90/d109/fea 549981 0 2024-04-26T04:42:25.156 INFO:tasks.workunit.client.1.smithi104.stdout:4/985: mkdir d4/d7/d1d/d1e/d24/d4a/d153 0 2024-04-26T04:42:25.156 INFO:tasks.workunit.client.0.smithi090.stdout:8/678: write dc/dd2/d39/f52 [3147839,68918] 0 2024-04-26T04:42:25.157 INFO:tasks.workunit.client.0.smithi090.stdout:5/573: dread d4/d1a/df/f22 [0,4194304] 0 2024-04-26T04:42:25.158 INFO:tasks.workunit.client.0.smithi090.stdout:8/679: mkdir dc/dd2/d3b/d3d/d97/dee 0 2024-04-26T04:42:25.158 INFO:tasks.workunit.client.0.smithi090.stdout:8/680: mknod dc/d34/cef 0 2024-04-26T04:42:25.159 INFO:tasks.workunit.client.1.smithi104.stdout:6/936: dwrite d0/d9/d13/d41/d96/dac/f4e [0,4194304] 0 2024-04-26T04:42:25.159 INFO:tasks.workunit.client.1.smithi104.stdout:6/937: write d0/d9/d13/d41/d82/dc8/d80/f61 [1563157,13298] 0 2024-04-26T04:42:25.159 INFO:tasks.workunit.client.1.smithi104.stdout:6/938: chown d0/dd8/de0/d102/fe8 14414 1 2024-04-26T04:42:25.160 INFO:tasks.workunit.client.1.smithi104.stdout:6/939: chown d0/d9/d13/cc7 30936 1 2024-04-26T04:42:25.162 INFO:tasks.workunit.client.0.smithi090.stdout:6/671: dwrite d9/d77/db0/db5/dda/d6d/d92/f26 [0,4194304] 0 2024-04-26T04:42:25.163 INFO:tasks.workunit.client.1.smithi104.stdout:7/861: dwrite d2/d4/d32/d3d/ffe [0,4194304] 0 2024-04-26T04:42:25.163 INFO:tasks.workunit.client.1.smithi104.stdout:7/862: readlink d2/d4/d32/d3d/d41/d5f/la5 0 2024-04-26T04:42:25.166 INFO:tasks.workunit.client.1.smithi104.stdout:8/802: dwrite d0/d1c/d27/d46/f78 [0,4194304] 0 2024-04-26T04:42:25.166 INFO:tasks.workunit.client.1.smithi104.stdout:8/803: write d0/d33/d35/d6d/f98 [750324,21107] 0 2024-04-26T04:42:25.166 INFO:tasks.workunit.client.1.smithi104.stdout:8/804: truncate d0/d33/d35/d6d/f95 1482148 0 2024-04-26T04:42:25.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/648: dwrite d0/d1/d4/fc5 [0,4194304] 0 2024-04-26T04:42:25.168 INFO:tasks.workunit.client.0.smithi090.stdout:7/649: write d0/d1/d4/d98/d65/dbb/fc9 [443052,117847] 0 2024-04-26T04:42:25.177 INFO:tasks.workunit.client.0.smithi090.stdout:6/672: symlink d9/d77/db0/db5/dda/d6d/leb 0 2024-04-26T04:42:25.178 INFO:tasks.workunit.client.0.smithi090.stdout:6/673: write d9/d1c/d31/d79/fba [523855,100290] 0 2024-04-26T04:42:25.189 INFO:tasks.workunit.client.0.smithi090.stdout:8/681: dwrite dc/dd2/d3b/d3d/f65 [0,4194304] 0 2024-04-26T04:42:25.189 INFO:tasks.workunit.client.0.smithi090.stdout:8/682: truncate dc/d34/d4d/fc4 52937 0 2024-04-26T04:42:25.193 INFO:tasks.workunit.client.1.smithi104.stdout:6/940: dwrite d0/d9/d13/d41/d82/dc8/f42 [0,4194304] 0 2024-04-26T04:42:25.194 INFO:tasks.workunit.client.1.smithi104.stdout:6/941: fsync d0/d9/d13/ff8 0 2024-04-26T04:42:25.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/650: dwrite d0/d1/d4/d3e/fcf [0,4194304] 0 2024-04-26T04:42:25.198 INFO:tasks.workunit.client.0.smithi090.stdout:7/651: fdatasync d0/d1/f61 0 2024-04-26T04:42:25.199 INFO:tasks.workunit.client.0.smithi090.stdout:6/674: dwrite d9/d21/d40/d4c/d6f/d9f/dae/fdd [4194304,4194304] 0 2024-04-26T04:42:25.202 INFO:tasks.workunit.client.0.smithi090.stdout:7/652: mknod d0/db/df/d4d/d64/d66/ced 0 2024-04-26T04:42:25.203 INFO:tasks.workunit.client.0.smithi090.stdout:6/675: creat d9/d77/d9a/dc8/fec x:0 0 0 2024-04-26T04:42:25.203 INFO:tasks.workunit.client.0.smithi090.stdout:6/676: creat d9/d1c/dc5/dd9/fed x:0 0 0 2024-04-26T04:42:25.204 INFO:tasks.workunit.client.0.smithi090.stdout:7/653: mknod d0/db/df/d4d/d7b/cee 0 2024-04-26T04:42:25.204 INFO:tasks.workunit.client.0.smithi090.stdout:7/654: write d0/db/df/d1c/fa6 [183826,114282] 0 2024-04-26T04:42:25.209 INFO:tasks.workunit.client.0.smithi090.stdout:8/683: dwrite dc/d1f/f21 [0,4194304] 0 2024-04-26T04:42:25.209 INFO:tasks.workunit.client.0.smithi090.stdout:8/684: chown dc/d5a/d5d/d73/ce2 1002582 1 2024-04-26T04:42:25.219 INFO:tasks.workunit.client.0.smithi090.stdout:2/578: sync 2024-04-26T04:42:25.219 INFO:tasks.workunit.client.0.smithi090.stdout:8/685: write dc/ff [2060363,67667] 0 2024-04-26T04:42:25.220 INFO:tasks.workunit.client.1.smithi104.stdout:4/986: truncate d4/d7/f2a 418484 0 2024-04-26T04:42:25.220 INFO:tasks.workunit.client.1.smithi104.stdout:4/987: dread - d4/d7/d1d/d1e/d24/d4a/f142 zero size 2024-04-26T04:42:25.220 INFO:tasks.workunit.client.1.smithi104.stdout:4/988: write d4/d20/d55/d56/f3e [5031426,35605] 0 2024-04-26T04:42:25.221 INFO:tasks.workunit.client.0.smithi090.stdout:7/655: dwrite d0/f45 [0,4194304] 0 2024-04-26T04:42:25.221 INFO:tasks.workunit.client.0.smithi090.stdout:7/656: readlink d0/d1/d4/d98/d65/db2/lb8 0 2024-04-26T04:42:25.222 INFO:tasks.workunit.client.0.smithi090.stdout:8/686: symlink dc/dd2/d4f/d58/lf0 0 2024-04-26T04:42:25.222 INFO:tasks.workunit.client.0.smithi090.stdout:8/687: chown dc/d34/d4d/f4e 109937 1 2024-04-26T04:42:25.222 INFO:tasks.workunit.client.0.smithi090.stdout:8/688: chown dc/dd2/d39/d55/d64/f44 15282 1 2024-04-26T04:42:25.223 INFO:tasks.workunit.client.0.smithi090.stdout:7/657: creat d0/db/d15/dbc/dd2/fef x:0 0 0 2024-04-26T04:42:25.223 INFO:tasks.workunit.client.0.smithi090.stdout:7/658: creat d0/d1/d4/d98/db1/ff0 x:0 0 0 2024-04-26T04:42:25.225 INFO:tasks.workunit.client.1.smithi104.stdout:0/850: truncate d0/d2/d21/d27/faa 697431 0 2024-04-26T04:42:25.225 INFO:tasks.workunit.client.1.smithi104.stdout:0/851: chown d0/d2/d21/d49/d5e/f7d 3759063 1 2024-04-26T04:42:25.225 INFO:tasks.workunit.client.1.smithi104.stdout:0/852: write d0/d2/d21/f97 [55824,40205] 0 2024-04-26T04:42:25.225 INFO:tasks.workunit.client.1.smithi104.stdout:0/853: chown d0/f10d 1 1 2024-04-26T04:42:25.225 INFO:tasks.workunit.client.1.smithi104.stdout:0/854: creat d0/d2/d10/d3d/dc0/df1/f11a x:0 0 0 2024-04-26T04:42:25.226 INFO:tasks.workunit.client.1.smithi104.stdout:7/863: mknod d2/d4/d32/d35/d37/c112 0 2024-04-26T04:42:25.227 INFO:tasks.workunit.client.0.smithi090.stdout:7/659: symlink d0/d1/da3/lf1 0 2024-04-26T04:42:25.227 INFO:tasks.workunit.client.0.smithi090.stdout:7/660: chown d0/d1/d4/d3e/l68 0 1 2024-04-26T04:42:25.227 INFO:tasks.workunit.client.0.smithi090.stdout:7/661: readlink d0/db/d53/db7/l80 0 2024-04-26T04:42:25.227 INFO:tasks.workunit.client.1.smithi104.stdout:8/805: getdents d0/d9a 0 2024-04-26T04:42:25.229 INFO:tasks.workunit.client.0.smithi090.stdout:7/662: mkdir d0/db/df/d4d/d64/df2 0 2024-04-26T04:42:25.230 INFO:tasks.workunit.client.1.smithi104.stdout:9/747: sync 2024-04-26T04:42:25.230 INFO:tasks.workunit.client.1.smithi104.stdout:6/942: unlink d0/d9/d13/d41/d96/dac/dec/f9b 0 2024-04-26T04:42:25.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/677: truncate d9/d1c/dc5/dd9/fe4 7435375 0 2024-04-26T04:42:25.231 INFO:tasks.workunit.client.1.smithi104.stdout:5/930: getdents d0/d1/d4/d2a/d2b/d37/d48/dc1/dfd 0 2024-04-26T04:42:25.232 INFO:tasks.workunit.client.0.smithi090.stdout:8/689: truncate dc/dd2/d3b/d3d/f65 1307800 0 2024-04-26T04:42:25.233 INFO:tasks.workunit.client.1.smithi104.stdout:7/864: dread d2/d4/d32/d35/f4d [0,4194304] 0 2024-04-26T04:42:25.234 INFO:tasks.workunit.client.1.smithi104.stdout:4/989: mknod d4/db/d6b/d141/c154 0 2024-04-26T04:42:25.234 INFO:tasks.workunit.client.1.smithi104.stdout:3/982: sync 2024-04-26T04:42:25.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/690: creat dc/dd2/d39/d55/d64/ff1 x:0 0 0 2024-04-26T04:42:25.234 INFO:tasks.workunit.client.0.smithi090.stdout:8/691: fdatasync dc/d34/d4d/f9e 0 2024-04-26T04:42:25.237 INFO:tasks.workunit.client.0.smithi090.stdout:8/692: creat dc/d5a/d5d/d73/dce/ff2 x:0 0 0 2024-04-26T04:42:25.238 INFO:tasks.workunit.client.0.smithi090.stdout:6/678: read d9/d21/d3b/f46 [1538606,5080] 0 2024-04-26T04:42:25.240 INFO:tasks.workunit.client.1.smithi104.stdout:8/806: creat d0/d33/d35/d6d/f105 x:0 0 0 2024-04-26T04:42:25.240 INFO:tasks.workunit.client.1.smithi104.stdout:8/807: creat d0/d33/d35/d6d/f106 x:0 0 0 2024-04-26T04:42:25.242 INFO:tasks.workunit.client.0.smithi090.stdout:6/679: link d9/f14 d9/d77/db0/db5/dda/d58/fee 0 2024-04-26T04:42:25.243 INFO:tasks.workunit.client.1.smithi104.stdout:9/748: rename d11/d15/d32/c3b to d11/d15/d32/d4c/d8a/cf2 0 2024-04-26T04:42:25.243 INFO:tasks.workunit.client.1.smithi104.stdout:9/749: write d11/f89 [1664493,80201] 0 2024-04-26T04:42:25.243 INFO:tasks.workunit.client.1.smithi104.stdout:9/750: read - d11/d15/d32/d35/d7b/fde zero size 2024-04-26T04:42:25.244 INFO:tasks.workunit.client.1.smithi104.stdout:6/943: mkdir d0/d9/d13/d41/d82/df7/dfb/d133 0 2024-04-26T04:42:25.245 INFO:tasks.workunit.client.0.smithi090.stdout:6/680: getdents d9/d1c/da2 0 2024-04-26T04:42:25.245 INFO:tasks.workunit.client.0.smithi090.stdout:6/681: chown d9/d1c/d31/d69/c89 260888767 1 2024-04-26T04:42:25.246 INFO:tasks.workunit.client.1.smithi104.stdout:5/931: mkdir d0/d1/d4/d2a/d6b/d73/d11d/d138 0 2024-04-26T04:42:25.247 INFO:tasks.workunit.client.1.smithi104.stdout:7/865: mknod d2/d4/d32/c113 0 2024-04-26T04:42:25.249 INFO:tasks.workunit.client.1.smithi104.stdout:4/990: link d4/d7/d1d/d1e/d24/fe4 d4/d7/d1d/d2e/d74/dbd/dd3/f155 0 2024-04-26T04:42:25.250 INFO:tasks.workunit.client.1.smithi104.stdout:0/855: truncate d0/d2/d21/d27/d1e/d57/d6c/f6a 500574 0 2024-04-26T04:42:25.255 INFO:tasks.workunit.client.1.smithi104.stdout:8/808: symlink d0/d9a/l107 0 2024-04-26T04:42:25.256 INFO:tasks.workunit.client.1.smithi104.stdout:8/809: chown d0/d1c/d2d/f32 700 1 2024-04-26T04:42:25.256 INFO:tasks.workunit.client.0.smithi090.stdout:2/579: dwrite db/d11/d14/d25/f37 [0,4194304] 0 2024-04-26T04:42:25.257 INFO:tasks.workunit.client.1.smithi104.stdout:9/751: write d11/d15/d32/d4c/d54/f7a [3901679,80785] 0 2024-04-26T04:42:25.258 INFO:tasks.workunit.client.0.smithi090.stdout:2/580: mknod db/d11/d14/d1b/cc1 0 2024-04-26T04:42:25.258 INFO:tasks.workunit.client.0.smithi090.stdout:2/581: write db/d11/d14/d27/f7b [150459,106334] 0 2024-04-26T04:42:25.258 INFO:tasks.workunit.client.0.smithi090.stdout:2/582: write db/d11/d14/d27/db0/faa [858910,77406] 0 2024-04-26T04:42:25.258 INFO:tasks.workunit.client.0.smithi090.stdout:2/583: chown db/d11/d14/d1b/f6c 155 1 2024-04-26T04:42:25.258 INFO:tasks.workunit.client.0.smithi090.stdout:2/584: chown db/d11/d14/d25/d2d/d48/c88 0 1 2024-04-26T04:42:25.259 INFO:tasks.workunit.client.1.smithi104.stdout:3/983: rename d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d49 to d5/d143/d152 0 2024-04-26T04:42:25.259 INFO:tasks.workunit.client.1.smithi104.stdout:3/984: write d5/db2/f1b [2587561,90539] 0 2024-04-26T04:42:25.259 INFO:tasks.workunit.client.1.smithi104.stdout:3/985: read - d5/d9/d80/dbc/dd4/f14c zero size 2024-04-26T04:42:25.259 INFO:tasks.workunit.client.1.smithi104.stdout:4/991: write d4/db/d6b/fa2 [2135749,43443] 0 2024-04-26T04:42:25.259 INFO:tasks.workunit.client.1.smithi104.stdout:4/992: truncate d4/db/d6b/d86/f101 2581334 0 2024-04-26T04:42:25.260 INFO:tasks.workunit.client.0.smithi090.stdout:2/585: symlink db/d11/d14/lc2 0 2024-04-26T04:42:25.260 INFO:tasks.workunit.client.1.smithi104.stdout:6/944: creat d0/d9/d13/d41/d96/dac/df1/f134 x:0 0 0 2024-04-26T04:42:25.261 INFO:tasks.workunit.client.1.smithi104.stdout:5/932: creat d0/d1/d4/d2a/d2b/d5c/f139 x:0 0 0 2024-04-26T04:42:25.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/586: rename db/d11/d14/d63/fac to db/d11/d14/d25/d86/fc3 0 2024-04-26T04:42:25.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/587: write db/d11/d14/d27/f7b [1204991,108756] 0 2024-04-26T04:42:25.262 INFO:tasks.workunit.client.1.smithi104.stdout:7/866: unlink d2/d4/d32/d3d/d41/d5f/d6c/fb9 0 2024-04-26T04:42:25.264 INFO:tasks.workunit.client.1.smithi104.stdout:8/810: mknod d0/d33/c108 0 2024-04-26T04:42:25.264 INFO:tasks.workunit.client.1.smithi104.stdout:8/811: creat d0/d1c/d27/d47/d67/d76/daf/f109 x:0 0 0 2024-04-26T04:42:25.265 INFO:tasks.workunit.client.1.smithi104.stdout:8/812: readlink d0/d1c/d27/d47/ldd 0 2024-04-26T04:42:25.266 INFO:tasks.workunit.client.1.smithi104.stdout:8/813: dread d0/d50/ff9 [0,4194304] 0 2024-04-26T04:42:25.266 INFO:tasks.workunit.client.1.smithi104.stdout:9/752: mkdir d11/d15/d1b/d6d/dd5/df3 0 2024-04-26T04:42:25.266 INFO:tasks.workunit.client.1.smithi104.stdout:9/753: dread - d11/d15/d1b/d6d/fcd zero size 2024-04-26T04:42:25.267 INFO:tasks.workunit.client.1.smithi104.stdout:4/993: readlink d4/d7/d1d/d2e/l5e 0 2024-04-26T04:42:25.268 INFO:tasks.workunit.client.1.smithi104.stdout:6/945: link d0/d9/d13/d41/d96/lf6 d0/d9/d13/dcd/dd9/df5/l135 0 2024-04-26T04:42:25.270 INFO:tasks.workunit.client.1.smithi104.stdout:8/814: symlink d0/d1c/d2d/d7a/l10a 0 2024-04-26T04:42:25.271 INFO:tasks.workunit.client.1.smithi104.stdout:4/994: symlink d4/d7/d1d/d8e/l156 0 2024-04-26T04:42:25.272 INFO:tasks.workunit.client.1.smithi104.stdout:5/933: symlink d0/d1/d1e/l13a 0 2024-04-26T04:42:25.272 INFO:tasks.workunit.client.1.smithi104.stdout:5/934: read - d0/d1/d4/d2a/d6b/d73/d11d/d125/db2/f12e zero size 2024-04-26T04:42:25.273 INFO:tasks.workunit.client.1.smithi104.stdout:7/867: write d2/d4/f74 [2219482,56693] 0 2024-04-26T04:42:25.273 INFO:tasks.workunit.client.1.smithi104.stdout:5/935: getdents d0/d1/d4/d2a/d2b 0 2024-04-26T04:42:25.275 INFO:tasks.workunit.client.1.smithi104.stdout:7/868: creat d2/d4/d32/d3d/d41/d5f/d6c/d94/f114 x:0 0 0 2024-04-26T04:42:25.275 INFO:tasks.workunit.client.1.smithi104.stdout:7/869: chown d2/d4/d5a/f104 1153330166 1 2024-04-26T04:42:25.275 INFO:tasks.workunit.client.1.smithi104.stdout:7/870: truncate d2/d4/d32/d35/d44/d5b/f7c 1143010 0 2024-04-26T04:42:25.275 INFO:tasks.workunit.client.1.smithi104.stdout:7/871: chown d2/d4/d32/d35/d37/c4c 30098256 1 2024-04-26T04:42:25.275 INFO:tasks.workunit.client.1.smithi104.stdout:7/872: chown d2/d4/d1d/d34/lf9 6062975 1 2024-04-26T04:42:25.276 INFO:tasks.workunit.client.1.smithi104.stdout:5/936: creat d0/d1/d4/d2a/d2b/d37/d48/dc1/f13b x:0 0 0 2024-04-26T04:42:25.276 INFO:tasks.workunit.client.1.smithi104.stdout:7/873: rmdir d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d 39 2024-04-26T04:42:25.278 INFO:tasks.workunit.client.1.smithi104.stdout:5/937: link d0/f5 d0/d1/d1e/df5/f13c 0 2024-04-26T04:42:25.279 INFO:tasks.workunit.client.1.smithi104.stdout:7/874: getdents d2/d4/d1d 0 2024-04-26T04:42:25.280 INFO:tasks.workunit.client.1.smithi104.stdout:7/875: getdents d2/d4 0 2024-04-26T04:42:25.281 INFO:tasks.workunit.client.1.smithi104.stdout:7/876: getdents d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/dd8 0 2024-04-26T04:42:25.281 INFO:tasks.workunit.client.1.smithi104.stdout:7/877: fdatasync d2/fb8 0 2024-04-26T04:42:25.282 INFO:tasks.workunit.client.1.smithi104.stdout:7/878: fsync d2/d4/d32/d3d/d41/d5f/d6d/fa3 0 2024-04-26T04:42:25.282 INFO:tasks.workunit.client.1.smithi104.stdout:7/879: creat d2/d4/d1d/d34/d7f/f115 x:0 0 0 2024-04-26T04:42:25.283 INFO:tasks.workunit.client.1.smithi104.stdout:6/946: dread d0/d9/d13/d41/d96/dac/dc4/f118 [0,4194304] 0 2024-04-26T04:42:25.285 INFO:tasks.workunit.client.1.smithi104.stdout:6/947: mkdir d0/d9/d13/d41/d96/ded/d10d/d71/d136 0 2024-04-26T04:42:25.286 INFO:tasks.workunit.client.1.smithi104.stdout:6/948: symlink d0/d9/d13/d41/d96/dac/l137 0 2024-04-26T04:42:25.288 INFO:tasks.workunit.client.1.smithi104.stdout:4/995: read d4/d20/d55/d56/f3e [636747,112456] 0 2024-04-26T04:42:25.289 INFO:tasks.workunit.client.1.smithi104.stdout:4/996: rename d4/d20/d55/d56/ff5 to d4/d7/f157 0 2024-04-26T04:42:25.290 INFO:tasks.workunit.client.1.smithi104.stdout:6/949: dread d0/d9/d13/d41/d96/ded/d10d/d71/da5/fb7 [0,4194304] 0 2024-04-26T04:42:25.294 INFO:tasks.workunit.client.1.smithi104.stdout:7/880: dread d2/d4/d1d/d34/d7f/faf [4194304,4194304] 0 2024-04-26T04:42:25.295 INFO:tasks.workunit.client.1.smithi104.stdout:7/881: mkdir d2/d4/d32/d35/d116 0 2024-04-26T04:42:25.296 INFO:tasks.workunit.client.1.smithi104.stdout:7/882: mkdir d2/d4/d32/d3d/d41/d7e/de0/d117 0 2024-04-26T04:42:25.296 INFO:tasks.workunit.client.1.smithi104.stdout:7/883: creat d2/d4/d32/d3d/d41/d7e/de0/d117/f118 x:0 0 0 2024-04-26T04:42:25.302 INFO:tasks.workunit.client.0.smithi090.stdout:7/663: dwrite d0/d1/d4/d3e/fcf [4194304,4194304] 0 2024-04-26T04:42:25.303 INFO:tasks.workunit.client.0.smithi090.stdout:7/664: readlink d0/d1/da3/lb4 0 2024-04-26T04:42:25.306 INFO:tasks.workunit.client.1.smithi104.stdout:7/884: dread d2/d4/d32/d35/d44/fa2 [0,4194304] 0 2024-04-26T04:42:25.306 INFO:tasks.workunit.client.1.smithi104.stdout:7/885: creat d2/d4/d32/d3d/d47/db1/dfa/f119 x:0 0 0 2024-04-26T04:42:25.306 INFO:tasks.workunit.client.0.smithi090.stdout:7/665: mkdir d0/db/d15/d29/df3 0 2024-04-26T04:42:25.308 INFO:tasks.workunit.client.1.smithi104.stdout:6/950: write d0/d9/d13/d41/d96/ded/d10d/d71/f21 [3490712,1141] 0 2024-04-26T04:42:25.309 INFO:tasks.workunit.client.1.smithi104.stdout:6/951: dread d0/d9/d13/d41/d82/dc8/d80/fa7 [0,4194304] 0 2024-04-26T04:42:25.310 INFO:tasks.workunit.client.1.smithi104.stdout:4/997: dread d4/d7/d1d/d1e/d8f/db6/ffa [0,4194304] 0 2024-04-26T04:42:25.310 INFO:tasks.workunit.client.1.smithi104.stdout:6/952: symlink d0/d9/d13/d41/d82/df7/l138 0 2024-04-26T04:42:25.311 INFO:tasks.workunit.client.1.smithi104.stdout:6/953: dread - d0/d9/f93 zero size 2024-04-26T04:42:25.311 INFO:tasks.workunit.client.0.smithi090.stdout:8/693: dwrite dc/d5a/d5d/f66 [0,4194304] 0 2024-04-26T04:42:25.311 INFO:tasks.workunit.client.0.smithi090.stdout:6/682: dwrite d9/d77/db0/db5/dda/d6d/d66/fb8 [0,4194304] 0 2024-04-26T04:42:25.312 INFO:tasks.workunit.client.0.smithi090.stdout:6/683: write d9/d21/d40/d4c/d6f/fb6 [369901,63772] 0 2024-04-26T04:42:25.313 INFO:tasks.workunit.client.1.smithi104.stdout:4/998: unlink d4/d20/f51 0 2024-04-26T04:42:25.313 INFO:tasks.workunit.client.1.smithi104.stdout:4/999: fdatasync d4/db/d6b/fc9 0 2024-04-26T04:42:25.314 INFO:tasks.workunit.client.1.smithi104.stdout:3/986: dwrite d5/db2/da3/dd0/ddb/d69/f6e [0,4194304] 0 2024-04-26T04:42:25.314 INFO:tasks.workunit.client.1.smithi104.stdout:5/938: dwrite d0/d1/f134 [0,4194304] 0 2024-04-26T04:42:25.314 INFO:tasks.workunit.client.1.smithi104.stdout:6/954: link d0/d9/d13/cc5 d0/d9/d13/dcd/dd9/d120/c139 0 2024-04-26T04:42:25.316 INFO:tasks.workunit.client.1.smithi104.stdout:8/815: dwrite d0/d1c/d27/d47/d67/d76/daf/feb [0,4194304] 0 2024-04-26T04:42:25.316 INFO:tasks.workunit.client.1.smithi104.stdout:8/816: chown d0/d33/fab 473 1 2024-04-26T04:42:25.317 INFO:tasks.workunit.client.0.smithi090.stdout:2/588: dread db/d11/d14/d27/db0/faa [0,4194304] 0 2024-04-26T04:42:25.318 INFO:tasks.workunit.client.0.smithi090.stdout:8/694: symlink dc/dd2/d4f/da6/lf3 0 2024-04-26T04:42:25.320 INFO:tasks.workunit.client.1.smithi104.stdout:3/987: unlink d5/db2/da3/dd0/ddb/d32/dd9/c151 0 2024-04-26T04:42:25.322 INFO:tasks.workunit.client.1.smithi104.stdout:7/886: dwrite d2/d4/d32/d3d/d47/db6/f10a [0,4194304] 0 2024-04-26T04:42:25.327 INFO:tasks.workunit.client.0.smithi090.stdout:7/666: truncate d0/f45 2658050 0 2024-04-26T04:42:25.329 INFO:tasks.workunit.client.0.smithi090.stdout:6/684: creat d9/d77/db0/db5/fef x:0 0 0 2024-04-26T04:42:25.330 INFO:tasks.workunit.client.1.smithi104.stdout:8/817: mknod d0/d1c/d2d/c10b 0 2024-04-26T04:42:25.330 INFO:tasks.workunit.client.1.smithi104.stdout:8/818: chown d0/d50/lba 2 1 2024-04-26T04:42:25.332 INFO:tasks.workunit.client.0.smithi090.stdout:2/589: unlink db/d11/d14/d1b/d20/d4c/d98/db4/cb6 0 2024-04-26T04:42:25.332 INFO:tasks.workunit.client.1.smithi104.stdout:3/988: mkdir d5/db2/d153 0 2024-04-26T04:42:25.333 INFO:tasks.workunit.client.1.smithi104.stdout:3/989: read d5/db2/da3/dd0/ddb/d69/fb6 [48192,52515] 0 2024-04-26T04:42:25.334 INFO:tasks.workunit.client.0.smithi090.stdout:7/667: creat d0/db/d15/d29/df3/ff4 x:0 0 0 2024-04-26T04:42:25.334 INFO:tasks.workunit.client.0.smithi090.stdout:7/668: truncate d0/d1/d4/d3e/f4b 2425860 0 2024-04-26T04:42:25.334 INFO:tasks.workunit.client.1.smithi104.stdout:5/939: getdents d0/d1/de6 0 2024-04-26T04:42:25.335 INFO:tasks.workunit.client.0.smithi090.stdout:6/685: symlink d9/d1c/lf0 0 2024-04-26T04:42:25.337 INFO:tasks.workunit.client.0.smithi090.stdout:2/590: rename db/dc/l82 to db/d11/d14/d25/d2d/lc4 0 2024-04-26T04:42:25.338 INFO:tasks.workunit.client.1.smithi104.stdout:6/955: rmdir d0/d9/d13/d41/d82/dc8/d80 39 2024-04-26T04:42:25.339 INFO:tasks.workunit.client.0.smithi090.stdout:8/695: symlink dc/dd2/d3b/lf4 0 2024-04-26T04:42:25.339 INFO:tasks.workunit.client.0.smithi090.stdout:8/696: readlink dc/d5a/l76 0 2024-04-26T04:42:25.339 INFO:tasks.workunit.client.0.smithi090.stdout:8/697: creat dc/d5a/ff5 x:0 0 0 2024-04-26T04:42:25.342 INFO:tasks.workunit.client.0.smithi090.stdout:8/698: rename dc/d5a/d5d/c7b to dc/d5a/d5d/cf6 0 2024-04-26T04:42:25.343 INFO:tasks.workunit.client.0.smithi090.stdout:0/685: sync 2024-04-26T04:42:25.344 INFO:tasks.workunit.client.0.smithi090.stdout:4/562: sync 2024-04-26T04:42:25.344 INFO:tasks.workunit.client.0.smithi090.stdout:0/686: write d0/d4b/d5d/d5c/d8a/d45/d53/f87 [43373,122629] 0 2024-04-26T04:42:25.344 INFO:tasks.workunit.client.0.smithi090.stdout:0/687: read d0/d4b/d5d/d5c/d8a/d45/f59 [2844898,39607] 0 2024-04-26T04:42:25.344 INFO:tasks.workunit.client.0.smithi090.stdout:0/688: chown d0/d4b/d5d/d48/d7d/fc3 2914 1 2024-04-26T04:42:25.344 INFO:tasks.workunit.client.0.smithi090.stdout:8/699: symlink dc/dd2/d4f/da6/db1/lf7 0 2024-04-26T04:42:25.346 INFO:tasks.workunit.client.1.smithi104.stdout:7/887: dwrite d2/d4/d32/d35/dbe/fc2 [0,4194304] 0 2024-04-26T04:42:25.346 INFO:tasks.workunit.client.1.smithi104.stdout:6/956: dread d0/f9c [0,4194304] 0 2024-04-26T04:42:25.347 INFO:tasks.workunit.client.0.smithi090.stdout:7/669: dread d0/db/d15/d29/fc [8388608,4194304] 0 2024-04-26T04:42:25.347 INFO:tasks.workunit.client.0.smithi090.stdout:7/670: creat d0/db/df/d1c/d23/d8d/ff5 x:0 0 0 2024-04-26T04:42:25.347 INFO:tasks.workunit.client.0.smithi090.stdout:4/563: mknod d3/d13/d64/dae/cb2 0 2024-04-26T04:42:25.350 INFO:tasks.workunit.client.0.smithi090.stdout:7/671: creat d0/db/d15/dbc/ff6 x:0 0 0 2024-04-26T04:42:25.351 INFO:tasks.workunit.client.1.smithi104.stdout:8/819: rename d0/d33/d35/d6d/f9c to d0/d50/dc4/dcf/f10c 0 2024-04-26T04:42:25.351 INFO:tasks.workunit.client.1.smithi104.stdout:8/820: write d0/d33/da6/fb6 [615079,17949] 0 2024-04-26T04:42:25.353 INFO:tasks.workunit.client.0.smithi090.stdout:4/564: link d3/db/d12/d20/d24/d43/d72/f8a d3/db/d12/d20/fb3 0 2024-04-26T04:42:25.356 INFO:tasks.workunit.client.0.smithi090.stdout:7/672: rename d0/db/df/d4d/d64/d66/ced to d0/d1/da3/cf7 0 2024-04-26T04:42:25.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/565: mknod d3/db/cb4 0 2024-04-26T04:42:25.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/566: chown d3/d13/f61 2763340 1 2024-04-26T04:42:25.358 INFO:tasks.workunit.client.0.smithi090.stdout:4/567: chown d3/db/d12/d20/d2d/la8 15525446 1 2024-04-26T04:42:25.360 INFO:tasks.workunit.client.0.smithi090.stdout:7/673: symlink d0/db/df/d1c/d23/lf8 0 2024-04-26T04:42:25.362 INFO:tasks.workunit.client.0.smithi090.stdout:4/568: rename d3/l18 to d3/db/d60/d5c/lb5 0 2024-04-26T04:42:25.363 INFO:tasks.workunit.client.0.smithi090.stdout:4/569: write d3/f6d [218357,95577] 0 2024-04-26T04:42:25.363 INFO:tasks.workunit.client.0.smithi090.stdout:4/570: readlink d3/db/d60/lb1 0 2024-04-26T04:42:25.363 INFO:tasks.workunit.client.0.smithi090.stdout:4/571: creat d3/db/d12/d20/d2d/d58/d6a/d74/fb6 x:0 0 0 2024-04-26T04:42:25.364 INFO:tasks.workunit.client.0.smithi090.stdout:2/591: dwrite db/f10 [4194304,4194304] 0 2024-04-26T04:42:25.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/572: rmdir d3/db/d12/d20/d2d/d58/d6a/d9c 0 2024-04-26T04:42:25.368 INFO:tasks.workunit.client.0.smithi090.stdout:2/592: mknod db/d59/d66/cc5 0 2024-04-26T04:42:25.368 INFO:tasks.workunit.client.0.smithi090.stdout:4/573: truncate d3/db/d12/d20/d2d/d58/d6a/f8b 1205605 0 2024-04-26T04:42:25.369 INFO:tasks.workunit.client.0.smithi090.stdout:4/574: stat d3/db/d12/d20/l44 0 2024-04-26T04:42:25.369 INFO:tasks.workunit.client.0.smithi090.stdout:4/575: creat d3/db/d12/d20/d45/fb7 x:0 0 0 2024-04-26T04:42:25.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/593: creat db/d11/d7f/fc6 x:0 0 0 2024-04-26T04:42:25.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/594: truncate db/d11/d14/d27/d2f/d70/d71/f9c 676888 0 2024-04-26T04:42:25.369 INFO:tasks.workunit.client.0.smithi090.stdout:2/595: dread - db/d11/d14/d1b/d20/d4c/fae zero size 2024-04-26T04:42:25.378 INFO:tasks.workunit.client.1.smithi104.stdout:5/940: symlink d0/d1/de6/d128/l13d 0 2024-04-26T04:42:25.382 INFO:tasks.workunit.client.0.smithi090.stdout:4/576: write d3/db/d12/d20/d2d/d58/d6a/d74/da2/f5a [976292,5997] 0 2024-04-26T04:42:25.384 INFO:tasks.workunit.client.1.smithi104.stdout:7/888: link d2/d4/d1d/d34/d7f/l10f d2/d4/d32/d35/dbe/l11a 0 2024-04-26T04:42:25.385 INFO:tasks.workunit.client.1.smithi104.stdout:6/957: mknod d0/d9/dc6/c13a 0 2024-04-26T04:42:25.385 INFO:tasks.workunit.client.1.smithi104.stdout:6/958: read - d0/d9/d25/d10c/f116 zero size 2024-04-26T04:42:25.387 INFO:tasks.workunit.client.0.smithi090.stdout:8/700: dwrite dc/dd2/f17 [0,4194304] 0 2024-04-26T04:42:25.387 INFO:tasks.workunit.client.1.smithi104.stdout:8/821: dwrite d0/d33/d35/d6d/def/ff6 [0,4194304] 0 2024-04-26T04:42:25.390 INFO:tasks.workunit.client.1.smithi104.stdout:5/941: dread d0/d1/d4/f63 [4194304,4194304] 0 2024-04-26T04:42:25.390 INFO:tasks.workunit.client.1.smithi104.stdout:5/942: write d0/d1/d64/fde [579742,93939] 0 2024-04-26T04:42:25.390 INFO:tasks.workunit.client.1.smithi104.stdout:5/943: fdatasync d0/d1/d4/d2a/d2b/d37/f57 0 2024-04-26T04:42:25.398 INFO:tasks.workunit.client.0.smithi090.stdout:8/701: link dc/d1f/ld3 dc/d1f/lf8 0 2024-04-26T04:42:25.398 INFO:tasks.workunit.client.0.smithi090.stdout:8/702: read dc/ff [2885219,78427] 0 2024-04-26T04:42:25.399 INFO:tasks.workunit.client.0.smithi090.stdout:8/703: mknod dc/dd2/d4f/da6/dd8/cf9 0 2024-04-26T04:42:25.399 INFO:tasks.workunit.client.0.smithi090.stdout:8/704: getdents dc/dd2/d4f/da6/db1/ddd 0 2024-04-26T04:42:25.400 INFO:tasks.workunit.client.0.smithi090.stdout:8/705: write dc/dd2/d39/d55/d64/f3e [6338137,121312] 0 2024-04-26T04:42:25.400 INFO:tasks.workunit.client.0.smithi090.stdout:8/706: write dc/d71/fae [9539,73993] 0 2024-04-26T04:42:25.400 INFO:tasks.workunit.client.0.smithi090.stdout:8/707: truncate dc/d1f/f21 2173097 0 2024-04-26T04:42:25.400 INFO:tasks.workunit.client.1.smithi104.stdout:7/889: symlink d2/d4/d32/d35/dbe/l11b 0 2024-04-26T04:42:25.402 INFO:tasks.workunit.client.0.smithi090.stdout:8/708: link dc/dd2/d4f/da6/dd8/fc5 dc/dd2/d3b/d3d/d97/dad/ffa 0 2024-04-26T04:42:25.403 INFO:tasks.workunit.client.1.smithi104.stdout:6/959: mkdir d0/d9/d13/d41/d82/d13b 0 2024-04-26T04:42:25.405 INFO:tasks.workunit.client.1.smithi104.stdout:8/822: unlink d0/d1c/d2d/c10b 0 2024-04-26T04:42:25.405 INFO:tasks.workunit.client.1.smithi104.stdout:7/890: creat d2/d4/f11c x:0 0 0 2024-04-26T04:42:25.407 INFO:tasks.workunit.client.1.smithi104.stdout:6/960: symlink d0/d9/dc6/l13c 0 2024-04-26T04:42:25.408 INFO:tasks.workunit.client.1.smithi104.stdout:8/823: unlink d0/l10 0 2024-04-26T04:42:25.409 INFO:tasks.workunit.client.0.smithi090.stdout:4/577: dwrite d3/db/d12/d20/d2d/d58/d6a/d74/da2/f5a [4194304,4194304] 0 2024-04-26T04:42:25.409 INFO:tasks.workunit.client.1.smithi104.stdout:6/961: mkdir d0/d9/d13/d41/d96/ded/d10d/d71/d3f/d13d 0 2024-04-26T04:42:25.409 INFO:tasks.workunit.client.1.smithi104.stdout:6/962: readlink d0/l76 0 2024-04-26T04:42:25.409 INFO:tasks.workunit.client.1.smithi104.stdout:6/963: chown d0/d9/d13/dcd/dd9/df5/c130 1525018 1 2024-04-26T04:42:25.410 INFO:tasks.workunit.client.1.smithi104.stdout:6/964: fdatasync d0/d9/d13/d41/d96/dac/dec/dbd/f112 0 2024-04-26T04:42:25.410 INFO:tasks.workunit.client.0.smithi090.stdout:4/578: rmdir d3/d13/d6c/d7e 39 2024-04-26T04:42:25.410 INFO:tasks.workunit.client.1.smithi104.stdout:6/965: symlink d0/d9/d13/d41/d96/ded/d10d/d71/da5/l13e 0 2024-04-26T04:42:25.411 INFO:tasks.workunit.client.1.smithi104.stdout:3/990: rmdir d5/db2 39 2024-04-26T04:42:25.412 INFO:tasks.workunit.client.1.smithi104.stdout:3/991: symlink d5/d13a/l154 0 2024-04-26T04:42:25.413 INFO:tasks.workunit.client.1.smithi104.stdout:6/966: dread d0/d9/d13/d41/d82/dc8/f84 [0,4194304] 0 2024-04-26T04:42:25.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/551: sync 2024-04-26T04:42:25.413 INFO:tasks.workunit.client.0.smithi090.stdout:9/469: sync 2024-04-26T04:42:25.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/552: dread - d0/d55/f63 zero size 2024-04-26T04:42:25.413 INFO:tasks.workunit.client.0.smithi090.stdout:1/553: chown d0/f2 83761382 1 2024-04-26T04:42:25.415 INFO:tasks.workunit.client.0.smithi090.stdout:9/470: symlink db/dd/d18/d95/la6 0 2024-04-26T04:42:25.415 INFO:tasks.workunit.client.0.smithi090.stdout:9/471: write db/dd/d44/f5e [1128461,29147] 0 2024-04-26T04:42:25.419 INFO:tasks.workunit.client.0.smithi090.stdout:8/709: read dc/dd2/d4f/f7f [2567555,81284] 0 2024-04-26T04:42:25.420 INFO:tasks.workunit.client.0.smithi090.stdout:8/710: truncate dc/d5a/fc8 959827 0 2024-04-26T04:42:25.420 INFO:tasks.workunit.client.0.smithi090.stdout:8/711: fsync dc/dd2/d4f/d58/fa5 0 2024-04-26T04:42:25.420 INFO:tasks.workunit.client.0.smithi090.stdout:9/472: mknod db/d5f/ca7 0 2024-04-26T04:42:25.420 INFO:tasks.workunit.client.1.smithi104.stdout:8/824: dread d0/d50/ffe [0,4194304] 0 2024-04-26T04:42:25.420 INFO:tasks.workunit.client.1.smithi104.stdout:8/825: creat d0/d33/da6/de0/f10d x:0 0 0 2024-04-26T04:42:25.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/554: mkdir d0/d4/d7/d4a/da8 0 2024-04-26T04:42:25.422 INFO:tasks.workunit.client.1.smithi104.stdout:8/826: mkdir d0/d1c/d27/d46/d10e 0 2024-04-26T04:42:25.424 INFO:tasks.workunit.client.0.smithi090.stdout:8/712: rename dc/dd2/d4f/da6/db1/lf7 to dc/dd2/d4f/da6/lfb 0 2024-04-26T04:42:25.425 INFO:tasks.workunit.client.0.smithi090.stdout:9/473: mkdir db/da3/da8 0 2024-04-26T04:42:25.428 INFO:tasks.workunit.client.0.smithi090.stdout:1/555: rmdir d0/d4/d7/d4a/d75/d80/d58/d7c 0 2024-04-26T04:42:25.428 INFO:tasks.workunit.client.0.smithi090.stdout:1/556: creat d0/d4/d7/d49/fa9 x:0 0 0 2024-04-26T04:42:25.429 INFO:tasks.workunit.client.0.smithi090.stdout:9/474: link db/dd/d44/f8e db/d16/fa9 0 2024-04-26T04:42:25.430 INFO:tasks.workunit.client.0.smithi090.stdout:9/475: chown db/dd/d18/d31/d6a/d70/l75 1005 1 2024-04-26T04:42:25.432 INFO:tasks.workunit.client.0.smithi090.stdout:1/557: rename d0/c1f to d0/d4/d43/caa 0 2024-04-26T04:42:25.432 INFO:tasks.workunit.client.0.smithi090.stdout:1/558: stat d0/f2 0 2024-04-26T04:42:25.433 INFO:tasks.workunit.client.0.smithi090.stdout:9/476: symlink db/laa 0 2024-04-26T04:42:25.434 INFO:tasks.workunit.client.0.smithi090.stdout:1/559: symlink d0/d4/d7/d17/lab 0 2024-04-26T04:42:25.435 INFO:tasks.workunit.client.0.smithi090.stdout:1/560: symlink d0/d4/d7/d4a/da8/lac 0 2024-04-26T04:42:25.523 INFO:tasks.workunit.client.1.smithi104.stdout:6/967: write d0/d9/d13/d41/d82/dc8/d80/fee [823777,20058] 0 2024-04-26T04:42:25.527 INFO:tasks.workunit.client.1.smithi104.stdout:6/968: symlink d0/d9/d13/d41/d96/ded/d10d/d47/l13f 0 2024-04-26T04:42:25.527 INFO:tasks.workunit.client.1.smithi104.stdout:6/969: write d0/d9/d13/d41/d96/dac/f11b [321092,111684] 0 2024-04-26T04:42:25.530 INFO:tasks.workunit.client.1.smithi104.stdout:6/970: rename d0/d9/d13/d41/d82/dc8/l81 to d0/d9/d13/d41/d96/l140 0 2024-04-26T04:42:25.532 INFO:tasks.workunit.client.1.smithi104.stdout:6/971: rename d0/d9/d13/d41/d96/ded/d10d/d71/d3f/l48 to d0/d9/d13/d41/d96/dac/l141 0 2024-04-26T04:42:25.534 INFO:tasks.workunit.client.0.smithi090.stdout:0/689: dwrite d0/d4b/f52 [0,4194304] 0 2024-04-26T04:42:25.534 INFO:tasks.workunit.client.1.smithi104.stdout:8/827: dread d0/d1c/d2d/f8c [0,4194304] 0 2024-04-26T04:42:25.534 INFO:tasks.workunit.client.1.smithi104.stdout:8/828: readlink d0/d33/l60 0 2024-04-26T04:42:25.535 INFO:tasks.workunit.client.1.smithi104.stdout:8/829: creat d0/d33/d35/d6d/da4/f10f x:0 0 0 2024-04-26T04:42:25.538 INFO:tasks.workunit.client.0.smithi090.stdout:0/690: rename d0/df/cc0 to d0/d4b/d5d/d48/d7d/cec 0 2024-04-26T04:42:25.552 INFO:tasks.workunit.client.0.smithi090.stdout:1/561: dread d0/f86 [0,4194304] 0 2024-04-26T04:42:25.552 INFO:tasks.workunit.client.0.smithi090.stdout:1/562: stat d0/d4/f9 0 2024-04-26T04:42:25.552 INFO:tasks.workunit.client.0.smithi090.stdout:0/691: dread d0/d4b/d6d/fc6 [0,4194304] 0 2024-04-26T04:42:25.553 INFO:tasks.workunit.client.0.smithi090.stdout:0/692: read - d0/d4b/d5d/d5c/f91 zero size 2024-04-26T04:42:25.553 INFO:tasks.workunit.client.0.smithi090.stdout:1/563: creat d0/d4/d7/d4a/da8/fad x:0 0 0 2024-04-26T04:42:25.553 INFO:tasks.workunit.client.0.smithi090.stdout:1/564: mknod d0/d4/d7/d17/d22/d31/cae 0 2024-04-26T04:42:25.554 INFO:tasks.workunit.client.0.smithi090.stdout:1/565: creat d0/d4/d7/d4a/faf x:0 0 0 2024-04-26T04:42:25.554 INFO:tasks.workunit.client.0.smithi090.stdout:2/596: dwrite db/d11/d14/d27/d2f/f32 [0,4194304] 0 2024-04-26T04:42:25.554 INFO:tasks.workunit.client.0.smithi090.stdout:2/597: write db/d11/d14/d27/d28/fa5 [5164264,6985] 0 2024-04-26T04:42:25.555 INFO:tasks.workunit.client.1.smithi104.stdout:3/992: dwrite d5/db2/fe2 [0,4194304] 0 2024-04-26T04:42:25.557 INFO:tasks.workunit.client.1.smithi104.stdout:3/993: getdents d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d 0 2024-04-26T04:42:25.561 INFO:tasks.workunit.client.1.smithi104.stdout:5/944: dwrite d0/f4e [0,4194304] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:4/579: dwrite d3/db/d12/d20/d45/fb7 [0,4194304] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:4/580: truncate d3/d13/d1d/f54 245916 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:9/477: dwrite db/dd/d18/d31/f3c [0,4194304] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/713: dwrite dc/dd2/d4f/d58/d95/f83 [0,4194304] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:9/478: write db/dd/d18/d38/f90 [545345,80787] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/714: write dc/dd2/d3b/f8b [295765,130579] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/715: write dc/dd2/d3b/fbf [1248459,77285] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/716: write dc/dd2/d4f/f96 [579418,27409] 0 2024-04-26T04:42:25.562 INFO:tasks.workunit.client.0.smithi090.stdout:8/717: truncate dc/d34/d4d/f4e 1074184 0 2024-04-26T04:42:25.563 INFO:tasks.workunit.client.1.smithi104.stdout:7/891: dwrite d2/d4/d1d/fa8 [0,4194304] 0 2024-04-26T04:42:25.563 INFO:tasks.workunit.client.1.smithi104.stdout:7/892: read - d2/d4/d32/d3d/d41/d5f/d6c/d89/fd0 zero size 2024-04-26T04:42:25.565 INFO:tasks.workunit.client.0.smithi090.stdout:1/566: dread d0/d4/d7/d17/d22/d31/f39 [0,4194304] 0 2024-04-26T04:42:25.565 INFO:tasks.workunit.client.0.smithi090.stdout:1/567: read - d0/d4/d7/d4a/f9d zero size 2024-04-26T04:42:25.570 INFO:tasks.workunit.client.1.smithi104.stdout:7/893: mkdir d2/d4/d32/d35/d44/d5b/db2/d11d 0 2024-04-26T04:42:25.570 INFO:tasks.workunit.client.1.smithi104.stdout:7/894: fdatasync d2/d4/d32/d3d/d41/d7e/de0/d117/f118 0 2024-04-26T04:42:25.571 INFO:tasks.workunit.client.1.smithi104.stdout:3/994: write d5/d9/fa [7190467,8760] 0 2024-04-26T04:42:25.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/581: unlink d3/db/d12/d20/d2d/d58/d6a/f8d 0 2024-04-26T04:42:25.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/582: chown d3/db/d12/d3b/f78 13 1 2024-04-26T04:42:25.574 INFO:tasks.workunit.client.0.smithi090.stdout:2/598: write db/d11/d14/d1b/d20/f24 [3149009,58219] 0 2024-04-26T04:42:25.575 INFO:tasks.workunit.client.1.smithi104.stdout:3/995: mknod d5/db2/da3/dd0/ddb/d32/d48/c155 0 2024-04-26T04:42:25.575 INFO:tasks.workunit.client.1.smithi104.stdout:3/996: write d5/db2/d60/faf [1189809,123689] 0 2024-04-26T04:42:25.575 INFO:tasks.workunit.client.1.smithi104.stdout:3/997: truncate d5/f23 1845976 0 2024-04-26T04:42:25.575 INFO:tasks.workunit.client.1.smithi104.stdout:3/998: dread - d5/db2/da3/dd0/ddb/d32/d48/dac/dd7/d7d/d8e/d13c/f140 zero size 2024-04-26T04:42:25.576 INFO:tasks.workunit.client.1.smithi104.stdout:3/999: chown d5/db2/da3/dd0/ddb/d32/c4c 16824451 1 2024-04-26T04:42:25.576 INFO:tasks.workunit.client.0.smithi090.stdout:8/718: mkdir dc/dd2/d4f/da6/dfc 0 2024-04-26T04:42:25.576 INFO:tasks.workunit.client.0.smithi090.stdout:8/719: stat dc/d34 0 2024-04-26T04:42:25.577 INFO:tasks.workunit.client.0.smithi090.stdout:1/568: rmdir d0/d4/d7/d4a/d75/d80/d58 39 2024-04-26T04:42:25.578 INFO:tasks.workunit.client.1.smithi104.stdout:8/830: dwrite d0/d1c/d2d/f32 [0,4194304] 0 2024-04-26T04:42:25.578 INFO:tasks.workunit.client.1.smithi104.stdout:8/831: write d0/f4d [4154698,100201] 0 2024-04-26T04:42:25.580 INFO:tasks.workunit.client.0.smithi090.stdout:8/720: unlink dc/d1f/f21 0 2024-04-26T04:42:25.581 INFO:tasks.workunit.client.0.smithi090.stdout:1/569: mkdir d0/d4/d7/d17/d22/d93/db0 0 2024-04-26T04:42:25.582 INFO:tasks.workunit.client.0.smithi090.stdout:1/570: stat d0/l10 0 2024-04-26T04:42:25.583 INFO:tasks.workunit.client.0.smithi090.stdout:8/721: link dc/dd2/d3b/la9 dc/dd2/d39/d55/lfd 0 2024-04-26T04:42:25.585 INFO:tasks.workunit.client.0.smithi090.stdout:1/571: symlink d0/d4/d7/d4a/d75/d80/d58/d97/lb1 0 2024-04-26T04:42:25.586 INFO:tasks.workunit.client.1.smithi104.stdout:6/972: dwrite d0/d9/d13/d41/d96/dac/dc4/fcf [0,4194304] 0 2024-04-26T04:42:25.586 INFO:tasks.workunit.client.1.smithi104.stdout:6/973: write d0/dd8/de0/d102/fde [5036857,122029] 0 2024-04-26T04:42:25.586 INFO:tasks.workunit.client.1.smithi104.stdout:5/945: dwrite d0/f93 [4194304,4194304] 0 2024-04-26T04:42:25.588 INFO:tasks.workunit.client.0.smithi090.stdout:4/583: read d3/db/d12/d20/d24/d43/d72/f80 [3364852,57112] 0 2024-04-26T04:42:25.588 INFO:tasks.workunit.client.1.smithi104.stdout:6/974: mkdir d0/d9/d13/d41/d96/ded/d10d/d71/d142 0 2024-04-26T04:42:25.588 INFO:tasks.workunit.client.1.smithi104.stdout:6/975: write d0/d9/d13/d41/f7a [824356,47574] 0 2024-04-26T04:42:25.589 INFO:tasks.workunit.client.0.smithi090.stdout:8/722: dread dc/dd2/d3b/d3d/d97/fac [0,4194304] 0 2024-04-26T04:42:25.589 INFO:tasks.workunit.client.0.smithi090.stdout:4/584: truncate d3/d13/d6c/f90 5987575 0 2024-04-26T04:42:25.589 INFO:tasks.workunit.client.1.smithi104.stdout:5/946: mknod d0/d1/d64/d26/ddc/c13e 0 2024-04-26T04:42:25.590 INFO:tasks.workunit.client.0.smithi090.stdout:8/723: rename dc/d71/lb2 to dc/dd2/d4f/db7/lfe 0 2024-04-26T04:42:25.592 INFO:tasks.workunit.client.1.smithi104.stdout:6/976: dread d0/d9/d13/d41/d82/dc8/fd6 [0,4194304] 0 2024-04-26T04:42:25.596 INFO:tasks.workunit.client.0.smithi090.stdout:4/585: dread d3/db/d12/f4e [0,4194304] 0 2024-04-26T04:42:25.596 INFO:tasks.workunit.client.0.smithi090.stdout:4/586: chown d3/d13/f87 10832638 1 2024-04-26T04:42:25.596 INFO:tasks.workunit.client.0.smithi090.stdout:4/587: write d3/db/d12/f4e [433966,62982] 0 2024-04-26T04:42:25.597 INFO:tasks.workunit.client.0.smithi090.stdout:4/588: mknod d3/db/d12/d20/d2d/d58/d6a/d74/d85/cb8 0 2024-04-26T04:42:25.599 INFO:tasks.workunit.client.0.smithi090.stdout:4/589: rename d3/db/d12/d20/fb3 to d3/db/d12/d20/d24/fb9 0 2024-04-26T04:42:25.601 INFO:tasks.workunit.client.0.smithi090.stdout:4/590: mkdir d3/d13/d6c/dba 0 2024-04-26T04:42:25.602 INFO:tasks.workunit.client.0.smithi090.stdout:9/479: dwrite db/dd/d18/d31/d6a/d70/f96 [0,4194304] 0 2024-04-26T04:42:25.602 INFO:tasks.workunit.client.0.smithi090.stdout:8/724: read dc/dd2/d4f/f96 [131130,67184] 0 2024-04-26T04:42:25.604 INFO:tasks.workunit.client.1.smithi104.stdout:6/977: creat d0/d9/d13/d41/d82/dc8/dba/daa/f143 x:0 0 0 2024-04-26T04:42:25.606 INFO:tasks.workunit.client.0.smithi090.stdout:4/591: unlink d3/d13/d1d/d50/d51/f97 0 2024-04-26T04:42:25.606 INFO:tasks.workunit.client.0.smithi090.stdout:2/599: dwrite db/d11/d14/d25/d41/fba [0,4194304] 0 2024-04-26T04:42:25.606 INFO:tasks.workunit.client.0.smithi090.stdout:2/600: truncate db/d11/d14/d27/d28/f61 546400 0 2024-04-26T04:42:25.606 INFO:tasks.workunit.client.0.smithi090.stdout:2/601: fsync db/dc/f19 0 2024-04-26T04:42:25.606 INFO:tasks.workunit.client.1.smithi104.stdout:5/947: dread d0/d1/d1e/df5/f13c [0,4194304] 0 2024-04-26T04:42:25.607 INFO:tasks.workunit.client.1.smithi104.stdout:8/832: read d0/d1c/d27/d47/d67/d76/d7f/ffc [4133859,87088] 0 2024-04-26T04:42:25.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/592: dread d3/db/d12/d20/d2d/d2e/f77 [0,4194304] 0 2024-04-26T04:42:25.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/593: truncate d3/db/d12/f5b 892263 0 2024-04-26T04:42:25.609 INFO:tasks.workunit.client.0.smithi090.stdout:4/594: fdatasync d3/db/d12/d20/d2d/d58/d6a/d74/da2/f49 0 2024-04-26T04:42:25.610 INFO:tasks.workunit.client.0.smithi090.stdout:9/480: symlink db/dd/d18/lab 0 2024-04-26T04:42:25.610 INFO:tasks.workunit.client.1.smithi104.stdout:6/978: creat d0/d9/d13/d41/d96/dac/dec/f144 x:0 0 0 2024-04-26T04:42:25.610 INFO:tasks.workunit.client.1.smithi104.stdout:0/856: sync 2024-04-26T04:42:25.610 INFO:tasks.workunit.client.1.smithi104.stdout:0/857: write d0/d2/d21/d27/d1e/d57/d6c/d85/fa2 [1112369,86786] 0 2024-04-26T04:42:25.610 INFO:tasks.workunit.client.1.smithi104.stdout:0/858: dread - d0/d2/d21/f118 zero size 2024-04-26T04:42:25.610 INFO:tasks.workunit.client.1.smithi104.stdout:9/754: sync 2024-04-26T04:42:25.611 INFO:tasks.workunit.client.1.smithi104.stdout:8/833: unlink d0/d1c/d2d/d7a/l102 0 2024-04-26T04:42:25.611 INFO:tasks.workunit.client.1.smithi104.stdout:0/859: creat d0/d2/d21/d49/d5e/d9c/d53/f11b x:0 0 0 2024-04-26T04:42:25.611 INFO:tasks.workunit.client.1.smithi104.stdout:0/860: fsync d0/d4/f11 0 2024-04-26T04:42:25.612 INFO:tasks.workunit.client.0.smithi090.stdout:8/725: stat dc/dd2/d39/d55/d64/le7 0 2024-04-26T04:42:25.612 INFO:tasks.workunit.client.0.smithi090.stdout:8/726: creat dc/d34/fff x:0 0 0 2024-04-26T04:42:25.612 INFO:tasks.workunit.client.0.smithi090.stdout:8/727: creat dc/d5a/d5d/d73/dce/f100 x:0 0 0 2024-04-26T04:42:25.612 INFO:tasks.workunit.client.1.smithi104.stdout:9/755: creat d11/d15/d32/dd6/ff4 x:0 0 0 2024-04-26T04:42:25.612 INFO:tasks.workunit.client.1.smithi104.stdout:9/756: creat d11/d15/d98/ff5 x:0 0 0 2024-04-26T04:42:25.614 INFO:tasks.workunit.client.1.smithi104.stdout:8/834: rename d0/d1c/f6e to d0/d1c/d59/f110 0 2024-04-26T04:42:25.615 INFO:tasks.workunit.client.1.smithi104.stdout:9/757: dread d11/d15/f6a [0,4194304] 0 2024-04-26T04:42:25.617 INFO:tasks.workunit.client.1.smithi104.stdout:0/861: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dba/f11c x:0 0 0 2024-04-26T04:42:25.617 INFO:tasks.workunit.client.0.smithi090.stdout:9/481: dread db/dd/d18/f43 [0,4194304] 0 2024-04-26T04:42:25.619 INFO:tasks.workunit.client.1.smithi104.stdout:8/835: dread d0/d33/d35/d6d/f95 [0,4194304] 0 2024-04-26T04:42:25.619 INFO:tasks.workunit.client.1.smithi104.stdout:8/836: write d0/d50/fb4 [5121223,26275] 0 2024-04-26T04:42:25.619 INFO:tasks.workunit.client.1.smithi104.stdout:8/837: dread - d0/d33/d35/d6d/f105 zero size 2024-04-26T04:42:25.619 INFO:tasks.workunit.client.1.smithi104.stdout:8/838: read d0/d33/d51/f71 [54863,39391] 0 2024-04-26T04:42:25.620 INFO:tasks.workunit.client.1.smithi104.stdout:9/758: link d11/d15/d1b/d6d/d9c/le5 d11/d1d/d34/d69/lf6 0 2024-04-26T04:42:25.620 INFO:tasks.workunit.client.1.smithi104.stdout:9/759: stat d11/d90 0 2024-04-26T04:42:25.622 INFO:tasks.workunit.client.0.smithi090.stdout:1/572: dwrite d0/d4/d7/f2c [0,4194304] 0 2024-04-26T04:42:25.623 INFO:tasks.workunit.client.0.smithi090.stdout:1/573: creat d0/d4/fb2 x:0 0 0 2024-04-26T04:42:25.628 INFO:tasks.workunit.client.1.smithi104.stdout:9/760: symlink d11/d15/d32/dd6/ded/lf7 0 2024-04-26T04:42:25.629 INFO:tasks.workunit.client.1.smithi104.stdout:8/839: dread d0/d1c/d27/d47/d67/d76/daf/fb2 [0,4194304] 0 2024-04-26T04:42:25.629 INFO:tasks.workunit.client.1.smithi104.stdout:8/840: chown d0/l4 3 1 2024-04-26T04:42:25.629 INFO:tasks.workunit.client.1.smithi104.stdout:8/841: fsync d0/d33/da6/fb6 0 2024-04-26T04:42:25.629 INFO:tasks.workunit.client.1.smithi104.stdout:9/761: mknod d11/d15/d1b/d6d/d9c/cf8 0 2024-04-26T04:42:25.630 INFO:tasks.workunit.client.1.smithi104.stdout:8/842: rmdir d0/d50/dc4 39 2024-04-26T04:42:25.631 INFO:tasks.workunit.client.1.smithi104.stdout:9/762: symlink d11/d1d/lf9 0 2024-04-26T04:42:25.631 INFO:tasks.workunit.client.0.smithi090.stdout:4/595: unlink d3/d13/d64/dae/cb2 0 2024-04-26T04:42:25.631 INFO:tasks.workunit.client.0.smithi090.stdout:4/596: creat d3/d13/d6c/fbb x:0 0 0 2024-04-26T04:42:25.632 INFO:tasks.workunit.client.0.smithi090.stdout:4/597: chown d3/l1a 239529690 1 2024-04-26T04:42:25.632 INFO:tasks.workunit.client.0.smithi090.stdout:4/598: creat d3/db/d12/d20/d24/da9/fbc x:0 0 0 2024-04-26T04:42:25.632 INFO:tasks.workunit.client.0.smithi090.stdout:4/599: getdents d3/d13/d6c/dba 0 2024-04-26T04:42:25.632 INFO:tasks.workunit.client.1.smithi104.stdout:8/843: rename d0/d1c/d27/d47/d67/c97 to d0/d50/dc4/c111 0 2024-04-26T04:42:25.632 INFO:tasks.workunit.client.1.smithi104.stdout:8/844: truncate d0/f1e 2251981 0 2024-04-26T04:42:25.633 INFO:tasks.workunit.client.1.smithi104.stdout:9/763: link d11/d1d/d34/db3/fee d11/d53/d84/ffa 0 2024-04-26T04:42:25.633 INFO:tasks.workunit.client.1.smithi104.stdout:9/764: stat d11/d1d/d34/d69/f78 0 2024-04-26T04:42:25.634 INFO:tasks.workunit.client.1.smithi104.stdout:8/845: rmdir d0/d1c/d59 39 2024-04-26T04:42:25.634 INFO:tasks.workunit.client.1.smithi104.stdout:8/846: write d0/d1c/d27/d47/f1d [886180,86327] 0 2024-04-26T04:42:25.635 INFO:tasks.workunit.client.0.smithi090.stdout:1/574: write d0/d4/f9 [2858216,64937] 0 2024-04-26T04:42:25.635 INFO:tasks.workunit.client.0.smithi090.stdout:1/575: chown d0/d4/d7/d4b/f5b 884 1 2024-04-26T04:42:25.635 INFO:tasks.workunit.client.1.smithi104.stdout:9/765: mkdir d11/d15/d1b/d6d/d9c/dfb 0 2024-04-26T04:42:25.636 INFO:tasks.workunit.client.0.smithi090.stdout:8/728: rename dc/dd2/d39/d55/d64/l45 to dc/dd2/l101 0 2024-04-26T04:42:25.637 INFO:tasks.workunit.client.1.smithi104.stdout:9/766: unlink d11/d1d/d34/f6c 0 2024-04-26T04:42:25.637 INFO:tasks.workunit.client.1.smithi104.stdout:9/767: readlink d11/d15/d1b/l2a 0 2024-04-26T04:42:25.637 INFO:tasks.workunit.client.1.smithi104.stdout:9/768: write d11/d1d/d34/d69/fa7 [602439,104045] 0 2024-04-26T04:42:25.641 INFO:tasks.workunit.client.1.smithi104.stdout:0/862: dwrite d0/d2/d10/d3d/db7/fd2 [0,4194304] 0 2024-04-26T04:42:25.641 INFO:tasks.workunit.client.1.smithi104.stdout:6/979: dwrite d0/d9/da8/fea [0,4194304] 0 2024-04-26T04:42:25.642 INFO:tasks.workunit.client.1.smithi104.stdout:6/980: dread d0/d9/d13/d41/d82/dc8/d80/fc2 [0,4194304] 0 2024-04-26T04:42:25.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/482: dwrite db/dd/d18/d38/f79 [0,4194304] 0 2024-04-26T04:42:25.646 INFO:tasks.workunit.client.1.smithi104.stdout:0/863: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/ded/d11d 0 2024-04-26T04:42:25.649 INFO:tasks.workunit.client.0.smithi090.stdout:4/600: stat d3/le 0 2024-04-26T04:42:25.652 INFO:tasks.workunit.client.0.smithi090.stdout:8/729: truncate dc/dd2/d4f/d58/d95/fa7 139897 0 2024-04-26T04:42:25.652 INFO:tasks.workunit.client.0.smithi090.stdout:8/730: truncate dc/d34/d4d/fc4 460437 0 2024-04-26T04:42:25.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/483: mknod db/cac 0 2024-04-26T04:42:25.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/484: readlink db/dd/d18/d95/la6 0 2024-04-26T04:42:25.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/485: readlink db/l69 0 2024-04-26T04:42:25.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/601: mknod d3/d13/cbd 0 2024-04-26T04:42:25.655 INFO:tasks.workunit.client.1.smithi104.stdout:0/864: dread d0/d4/f29 [0,4194304] 0 2024-04-26T04:42:25.658 INFO:tasks.workunit.client.0.smithi090.stdout:1/576: symlink d0/d4/d7/d17/d22/lb3 0 2024-04-26T04:42:25.658 INFO:tasks.workunit.client.0.smithi090.stdout:9/486: creat db/dd/d18/d31/d6a/d70/d9d/fad x:0 0 0 2024-04-26T04:42:25.659 INFO:tasks.workunit.client.0.smithi090.stdout:9/487: fdatasync db/dd/d44/f8e 0 2024-04-26T04:42:25.660 INFO:tasks.workunit.client.0.smithi090.stdout:1/577: symlink d0/d4/d7/d4b/d53/lb4 0 2024-04-26T04:42:25.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/574: sync 2024-04-26T04:42:25.661 INFO:tasks.workunit.client.0.smithi090.stdout:5/575: write d4/d1a/d1f/d53/d59/d5e/f6a [301861,109487] 0 2024-04-26T04:42:25.661 INFO:tasks.workunit.client.0.smithi090.stdout:5/576: chown d4/d1a/d1f/d53 952 1 2024-04-26T04:42:25.664 INFO:tasks.workunit.client.0.smithi090.stdout:1/578: link d0/d4/d43/c7e d0/cb5 0 2024-04-26T04:42:25.664 INFO:tasks.workunit.client.0.smithi090.stdout:1/579: read - d0/d4/d7/d4a/f9d zero size 2024-04-26T04:42:25.664 INFO:tasks.workunit.client.0.smithi090.stdout:1/580: write d0/d4/d62/d78/f82 [173837,41736] 0 2024-04-26T04:42:25.665 INFO:tasks.workunit.client.0.smithi090.stdout:5/577: mknod d4/d1a/d2a/d79/d90/cb7 0 2024-04-26T04:42:25.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/578: rename d4/d1a/d1f/d53/d59/d95 to d4/d1a/d45/db8 0 2024-04-26T04:42:25.668 INFO:tasks.workunit.client.0.smithi090.stdout:5/579: write d4/d1a/df/f8d [3091792,71172] 0 2024-04-26T04:42:25.669 INFO:tasks.workunit.client.1.smithi104.stdout:8/847: dwrite d0/d1c/d27/f86 [0,4194304] 0 2024-04-26T04:42:25.670 INFO:tasks.workunit.client.0.smithi090.stdout:1/581: dread d0/d4/d62/d78/f82 [0,4194304] 0 2024-04-26T04:42:25.671 INFO:tasks.workunit.client.0.smithi090.stdout:3/557: sync 2024-04-26T04:42:25.673 INFO:tasks.workunit.client.1.smithi104.stdout:8/848: unlink d0/d9a/caa 0 2024-04-26T04:42:25.675 INFO:tasks.workunit.client.1.smithi104.stdout:8/849: mkdir d0/d1c/d2d/d7a/d112 0 2024-04-26T04:42:25.676 INFO:tasks.workunit.client.1.smithi104.stdout:8/850: mkdir d0/d33/d35/d113 0 2024-04-26T04:42:25.683 INFO:tasks.workunit.client.0.smithi090.stdout:5/580: write d4/d1a/d3f/f65 [1740016,123648] 0 2024-04-26T04:42:25.685 INFO:tasks.workunit.client.0.smithi090.stdout:5/581: symlink d4/d1a/d2a/d79/d90/d56/d80/lb9 0 2024-04-26T04:42:25.687 INFO:tasks.workunit.client.0.smithi090.stdout:5/582: creat d4/d1a/fba x:0 0 0 2024-04-26T04:42:25.689 INFO:tasks.workunit.client.1.smithi104.stdout:9/769: dwrite d11/d15/d32/d35/d7b/fde [0,4194304] 0 2024-04-26T04:42:25.689 INFO:tasks.workunit.client.0.smithi090.stdout:3/558: read d1/d17/d36/d33/d43/db0/f14 [2584194,95333] 0 2024-04-26T04:42:25.689 INFO:tasks.workunit.client.0.smithi090.stdout:3/559: write d1/d9/fb [9267700,58890] 0 2024-04-26T04:42:25.690 INFO:tasks.workunit.client.0.smithi090.stdout:3/560: mknod d1/d17/d36/d33/d43/cbd 0 2024-04-26T04:42:25.691 INFO:tasks.workunit.client.0.smithi090.stdout:3/561: creat d1/d17/d36/d2a/d3a/fbe x:0 0 0 2024-04-26T04:42:25.695 INFO:tasks.workunit.client.0.smithi090.stdout:1/582: read d0/d4/d7/d17/d22/f52 [837963,128326] 0 2024-04-26T04:42:25.696 INFO:tasks.workunit.client.1.smithi104.stdout:9/770: dread d11/d15/d32/f56 [0,4194304] 0 2024-04-26T04:42:25.697 INFO:tasks.workunit.client.0.smithi090.stdout:1/583: mknod d0/d4/d7/d4a/d75/d80/d58/cb6 0 2024-04-26T04:42:25.698 INFO:tasks.workunit.client.0.smithi090.stdout:1/584: creat d0/d4/d7/d4a/d75/d80/d58/d97/fb7 x:0 0 0 2024-04-26T04:42:25.698 INFO:tasks.workunit.client.0.smithi090.stdout:1/585: truncate d0/d55/f83 1039077 0 2024-04-26T04:42:25.699 INFO:tasks.workunit.client.0.smithi090.stdout:5/583: dread d4/d1a/d2a/d79/d90/d56/f6f [0,4194304] 0 2024-04-26T04:42:25.699 INFO:tasks.workunit.client.0.smithi090.stdout:5/584: fdatasync d4/d1a/d45/db8/f9e 0 2024-04-26T04:42:25.700 INFO:tasks.workunit.client.1.smithi104.stdout:9/771: rename d11/d1d/d34/l42 to d11/d1d/d34/lfc 0 2024-04-26T04:42:25.701 INFO:tasks.workunit.client.0.smithi090.stdout:5/585: symlink d4/d1a/df/d36/d8e/lbb 0 2024-04-26T04:42:25.701 INFO:tasks.workunit.client.0.smithi090.stdout:5/586: chown d4/d1a/d3f/l40 28 1 2024-04-26T04:42:25.701 INFO:tasks.workunit.client.1.smithi104.stdout:6/981: dwrite d0/d9/d13/d41/d96/ded/d10d/f9a [0,4194304] 0 2024-04-26T04:42:25.701 INFO:tasks.workunit.client.1.smithi104.stdout:6/982: write d0/d9/d13/d41/d82/dc8/dba/daa/feb [43247,100828] 0 2024-04-26T04:42:25.701 INFO:tasks.workunit.client.1.smithi104.stdout:9/772: truncate d11/d15/d32/d4c/d54/f61 658693 0 2024-04-26T04:42:25.704 INFO:tasks.workunit.client.0.smithi090.stdout:8/731: dwrite dc/d1f/fcf [4194304,4194304] 0 2024-04-26T04:42:25.706 INFO:tasks.workunit.client.1.smithi104.stdout:9/773: mkdir d11/d90/dfd 0 2024-04-26T04:42:25.706 INFO:tasks.workunit.client.0.smithi090.stdout:4/602: dwrite d3/db/d12/d20/d24/d43/f91 [0,4194304] 0 2024-04-26T04:42:25.706 INFO:tasks.workunit.client.0.smithi090.stdout:9/488: dwrite db/dd/d18/d31/f4a [0,4194304] 0 2024-04-26T04:42:25.707 INFO:tasks.workunit.client.1.smithi104.stdout:0/865: dwrite d0/d2/d21/d27/f83 [0,4194304] 0 2024-04-26T04:42:25.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/732: symlink dc/dd2/d4f/d58/d95/l102 0 2024-04-26T04:42:25.709 INFO:tasks.workunit.client.1.smithi104.stdout:0/866: mkdir d0/d2/d21/d27/d1e/d57/d6c/d85/df4/d11e 0 2024-04-26T04:42:25.709 INFO:tasks.workunit.client.1.smithi104.stdout:9/774: dread d11/d15/d32/d35/d7b/f82 [0,4194304] 0 2024-04-26T04:42:25.710 INFO:tasks.workunit.client.0.smithi090.stdout:4/603: mkdir d3/db/d12/d20/d2d/dbe 0 2024-04-26T04:42:25.711 INFO:tasks.workunit.client.1.smithi104.stdout:0/867: mkdir d0/d2/d21/d49/d5e/d9c/db2/d11f 0 2024-04-26T04:42:25.712 INFO:tasks.workunit.client.1.smithi104.stdout:0/868: truncate d0/d2/d21/fb1 987167 0 2024-04-26T04:42:25.712 INFO:tasks.workunit.client.1.smithi104.stdout:9/775: getdents d11/d15/d1b/d6d 0 2024-04-26T04:42:25.713 INFO:tasks.workunit.client.0.smithi090.stdout:9/489: mknod db/cae 0 2024-04-26T04:42:25.714 INFO:tasks.workunit.client.1.smithi104.stdout:9/776: creat d11/d15/d32/dd6/ded/ffe x:0 0 0 2024-04-26T04:42:25.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/733: creat dc/dd2/d4f/da6/dd8/f103 x:0 0 0 2024-04-26T04:42:25.715 INFO:tasks.workunit.client.0.smithi090.stdout:8/734: chown dc/dd2/d3b/cbc 15829740 1 2024-04-26T04:42:25.715 INFO:tasks.workunit.client.1.smithi104.stdout:9/777: getdents d11/d15/d32/d4c 0 2024-04-26T04:42:25.717 INFO:tasks.workunit.client.1.smithi104.stdout:9/778: creat d11/d15/d32/d4c/d54/ddc/fff x:0 0 0 2024-04-26T04:42:25.717 INFO:tasks.workunit.client.1.smithi104.stdout:9/779: readlink d11/d15/da3/lad 0 2024-04-26T04:42:25.718 INFO:tasks.workunit.client.0.smithi090.stdout:8/735: rename dc/dd2/d39/d55/d64/c8c to dc/dd2/d4f/da6/dd0/c104 0 2024-04-26T04:42:25.719 INFO:tasks.workunit.client.1.smithi104.stdout:8/851: dwrite d0/f2e [0,4194304] 0 2024-04-26T04:42:25.719 INFO:tasks.workunit.client.1.smithi104.stdout:8/852: chown d0/d33/f1f 7464 1 2024-04-26T04:42:25.719 INFO:tasks.workunit.client.1.smithi104.stdout:9/780: symlink d11/d15/d32/d4c/d54/l100 0 2024-04-26T04:42:25.719 INFO:tasks.workunit.client.1.smithi104.stdout:9/781: fsync d11/d15/da3/fd1 0 2024-04-26T04:42:25.720 INFO:tasks.workunit.client.0.smithi090.stdout:8/736: truncate dc/d5a/d5d/f66 2829271 0 2024-04-26T04:42:25.722 INFO:tasks.workunit.client.0.smithi090.stdout:9/490: read db/dd/d18/d31/f4a [3720185,96276] 0 2024-04-26T04:42:25.725 INFO:tasks.workunit.client.1.smithi104.stdout:8/853: symlink d0/d33/d35/l114 0 2024-04-26T04:42:25.729 INFO:tasks.workunit.client.0.smithi090.stdout:8/737: dread dc/d1f/f27 [0,4194304] 0 2024-04-26T04:42:25.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/604: write d3/db/d12/d20/d45/f96 [2367233,69872] 0 2024-04-26T04:42:25.731 INFO:tasks.workunit.client.0.smithi090.stdout:4/605: fdatasync d3/f9 0 2024-04-26T04:42:25.731 INFO:tasks.workunit.client.0.smithi090.stdout:8/738: getdents dc/dd2/d39/d99 0 2024-04-26T04:42:25.732 INFO:tasks.workunit.client.1.smithi104.stdout:8/854: dread d0/d1c/d27/d47/d67/d76/d7f/ffc [0,4194304] 0 2024-04-26T04:42:25.733 INFO:tasks.workunit.client.0.smithi090.stdout:4/606: truncate d3/db/d12/f10 2938507 0 2024-04-26T04:42:25.734 INFO:tasks.workunit.client.1.smithi104.stdout:8/855: symlink d0/d33/d35/l115 0 2024-04-26T04:42:25.734 INFO:tasks.workunit.client.1.smithi104.stdout:8/856: creat d0/d1c/d27/d47/d67/d76/d7f/f116 x:0 0 0 2024-04-26T04:42:25.735 INFO:tasks.workunit.client.0.smithi090.stdout:8/739: truncate dc/d5a/f6a 940453 0 2024-04-26T04:42:25.735 INFO:tasks.workunit.client.0.smithi090.stdout:8/740: creat dc/d71/f105 x:0 0 0 2024-04-26T04:42:25.737 INFO:tasks.workunit.client.1.smithi104.stdout:8/857: creat d0/d33/d35/d6d/def/df7/f117 x:0 0 0 2024-04-26T04:42:25.737 INFO:tasks.workunit.client.0.smithi090.stdout:8/741: mkdir dc/dd2/d4f/d58/d95/d106 0 2024-04-26T04:42:25.737 INFO:tasks.workunit.client.0.smithi090.stdout:8/742: chown dc/dd2/d4f/d58/d95/f83 43520 1 2024-04-26T04:42:25.738 INFO:tasks.workunit.client.1.smithi104.stdout:8/858: read d0/d1c/d27/d47/d67/d76/d7f/fc2 [69481,121035] 0 2024-04-26T04:42:25.739 INFO:tasks.workunit.client.0.smithi090.stdout:8/743: creat dc/dd2/d4f/d58/d95/d106/f107 x:0 0 0 2024-04-26T04:42:25.739 INFO:tasks.workunit.client.0.smithi090.stdout:8/744: chown dc/dd2/d39/d55/d64/ff1 1 1 2024-04-26T04:42:25.739 INFO:tasks.workunit.client.1.smithi104.stdout:8/859: creat d0/d1c/d27/d47/d67/d76/daf/dd4/f118 x:0 0 0 2024-04-26T04:42:25.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/745: link dc/dd2/d3b/d3d/db8/ddc/ce1 dc/dd2/d4f/db7/c108 0 2024-04-26T04:42:25.741 INFO:tasks.workunit.client.0.smithi090.stdout:8/746: write dc/dd2/d4f/d58/d95/d106/f107 [821013,129302] 0 2024-04-26T04:42:25.741 INFO:tasks.workunit.client.1.smithi104.stdout:8/860: unlink d0/d33/d35/l115 0 2024-04-26T04:42:25.743 INFO:tasks.workunit.client.0.smithi090.stdout:8/747: unlink dc/d1f/f27 0 2024-04-26T04:42:25.750 INFO:tasks.workunit.client.0.smithi090.stdout:3/562: dwrite d1/d17/f1a [0,4194304] 0 2024-04-26T04:42:25.753 INFO:tasks.workunit.client.0.smithi090.stdout:3/563: creat d1/d17/d52/d7d/fbf x:0 0 0 2024-04-26T04:42:25.760 INFO:tasks.workunit.client.0.smithi090.stdout:1/586: dwrite d0/d4/d7/d49/fa9 [0,4194304] 0 2024-04-26T04:42:25.761 INFO:tasks.workunit.client.0.smithi090.stdout:3/564: write d1/d9/fa [1340488,63488] 0 2024-04-26T04:42:25.761 INFO:tasks.workunit.client.0.smithi090.stdout:5/587: dwrite d4/d1a/df/d4a/f7a [0,4194304] 0 2024-04-26T04:42:25.761 INFO:tasks.workunit.client.0.smithi090.stdout:3/565: chown d1/d17/d36/d33/d43/cbd 46542554 1 2024-04-26T04:42:25.761 INFO:tasks.workunit.client.0.smithi090.stdout:5/588: truncate d4/d1a/df/d36/f4d 5276567 0 2024-04-26T04:42:25.761 INFO:tasks.workunit.client.1.smithi104.stdout:0/869: dwrite d0/d2/d21/d27/fd4 [0,4194304] 0 2024-04-26T04:42:25.762 INFO:tasks.workunit.client.0.smithi090.stdout:9/491: dwrite db/f84 [4194304,4194304] 0 2024-04-26T04:42:25.765 INFO:tasks.workunit.client.1.smithi104.stdout:0/870: creat d0/d2/d21/d49/d5e/d9c/f120 x:0 0 0 2024-04-26T04:42:25.765 INFO:tasks.workunit.client.1.smithi104.stdout:0/871: write d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fe1 [747612,37896] 0 2024-04-26T04:42:25.767 INFO:tasks.workunit.client.0.smithi090.stdout:5/589: mknod d4/d1a/d1f/d53/d59/d5e/da0/cbc 0 2024-04-26T04:42:25.767 INFO:tasks.workunit.client.0.smithi090.stdout:5/590: fsync d4/d1a/d2a/d79/d90/f96 0 2024-04-26T04:42:25.768 INFO:tasks.workunit.client.1.smithi104.stdout:0/872: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d121 0 2024-04-26T04:42:25.768 INFO:tasks.workunit.client.1.smithi104.stdout:0/873: truncate d0/d2/d21/d27/d1e/d57/d6c/d85/fbd 5056198 0 2024-04-26T04:42:25.769 INFO:tasks.workunit.client.1.smithi104.stdout:0/874: fdatasync d0/f4d 0 2024-04-26T04:42:25.769 INFO:tasks.workunit.client.0.smithi090.stdout:4/607: dwrite d3/f27 [0,4194304] 0 2024-04-26T04:42:25.769 INFO:tasks.workunit.client.0.smithi090.stdout:8/748: dwrite dc/d1f/f36 [0,4194304] 0 2024-04-26T04:42:25.769 INFO:tasks.workunit.client.0.smithi090.stdout:8/749: readlink dc/d5a/lda 0 2024-04-26T04:42:25.769 INFO:tasks.workunit.client.0.smithi090.stdout:3/566: dread d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:25.770 INFO:tasks.workunit.client.1.smithi104.stdout:0/875: creat d0/d2/d21/d27/d1e/f122 x:0 0 0 2024-04-26T04:42:25.779 INFO:tasks.workunit.client.0.smithi090.stdout:5/591: dread d4/d1a/d2a/d79/d90/d56/f75 [0,4194304] 0 2024-04-26T04:42:25.783 INFO:tasks.workunit.client.0.smithi090.stdout:8/750: getdents dc/dd2/d3b/d3d/db8/ddc 0 2024-04-26T04:42:25.784 INFO:tasks.workunit.client.0.smithi090.stdout:3/567: mknod d1/d17/d36/d2a/d3a/d5d/d82/cc0 0 2024-04-26T04:42:25.786 INFO:tasks.workunit.client.0.smithi090.stdout:9/492: rmdir db/dd/d18/d38/d39 39 2024-04-26T04:42:25.787 INFO:tasks.workunit.client.0.smithi090.stdout:3/568: symlink d1/d17/d52/d7d/lc1 0 2024-04-26T04:42:25.788 INFO:tasks.workunit.client.0.smithi090.stdout:3/569: truncate d1/d17/d36/f24 1265211 0 2024-04-26T04:42:25.789 INFO:tasks.workunit.client.0.smithi090.stdout:3/570: rmdir d1/d17/d36/d33/d43/db0/d2c/d97 39 2024-04-26T04:42:25.789 INFO:tasks.workunit.client.0.smithi090.stdout:3/571: stat d1/d17/d36/d33/d68/dae/fb3 0 2024-04-26T04:42:25.791 INFO:tasks.workunit.client.0.smithi090.stdout:3/572: creat d1/d17/d36/dab/fc2 x:0 0 0 2024-04-26T04:42:25.791 INFO:tasks.workunit.client.0.smithi090.stdout:3/573: creat d1/d17/d36/d33/d42/fc3 x:0 0 0 2024-04-26T04:42:25.791 INFO:tasks.workunit.client.1.smithi104.stdout:0/876: dwrite d0/d2/d21/d49/d5e/d9c/d53/d5d/d6f/f8c [0,4194304] 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/751: write dc/dd2/d4f/f9f [2683747,21121] 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/752: fsync dc/dd2/d4f/d58/fa5 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/753: rename dc/dd2/d3b/d3d/ccb to dc/dd2/d3b/d3d/d97/dad/c109 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/754: write dc/dd2/d3b/fbf [2022382,52428] 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/755: write dc/d5a/d5d/f69 [1594464,32695] 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/756: dread dc/dd2/d39/f81 [0,4194304] 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/757: truncate dc/d34/fff 1013366 0 2024-04-26T04:42:25.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/758: chown dc/dd2/d4f/d58/d95/d106/f107 216608434 1 2024-04-26T04:42:25.802 INFO:tasks.workunit.client.1.smithi104.stdout:0/877: write d0/d2/fe5 [1298637,89851] 0 2024-04-26T04:42:25.802 INFO:tasks.workunit.client.0.smithi090.stdout:3/574: write d1/d17/f4f [638685,66771] 0 2024-04-26T04:42:25.802 INFO:tasks.workunit.client.0.smithi090.stdout:8/759: mknod dc/dd2/d4f/db7/c10a 0 2024-04-26T04:42:25.802 INFO:tasks.workunit.client.0.smithi090.stdout:8/760: dread - dc/dd2/d4f/da6/dd8/f103 zero size 2024-04-26T04:42:25.803 INFO:tasks.workunit.client.1.smithi104.stdout:0/878: truncate d0/d4/f29 754529 0 2024-04-26T04:42:25.805 INFO:tasks.workunit.client.0.smithi090.stdout:9/493: dwrite db/dd/d18/f3a [0,4194304] 0 2024-04-26T04:42:25.805 INFO:tasks.workunit.client.0.smithi090.stdout:9/494: truncate db/dd/d18/d31/d6a/f92 769239 0 2024-04-26T04:42:25.805 INFO:tasks.workunit.client.0.smithi090.stdout:9/495: fdatasync db/dd/d18/f43 0 2024-04-26T04:42:25.805 INFO:tasks.workunit.client.0.smithi090.stdout:9/496: chown db/d16/f33 8 1 2024-04-26T04:42:25.805 INFO:tasks.workunit.client.0.smithi090.stdout:3/575: symlink d1/d17/d36/d33/d43/db0/d2c/lc4 0 2024-04-26T04:42:25.806 INFO:tasks.workunit.client.1.smithi104.stdout:0/879: rename d0/d2/d21/d49/d5e/d9c/d53/d5d/f88 to d0/d2/d21/d49/d5e/d9c/db2/f123 0 2024-04-26T04:42:25.807 INFO:tasks.workunit.client.0.smithi090.stdout:9/497: creat db/dd/d44/d82/d85/faf x:0 0 0 2024-04-26T04:42:25.808 INFO:tasks.workunit.client.1.smithi104.stdout:0/880: mkdir d0/d2/d21/d27/d1e/d57/d6c/d85/df4/d124 0 2024-04-26T04:42:25.809 INFO:tasks.workunit.client.0.smithi090.stdout:9/498: mknod db/dd/d18/d31/d6a/cb0 0 2024-04-26T04:42:25.809 INFO:tasks.workunit.client.0.smithi090.stdout:9/499: chown db/dd/d44/f8e 3055298 1 2024-04-26T04:42:25.810 INFO:tasks.workunit.client.0.smithi090.stdout:9/500: truncate db/f3d 4021981 0 2024-04-26T04:42:25.813 INFO:tasks.workunit.client.1.smithi104.stdout:0/881: rmdir d0/d2/d21/d49/d5e/d9c/db2/d11f 0 2024-04-26T04:42:25.814 INFO:tasks.workunit.client.0.smithi090.stdout:9/501: symlink db/dd/d18/d31/d6a/d70/d9d/lb1 0 2024-04-26T04:42:25.814 INFO:tasks.workunit.client.1.smithi104.stdout:0/882: mkdir d0/d2/d10/d3d/dc0/df1/d125 0 2024-04-26T04:42:25.817 INFO:tasks.workunit.client.1.smithi104.stdout:0/883: getdents d0/d2/d21 0 2024-04-26T04:42:25.817 INFO:tasks.workunit.client.1.smithi104.stdout:0/884: rmdir d0/d2/d21/d27/d1e/d57/d6c 39 2024-04-26T04:42:25.819 INFO:tasks.workunit.client.0.smithi090.stdout:6/686: sync 2024-04-26T04:42:25.820 INFO:tasks.workunit.client.1.smithi104.stdout:0/885: write d0/d2/d21/d27/d1e/f2c [1592410,75153] 0 2024-04-26T04:42:25.821 INFO:tasks.workunit.client.1.smithi104.stdout:0/886: getdents d0/d2/d21/d27/d1e/d57/d6c/d113 0 2024-04-26T04:42:25.821 INFO:tasks.workunit.client.1.smithi104.stdout:0/887: symlink d0/d2/d21/d27/d1e/d57/d6c/dd1/l126 0 2024-04-26T04:42:25.822 INFO:tasks.workunit.client.1.smithi104.stdout:0/888: link d0/d2/d21/fb1 d0/d2/da0/f127 0 2024-04-26T04:42:25.824 INFO:tasks.workunit.client.0.smithi090.stdout:6/687: rmdir d9/d1c 39 2024-04-26T04:42:25.827 INFO:tasks.workunit.client.1.smithi104.stdout:0/889: dread d0/d2/d21/d27/d1e/d57/d6c/d85/fa2 [0,4194304] 0 2024-04-26T04:42:25.827 INFO:tasks.workunit.client.1.smithi104.stdout:0/890: fsync d0/f4d 0 2024-04-26T04:42:25.828 INFO:tasks.workunit.client.1.smithi104.stdout:0/891: rename d0/d2/d21/d49/d5e/d9c/d53/d5d/lb8 to d0/d2/d21/d49/d5e/d9c/db2/l128 0 2024-04-26T04:42:25.830 INFO:tasks.workunit.client.0.smithi090.stdout:3/576: dwrite d1/d5c/f99 [0,4194304] 0 2024-04-26T04:42:25.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/761: dwrite dc/d5a/ff5 [0,4194304] 0 2024-04-26T04:42:25.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/762: stat dc/dd2/d3b 0 2024-04-26T04:42:25.833 INFO:tasks.workunit.client.0.smithi090.stdout:3/577: mknod d1/d17/d36/d2a/d3a/d5d/cc5 0 2024-04-26T04:42:25.836 INFO:tasks.workunit.client.0.smithi090.stdout:6/688: dread d9/d1c/da2/faa [0,4194304] 0 2024-04-26T04:42:25.836 INFO:tasks.workunit.client.0.smithi090.stdout:6/689: getdents d9/d3c/db4/ddb 0 2024-04-26T04:42:25.838 INFO:tasks.workunit.client.1.smithi104.stdout:0/892: creat d0/d2/d10/f129 x:0 0 0 2024-04-26T04:42:25.850 INFO:tasks.workunit.client.0.smithi090.stdout:6/690: write d9/d21/d40/f4b [2906280,68912] 0 2024-04-26T04:42:25.851 INFO:tasks.workunit.client.0.smithi090.stdout:6/691: link d9/d1c/c2c d9/d3c/db4/cf1 0 2024-04-26T04:42:25.853 INFO:tasks.workunit.client.0.smithi090.stdout:6/692: dread d9/f11 [0,4194304] 0 2024-04-26T04:42:25.853 INFO:tasks.workunit.client.0.smithi090.stdout:6/693: write d9/f5c [535587,34092] 0 2024-04-26T04:42:25.855 INFO:tasks.workunit.client.0.smithi090.stdout:6/694: unlink d9/d77/d9a/dc8/fec 0 2024-04-26T04:42:25.855 INFO:tasks.workunit.client.0.smithi090.stdout:6/695: chown d9/d1c/f80 28 1 2024-04-26T04:42:25.855 INFO:tasks.workunit.client.0.smithi090.stdout:6/696: write d9/f1d [1590071,53150] 0 2024-04-26T04:42:25.856 INFO:tasks.workunit.client.0.smithi090.stdout:6/697: creat d9/d1c/d31/d69/ff2 x:0 0 0 2024-04-26T04:42:25.856 INFO:tasks.workunit.client.0.smithi090.stdout:8/763: dwrite dc/f1e [0,4194304] 0 2024-04-26T04:42:25.858 INFO:tasks.workunit.client.0.smithi090.stdout:8/764: creat dc/dd2/d4f/d58/d95/f10b x:0 0 0 2024-04-26T04:42:25.859 INFO:tasks.workunit.client.0.smithi090.stdout:9/502: dread db/d16/f9a [0,4194304] 0 2024-04-26T04:42:25.860 INFO:tasks.workunit.client.0.smithi090.stdout:9/503: fdatasync db/dd/ff 0 2024-04-26T04:42:25.861 INFO:tasks.workunit.client.0.smithi090.stdout:9/504: mkdir db/dd/d18/d31/d99/db2 0 2024-04-26T04:42:25.862 INFO:tasks.workunit.client.0.smithi090.stdout:3/578: dwrite d1/d59/d5e/f62 [0,4194304] 0 2024-04-26T04:42:25.865 INFO:tasks.workunit.client.0.smithi090.stdout:6/698: dread d9/d1c/d31/f95 [0,4194304] 0 2024-04-26T04:42:25.867 INFO:tasks.workunit.client.1.smithi104.stdout:0/893: dwrite d0/d2/d10/f64 [0,4194304] 0 2024-04-26T04:42:25.868 INFO:tasks.workunit.client.0.smithi090.stdout:3/579: rmdir d1/d17/d36/d33/d68/d71/d75 0 2024-04-26T04:42:25.868 INFO:tasks.workunit.client.0.smithi090.stdout:6/699: write d9/d1c/da2/faa [3048525,63700] 0 2024-04-26T04:42:25.869 INFO:tasks.workunit.client.1.smithi104.stdout:0/894: creat d0/d2/d21/d27/d1e/d57/d6c/dd1/d7c/f12a x:0 0 0 2024-04-26T04:42:25.870 INFO:tasks.workunit.client.0.smithi090.stdout:3/580: symlink d1/d17/d36/d33/daa/lc6 0 2024-04-26T04:42:25.872 INFO:tasks.workunit.client.0.smithi090.stdout:6/700: truncate d9/d77/db0/db5/dda/d6d/f93 3520661 0 2024-04-26T04:42:25.872 INFO:tasks.workunit.client.0.smithi090.stdout:6/701: chown d9/f5c 2112536 1 2024-04-26T04:42:25.872 INFO:tasks.workunit.client.0.smithi090.stdout:6/702: chown d9/f11 43715176 1 2024-04-26T04:42:25.873 INFO:tasks.workunit.client.0.smithi090.stdout:3/581: mkdir d1/d17/d36/d2a/dc7 0 2024-04-26T04:42:25.873 INFO:tasks.workunit.client.0.smithi090.stdout:3/582: readlink d1/d17/d36/d33/d43/db0/d1b/l4d 0 2024-04-26T04:42:25.882 INFO:tasks.workunit.client.1.smithi104.stdout:5/948: sync 2024-04-26T04:42:25.882 INFO:tasks.workunit.client.1.smithi104.stdout:7/895: sync 2024-04-26T04:42:25.882 INFO:tasks.workunit.client.1.smithi104.stdout:7/896: chown d2/d4/d32/d60/f82 15 1 2024-04-26T04:42:25.882 INFO:tasks.workunit.client.1.smithi104.stdout:5/949: truncate d0/d1/d64/d26/fe7 878923 0 2024-04-26T04:42:25.882 INFO:tasks.workunit.client.1.smithi104.stdout:5/950: truncate d0/d1/d64/d26/fe7 1892187 0 2024-04-26T04:42:25.883 INFO:tasks.workunit.client.0.smithi090.stdout:9/505: dwrite db/dd/d18/d38/f79 [0,4194304] 0 2024-04-26T04:42:25.884 INFO:tasks.workunit.client.1.smithi104.stdout:5/951: unlink d0/d1/d64/d26/l2c 0 2024-04-26T04:42:25.885 INFO:tasks.workunit.client.1.smithi104.stdout:7/897: mknod d2/d4/d32/d3d/d41/d5f/d6d/c11e 0 2024-04-26T04:42:25.886 INFO:tasks.workunit.client.1.smithi104.stdout:7/898: creat d2/d4/d32/d35/d44/f11f x:0 0 0 2024-04-26T04:42:25.886 INFO:tasks.workunit.client.1.smithi104.stdout:7/899: truncate d2/d4/d32/d3d/d47/db1/dfa/f119 34615 0 2024-04-26T04:42:25.890 INFO:tasks.workunit.client.0.smithi090.stdout:9/506: getdents db/dd/d44 0 2024-04-26T04:42:25.890 INFO:tasks.workunit.client.0.smithi090.stdout:9/507: fsync db/dd/d18/f3a 0 2024-04-26T04:42:25.892 INFO:tasks.workunit.client.0.smithi090.stdout:9/508: rename db/fa2 to db/dd/d18/d31/d99/db2/fb3 0 2024-04-26T04:42:25.898 INFO:tasks.workunit.client.1.smithi104.stdout:0/895: dwrite d0/d2/d21/d27/f7b [0,4194304] 0 2024-04-26T04:42:25.898 INFO:tasks.workunit.client.0.smithi090.stdout:9/509: dread db/f87 [0,4194304] 0 2024-04-26T04:42:25.899 INFO:tasks.workunit.client.0.smithi090.stdout:9/510: rmdir db/dd/d18/d38/d39 39 2024-04-26T04:42:25.899 INFO:tasks.workunit.client.0.smithi090.stdout:9/511: write db/dd/d44/f56 [584303,116692] 0 2024-04-26T04:42:25.905 INFO:tasks.workunit.client.0.smithi090.stdout:3/583: dwrite d1/d17/f1f [0,4194304] 0 2024-04-26T04:42:25.905 INFO:tasks.workunit.client.0.smithi090.stdout:3/584: write d1/d17/d36/f40 [4692927,113025] 0 2024-04-26T04:42:25.905 INFO:tasks.workunit.client.0.smithi090.stdout:7/674: sync 2024-04-26T04:42:25.905 INFO:tasks.workunit.client.0.smithi090.stdout:0/693: sync 2024-04-26T04:42:25.909 INFO:tasks.workunit.client.0.smithi090.stdout:3/585: creat d1/d17/d36/d33/d43/db0/d1b/d58/da0/fc8 x:0 0 0 2024-04-26T04:42:25.909 INFO:tasks.workunit.client.0.smithi090.stdout:3/586: stat d1/d17/d36/d33/d42/d70 0 2024-04-26T04:42:25.910 INFO:tasks.workunit.client.0.smithi090.stdout:7/675: truncate d0/d1/d4/f2e 11522584 0 2024-04-26T04:42:25.911 INFO:tasks.workunit.client.1.smithi104.stdout:5/952: dwrite d0/d1/d4/d2a/d6b/d91/dbd/ffa [0,4194304] 0 2024-04-26T04:42:25.911 INFO:tasks.workunit.client.1.smithi104.stdout:5/953: chown d0/d1/d4/d2a/d2b/d37/d48/dc6 376590371 1 2024-04-26T04:42:25.912 INFO:tasks.workunit.client.1.smithi104.stdout:5/954: chown d0/d41/d4f/c112 1555 1 2024-04-26T04:42:25.912 INFO:tasks.workunit.client.1.smithi104.stdout:5/955: chown d0/d41/d4f/df2/d108/f11b 184759 1 2024-04-26T04:42:25.913 INFO:tasks.workunit.client.1.smithi104.stdout:5/956: symlink d0/d1/de6/l13f 0 2024-04-26T04:42:25.913 INFO:tasks.workunit.client.1.smithi104.stdout:5/957: fsync d0/d1/d4/d2a/d6b/f119 0 2024-04-26T04:42:25.913 INFO:tasks.workunit.client.1.smithi104.stdout:5/958: creat d0/d1/d64/d26/f140 x:0 0 0 2024-04-26T04:42:25.914 INFO:tasks.workunit.client.1.smithi104.stdout:5/959: write d0/d1/d4/d2a/d2b/d37/f7c [734662,3631] 0 2024-04-26T04:42:25.914 INFO:tasks.workunit.client.1.smithi104.stdout:5/960: chown d0/d1/f134 31 1 2024-04-26T04:42:25.920 INFO:tasks.workunit.client.0.smithi090.stdout:7/676: write d0/f45 [621688,37665] 0 2024-04-26T04:42:25.921 INFO:tasks.workunit.client.1.smithi104.stdout:7/900: dwrite d2/d4/d5a/f99 [0,4194304] 0 2024-04-26T04:42:25.921 INFO:tasks.workunit.client.1.smithi104.stdout:7/901: fsync d2/d4/d32/d35/dbe/fc2 0 2024-04-26T04:42:25.921 INFO:tasks.workunit.client.1.smithi104.stdout:7/902: readlink d2/d4/d1d/d34/lf9 0 2024-04-26T04:42:25.922 INFO:tasks.workunit.client.1.smithi104.stdout:7/903: fsync d2/d4/d32/d3d/d41/d7e/de0/d117/f118 0 2024-04-26T04:42:25.922 INFO:tasks.workunit.client.1.smithi104.stdout:7/904: write d2/d4/d32/d3d/d41/d5f/d6c/d89/fdc [782909,88037] 0 2024-04-26T04:42:25.922 INFO:tasks.workunit.client.1.smithi104.stdout:7/905: truncate d2/d4/d32/d35/d44/d5b/fd4 890300 0 2024-04-26T04:42:25.922 INFO:tasks.workunit.client.1.smithi104.stdout:7/906: write d2/d4/d32/d3d/d41/d5f/d6d/fe6 [143608,5187] 0 2024-04-26T04:42:25.922 INFO:tasks.workunit.client.0.smithi090.stdout:7/677: unlink d0/db/d53/db7/f32 0 2024-04-26T04:42:25.923 INFO:tasks.workunit.client.1.smithi104.stdout:0/896: dwrite d0/d2/f104 [0,4194304] 0 2024-04-26T04:42:25.923 INFO:tasks.workunit.client.1.smithi104.stdout:0/897: stat d0/d2/d21/d49/d5e/d9c/df7 0 2024-04-26T04:42:25.923 INFO:tasks.workunit.client.1.smithi104.stdout:7/907: symlink d2/d4/d32/d3d/d41/d7e/de0/l120 0 2024-04-26T04:42:25.924 INFO:tasks.workunit.client.0.smithi090.stdout:9/512: rename db/dd/d18/d31 to db/dd/d44/d82/d85/db4 0 2024-04-26T04:42:25.924 INFO:tasks.workunit.client.0.smithi090.stdout:9/513: chown db/c10 84540 1 2024-04-26T04:42:25.925 INFO:tasks.workunit.client.0.smithi090.stdout:3/587: rmdir d1/d17/d36/d33/d43/db0/d1b/d58/da0 39 2024-04-26T04:42:25.926 INFO:tasks.workunit.client.1.smithi104.stdout:5/961: dread d0/f4e [0,4194304] 0 2024-04-26T04:42:25.929 INFO:tasks.workunit.client.1.smithi104.stdout:0/898: creat d0/d2/d21/d27/d1e/d57/d6c/d85/d9a/f12b x:0 0 0 2024-04-26T04:42:25.929 INFO:tasks.workunit.client.0.smithi090.stdout:0/694: rename d0/d4b/d5d/d48/d7d/fc3 to d0/d4b/db3/fed 0 2024-04-26T04:42:25.932 INFO:tasks.workunit.client.0.smithi090.stdout:9/514: creat db/dd/d44/d82/d85/db4/d6a/fb5 x:0 0 0 2024-04-26T04:42:25.933 INFO:tasks.workunit.client.1.smithi104.stdout:7/908: dread d2/d4/d5a/f68 [0,4194304] 0 2024-04-26T04:42:25.934 INFO:tasks.workunit.client.0.smithi090.stdout:9/515: mkdir db/dd/d44/d82/d85/db4/d6a/db6 0 2024-04-26T04:42:25.934 INFO:tasks.workunit.client.1.smithi104.stdout:7/909: read d2/d4/d32/d35/d44/d5b/f7c [1135580,38020] 0 2024-04-26T04:42:25.934 INFO:tasks.workunit.client.1.smithi104.stdout:7/910: readlink d2/d4/d1d/l45 0 2024-04-26T04:42:25.934 INFO:tasks.workunit.client.1.smithi104.stdout:7/911: chown d2/d4/d5a/f104 7 1 2024-04-26T04:42:25.935 INFO:tasks.workunit.client.0.smithi090.stdout:9/516: rmdir db/dd/d44/d82 39 2024-04-26T04:42:25.935 INFO:tasks.workunit.client.1.smithi104.stdout:7/912: truncate d2/d4/d32/d3d/f106 2915874 0 2024-04-26T04:42:25.936 INFO:tasks.workunit.client.1.smithi104.stdout:0/899: dread d0/d2/fe5 [0,4194304] 0 2024-04-26T04:42:25.936 INFO:tasks.workunit.client.1.smithi104.stdout:0/900: write d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/fda [1954180,119168] 0 2024-04-26T04:42:25.936 INFO:tasks.workunit.client.1.smithi104.stdout:0/901: stat d0/d2/d21/d27/d1e/d57/d6c/dd1/d7c/cd7 0 2024-04-26T04:42:25.936 INFO:tasks.workunit.client.1.smithi104.stdout:0/902: write d0/d2/d21/d49/d5e/d9c/dbe/fbf [21261,108591] 0 2024-04-26T04:42:25.936 INFO:tasks.workunit.client.1.smithi104.stdout:0/903: write d0/d2/d21/d49/f7e [355453,65849] 0 2024-04-26T04:42:25.936 INFO:tasks.workunit.client.1.smithi104.stdout:0/904: read d0/d2/fe5 [3977179,1679] 0 2024-04-26T04:42:25.937 INFO:tasks.workunit.client.0.smithi090.stdout:9/517: symlink db/dd/d18/d95/lb7 0 2024-04-26T04:42:25.939 INFO:tasks.workunit.client.1.smithi104.stdout:0/905: link d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/lef d0/d2/d10/l12c 0 2024-04-26T04:42:25.939 INFO:tasks.workunit.client.1.smithi104.stdout:0/906: stat d0/d2/d21/d49/d5e/d9c/db2 0 2024-04-26T04:42:25.939 INFO:tasks.workunit.client.1.smithi104.stdout:0/907: write d0/d2/d21/d27/d1e/d57/d6c/dd1/fe2 [3161645,119274] 0 2024-04-26T04:42:25.941 INFO:tasks.workunit.client.1.smithi104.stdout:0/908: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d121/f12d x:0 0 0 2024-04-26T04:42:25.942 INFO:tasks.workunit.client.1.smithi104.stdout:7/913: rename d2/d4/d32/d3d/d41/d5f/d6c/d94 to d2/d4/d1d/d121 0 2024-04-26T04:42:25.944 INFO:tasks.workunit.client.0.smithi090.stdout:7/678: dwrite d0/db/d15/d29/fa2 [0,4194304] 0 2024-04-26T04:42:25.947 INFO:tasks.workunit.client.1.smithi104.stdout:5/962: dwrite d0/d1/d4/d90/d109/dfe/fc0 [0,4194304] 0 2024-04-26T04:42:25.951 INFO:tasks.workunit.client.0.smithi090.stdout:9/518: fsync db/dd/d44/d82/d85/db4/d6a/fb5 0 2024-04-26T04:42:25.951 INFO:tasks.workunit.client.0.smithi090.stdout:9/519: readlink db/dd/d44/l29 0 2024-04-26T04:42:25.953 INFO:tasks.workunit.client.1.smithi104.stdout:5/963: mknod d0/d1/d4/d2a/d6b/d116/d126/c141 0 2024-04-26T04:42:25.957 INFO:tasks.workunit.client.1.smithi104.stdout:5/964: creat d0/d1/d1e/df5/f142 x:0 0 0 2024-04-26T04:42:25.959 INFO:tasks.workunit.client.1.smithi104.stdout:5/965: write d0/d1/d64/d26/d36/da7/db3/fb4 [367708,4995] 0 2024-04-26T04:42:25.961 INFO:tasks.workunit.client.1.smithi104.stdout:5/966: symlink d0/d1/d4/d2a/d2b/d37/d48/dc1/dfd/l143 0 2024-04-26T04:42:25.961 INFO:tasks.workunit.client.1.smithi104.stdout:5/967: fdatasync d0/d1/d4/d2a/d2b/d37/d48/f70 0 2024-04-26T04:42:25.962 INFO:tasks.workunit.client.1.smithi104.stdout:5/968: getdents d0/d1/d64/d26 0 2024-04-26T04:42:25.963 INFO:tasks.workunit.client.1.smithi104.stdout:5/969: chown d0/d1/d64/d26/d36/da7/fee 681302169 1 2024-04-26T04:42:25.963 INFO:tasks.workunit.client.1.smithi104.stdout:5/970: mkdir d0/d1/d64/d26/d144 0 2024-04-26T04:42:25.963 INFO:tasks.workunit.client.1.smithi104.stdout:5/971: write d0/d1/d4/d90/d109/fea [768589,6343] 0 2024-04-26T04:42:25.965 INFO:tasks.workunit.client.1.smithi104.stdout:0/909: dwrite d0/d2/d10/d3d/db7/fd2 [0,4194304] 0 2024-04-26T04:42:25.965 INFO:tasks.workunit.client.1.smithi104.stdout:5/972: creat d0/d1/d64/d26/d36/da7/db3/f145 x:0 0 0 2024-04-26T04:42:25.968 INFO:tasks.workunit.client.1.smithi104.stdout:5/973: creat d0/d1/d4/d2a/d6b/d116/f146 x:0 0 0 2024-04-26T04:42:25.968 INFO:tasks.workunit.client.1.smithi104.stdout:5/974: mknod d0/d1/d4/d2a/d2b/c147 0 2024-04-26T04:42:25.970 INFO:tasks.workunit.client.1.smithi104.stdout:5/975: rename d0/d1/d1e/f30 to d0/d1/f148 0 2024-04-26T04:42:25.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:25 smithi090 ceph-mon[26648]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:25.973 INFO:tasks.workunit.client.1.smithi104.stdout:7/914: dwrite d2/d4/d1d/d34/d7f/faf [8388608,4194304] 0 2024-04-26T04:42:25.973 INFO:tasks.workunit.client.1.smithi104.stdout:7/915: creat d2/d4/d32/d3d/d41/d7e/de0/d117/f122 x:0 0 0 2024-04-26T04:42:25.973 INFO:tasks.workunit.client.1.smithi104.stdout:7/916: fsync d2/d4/d5a/da0/fab 0 2024-04-26T04:42:25.973 INFO:tasks.workunit.client.1.smithi104.stdout:7/917: fdatasync d2/d4/d32/d3d/d41/d7e/de0/d117/f118 0 2024-04-26T04:42:25.973 INFO:tasks.workunit.client.0.smithi090.stdout:9/520: dwrite db/dd/d44/d82/d85/db4/d99/db2/fb3 [0,4194304] 0 2024-04-26T04:42:25.977 INFO:tasks.workunit.client.0.smithi090.stdout:9/521: mknod db/dd/d18/d38/d39/cb8 0 2024-04-26T04:42:25.977 INFO:tasks.workunit.client.1.smithi104.stdout:5/976: dread d0/d1/d64/d26/d36/fcc [0,4194304] 0 2024-04-26T04:42:25.979 INFO:tasks.workunit.client.0.smithi090.stdout:9/522: creat db/dd/d18/d95/fb9 x:0 0 0 2024-04-26T04:42:25.981 INFO:tasks.workunit.client.0.smithi090.stdout:9/523: mkdir db/dd/d18/d38/d39/dba 0 2024-04-26T04:42:25.981 INFO:tasks.workunit.client.0.smithi090.stdout:9/524: creat db/dd/d18/d57/fbb x:0 0 0 2024-04-26T04:42:25.983 INFO:tasks.workunit.client.0.smithi090.stdout:9/525: symlink db/dd/d18/d57/lbc 0 2024-04-26T04:42:25.985 INFO:tasks.workunit.client.0.smithi090.stdout:9/526: dread db/dd/d44/f34 [0,4194304] 0 2024-04-26T04:42:25.985 INFO:tasks.workunit.client.0.smithi090.stdout:9/527: chown db/dd/f60 95979 1 2024-04-26T04:42:25.986 INFO:tasks.workunit.client.1.smithi104.stdout:7/918: dread d2/d4/d32/d3d/fb4 [0,4194304] 0 2024-04-26T04:42:25.986 INFO:tasks.workunit.client.0.smithi090.stdout:9/528: mkdir db/dd/d44/d82/dbd 0 2024-04-26T04:42:25.988 INFO:tasks.workunit.client.0.smithi090.stdout:9/529: write db/dd/d44/f5e [237913,125836] 0 2024-04-26T04:42:25.988 INFO:tasks.workunit.client.0.smithi090.stdout:9/530: write db/f1b [3418186,8794] 0 2024-04-26T04:42:25.989 INFO:tasks.workunit.client.0.smithi090.stdout:2/602: sync 2024-04-26T04:42:25.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/531: symlink db/dd/d44/d82/d85/db4/d6a/d70/d9d/lbe 0 2024-04-26T04:42:25.992 INFO:tasks.workunit.client.1.smithi104.stdout:5/977: symlink d0/d1/d4/d2a/d6b/d116/db5/l149 0 2024-04-26T04:42:25.992 INFO:tasks.workunit.client.1.smithi104.stdout:5/978: write d0/f6 [57442,3446] 0 2024-04-26T04:42:25.992 INFO:tasks.workunit.client.1.smithi104.stdout:0/910: dwrite d0/d2/d21/d27/d1e/d57/f8f [0,4194304] 0 2024-04-26T04:42:25.993 INFO:tasks.workunit.client.1.smithi104.stdout:0/911: dread - d0/d2/d21/d27/d1e/f122 zero size 2024-04-26T04:42:25.993 INFO:tasks.workunit.client.1.smithi104.stdout:0/912: stat d0/d2/d21/d27/d1e/d57/d6c/dd1/dcf 0 2024-04-26T04:42:25.993 INFO:tasks.workunit.client.1.smithi104.stdout:0/913: chown d0/d2/d21/d27/d1e/l117 743615936 1 2024-04-26T04:42:25.993 INFO:tasks.workunit.client.1.smithi104.stdout:0/914: chown d0/d2/d21/d49/ffb 1897697 1 2024-04-26T04:42:25.994 INFO:tasks.workunit.client.1.smithi104.stdout:5/979: rename d0/d1/d64/c6e to d0/d41/c14a 0 2024-04-26T04:42:25.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/603: dread db/d11/d14/d25/f49 [0,4194304] 0 2024-04-26T04:42:25.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/604: chown db/d11/d14 60 1 2024-04-26T04:42:26.000 INFO:tasks.workunit.client.1.smithi104.stdout:6/983: sync 2024-04-26T04:42:26.000 INFO:tasks.workunit.client.1.smithi104.stdout:9/782: sync 2024-04-26T04:42:26.000 INFO:tasks.workunit.client.1.smithi104.stdout:8/861: sync 2024-04-26T04:42:26.000 INFO:tasks.workunit.client.1.smithi104.stdout:8/862: chown d0/d1c/d27/d47/d67/d76/fbf 62662 1 2024-04-26T04:42:26.001 INFO:tasks.workunit.client.0.smithi090.stdout:2/605: link l6 db/d59/lc7 0 2024-04-26T04:42:26.001 INFO:tasks.workunit.client.1.smithi104.stdout:9/783: mknod d11/d53/d84/c101 0 2024-04-26T04:42:26.001 INFO:tasks.workunit.client.1.smithi104.stdout:8/863: mkdir d0/d1c/d27/d119 0 2024-04-26T04:42:26.002 INFO:tasks.workunit.client.1.smithi104.stdout:9/784: mkdir d11/d15/d1b/d6d/d9c/dfb/d102 0 2024-04-26T04:42:26.003 INFO:tasks.workunit.client.1.smithi104.stdout:0/915: dread d0/d2/d21/d27/f82 [0,4194304] 0 2024-04-26T04:42:26.004 INFO:tasks.workunit.client.1.smithi104.stdout:0/916: write d0/d2/d10/d84/fb0 [77769,76331] 0 2024-04-26T04:42:26.004 INFO:tasks.workunit.client.1.smithi104.stdout:8/864: rename d0/d1c/d2d/d7a/c8e to d0/d1c/d27/d119/c11a 0 2024-04-26T04:42:26.005 INFO:tasks.workunit.client.1.smithi104.stdout:0/917: mknod d0/d2/d10/d3d/db7/c12e 0 2024-04-26T04:42:26.005 INFO:tasks.workunit.client.1.smithi104.stdout:0/918: dread - d0/d2/d21/f118 zero size 2024-04-26T04:42:26.006 INFO:tasks.workunit.client.1.smithi104.stdout:0/919: rmdir d0/d2/d21/d27/d1e/d57/d6c/d85/df4 39 2024-04-26T04:42:26.008 INFO:tasks.workunit.client.1.smithi104.stdout:8/865: rmdir d0/d33/d35/d6d 39 2024-04-26T04:42:26.008 INFO:tasks.workunit.client.1.smithi104.stdout:8/866: dread - d0/d1c/d27/f90 zero size 2024-04-26T04:42:26.009 INFO:tasks.workunit.client.1.smithi104.stdout:0/920: dread d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fe1 [0,4194304] 0 2024-04-26T04:42:26.010 INFO:tasks.workunit.client.1.smithi104.stdout:8/867: creat d0/d1c/d2d/d7a/d112/f11b x:0 0 0 2024-04-26T04:42:26.010 INFO:tasks.workunit.client.1.smithi104.stdout:8/868: creat d0/d9a/dce/f11c x:0 0 0 2024-04-26T04:42:26.010 INFO:tasks.workunit.client.1.smithi104.stdout:0/921: getdents d0/d4/ddb 0 2024-04-26T04:42:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:25 smithi104 ceph-mon[32369]: from='client.14616 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:26.011 INFO:tasks.workunit.client.1.smithi104.stdout:6/984: dread d0/d9/d13/dcd/dd9/f10a [0,4194304] 0 2024-04-26T04:42:26.012 INFO:tasks.workunit.client.1.smithi104.stdout:0/922: link d0/d2/l9 d0/d2/d21/d27/d1e/d57/d6c/dd1/d7c/l12f 0 2024-04-26T04:42:26.012 INFO:tasks.workunit.client.1.smithi104.stdout:9/785: write d11/d15/d32/d35/f80 [3330451,84576] 0 2024-04-26T04:42:26.014 INFO:tasks.workunit.client.1.smithi104.stdout:9/786: unlink d11/d90/cdd 0 2024-04-26T04:42:26.014 INFO:tasks.workunit.client.1.smithi104.stdout:0/923: getdents d0/d2/d21/d27/d1e 0 2024-04-26T04:42:26.017 INFO:tasks.workunit.client.1.smithi104.stdout:8/869: dread d0/d1c/d27/f40 [0,4194304] 0 2024-04-26T04:42:26.019 INFO:tasks.workunit.client.1.smithi104.stdout:8/870: link d0/d33/d35/fc8 d0/d1c/d59/f11d 0 2024-04-26T04:42:26.019 INFO:tasks.workunit.client.1.smithi104.stdout:8/871: readlink d0/d33/d51/l88 0 2024-04-26T04:42:26.020 INFO:tasks.workunit.client.1.smithi104.stdout:0/924: dread d0/d2/d21/d49/d5e/d9c/db2/f123 [0,4194304] 0 2024-04-26T04:42:26.020 INFO:tasks.workunit.client.1.smithi104.stdout:8/872: mkdir d0/d1c/d27/d11e 0 2024-04-26T04:42:26.024 INFO:tasks.workunit.client.0.smithi090.stdout:2/606: write db/d11/d14/d25/f37 [1706692,15352] 0 2024-04-26T04:42:26.028 INFO:tasks.workunit.client.1.smithi104.stdout:7/919: dwrite d2/d4/d32/d3d/d47/f61 [0,4194304] 0 2024-04-26T04:42:26.028 INFO:tasks.workunit.client.1.smithi104.stdout:6/985: dread d0/d9/d13/d41/d96/ded/d10d/d71/f21 [0,4194304] 0 2024-04-26T04:42:26.029 INFO:tasks.workunit.client.1.smithi104.stdout:7/920: mknod d2/d4/d32/d3d/d47/db6/dce/c123 0 2024-04-26T04:42:26.029 INFO:tasks.workunit.client.1.smithi104.stdout:7/921: creat d2/d4/d1d/d34/f124 x:0 0 0 2024-04-26T04:42:26.030 INFO:tasks.workunit.client.1.smithi104.stdout:6/986: mkdir d0/d9/d13/d145 0 2024-04-26T04:42:26.032 INFO:tasks.workunit.client.1.smithi104.stdout:7/922: creat d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/dd8/f125 x:0 0 0 2024-04-26T04:42:26.033 INFO:tasks.workunit.client.0.smithi090.stdout:9/532: dwrite db/f41 [0,4194304] 0 2024-04-26T04:42:26.033 INFO:tasks.workunit.client.1.smithi104.stdout:6/987: readlink d0/d9/d13/d41/d96/ded/d10d/d47/l95 0 2024-04-26T04:42:26.033 INFO:tasks.workunit.client.1.smithi104.stdout:5/980: dwrite d0/d1/d4/d2a/d2b/d37/f111 [0,4194304] 0 2024-04-26T04:42:26.033 INFO:tasks.workunit.client.1.smithi104.stdout:5/981: readlink d0/d1/d1e/d98/dd4/l12c 0 2024-04-26T04:42:26.034 INFO:tasks.workunit.client.1.smithi104.stdout:7/923: mknod d2/d4/d1d/d121/d10c/c126 0 2024-04-26T04:42:26.035 INFO:tasks.workunit.client.0.smithi090.stdout:9/533: creat db/dd/d44/d82/d85/db4/d6a/d70/d9d/fbf x:0 0 0 2024-04-26T04:42:26.035 INFO:tasks.workunit.client.0.smithi090.stdout:9/534: write f9 [4960048,119180] 0 2024-04-26T04:42:26.037 INFO:tasks.workunit.client.1.smithi104.stdout:6/988: unlink d0/d9/f3b 0 2024-04-26T04:42:26.041 INFO:tasks.workunit.client.1.smithi104.stdout:7/924: truncate d2/d4/d1d/d34/f40 3403928 0 2024-04-26T04:42:26.041 INFO:tasks.workunit.client.0.smithi090.stdout:2/607: getdents db/d59 0 2024-04-26T04:42:26.041 INFO:tasks.workunit.client.0.smithi090.stdout:2/608: chown db/d11/d14/lc2 1878705079 1 2024-04-26T04:42:26.041 INFO:tasks.workunit.client.1.smithi104.stdout:7/925: fdatasync d2/d4/d32/d35/f4d 0 2024-04-26T04:42:26.042 INFO:tasks.workunit.client.1.smithi104.stdout:9/787: dwrite d11/d15/d1b/f37 [0,4194304] 0 2024-04-26T04:42:26.042 INFO:tasks.workunit.client.0.smithi090.stdout:2/609: getdents db/d11/d14/d27/d2f 0 2024-04-26T04:42:26.044 INFO:tasks.workunit.client.0.smithi090.stdout:2/610: link db/d11/d14/d27/d2f/d42/d5c/f4f db/d11/d14/d27/d2f/d42/fc8 0 2024-04-26T04:42:26.047 INFO:tasks.workunit.client.1.smithi104.stdout:9/788: dread d11/f89 [0,4194304] 0 2024-04-26T04:42:26.050 INFO:tasks.workunit.client.1.smithi104.stdout:7/926: rename d2/d4/d1d/d34/d7f to d2/d4/d32/d3d/d41/d7e/de0/d127 0 2024-04-26T04:42:26.050 INFO:tasks.workunit.client.1.smithi104.stdout:7/927: creat d2/d4/d32/d3d/d47/db1/dfa/f128 x:0 0 0 2024-04-26T04:42:26.059 INFO:tasks.workunit.client.0.smithi090.stdout:9/535: rename db/dd/d44/d82/d85/db4/d6a/d70/d9d to db/dd/d18/d38/d39/d6b/dc0 0 2024-04-26T04:42:26.059 INFO:tasks.workunit.client.0.smithi090.stdout:9/536: fsync db/dd/f8d 0 2024-04-26T04:42:26.059 INFO:tasks.workunit.client.0.smithi090.stdout:9/537: stat db/f1b 0 2024-04-26T04:42:26.059 INFO:tasks.workunit.client.0.smithi090.stdout:2/611: write db/d11/d14/d27/d28/f61 [84130,102983] 0 2024-04-26T04:42:26.062 INFO:tasks.workunit.client.0.smithi090.stdout:9/538: rename db/dd/d18/d57/l9f to db/dd/d44/d82/d85/db4/d6a/lc1 0 2024-04-26T04:42:26.065 INFO:tasks.workunit.client.0.smithi090.stdout:9/539: truncate db/d16/f9a 1893937 0 2024-04-26T04:42:26.065 INFO:tasks.workunit.client.0.smithi090.stdout:9/540: creat db/d16/fc2 x:0 0 0 2024-04-26T04:42:26.066 INFO:tasks.workunit.client.0.smithi090.stdout:9/541: dread - db/d16/fa9 zero size 2024-04-26T04:42:26.072 INFO:tasks.workunit.client.0.smithi090.stdout:2/612: dread db/dc/f8e [0,4194304] 0 2024-04-26T04:42:26.072 INFO:tasks.workunit.client.0.smithi090.stdout:2/613: write db/d11/d14/d1b/f51 [13699,50983] 0 2024-04-26T04:42:26.073 INFO:tasks.workunit.client.1.smithi104.stdout:6/989: dwrite d0/d9/d25/f39 [0,4194304] 0 2024-04-26T04:42:26.076 INFO:tasks.workunit.client.1.smithi104.stdout:0/925: dwrite d0/d2/d21/d27/f83 [0,4194304] 0 2024-04-26T04:42:26.076 INFO:tasks.workunit.client.1.smithi104.stdout:8/873: dwrite d0/d1c/d59/ffa [0,4194304] 0 2024-04-26T04:42:26.076 INFO:tasks.workunit.client.0.smithi090.stdout:2/614: unlink db/d11/d14/d27/d2f/d42/d5c/la8 0 2024-04-26T04:42:26.077 INFO:tasks.workunit.client.1.smithi104.stdout:5/982: dwrite d0/d1/d4/d2a/d6b/d73/d95/fba [0,4194304] 0 2024-04-26T04:42:26.079 INFO:tasks.workunit.client.0.smithi090.stdout:2/615: rename db/d11/cbb to db/d11/d7f/cc9 0 2024-04-26T04:42:26.079 INFO:tasks.workunit.client.1.smithi104.stdout:7/928: dwrite d2/d4/d32/fcf [0,4194304] 0 2024-04-26T04:42:26.080 INFO:tasks.workunit.client.0.smithi090.stdout:2/616: creat db/d11/d14/fca x:0 0 0 2024-04-26T04:42:26.086 INFO:tasks.workunit.client.0.smithi090.stdout:2/617: write db/dc/f8e [2538940,99257] 0 2024-04-26T04:42:26.088 INFO:tasks.workunit.client.0.smithi090.stdout:2/618: truncate db/d11/d14/d27/d2f/d70/d71/f83 3603207 0 2024-04-26T04:42:26.090 INFO:tasks.workunit.client.1.smithi104.stdout:5/983: write d0/d1/f65 [3510964,33942] 0 2024-04-26T04:42:26.090 INFO:tasks.workunit.client.1.smithi104.stdout:5/984: dread - d0/d1/d64/d26/d36/da7/fee zero size 2024-04-26T04:42:26.091 INFO:tasks.workunit.client.0.smithi090.stdout:2/619: rmdir db/d11/d14/d27/d2f/dbc 0 2024-04-26T04:42:26.091 INFO:tasks.workunit.client.0.smithi090.stdout:2/620: write db/d11/d7f/fa1 [1264889,95065] 0 2024-04-26T04:42:26.092 INFO:tasks.workunit.client.1.smithi104.stdout:9/789: symlink d11/d15/d32/dd6/l103 0 2024-04-26T04:42:26.093 INFO:tasks.workunit.client.0.smithi090.stdout:2/621: symlink db/d11/d14/d1b/d20/d4c/d98/lcb 0 2024-04-26T04:42:26.093 INFO:tasks.workunit.client.0.smithi090.stdout:2/622: stat db/d11/d14/d27/d2f/d70 0 2024-04-26T04:42:26.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/623: truncate db/d59/f78 183318 0 2024-04-26T04:42:26.095 INFO:tasks.workunit.client.0.smithi090.stdout:2/624: creat db/fcc x:0 0 0 2024-04-26T04:42:26.097 INFO:tasks.workunit.client.0.smithi090.stdout:9/542: dread db/f1b [0,4194304] 0 2024-04-26T04:42:26.097 INFO:tasks.workunit.client.1.smithi104.stdout:6/990: dwrite d0/d9/d25/f68 [0,4194304] 0 2024-04-26T04:42:26.097 INFO:tasks.workunit.client.1.smithi104.stdout:6/991: readlink d0/d9/d13/d41/d82/df7/l138 0 2024-04-26T04:42:26.104 INFO:tasks.workunit.client.0.smithi090.stdout:2/625: write db/d11/d14/d25/d86/fc3 [3407404,117663] 0 2024-04-26T04:42:26.105 INFO:tasks.workunit.client.0.smithi090.stdout:2/626: readlink db/d11/d14/d1b/d20/d4c/l75 0 2024-04-26T04:42:26.110 INFO:tasks.workunit.client.0.smithi090.stdout:2/627: read db/d11/d7f/fa1 [436332,39190] 0 2024-04-26T04:42:26.111 INFO:tasks.workunit.client.0.smithi090.stdout:2/628: symlink db/d59/d66/d77/lcd 0 2024-04-26T04:42:26.123 INFO:tasks.workunit.client.1.smithi104.stdout:6/992: dwrite d0/d9/d13/d41/d96/ded/d10d/d71/fc1 [0,4194304] 0 2024-04-26T04:42:26.123 INFO:tasks.workunit.client.0.smithi090.stdout:2/629: fsync db/d11/d7f/fa1 0 2024-04-26T04:42:26.123 INFO:tasks.workunit.client.0.smithi090.stdout:9/543: dwrite db/d16/f5a [0,4194304] 0 2024-04-26T04:42:26.123 INFO:tasks.workunit.client.0.smithi090.stdout:2/630: write db/d11/d14/d25/f9a [976821,91529] 0 2024-04-26T04:42:26.126 INFO:tasks.workunit.client.0.smithi090.stdout:2/631: symlink db/dc/lce 0 2024-04-26T04:42:26.153 INFO:tasks.workunit.client.0.smithi090.stdout:9/544: dwrite db/dd/d44/d82/d85/faf [0,4194304] 0 2024-04-26T04:42:26.154 INFO:tasks.workunit.client.0.smithi090.stdout:9/545: fsync db/dd/d18/d38/d39/d6b/dc0/fad 0 2024-04-26T04:42:26.154 INFO:tasks.workunit.client.0.smithi090.stdout:9/546: chown db/f35 62184 1 2024-04-26T04:42:26.154 INFO:tasks.workunit.client.0.smithi090.stdout:2/632: dwrite db/d11/d14/fc0 [0,4194304] 0 2024-04-26T04:42:26.162 INFO:tasks.workunit.client.0.smithi090.stdout:9/547: read db/dd/ff [1584976,16886] 0 2024-04-26T04:42:26.164 INFO:tasks.workunit.client.0.smithi090.stdout:9/548: creat db/dd/d44/d82/d85/db4/d99/fc3 x:0 0 0 2024-04-26T04:42:26.164 INFO:tasks.workunit.client.0.smithi090.stdout:9/549: chown db/dd/d18/lab 53846 1 2024-04-26T04:42:26.167 INFO:tasks.workunit.client.0.smithi090.stdout:1/587: sync 2024-04-26T04:42:26.167 INFO:tasks.workunit.client.0.smithi090.stdout:4/608: sync 2024-04-26T04:42:26.167 INFO:tasks.workunit.client.0.smithi090.stdout:1/588: stat d0/d4/d7/d49/c5d 0 2024-04-26T04:42:26.168 INFO:tasks.workunit.client.0.smithi090.stdout:1/589: mkdir d0/d4/d7/d17/d22/d93/db8 0 2024-04-26T04:42:26.169 INFO:tasks.workunit.client.0.smithi090.stdout:1/590: chown d0/d4/d7/d4a/c61 1797 1 2024-04-26T04:42:26.169 INFO:tasks.workunit.client.0.smithi090.stdout:1/591: creat d0/d55/fb9 x:0 0 0 2024-04-26T04:42:26.169 INFO:tasks.workunit.client.0.smithi090.stdout:1/592: readlink d0/d4/d7/d17/l8b 0 2024-04-26T04:42:26.169 INFO:tasks.workunit.client.0.smithi090.stdout:1/593: creat d0/d4/d7/d4a/da8/fba x:0 0 0 2024-04-26T04:42:26.169 INFO:tasks.workunit.client.0.smithi090.stdout:1/594: write d0/d4/d43/f9f [239570,4029] 0 2024-04-26T04:42:26.171 INFO:tasks.workunit.client.0.smithi090.stdout:2/633: dwrite db/d11/d14/d27/d2f/f8d [0,4194304] 0 2024-04-26T04:42:26.172 INFO:tasks.workunit.client.0.smithi090.stdout:1/595: dread d0/d4/d62/d78/f82 [0,4194304] 0 2024-04-26T04:42:26.174 INFO:tasks.workunit.client.0.smithi090.stdout:2/634: stat ca 0 2024-04-26T04:42:26.176 INFO:tasks.workunit.client.0.smithi090.stdout:1/596: getdents d0/d4/d43 0 2024-04-26T04:42:26.176 INFO:tasks.workunit.client.0.smithi090.stdout:1/597: stat d0/f86 0 2024-04-26T04:42:26.177 INFO:tasks.workunit.client.0.smithi090.stdout:2/635: getdents db/d11/d14/d25/d2d/d48/d81 0 2024-04-26T04:42:26.177 INFO:tasks.workunit.client.0.smithi090.stdout:2/636: chown db/c52 22 1 2024-04-26T04:42:26.179 INFO:tasks.workunit.client.0.smithi090.stdout:1/598: link d0/d4/d7/d17/d22/f74 d0/d4/d7/d17/d22/d93/fbb 0 2024-04-26T04:42:26.180 INFO:tasks.workunit.client.0.smithi090.stdout:2/637: symlink db/d59/d66/lcf 0 2024-04-26T04:42:26.182 INFO:tasks.workunit.client.0.smithi090.stdout:1/599: write d0/d4/d7/f33 [1709157,13118] 0 2024-04-26T04:42:26.184 INFO:tasks.workunit.client.0.smithi090.stdout:1/600: mknod d0/d4/d7/d4b/d53/d7b/cbc 0 2024-04-26T04:42:26.185 INFO:tasks.workunit.client.0.smithi090.stdout:1/601: unlink d0/d4/f9 0 2024-04-26T04:42:26.186 INFO:tasks.workunit.client.0.smithi090.stdout:1/602: mkdir d0/d55/dbd 0 2024-04-26T04:42:26.187 INFO:tasks.workunit.client.0.smithi090.stdout:1/603: symlink d0/d4/d62/lbe 0 2024-04-26T04:42:26.188 INFO:tasks.workunit.client.0.smithi090.stdout:1/604: getdents d0/d4/d62/d78 0 2024-04-26T04:42:26.188 INFO:tasks.workunit.client.0.smithi090.stdout:1/605: read - d0/d4/d7/d4a/d75/d80/f9e zero size 2024-04-26T04:42:26.189 INFO:tasks.workunit.client.0.smithi090.stdout:1/606: rmdir d0/d4/d7/d17/d22/d93 39 2024-04-26T04:42:26.191 INFO:tasks.workunit.client.0.smithi090.stdout:2/638: write db/d11/d14/d25/d2d/d48/d5b/f7d [2148977,120221] 0 2024-04-26T04:42:26.191 INFO:tasks.workunit.client.0.smithi090.stdout:2/639: stat db/d11/d14/d27/d2f/d42/d5c/f80 0 2024-04-26T04:42:26.193 INFO:tasks.workunit.client.0.smithi090.stdout:9/550: dwrite db/dd/f1d [0,4194304] 0 2024-04-26T04:42:26.194 INFO:tasks.workunit.client.0.smithi090.stdout:9/551: read db/dd/d44/f5e [119345,102329] 0 2024-04-26T04:42:26.194 INFO:tasks.workunit.client.0.smithi090.stdout:2/640: link db/d59/d66/cad db/d59/cd0 0 2024-04-26T04:42:26.194 INFO:tasks.workunit.client.0.smithi090.stdout:4/609: dwrite d3/db/d12/d20/d24/d43/d72/f9b [0,4194304] 0 2024-04-26T04:42:26.195 INFO:tasks.workunit.client.0.smithi090.stdout:2/641: mkdir db/d11/d14/d27/d28/dd1 0 2024-04-26T04:42:26.199 INFO:tasks.workunit.client.0.smithi090.stdout:4/610: creat d3/db/d12/d20/d2d/d58/fbf x:0 0 0 2024-04-26T04:42:26.199 INFO:tasks.workunit.client.0.smithi090.stdout:4/611: stat d3/d13/d1d/d50/d51 0 2024-04-26T04:42:26.200 INFO:tasks.workunit.client.0.smithi090.stdout:2/642: creat db/d11/d14/d1b/d20/da7/fd2 x:0 0 0 2024-04-26T04:42:26.201 INFO:tasks.workunit.client.0.smithi090.stdout:2/643: symlink db/d11/d14/d27/ld3 0 2024-04-26T04:42:26.201 INFO:tasks.workunit.client.0.smithi090.stdout:2/644: fsync db/d11/d14/d25/d41/fba 0 2024-04-26T04:42:26.206 INFO:tasks.workunit.client.1.smithi104.stdout:7/929: truncate d2/fe8 1899389 0 2024-04-26T04:42:26.213 INFO:tasks.workunit.client.0.smithi090.stdout:2/645: dread db/d59/d66/d69/fa6 [0,4194304] 0 2024-04-26T04:42:26.215 INFO:tasks.workunit.client.0.smithi090.stdout:2/646: symlink db/d11/d14/d27/ld4 0 2024-04-26T04:42:26.217 INFO:tasks.workunit.client.0.smithi090.stdout:2/647: rename db/d11/d14/d25/f37 to db/d11/d14/d63/fd5 0 2024-04-26T04:42:26.218 INFO:tasks.workunit.client.0.smithi090.stdout:4/612: dwrite d3/db/d12/f40 [0,4194304] 0 2024-04-26T04:42:26.218 INFO:tasks.workunit.client.0.smithi090.stdout:2/648: symlink db/d11/d14/d1b/d20/ld6 0 2024-04-26T04:42:26.220 INFO:tasks.workunit.client.0.smithi090.stdout:2/649: truncate db/d11/d14/d27/db0/faa 117582 0 2024-04-26T04:42:26.220 INFO:tasks.workunit.client.0.smithi090.stdout:2/650: read db/d11/d14/d1b/f51 [2824616,90567] 0 2024-04-26T04:42:26.221 INFO:tasks.workunit.client.0.smithi090.stdout:2/651: chown db/d11/d14/d1b/d20 17886707 1 2024-04-26T04:42:26.225 INFO:tasks.workunit.client.0.smithi090.stdout:4/613: dread d3/db/d12/d20/d24/f32 [0,4194304] 0 2024-04-26T04:42:26.228 INFO:tasks.workunit.client.1.smithi104.stdout:5/985: link d0/d1/d4/d90/d109/fbe d0/d1/d4/d2a/d6b/d116/f14b 0 2024-04-26T04:42:26.231 INFO:tasks.workunit.client.1.smithi104.stdout:9/790: mkdir d11/d1d/d34/d69/d104 0 2024-04-26T04:42:26.241 INFO:tasks.workunit.client.1.smithi104.stdout:0/926: rmdir d0/d2/d21/d49/d5e/d9c/d53 39 2024-04-26T04:42:26.243 INFO:tasks.workunit.client.0.smithi090.stdout:9/552: dwrite db/dd/d18/d38/d39/f7f [0,4194304] 0 2024-04-26T04:42:26.243 INFO:tasks.workunit.client.1.smithi104.stdout:7/930: creat d2/d4/d32/d60/dff/f129 x:0 0 0 2024-04-26T04:42:26.245 INFO:tasks.workunit.client.1.smithi104.stdout:5/986: link d0/d1/d4/d2a/d2b/d37/d48/ce0 d0/d1/d4/d2a/d2b/d37/d48/dc1/d113/c14c 0 2024-04-26T04:42:26.246 INFO:tasks.workunit.client.1.smithi104.stdout:9/791: getdents d11/d15/d1b/d6d/d9c/dbe 0 2024-04-26T04:42:26.248 INFO:tasks.workunit.client.1.smithi104.stdout:0/927: mkdir d0/d2/d21/d130 0 2024-04-26T04:42:26.249 INFO:tasks.workunit.client.0.smithi090.stdout:9/553: creat db/dd/d18/d38/d39/fc4 x:0 0 0 2024-04-26T04:42:26.250 INFO:tasks.workunit.client.0.smithi090.stdout:8/765: sync 2024-04-26T04:42:26.251 INFO:tasks.workunit.client.0.smithi090.stdout:5/592: sync 2024-04-26T04:42:26.251 INFO:tasks.workunit.client.0.smithi090.stdout:5/593: fdatasync d4/d1a/d45/fa6 0 2024-04-26T04:42:26.253 INFO:tasks.workunit.client.0.smithi090.stdout:5/594: rmdir d4/d1a/df/d36/d8e 39 2024-04-26T04:42:26.253 INFO:tasks.workunit.client.0.smithi090.stdout:5/595: chown d4/d1a/fba 1 1 2024-04-26T04:42:26.254 INFO:tasks.workunit.client.0.smithi090.stdout:4/614: dwrite d3/db/d12/f9a [0,4194304] 0 2024-04-26T04:42:26.254 INFO:tasks.workunit.client.1.smithi104.stdout:0/928: unlink d0/d2/d21/d27/d1e/d57/c10a 0 2024-04-26T04:42:26.254 INFO:tasks.workunit.client.1.smithi104.stdout:0/929: truncate d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dba/f11c 960618 0 2024-04-26T04:42:26.255 INFO:tasks.workunit.client.0.smithi090.stdout:2/652: dwrite db/d11/d14/fca [0,4194304] 0 2024-04-26T04:42:26.255 INFO:tasks.workunit.client.0.smithi090.stdout:9/554: dread db/dd/d18/f2a [0,4194304] 0 2024-04-26T04:42:26.256 INFO:tasks.workunit.client.0.smithi090.stdout:4/615: rename d3/db/d12/d20/d24/d43/f91 to d3/d13/d64/fc0 0 2024-04-26T04:42:26.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/596: dread d4/d1a/d3f/f52 [0,4194304] 0 2024-04-26T04:42:26.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/597: chown d4/d1a/d2a/d79/d90/d56/d80 57 1 2024-04-26T04:42:26.266 INFO:tasks.workunit.client.0.smithi090.stdout:2/653: mknod db/d11/d14/d25/d2d/d48/cd7 0 2024-04-26T04:42:26.267 INFO:tasks.workunit.client.0.smithi090.stdout:9/555: mknod db/da3/da8/cc5 0 2024-04-26T04:42:26.268 INFO:tasks.workunit.client.0.smithi090.stdout:4/616: unlink d3/db/d12/d20/d2d/d58/d6a/d74/fa4 0 2024-04-26T04:42:26.273 INFO:tasks.workunit.client.0.smithi090.stdout:4/617: dread d3/db/d12/d3b/f75 [0,4194304] 0 2024-04-26T04:42:26.273 INFO:tasks.workunit.client.0.smithi090.stdout:9/556: dread db/dd/d44/d82/d85/db4/d6a/f92 [0,4194304] 0 2024-04-26T04:42:26.273 INFO:tasks.workunit.client.0.smithi090.stdout:9/557: truncate db/dd/d18/d38/f79 4233623 0 2024-04-26T04:42:26.274 INFO:tasks.workunit.client.0.smithi090.stdout:9/558: dread db/dd/d18/d38/f9b [0,4194304] 0 2024-04-26T04:42:26.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/618: mkdir d3/d13/d64/dc1 0 2024-04-26T04:42:26.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/619: fdatasync d3/db/d12/d3b/f78 0 2024-04-26T04:42:26.276 INFO:tasks.workunit.client.1.smithi104.stdout:7/931: dwrite d2/d4/d1d/f90 [4194304,4194304] 0 2024-04-26T04:42:26.278 INFO:tasks.workunit.client.0.smithi090.stdout:8/766: dwrite dc/d5a/d5d/d73/dce/f100 [0,4194304] 0 2024-04-26T04:42:26.279 INFO:tasks.workunit.client.0.smithi090.stdout:4/620: mknod d3/db/d12/d20/d24/d43/cc2 0 2024-04-26T04:42:26.281 INFO:tasks.workunit.client.1.smithi104.stdout:8/874: mkdir d0/d1c/d27/d47/d67/d76/d11f 0 2024-04-26T04:42:26.282 INFO:tasks.workunit.client.1.smithi104.stdout:8/875: truncate d0/d1c/d27/f72 1419486 0 2024-04-26T04:42:26.283 INFO:tasks.workunit.client.0.smithi090.stdout:8/767: mknod dc/dd2/d39/c10c 0 2024-04-26T04:42:26.283 INFO:tasks.workunit.client.0.smithi090.stdout:8/768: stat dc/dd2/d4f/f96 0 2024-04-26T04:42:26.284 INFO:tasks.workunit.client.0.smithi090.stdout:9/559: write db/dd/f8d [1382812,62606] 0 2024-04-26T04:42:26.285 INFO:tasks.workunit.client.1.smithi104.stdout:8/876: unlink d0/d1c/d27/d47/d67/d76/daf/dd4/ced 0 2024-04-26T04:42:26.287 INFO:tasks.workunit.client.0.smithi090.stdout:8/769: rmdir dc/dd2/d4f/da6/dd8/dea 0 2024-04-26T04:42:26.287 INFO:tasks.workunit.client.0.smithi090.stdout:8/770: chown dc/d5a/d5d/cf6 3792 1 2024-04-26T04:42:26.287 INFO:tasks.workunit.client.0.smithi090.stdout:8/771: write dc/d5a/d5d/fc0 [803288,85001] 0 2024-04-26T04:42:26.287 INFO:tasks.workunit.client.0.smithi090.stdout:4/621: rmdir d3/d13/d6c 39 2024-04-26T04:42:26.288 INFO:tasks.workunit.client.1.smithi104.stdout:0/930: mkdir d0/d2/d131 0 2024-04-26T04:42:26.288 INFO:tasks.workunit.client.0.smithi090.stdout:9/560: symlink db/da3/lc6 0 2024-04-26T04:42:26.289 INFO:tasks.workunit.client.0.smithi090.stdout:9/561: readlink db/dd/d18/d38/d39/l45 0 2024-04-26T04:42:26.289 INFO:tasks.workunit.client.0.smithi090.stdout:9/562: chown db/dd/d44/c8a 4181225 1 2024-04-26T04:42:26.290 INFO:tasks.workunit.client.0.smithi090.stdout:7/679: sync 2024-04-26T04:42:26.290 INFO:tasks.workunit.client.0.smithi090.stdout:6/703: sync 2024-04-26T04:42:26.290 INFO:tasks.workunit.client.0.smithi090.stdout:3/588: sync 2024-04-26T04:42:26.293 INFO:tasks.workunit.client.0.smithi090.stdout:7/680: mkdir d0/db/df/d4d/d64/df2/df9 0 2024-04-26T04:42:26.293 INFO:tasks.workunit.client.0.smithi090.stdout:7/681: creat d0/db/d53/db7/d4f/ffa x:0 0 0 2024-04-26T04:42:26.294 INFO:tasks.workunit.client.0.smithi090.stdout:8/772: read dc/d5a/f6a [618945,73863] 0 2024-04-26T04:42:26.294 INFO:tasks.workunit.client.0.smithi090.stdout:8/773: write dc/dd2/d3b/d3d/d97/fa2 [2716433,82192] 0 2024-04-26T04:42:26.295 INFO:tasks.workunit.client.0.smithi090.stdout:6/704: rename d9/d77/db0/db5/dda/d6d/c82 to d9/d77/db0/db5/dca/dd8/cf3 0 2024-04-26T04:42:26.295 INFO:tasks.workunit.client.0.smithi090.stdout:6/705: chown d9/d21/d40/d4c/d6f/d9f/dae 26 1 2024-04-26T04:42:26.296 INFO:tasks.workunit.client.0.smithi090.stdout:6/706: fdatasync d9/d21/d40/d4c/f59 0 2024-04-26T04:42:26.297 INFO:tasks.workunit.client.1.smithi104.stdout:0/931: write d0/d2/d21/d27/d1e/d57/d6c/dd1/f34 [1400406,83043] 0 2024-04-26T04:42:26.297 INFO:tasks.workunit.client.0.smithi090.stdout:7/682: mknod d0/db/de6/cfb 0 2024-04-26T04:42:26.297 INFO:tasks.workunit.client.0.smithi090.stdout:0/695: sync 2024-04-26T04:42:26.297 INFO:tasks.workunit.client.0.smithi090.stdout:0/696: truncate d0/d4b/d5d/d5c/d8a/d45/d53/fa3 701860 0 2024-04-26T04:42:26.298 INFO:tasks.workunit.client.1.smithi104.stdout:8/877: dread d0/d1c/d27/f86 [0,4194304] 0 2024-04-26T04:42:26.298 INFO:tasks.workunit.client.1.smithi104.stdout:8/878: dread - d0/d1c/d59/d85/fc3 zero size 2024-04-26T04:42:26.299 INFO:tasks.workunit.client.0.smithi090.stdout:8/774: link dc/dd2/c23 dc/dd2/d4f/da6/db1/c10d 0 2024-04-26T04:42:26.299 INFO:tasks.workunit.client.1.smithi104.stdout:0/932: getdents d0 0 2024-04-26T04:42:26.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/707: symlink d9/d77/db0/db5/dda/lf4 0 2024-04-26T04:42:26.301 INFO:tasks.workunit.client.0.smithi090.stdout:6/708: fdatasync d9/d77/db0/db5/dda/d6d/d66/fb8 0 2024-04-26T04:42:26.301 INFO:tasks.workunit.client.1.smithi104.stdout:8/879: mknod d0/d33/d51/c120 0 2024-04-26T04:42:26.303 INFO:tasks.workunit.client.1.smithi104.stdout:8/880: readlink d0/d33/d51/lf3 0 2024-04-26T04:42:26.306 INFO:tasks.workunit.client.0.smithi090.stdout:6/709: write d9/d77/db0/db5/dda/d58/f6c [1842378,83776] 0 2024-04-26T04:42:26.306 INFO:tasks.workunit.client.0.smithi090.stdout:6/710: creat d9/d21/d40/d4c/ff5 x:0 0 0 2024-04-26T04:42:26.306 INFO:tasks.workunit.client.0.smithi090.stdout:0/697: rename d0/d4b/d5d/d5c/f5e to d0/fee 0 2024-04-26T04:42:26.306 INFO:tasks.workunit.client.1.smithi104.stdout:0/933: creat d0/d2/d21/d130/f132 x:0 0 0 2024-04-26T04:42:26.306 INFO:tasks.workunit.client.1.smithi104.stdout:0/934: chown d0/d2/d21/d27/d1e/f96 1 1 2024-04-26T04:42:26.306 INFO:tasks.workunit.client.1.smithi104.stdout:0/935: write d0/d2/d10/d3d/f8d [8945471,38633] 0 2024-04-26T04:42:26.307 INFO:tasks.workunit.client.1.smithi104.stdout:9/792: unlink d11/d15/f30 0 2024-04-26T04:42:26.307 INFO:tasks.workunit.client.1.smithi104.stdout:9/793: getdents d11/d15/d32/d35 0 2024-04-26T04:42:26.314 INFO:tasks.workunit.client.1.smithi104.stdout:6/993: creat d0/d9/d13/f146 x:0 0 0 2024-04-26T04:42:26.314 INFO:tasks.workunit.client.1.smithi104.stdout:6/994: chown d0/d9/d13/d41/d82/df7/l138 53458656 1 2024-04-26T04:42:26.316 INFO:tasks.workunit.client.0.smithi090.stdout:5/598: dwrite d4/d1a/d1f/d53/d59/d5e/f6a [0,4194304] 0 2024-04-26T04:42:26.316 INFO:tasks.workunit.client.1.smithi104.stdout:8/881: dread d0/d1c/d27/d47/db1/fc9 [0,4194304] 0 2024-04-26T04:42:26.316 INFO:tasks.workunit.client.1.smithi104.stdout:8/882: fdatasync d0/d1c/d27/d47/d67/d76/d7f/f116 0 2024-04-26T04:42:26.317 INFO:tasks.workunit.client.1.smithi104.stdout:8/883: truncate d0/d1c/d27/d47/d67/d76/daf/ffb 502439 0 2024-04-26T04:42:26.317 INFO:tasks.workunit.client.1.smithi104.stdout:8/884: readlink d0/d33/d35/d6d/lc7 0 2024-04-26T04:42:26.317 INFO:tasks.workunit.client.1.smithi104.stdout:6/995: truncate d0/d9/d13/d41/d82/dc8/f42 460322 0 2024-04-26T04:42:26.320 INFO:tasks.workunit.client.1.smithi104.stdout:8/885: mknod d0/d9a/dce/c121 0 2024-04-26T04:42:26.320 INFO:tasks.workunit.client.0.smithi090.stdout:5/599: creat d4/dab/fbd x:0 0 0 2024-04-26T04:42:26.321 INFO:tasks.workunit.client.0.smithi090.stdout:5/600: symlink d4/d1a/d45/db8/lbe 0 2024-04-26T04:42:26.322 INFO:tasks.workunit.client.0.smithi090.stdout:5/601: chown d4/d1a/d2a/d79/d90/d49/c62 2096913338 1 2024-04-26T04:42:26.322 INFO:tasks.workunit.client.1.smithi104.stdout:6/996: mkdir d0/d9/d13/dcd/dd9/d120/d147 0 2024-04-26T04:42:26.322 INFO:tasks.workunit.client.1.smithi104.stdout:6/997: chown d0/d9/da8 13830100 1 2024-04-26T04:42:26.324 INFO:tasks.workunit.client.0.smithi090.stdout:5/602: creat d4/d1a/df/d36/fbf x:0 0 0 2024-04-26T04:42:26.324 INFO:tasks.workunit.client.0.smithi090.stdout:8/775: write dc/dd2/d3b/d3d/f9c [678544,84675] 0 2024-04-26T04:42:26.324 INFO:tasks.workunit.client.1.smithi104.stdout:8/886: unlink d0/d33/d35/d6d/lc7 0 2024-04-26T04:42:26.325 INFO:tasks.workunit.client.1.smithi104.stdout:7/932: dwrite d2/d4/d32/d3d/d47/db1/dfa/f119 [0,4194304] 0 2024-04-26T04:42:26.328 INFO:tasks.workunit.client.1.smithi104.stdout:8/887: getdents d0/d1c 0 2024-04-26T04:42:26.328 INFO:tasks.workunit.client.1.smithi104.stdout:8/888: dread - d0/d1c/d59/f11d zero size 2024-04-26T04:42:26.328 INFO:tasks.workunit.client.1.smithi104.stdout:8/889: write d0/d1c/fd0 [1667242,26128] 0 2024-04-26T04:42:26.328 INFO:tasks.workunit.client.1.smithi104.stdout:8/890: getdents d0/d1c 0 2024-04-26T04:42:26.329 INFO:tasks.workunit.client.1.smithi104.stdout:8/891: creat d0/d1c/d27/f122 x:0 0 0 2024-04-26T04:42:26.331 INFO:tasks.workunit.client.1.smithi104.stdout:8/892: creat d0/d1c/d27/d47/f123 x:0 0 0 2024-04-26T04:42:26.351 INFO:tasks.workunit.client.0.smithi090.stdout:3/589: dwrite d1/d17/f1f [0,4194304] 0 2024-04-26T04:42:26.352 INFO:tasks.workunit.client.0.smithi090.stdout:2/654: dwrite db/d11/d14/d25/d65/f9d [0,4194304] 0 2024-04-26T04:42:26.353 INFO:tasks.workunit.client.0.smithi090.stdout:4/622: dwrite d3/db/d12/f10 [0,4194304] 0 2024-04-26T04:42:26.354 INFO:tasks.workunit.client.0.smithi090.stdout:4/623: readlink d3/l5 0 2024-04-26T04:42:26.354 INFO:tasks.workunit.client.0.smithi090.stdout:3/590: unlink d1/d17/d36/d2a/d3a/d5d/d82/cc0 0 2024-04-26T04:42:26.354 INFO:tasks.workunit.client.0.smithi090.stdout:8/776: dread dc/d34/f93 [0,4194304] 0 2024-04-26T04:42:26.354 INFO:tasks.workunit.client.0.smithi090.stdout:8/777: fsync dc/f30 0 2024-04-26T04:42:26.354 INFO:tasks.workunit.client.1.smithi104.stdout:5/987: symlink d0/d1/d64/l14d 0 2024-04-26T04:42:26.355 INFO:tasks.workunit.client.1.smithi104.stdout:5/988: fsync d0/d1/d1e/df5/f142 0 2024-04-26T04:42:26.355 INFO:tasks.workunit.client.0.smithi090.stdout:2/655: mkdir db/dc/d6f/dd8 0 2024-04-26T04:42:26.356 INFO:tasks.workunit.client.0.smithi090.stdout:4/624: creat d3/db/d12/d20/d2d/d58/d6a/fc3 x:0 0 0 2024-04-26T04:42:26.356 INFO:tasks.workunit.client.0.smithi090.stdout:4/625: getdents d3/db/d12/d99 0 2024-04-26T04:42:26.356 INFO:tasks.workunit.client.0.smithi090.stdout:4/626: creat d3/db/d12/d20/d2d/d58/fc4 x:0 0 0 2024-04-26T04:42:26.357 INFO:tasks.workunit.client.0.smithi090.stdout:8/778: mknod dc/dd2/d4f/d58/d95/d106/c10e 0 2024-04-26T04:42:26.357 INFO:tasks.workunit.client.0.smithi090.stdout:8/779: chown dc/d71/ld4 20787 1 2024-04-26T04:42:26.357 INFO:tasks.workunit.client.0.smithi090.stdout:8/780: chown dc/d5a 8011992 1 2024-04-26T04:42:26.358 INFO:tasks.workunit.client.0.smithi090.stdout:2/656: creat db/d11/d14/d27/d2f/d42/fd9 x:0 0 0 2024-04-26T04:42:26.358 INFO:tasks.workunit.client.0.smithi090.stdout:2/657: write db/d59/f62 [3841670,38842] 0 2024-04-26T04:42:26.360 INFO:tasks.workunit.client.0.smithi090.stdout:2/658: creat db/d11/d14/d27/d2f/fda x:0 0 0 2024-04-26T04:42:26.360 INFO:tasks.workunit.client.0.smithi090.stdout:2/659: fdatasync db/d11/d14/d25/d2d/d48/d5b/f7d 0 2024-04-26T04:42:26.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/660: rmdir db/d11/d14/d1b/d20/d4c 39 2024-04-26T04:42:26.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/661: write db/d11/d14/d27/d28/f3d [3053495,65121] 0 2024-04-26T04:42:26.364 INFO:tasks.workunit.client.0.smithi090.stdout:4/627: dread d3/d13/d6c/f90 [0,4194304] 0 2024-04-26T04:42:26.364 INFO:tasks.workunit.client.0.smithi090.stdout:4/628: chown f0 137 1 2024-04-26T04:42:26.366 INFO:tasks.workunit.client.0.smithi090.stdout:2/662: rename db/d11/d14/d27/db0/faa to db/d59/d66/fdb 0 2024-04-26T04:42:26.370 INFO:tasks.workunit.client.0.smithi090.stdout:3/591: write d1/d9/f12 [102188,82543] 0 2024-04-26T04:42:26.370 INFO:tasks.workunit.client.0.smithi090.stdout:3/592: read d1/d17/d36/f65 [1632758,79297] 0 2024-04-26T04:42:26.374 INFO:tasks.workunit.client.0.smithi090.stdout:3/593: rename d1/d17/d52 to d1/d59/dc9 0 2024-04-26T04:42:26.375 INFO:tasks.workunit.client.0.smithi090.stdout:4/629: dread d3/d13/f61 [0,4194304] 0 2024-04-26T04:42:26.377 INFO:tasks.workunit.client.0.smithi090.stdout:6/711: dwrite d9/d21/d40/d4c/d6f/d9f/fcd [0,4194304] 0 2024-04-26T04:42:26.380 INFO:tasks.workunit.client.0.smithi090.stdout:6/712: symlink d9/d77/lf6 0 2024-04-26T04:42:26.380 INFO:tasks.workunit.client.0.smithi090.stdout:4/630: dread d3/d13/d6c/f90 [0,4194304] 0 2024-04-26T04:42:26.381 INFO:tasks.workunit.client.0.smithi090.stdout:6/713: mknod d9/d21/d40/d4c/cf7 0 2024-04-26T04:42:26.381 INFO:tasks.workunit.client.0.smithi090.stdout:6/714: stat d9/d1c/dc5/dd9/fed 0 2024-04-26T04:42:26.382 INFO:tasks.workunit.client.0.smithi090.stdout:6/715: creat d9/d77/d9a/ff8 x:0 0 0 2024-04-26T04:42:26.384 INFO:tasks.workunit.client.0.smithi090.stdout:0/698: dwrite d0/d4b/d5d/fe7 [0,4194304] 0 2024-04-26T04:42:26.386 INFO:tasks.workunit.client.1.smithi104.stdout:0/936: dwrite d0/d2/d21/d27/d1e/f65 [4194304,4194304] 0 2024-04-26T04:42:26.387 INFO:tasks.workunit.client.0.smithi090.stdout:6/716: dread d9/d77/db0/db5/fa3 [0,4194304] 0 2024-04-26T04:42:26.387 INFO:tasks.workunit.client.0.smithi090.stdout:4/631: dread d3/db/d12/d20/d2d/d58/d6a/f8b [0,4194304] 0 2024-04-26T04:42:26.389 INFO:tasks.workunit.client.0.smithi090.stdout:4/632: symlink d3/db/d12/d20/d2d/d93/lc5 0 2024-04-26T04:42:26.389 INFO:tasks.workunit.client.0.smithi090.stdout:4/633: dread - d3/db/d12/d20/d2d/d58/fc4 zero size 2024-04-26T04:42:26.390 INFO:tasks.workunit.client.1.smithi104.stdout:0/937: rename d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/lc3 to d0/d2/d131/l133 0 2024-04-26T04:42:26.390 INFO:tasks.workunit.client.1.smithi104.stdout:0/938: dread - d0/d2/d10/d84/fe7 zero size 2024-04-26T04:42:26.391 INFO:tasks.workunit.client.1.smithi104.stdout:0/939: symlink d0/d2/d10/d3d/l134 0 2024-04-26T04:42:26.391 INFO:tasks.workunit.client.1.smithi104.stdout:0/940: stat d0/d4/d3b/cde 0 2024-04-26T04:42:26.392 INFO:tasks.workunit.client.0.smithi090.stdout:4/634: rename d3/db/d12/d20/d24/fb9 to d3/db/d12/d20/d24/fc6 0 2024-04-26T04:42:26.393 INFO:tasks.workunit.client.1.smithi104.stdout:8/893: dwrite d0/d33/da6/fa8 [0,4194304] 0 2024-04-26T04:42:26.393 INFO:tasks.workunit.client.0.smithi090.stdout:7/683: dwrite d0/d1/d4/d98/d65/dbb/fc9 [0,4194304] 0 2024-04-26T04:42:26.394 INFO:tasks.workunit.client.0.smithi090.stdout:7/684: creat d0/db/d15/d29/df3/ffc x:0 0 0 2024-04-26T04:42:26.396 INFO:tasks.workunit.client.0.smithi090.stdout:8/781: dwrite dc/d5a/fc8 [0,4194304] 0 2024-04-26T04:42:26.396 INFO:tasks.workunit.client.1.smithi104.stdout:6/998: dwrite d0/d9/dc6/f119 [0,4194304] 0 2024-04-26T04:42:26.400 INFO:tasks.workunit.client.1.smithi104.stdout:7/933: dwrite d2/fe8 [0,4194304] 0 2024-04-26T04:42:26.400 INFO:tasks.workunit.client.0.smithi090.stdout:0/699: write d0/df/f25 [1002313,103403] 0 2024-04-26T04:42:26.400 INFO:tasks.workunit.client.0.smithi090.stdout:0/700: chown d0/d4b/d5d/d5c/d8a/d45/f59 119 1 2024-04-26T04:42:26.402 INFO:tasks.workunit.client.1.smithi104.stdout:6/999: dread d0/d9/d13/d41/d96/ded/d10d/d71/da5/fb7 [0,4194304] 0 2024-04-26T04:42:26.405 INFO:tasks.workunit.client.1.smithi104.stdout:7/934: symlink d2/l12a 0 2024-04-26T04:42:26.406 INFO:tasks.workunit.client.1.smithi104.stdout:9/794: dwrite d11/d1d/d34/db3/fee [0,4194304] 0 2024-04-26T04:42:26.406 INFO:tasks.workunit.client.0.smithi090.stdout:7/685: write d0/d1/d4/d98/d4a/fbe [3147809,53140] 0 2024-04-26T04:42:26.407 INFO:tasks.workunit.client.1.smithi104.stdout:7/935: mknod d2/d4/d1d/d121/d10c/c12b 0 2024-04-26T04:42:26.409 INFO:tasks.workunit.client.1.smithi104.stdout:9/795: rmdir d11/d90 39 2024-04-26T04:42:26.409 INFO:tasks.workunit.client.1.smithi104.stdout:9/796: chown d11/d15/d32/d4c/d54/f6e 1339183418 1 2024-04-26T04:42:26.409 INFO:tasks.workunit.client.1.smithi104.stdout:5/989: dwrite d0/f4e [0,4194304] 0 2024-04-26T04:42:26.409 INFO:tasks.workunit.client.1.smithi104.stdout:0/941: dread d0/d2/d21/d49/d5e/d9c/f4b [0,4194304] 0 2024-04-26T04:42:26.411 INFO:tasks.workunit.client.0.smithi090.stdout:8/782: mknod dc/dd2/d3b/d3d/db8/ddc/c10f 0 2024-04-26T04:42:26.411 INFO:tasks.workunit.client.0.smithi090.stdout:8/783: fsync dc/dd2/d39/fb9 0 2024-04-26T04:42:26.411 INFO:tasks.workunit.client.0.smithi090.stdout:8/784: creat dc/d5a/f110 x:0 0 0 2024-04-26T04:42:26.412 INFO:tasks.workunit.client.1.smithi104.stdout:9/797: rename d11/d15/da3/fd1 to d11/d15/d1b/d6d/f105 0 2024-04-26T04:42:26.413 INFO:tasks.workunit.client.1.smithi104.stdout:5/990: unlink d0/d1/c34 0 2024-04-26T04:42:26.415 INFO:tasks.workunit.client.0.smithi090.stdout:2/663: dwrite db/d59/f62 [0,4194304] 0 2024-04-26T04:42:26.416 INFO:tasks.workunit.client.1.smithi104.stdout:9/798: mknod d11/d15/d32/d4c/d54/ddc/c106 0 2024-04-26T04:42:26.419 INFO:tasks.workunit.client.0.smithi090.stdout:8/785: dread dc/d34/d4d/f4e [0,4194304] 0 2024-04-26T04:42:26.420 INFO:tasks.workunit.client.1.smithi104.stdout:0/942: dread d0/d2/d21/d27/d1e/d57/d6c/f6a [0,4194304] 0 2024-04-26T04:42:26.425 INFO:tasks.workunit.client.0.smithi090.stdout:2/664: link db/d11/d14/d27/d2f/d42/c57 db/d11/d14/d1b/cdc 0 2024-04-26T04:42:26.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/686: write d0/d1/d4/d3e/f90 [7302557,7264] 0 2024-04-26T04:42:26.426 INFO:tasks.workunit.client.0.smithi090.stdout:7/687: truncate d0/db/d53/db7/d4f/ffa 848108 0 2024-04-26T04:42:26.427 INFO:tasks.workunit.client.0.smithi090.stdout:8/786: mkdir dc/dd2/d3b/d3d/d111 0 2024-04-26T04:42:26.429 INFO:tasks.workunit.client.0.smithi090.stdout:0/701: mknod d0/df/cef 0 2024-04-26T04:42:26.430 INFO:tasks.workunit.client.1.smithi104.stdout:7/936: dwrite d2/d4/d32/d3d/f50 [0,4194304] 0 2024-04-26T04:42:26.431 INFO:tasks.workunit.client.0.smithi090.stdout:2/665: rename db/d11/d14/d27/d28/fa5 to db/d11/d14/d27/d2f/d42/fdd 0 2024-04-26T04:42:26.432 INFO:tasks.workunit.client.0.smithi090.stdout:2/666: write db/d11/d14/d27/f38 [1085991,81091] 0 2024-04-26T04:42:26.432 INFO:tasks.workunit.client.0.smithi090.stdout:2/667: fdatasync db/dc/f3c 0 2024-04-26T04:42:26.432 INFO:tasks.workunit.client.0.smithi090.stdout:4/635: dwrite d3/db/d12/d20/d2d/d58/d6a/d74/fb6 [0,4194304] 0 2024-04-26T04:42:26.432 INFO:tasks.workunit.client.0.smithi090.stdout:4/636: fdatasync d3/db/d12/d20/d24/d43/d72/f80 0 2024-04-26T04:42:26.432 INFO:tasks.workunit.client.0.smithi090.stdout:6/717: dwrite d9/d77/f9d [0,4194304] 0 2024-04-26T04:42:26.432 INFO:tasks.workunit.client.0.smithi090.stdout:4/637: chown d3/db/d12/d20/d24/da9/fbc 157 1 2024-04-26T04:42:26.433 INFO:tasks.workunit.client.0.smithi090.stdout:8/787: rmdir dc/dd2/d39/d55 39 2024-04-26T04:42:26.434 INFO:tasks.workunit.client.0.smithi090.stdout:0/702: mknod d0/d4b/d5d/d5c/dc4/cf0 0 2024-04-26T04:42:26.442 INFO:tasks.workunit.client.1.smithi104.stdout:7/937: write d2/d4/d32/d35/d37/f6f [3103377,34124] 0 2024-04-26T04:42:26.445 INFO:tasks.workunit.client.0.smithi090.stdout:6/718: creat d9/d1c/ff9 x:0 0 0 2024-04-26T04:42:26.445 INFO:tasks.workunit.client.1.smithi104.stdout:0/943: write d0/d2/d21/d27/d1e/d57/f8f [216062,71765] 0 2024-04-26T04:42:26.446 INFO:tasks.workunit.client.0.smithi090.stdout:8/788: mknod dc/dd2/d4f/da6/dd8/c112 0 2024-04-26T04:42:26.446 INFO:tasks.workunit.client.0.smithi090.stdout:8/789: chown dc/d34/l38 299081976 1 2024-04-26T04:42:26.446 INFO:tasks.workunit.client.0.smithi090.stdout:8/790: chown dc/dd2/d4f/f96 8407 1 2024-04-26T04:42:26.448 INFO:tasks.workunit.client.0.smithi090.stdout:0/703: rmdir d0/d4b/db3 39 2024-04-26T04:42:26.449 INFO:tasks.workunit.client.0.smithi090.stdout:2/668: getdents db/d11/d14/d27/d2f/d70/d71 0 2024-04-26T04:42:26.451 INFO:tasks.workunit.client.0.smithi090.stdout:6/719: rename d9/d21/d40/d4c/d6f/fa7 to d9/d77/db0/db5/dda/d6d/d66/ffa 0 2024-04-26T04:42:26.453 INFO:tasks.workunit.client.0.smithi090.stdout:8/791: mknod dc/dd2/d4f/da6/db1/ddd/c113 0 2024-04-26T04:42:26.454 INFO:tasks.workunit.client.0.smithi090.stdout:0/704: creat d0/d4b/d5d/d5c/d8a/ff1 x:0 0 0 2024-04-26T04:42:26.454 INFO:tasks.workunit.client.0.smithi090.stdout:0/705: stat d0/d4b/d5d/d5c/d8a/d45/d53/d8e/c9f 0 2024-04-26T04:42:26.454 INFO:tasks.workunit.client.0.smithi090.stdout:0/706: truncate d0/d4b/db3/fcb 470568 0 2024-04-26T04:42:26.455 INFO:tasks.workunit.client.0.smithi090.stdout:2/669: link db/d11/d14/d1b/d20/d4c/cbd db/dc/d6f/cde 0 2024-04-26T04:42:26.456 INFO:tasks.workunit.client.0.smithi090.stdout:6/720: truncate d9/d1c/f2b 3898066 0 2024-04-26T04:42:26.457 INFO:tasks.workunit.client.0.smithi090.stdout:8/792: truncate dc/f30 3505035 0 2024-04-26T04:42:26.458 INFO:tasks.workunit.client.0.smithi090.stdout:8/793: chown dc/dd2/d3b/la9 152 1 2024-04-26T04:42:26.459 INFO:tasks.workunit.client.0.smithi090.stdout:0/707: dread d0/d4b/d5d/d5c/d8a/d45/d53/fa3 [0,4194304] 0 2024-04-26T04:42:26.460 INFO:tasks.workunit.client.1.smithi104.stdout:5/991: dwrite d0/d1/d4/d2a/d6b/d91/fb0 [0,4194304] 0 2024-04-26T04:42:26.460 INFO:tasks.workunit.client.1.smithi104.stdout:9/799: dwrite d11/d1d/d34/d69/fce [0,4194304] 0 2024-04-26T04:42:26.461 INFO:tasks.workunit.client.0.smithi090.stdout:6/721: creat d9/d1c/d31/d79/ffb x:0 0 0 2024-04-26T04:42:26.461 INFO:tasks.workunit.client.0.smithi090.stdout:6/722: dread - d9/d1c/dc5/dd9/fe8 zero size 2024-04-26T04:42:26.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/638: dwrite d3/d13/f61 [0,4194304] 0 2024-04-26T04:42:26.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/639: fdatasync d3/db/d12/d20/d2d/d58/f63 0 2024-04-26T04:42:26.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/670: getdents db/d11/d14/d25/d2d/d48 0 2024-04-26T04:42:26.467 INFO:tasks.workunit.client.0.smithi090.stdout:4/640: mknod d3/db/d12/d20/d2d/d2e/cc7 0 2024-04-26T04:42:26.467 INFO:tasks.workunit.client.0.smithi090.stdout:4/641: fdatasync d3/db/d12/d20/d2d/d58/d6a/fc3 0 2024-04-26T04:42:26.467 INFO:tasks.workunit.client.0.smithi090.stdout:4/642: chown d3/db/d12/d20/d2d/d93 58 1 2024-04-26T04:42:26.469 INFO:tasks.workunit.client.0.smithi090.stdout:2/671: mkdir db/d11/d14/d27/d28/dd1/ddf 0 2024-04-26T04:42:26.470 INFO:tasks.workunit.client.1.smithi104.stdout:9/800: dread d11/f52 [0,4194304] 0 2024-04-26T04:42:26.471 INFO:tasks.workunit.client.1.smithi104.stdout:9/801: dread - d11/d90/f99 zero size 2024-04-26T04:42:26.473 INFO:tasks.workunit.client.0.smithi090.stdout:0/708: rename d0/d4b/d5d/d5c/d8a/d45/d53/dc2/dd7 to d0/d4b/d6d/df2 0 2024-04-26T04:42:26.477 INFO:tasks.workunit.client.0.smithi090.stdout:8/794: dwrite dc/d71/fae [0,4194304] 0 2024-04-26T04:42:26.477 INFO:tasks.workunit.client.0.smithi090.stdout:8/795: write dc/dd2/d3b/d3d/f5e [242524,76047] 0 2024-04-26T04:42:26.477 INFO:tasks.workunit.client.0.smithi090.stdout:0/709: rename d0/d4b/d5d/d5c/d8a/f43 to d0/d4b/db3/dcc/ff3 0 2024-04-26T04:42:26.477 INFO:tasks.workunit.client.0.smithi090.stdout:0/710: write d0/d4b/d5d/fb1 [552932,116593] 0 2024-04-26T04:42:26.485 INFO:tasks.workunit.client.0.smithi090.stdout:8/796: unlink dc/d1f/fa8 0 2024-04-26T04:42:26.487 INFO:tasks.workunit.client.0.smithi090.stdout:0/711: rename d0/d4b/d5d/d5c/lc7 to d0/d4b/d5d/d5c/dc4/lf4 0 2024-04-26T04:42:26.487 INFO:tasks.workunit.client.0.smithi090.stdout:0/712: dread - d0/d4b/feb zero size 2024-04-26T04:42:26.487 INFO:tasks.workunit.client.1.smithi104.stdout:7/938: creat d2/d4/d32/d35/d116/f12c x:0 0 0 2024-04-26T04:42:26.489 INFO:tasks.workunit.client.0.smithi090.stdout:8/797: rename dc/dd2/d3b/d3d/d97/dad/ce3 to dc/dd2/d4f/da6/dfc/c114 0 2024-04-26T04:42:26.490 INFO:tasks.workunit.client.1.smithi104.stdout:0/944: creat d0/d2/d10/d3d/f135 x:0 0 0 2024-04-26T04:42:26.490 INFO:tasks.workunit.client.1.smithi104.stdout:0/945: stat d0/d2/d21/d27/d1e/d111/l115 0 2024-04-26T04:42:26.491 INFO:tasks.workunit.client.0.smithi090.stdout:8/798: truncate dc/dd2/d4f/d58/d95/f8a 2619706 0 2024-04-26T04:42:26.491 INFO:tasks.workunit.client.0.smithi090.stdout:8/799: chown dc/dd2/d4f/d58/d95/d106/f107 1 1 2024-04-26T04:42:26.492 INFO:tasks.workunit.client.1.smithi104.stdout:0/946: unlink d0/d2/d10/c63 0 2024-04-26T04:42:26.492 INFO:tasks.workunit.client.1.smithi104.stdout:0/947: truncate d0/d2/d10/d3d/f135 617671 0 2024-04-26T04:42:26.493 INFO:tasks.workunit.client.0.smithi090.stdout:8/800: link dc/d1f/f36 dc/dd2/d4f/da6/dd8/f115 0 2024-04-26T04:42:26.493 INFO:tasks.workunit.client.0.smithi090.stdout:8/801: creat dc/dd2/d4f/da6/f116 x:0 0 0 2024-04-26T04:42:26.494 INFO:tasks.workunit.client.1.smithi104.stdout:0/948: link d0/d4/f11 d0/d2/d21/d27/d1e/d57/d6c/d113/f136 0 2024-04-26T04:42:26.494 INFO:tasks.workunit.client.1.smithi104.stdout:0/949: stat d0/d2/d21/d49/d5e/d9c/db2/lf0 0 2024-04-26T04:42:26.494 INFO:tasks.workunit.client.1.smithi104.stdout:0/950: stat d0/d2/d10/d3d/f135 0 2024-04-26T04:42:26.495 INFO:tasks.workunit.client.0.smithi090.stdout:8/802: symlink dc/dd2/d4f/l117 0 2024-04-26T04:42:26.495 INFO:tasks.workunit.client.0.smithi090.stdout:8/803: truncate dc/d5a/f110 176106 0 2024-04-26T04:42:26.495 INFO:tasks.workunit.client.0.smithi090.stdout:8/804: chown dc/dd2/d39/fb9 1181 1 2024-04-26T04:42:26.496 INFO:tasks.workunit.client.0.smithi090.stdout:8/805: dread dc/dd2/d39/fba [0,4194304] 0 2024-04-26T04:42:26.499 INFO:tasks.workunit.client.1.smithi104.stdout:7/939: read d2/d4/f11 [914474,71672] 0 2024-04-26T04:42:26.502 INFO:tasks.workunit.client.0.smithi090.stdout:0/713: dread d0/d4b/d5d/d5c/d8a/f94 [0,4194304] 0 2024-04-26T04:42:26.506 INFO:tasks.workunit.client.1.smithi104.stdout:5/992: dwrite d0/d1/d4/d2a/d6b/d91/dbd/ffa [0,4194304] 0 2024-04-26T04:42:26.506 INFO:tasks.workunit.client.0.smithi090.stdout:0/714: mknod d0/cf5 0 2024-04-26T04:42:26.507 INFO:tasks.workunit.client.1.smithi104.stdout:0/951: write d0/f10d [3408415,47450] 0 2024-04-26T04:42:26.507 INFO:tasks.workunit.client.1.smithi104.stdout:9/802: dwrite d11/d15/d32/f83 [0,4194304] 0 2024-04-26T04:42:26.507 INFO:tasks.workunit.client.1.smithi104.stdout:9/803: write d11/d1d/fbd [362630,69756] 0 2024-04-26T04:42:26.508 INFO:tasks.workunit.client.0.smithi090.stdout:2/672: dwrite db/d11/d14/d27/d2f/d42/d5c/f4f [0,4194304] 0 2024-04-26T04:42:26.508 INFO:tasks.workunit.client.0.smithi090.stdout:2/673: getdents db/d11/d14/d27/d28/dd1/ddf 0 2024-04-26T04:42:26.508 INFO:tasks.workunit.client.0.smithi090.stdout:0/715: creat d0/d4b/d5d/d5c/d8a/d45/d53/dc5/ff6 x:0 0 0 2024-04-26T04:42:26.510 INFO:tasks.workunit.client.0.smithi090.stdout:4/643: dwrite d3/db/d12/d20/d24/d43/d72/f7c [0,4194304] 0 2024-04-26T04:42:26.510 INFO:tasks.workunit.client.0.smithi090.stdout:4/644: write d3/db/d12/d20/d2d/d58/d6a/d74/da2/f37 [813356,35692] 0 2024-04-26T04:42:26.510 INFO:tasks.workunit.client.0.smithi090.stdout:4/645: write d3/f48 [894356,28647] 0 2024-04-26T04:42:26.510 INFO:tasks.workunit.client.1.smithi104.stdout:7/940: dread d2/d4/d32/d60/f8c [0,4194304] 0 2024-04-26T04:42:26.511 INFO:tasks.workunit.client.0.smithi090.stdout:8/806: dread dc/dd2/d4f/da6/dd8/fa4 [0,4194304] 0 2024-04-26T04:42:26.511 INFO:tasks.workunit.client.0.smithi090.stdout:8/807: write dc/dd2/d39/f52 [3736482,50666] 0 2024-04-26T04:42:26.512 INFO:tasks.workunit.client.1.smithi104.stdout:0/952: link d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/ffd d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/dcc/f137 0 2024-04-26T04:42:26.512 INFO:tasks.workunit.client.1.smithi104.stdout:0/953: fsync d0/d2/d21/d27/d1e/d57/d6c/d85/fa2 0 2024-04-26T04:42:26.512 INFO:tasks.workunit.client.1.smithi104.stdout:0/954: dread - d0/d2/d21/d49/d5e/d9c/f120 zero size 2024-04-26T04:42:26.513 INFO:tasks.workunit.client.1.smithi104.stdout:9/804: creat d11/d15/d1b/d6d/dd5/f107 x:0 0 0 2024-04-26T04:42:26.515 INFO:tasks.workunit.client.0.smithi090.stdout:4/646: dread d3/db/d12/d20/d24/f32 [0,4194304] 0 2024-04-26T04:42:26.515 INFO:tasks.workunit.client.1.smithi104.stdout:7/941: symlink d2/d4/d32/d3d/d41/dbd/l12d 0 2024-04-26T04:42:26.515 INFO:tasks.workunit.client.0.smithi090.stdout:8/808: creat dc/dd2/d3b/d3d/d111/f118 x:0 0 0 2024-04-26T04:42:26.517 INFO:tasks.workunit.client.1.smithi104.stdout:9/805: creat d11/d53/f108 x:0 0 0 2024-04-26T04:42:26.517 INFO:tasks.workunit.client.0.smithi090.stdout:0/716: mknod d0/de/cf7 0 2024-04-26T04:42:26.517 INFO:tasks.workunit.client.0.smithi090.stdout:0/717: write d0/d4b/d6d/df2/fe6 [29709,9542] 0 2024-04-26T04:42:26.517 INFO:tasks.workunit.client.0.smithi090.stdout:0/718: chown d0/d4b/d5d/d5c/d8a/d45/d53/fba 14165031 1 2024-04-26T04:42:26.519 INFO:tasks.workunit.client.0.smithi090.stdout:4/647: write d3/db/d12/d20/d24/f52 [2185041,128978] 0 2024-04-26T04:42:26.520 INFO:tasks.workunit.client.1.smithi104.stdout:7/942: rmdir d2/d4/d32/d60 39 2024-04-26T04:42:26.520 INFO:tasks.workunit.client.1.smithi104.stdout:5/993: read d0/d41/f74 [4052098,20121] 0 2024-04-26T04:42:26.521 INFO:tasks.workunit.client.1.smithi104.stdout:0/955: dread d0/f91 [0,4194304] 0 2024-04-26T04:42:26.522 INFO:tasks.workunit.client.0.smithi090.stdout:8/809: mkdir dc/dd2/d4f/d58/d95/d106/d119 0 2024-04-26T04:42:26.522 INFO:tasks.workunit.client.0.smithi090.stdout:8/810: fsync dc/d5a/d5d/d73/dce/ff2 0 2024-04-26T04:42:26.522 INFO:tasks.workunit.client.1.smithi104.stdout:7/943: unlink d2/cc5 0 2024-04-26T04:42:26.522 INFO:tasks.workunit.client.1.smithi104.stdout:7/944: chown d2/d4/l25 15 1 2024-04-26T04:42:26.523 INFO:tasks.workunit.client.0.smithi090.stdout:8/811: dread dc/dd2/d39/f81 [0,4194304] 0 2024-04-26T04:42:26.523 INFO:tasks.workunit.client.0.smithi090.stdout:0/719: creat d0/d4b/ff8 x:0 0 0 2024-04-26T04:42:26.525 INFO:tasks.workunit.client.1.smithi104.stdout:7/945: rename d2/d4/d1d/d34/db0/fd5 to d2/d4/d32/d60/f12e 0 2024-04-26T04:42:26.526 INFO:tasks.workunit.client.0.smithi090.stdout:2/674: dwrite db/d59/f62 [0,4194304] 0 2024-04-26T04:42:26.526 INFO:tasks.workunit.client.0.smithi090.stdout:2/675: creat db/d11/d14/d1b/d20/fe0 x:0 0 0 2024-04-26T04:42:26.527 INFO:tasks.workunit.client.0.smithi090.stdout:4/648: mkdir d3/d13/d6c/dc8 0 2024-04-26T04:42:26.528 INFO:tasks.workunit.client.1.smithi104.stdout:7/946: rename d2/d4/d32/d3d/d41/d5f/d6d/fa3 to d2/f12f 0 2024-04-26T04:42:26.528 INFO:tasks.workunit.client.0.smithi090.stdout:8/812: link dc/dd2/d39/c10c dc/dd2/d3b/d3d/d97/dee/c11a 0 2024-04-26T04:42:26.529 INFO:tasks.workunit.client.0.smithi090.stdout:8/813: fsync dc/dd2/d4f/f7f 0 2024-04-26T04:42:26.533 INFO:tasks.workunit.client.1.smithi104.stdout:5/994: dread d0/d1/d4/d2a/d6b/d91/dbd/ffa [0,4194304] 0 2024-04-26T04:42:26.533 INFO:tasks.workunit.client.0.smithi090.stdout:4/649: mkdir d3/d13/d64/dae/dc9 0 2024-04-26T04:42:26.534 INFO:tasks.workunit.client.1.smithi104.stdout:9/806: dwrite d11/d15/f59 [0,4194304] 0 2024-04-26T04:42:26.536 INFO:tasks.workunit.client.1.smithi104.stdout:7/947: fsync d2/d4/d32/d60/f12e 0 2024-04-26T04:42:26.537 INFO:tasks.workunit.client.0.smithi090.stdout:8/814: rename dc/d71/c82 to dc/d34/c11b 0 2024-04-26T04:42:26.537 INFO:tasks.workunit.client.0.smithi090.stdout:8/815: write dc/dd2/d4f/f96 [94546,66482] 0 2024-04-26T04:42:26.541 INFO:tasks.workunit.client.1.smithi104.stdout:0/956: dread d0/d4/ddb/f112 [0,4194304] 0 2024-04-26T04:42:26.541 INFO:tasks.workunit.client.1.smithi104.stdout:0/957: fsync d0/d2/d10/fcd 0 2024-04-26T04:42:26.542 INFO:tasks.workunit.client.1.smithi104.stdout:0/958: truncate d0/d2/d21/d27/d1e/d57/d6c/dd1/f6b 7227757 0 2024-04-26T04:42:26.542 INFO:tasks.workunit.client.1.smithi104.stdout:5/995: dread d0/d1/d4/d90/d109/fea [0,4194304] 0 2024-04-26T04:42:26.543 INFO:tasks.workunit.client.1.smithi104.stdout:0/959: symlink d0/d2/d21/d27/d1e/l138 0 2024-04-26T04:42:26.544 INFO:tasks.workunit.client.1.smithi104.stdout:7/948: dread d2/d4/d32/d60/f8c [0,4194304] 0 2024-04-26T04:42:26.546 INFO:tasks.workunit.client.1.smithi104.stdout:5/996: creat d0/d1/d64/d26/ddc/f14e x:0 0 0 2024-04-26T04:42:26.546 INFO:tasks.workunit.client.1.smithi104.stdout:5/997: write d0/f4e [4697781,39084] 0 2024-04-26T04:42:26.546 INFO:tasks.workunit.client.1.smithi104.stdout:5/998: write d0/f88 [339612,9020] 0 2024-04-26T04:42:26.547 INFO:tasks.workunit.client.1.smithi104.stdout:7/949: symlink d2/d4/d32/d3d/d47/db6/dce/d111/l130 0 2024-04-26T04:42:26.548 INFO:tasks.workunit.client.1.smithi104.stdout:9/807: read d11/d15/d1b/f37 [36018,108842] 0 2024-04-26T04:42:26.548 INFO:tasks.workunit.client.0.smithi090.stdout:2/676: dwrite db/d11/d14/d25/d2d/f91 [0,4194304] 0 2024-04-26T04:42:26.549 INFO:tasks.workunit.client.1.smithi104.stdout:7/950: creat d2/d4/d32/d3d/d47/f131 x:0 0 0 2024-04-26T04:42:26.549 INFO:tasks.workunit.client.1.smithi104.stdout:5/999: mkdir d0/d1/d64/d26/d36/d14f 0 2024-04-26T04:42:26.550 INFO:tasks.workunit.client.1.smithi104.stdout:9/808: mknod d11/d15/d32/d4c/d54/ddc/c109 0 2024-04-26T04:42:26.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/677: creat db/d11/d14/d27/fe1 x:0 0 0 2024-04-26T04:42:26.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/678: creat db/d11/d14/d25/d2d/fe2 x:0 0 0 2024-04-26T04:42:26.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/679: fsync db/f84 0 2024-04-26T04:42:26.551 INFO:tasks.workunit.client.0.smithi090.stdout:2/680: fdatasync db/d11/d14/d25/d2d/d48/fbf 0 2024-04-26T04:42:26.552 INFO:tasks.workunit.client.1.smithi104.stdout:7/951: mkdir d2/d4/d5a/da0/d132 0 2024-04-26T04:42:26.552 INFO:tasks.workunit.client.0.smithi090.stdout:4/650: dread d3/f7d [0,4194304] 0 2024-04-26T04:42:26.553 INFO:tasks.workunit.client.1.smithi104.stdout:9/809: rename d11/d1d/fbd to d11/d15/d1b/f10a 0 2024-04-26T04:42:26.554 INFO:tasks.workunit.client.0.smithi090.stdout:4/651: truncate d3/db/d12/f9a 1795003 0 2024-04-26T04:42:26.554 INFO:tasks.workunit.client.0.smithi090.stdout:4/652: write d3/db/d12/d20/d2d/d58/d6a/f8b [295217,49624] 0 2024-04-26T04:42:26.554 INFO:tasks.workunit.client.0.smithi090.stdout:4/653: getdents d3/d13/d64/dc1 0 2024-04-26T04:42:26.555 INFO:tasks.workunit.client.1.smithi104.stdout:9/810: unlink d11/d1d/d34/c76 0 2024-04-26T04:42:26.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/563: sync 2024-04-26T04:42:26.561 INFO:tasks.workunit.client.0.smithi090.stdout:1/607: sync 2024-04-26T04:42:26.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/564: chown db/d16/f9a 1585382 1 2024-04-26T04:42:26.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/565: chown db/dd/d18 1800730828 1 2024-04-26T04:42:26.561 INFO:tasks.workunit.client.0.smithi090.stdout:9/566: write db/dd/ff [4644997,116316] 0 2024-04-26T04:42:26.564 INFO:tasks.workunit.client.1.smithi104.stdout:0/960: dwrite d0/d2/d21/d27/faa [0,4194304] 0 2024-04-26T04:42:26.564 INFO:tasks.workunit.client.0.smithi090.stdout:4/654: dread d3/db/d12/d20/d45/f96 [0,4194304] 0 2024-04-26T04:42:26.564 INFO:tasks.workunit.client.0.smithi090.stdout:1/608: link d0/d4/d7/d17/f29 d0/d4/d7/d4a/d75/d80/d58/fbf 0 2024-04-26T04:42:26.565 INFO:tasks.workunit.client.0.smithi090.stdout:1/609: chown d0 14755102 1 2024-04-26T04:42:26.566 INFO:tasks.workunit.client.0.smithi090.stdout:9/567: symlink db/dd/d18/d95/lc7 0 2024-04-26T04:42:26.566 INFO:tasks.workunit.client.0.smithi090.stdout:9/568: fdatasync db/dd/d44/d82/d85/db4/d6a/d70/f96 0 2024-04-26T04:42:26.570 INFO:tasks.workunit.client.0.smithi090.stdout:4/655: rename d3/d13/d64/f79 to d3/d13/d64/dae/da5/fca 0 2024-04-26T04:42:26.570 INFO:tasks.workunit.client.1.smithi104.stdout:0/961: dread d0/d2/d21/fb1 [0,4194304] 0 2024-04-26T04:42:26.571 INFO:tasks.workunit.client.0.smithi090.stdout:9/569: creat db/dd/d44/d82/d85/db4/fc8 x:0 0 0 2024-04-26T04:42:26.573 INFO:tasks.workunit.client.1.smithi104.stdout:0/962: truncate d0/d2/d21/d27/d1e/ffa 5089099 0 2024-04-26T04:42:26.573 INFO:tasks.workunit.client.1.smithi104.stdout:0/963: write d0/d2/d21/d49/d5e/d9c/f120 [1030875,93050] 0 2024-04-26T04:42:26.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/656: mkdir d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb 0 2024-04-26T04:42:26.577 INFO:tasks.workunit.client.0.smithi090.stdout:9/570: creat db/dd/d44/d82/d85/db4/d6a/d70/fc9 x:0 0 0 2024-04-26T04:42:26.579 INFO:tasks.workunit.client.0.smithi090.stdout:9/571: stat db/dd/d44/l30 0 2024-04-26T04:42:26.579 INFO:tasks.workunit.client.0.smithi090.stdout:9/572: fsync db/dd/d18/d38/d39/fc4 0 2024-04-26T04:42:26.610 INFO:tasks.workunit.client.1.smithi104.stdout:7/952: dwrite d2/ff1 [0,4194304] 0 2024-04-26T04:42:26.610 INFO:tasks.workunit.client.1.smithi104.stdout:9/811: dwrite d11/d15/d1b/fa4 [0,4194304] 0 2024-04-26T04:42:26.610 INFO:tasks.workunit.client.1.smithi104.stdout:9/812: creat d11/d15/d32/d4c/d54/f10b x:0 0 0 2024-04-26T04:42:26.610 INFO:tasks.workunit.client.1.smithi104.stdout:9/813: creat d11/d15/d98/f10c x:0 0 0 2024-04-26T04:42:26.612 INFO:tasks.workunit.client.0.smithi090.stdout:1/610: dwrite d0/d4/d7/d17/d22/d31/f34 [0,4194304] 0 2024-04-26T04:42:26.613 INFO:tasks.workunit.client.1.smithi104.stdout:7/953: creat d2/d4/d1d/d34/db0/f133 x:0 0 0 2024-04-26T04:42:26.613 INFO:tasks.workunit.client.0.smithi090.stdout:4/657: dwrite d3/db/d12/d20/d24/f32 [0,4194304] 0 2024-04-26T04:42:26.614 INFO:tasks.workunit.client.0.smithi090.stdout:4/658: dread - d3/db/d12/d20/d24/d43/d72/f8a zero size 2024-04-26T04:42:26.614 INFO:tasks.workunit.client.0.smithi090.stdout:9/573: dwrite db/d16/f5a [4194304,4194304] 0 2024-04-26T04:42:26.614 INFO:tasks.workunit.client.0.smithi090.stdout:9/574: chown db/dd/d44/d82/d85/db4/d99 54270 1 2024-04-26T04:42:26.614 INFO:tasks.workunit.client.0.smithi090.stdout:4/659: mknod d3/db/d60/ccc 0 2024-04-26T04:42:26.615 INFO:tasks.workunit.client.0.smithi090.stdout:1/611: dread d0/f86 [4194304,4194304] 0 2024-04-26T04:42:26.615 INFO:tasks.workunit.client.0.smithi090.stdout:9/575: mknod db/dd/d18/d57/cca 0 2024-04-26T04:42:26.617 INFO:tasks.workunit.client.0.smithi090.stdout:4/660: creat d3/d13/d1d/d50/d51/fcd x:0 0 0 2024-04-26T04:42:26.618 INFO:tasks.workunit.client.0.smithi090.stdout:9/576: mknod db/dd/d44/d82/d85/db4/d58/ccb 0 2024-04-26T04:42:26.619 INFO:tasks.workunit.client.0.smithi090.stdout:9/577: mkdir db/dd/d18/d57/dcc 0 2024-04-26T04:42:26.620 INFO:tasks.workunit.client.0.smithi090.stdout:9/578: mkdir db/dd/d44/d82/d85/db4/d99/db2/dcd 0 2024-04-26T04:42:26.630 INFO:tasks.workunit.client.1.smithi104.stdout:7/954: truncate d2/d4/d32/d35/d44/d5b/fd4 4308 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/579: rmdir db/dd/d44/d82/d85/db4/d6a/d70 39 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/580: chown db/f13 945856 1 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/581: stat db/laa 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/582: fdatasync db/dd/d44/d82/d85/db4/d58/f62 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:9/583: stat db/c10 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/612: dread d0/d4/d7/d17/f21 [0,4194304] 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/613: mknod d0/d4/d7/d4a/d75/cc0 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/614: fdatasync d0/d4/d7/d17/f3c 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/615: creat d0/d4/d7/d17/d22/d31/fc1 x:0 0 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/616: fdatasync d0/d4/d7/d17/d22/d31/f6c 0 2024-04-26T04:42:26.631 INFO:tasks.workunit.client.0.smithi090.stdout:4/661: dread d3/d13/d64/dae/da5/fca [0,4194304] 0 2024-04-26T04:42:26.633 INFO:tasks.workunit.client.1.smithi104.stdout:7/955: dread d2/d4/d32/d35/d37/f6f [0,4194304] 0 2024-04-26T04:42:26.633 INFO:tasks.workunit.client.1.smithi104.stdout:7/956: dread - d2/d4/d1d/d34/db0/f133 zero size 2024-04-26T04:42:26.636 INFO:tasks.workunit.client.0.smithi090.stdout:9/584: dread db/dd/d18/f3a [0,4194304] 0 2024-04-26T04:42:26.639 INFO:tasks.workunit.client.0.smithi090.stdout:9/585: rename db/cae to db/dd/d18/d38/d39/d6b/dc0/cce 0 2024-04-26T04:42:26.644 INFO:tasks.workunit.client.0.smithi090.stdout:9/586: dread db/d16/f33 [0,4194304] 0 2024-04-26T04:42:26.650 INFO:tasks.workunit.client.0.smithi090.stdout:9/587: dread db/f47 [0,4194304] 0 2024-04-26T04:42:26.650 INFO:tasks.workunit.client.0.smithi090.stdout:9/588: stat db/dd/l24 0 2024-04-26T04:42:26.654 INFO:tasks.workunit.client.0.smithi090.stdout:1/617: dwrite d0/d4/d7/d17/d22/f74 [0,4194304] 0 2024-04-26T04:42:26.655 INFO:tasks.workunit.client.0.smithi090.stdout:4/662: dwrite d3/d13/f87 [0,4194304] 0 2024-04-26T04:42:26.656 INFO:tasks.workunit.client.0.smithi090.stdout:1/618: symlink d0/d4/d7/d17/d22/d93/db8/lc2 0 2024-04-26T04:42:26.657 INFO:tasks.workunit.client.0.smithi090.stdout:1/619: creat d0/fc3 x:0 0 0 2024-04-26T04:42:26.660 INFO:tasks.workunit.client.0.smithi090.stdout:9/589: dread db/f41 [0,4194304] 0 2024-04-26T04:42:26.663 INFO:tasks.workunit.client.0.smithi090.stdout:9/590: dread db/dd/d44/f34 [0,4194304] 0 2024-04-26T04:42:26.663 INFO:tasks.workunit.client.0.smithi090.stdout:9/591: chown db/dd/d18/d38/d39/c7d 2 1 2024-04-26T04:42:26.663 INFO:tasks.workunit.client.0.smithi090.stdout:9/592: chown db/dd/d18/d38/d39/d6b/dc0/lb1 112 1 2024-04-26T04:42:26.663 INFO:tasks.workunit.client.0.smithi090.stdout:9/593: creat db/dd/d44/d82/d85/db4/d99/fcf x:0 0 0 2024-04-26T04:42:26.664 INFO:tasks.workunit.client.0.smithi090.stdout:9/594: stat db/dd/d18/d38/d39/d3f 0 2024-04-26T04:42:26.664 INFO:tasks.workunit.client.0.smithi090.stdout:9/595: truncate db/dd/d44/d82/d85/db4/d58/f6c 1109985 0 2024-04-26T04:42:26.664 INFO:tasks.workunit.client.0.smithi090.stdout:9/596: dread - db/dd/d44/d82/d85/db4/d58/f62 zero size 2024-04-26T04:42:26.673 INFO:tasks.workunit.client.0.smithi090.stdout:1/620: dwrite d0/d4/d7/d17/f21 [0,4194304] 0 2024-04-26T04:42:26.677 INFO:tasks.workunit.client.0.smithi090.stdout:1/621: dread d0/d4/d7/d4a/d75/d80/d58/fbf [0,4194304] 0 2024-04-26T04:42:26.678 INFO:tasks.workunit.client.0.smithi090.stdout:1/622: mknod d0/d4/d7/d49/cc4 0 2024-04-26T04:42:26.684 INFO:tasks.workunit.client.1.smithi104.stdout:7/957: symlink d2/d4/d5a/da0/l134 0 2024-04-26T04:42:26.684 INFO:tasks.workunit.client.1.smithi104.stdout:7/958: getdents d2/d4/d32/d3d/d41/d5f 0 2024-04-26T04:42:26.687 INFO:tasks.workunit.client.1.smithi104.stdout:9/814: rmdir d11/d15/d32/d4c/d54 39 2024-04-26T04:42:26.694 INFO:tasks.workunit.client.0.smithi090.stdout:1/623: dwrite d0/d4/fa4 [0,4194304] 0 2024-04-26T04:42:26.694 INFO:tasks.workunit.client.0.smithi090.stdout:1/624: dread - d0/d55/fb9 zero size 2024-04-26T04:42:26.698 INFO:tasks.workunit.client.0.smithi090.stdout:1/625: creat d0/d4/d43/fc5 x:0 0 0 2024-04-26T04:42:26.700 INFO:tasks.workunit.client.1.smithi104.stdout:9/815: dread d11/d1d/d34/db3/fee [0,4194304] 0 2024-04-26T04:42:26.700 INFO:tasks.workunit.client.1.smithi104.stdout:9/816: fsync d11/d1d/f49 0 2024-04-26T04:42:26.700 INFO:tasks.workunit.client.1.smithi104.stdout:8/894: sync 2024-04-26T04:42:26.700 INFO:tasks.workunit.client.1.smithi104.stdout:0/964: sync 2024-04-26T04:42:26.700 INFO:tasks.workunit.client.1.smithi104.stdout:8/895: fsync d0/d1c/f4c 0 2024-04-26T04:42:26.700 INFO:tasks.workunit.client.1.smithi104.stdout:9/817: stat d11/d15/d32/d35/l38 0 2024-04-26T04:42:26.702 INFO:tasks.workunit.client.1.smithi104.stdout:8/896: unlink d0/l19 0 2024-04-26T04:42:26.703 INFO:tasks.workunit.client.1.smithi104.stdout:0/965: symlink d0/d2/d21/d27/d1e/d57/l139 0 2024-04-26T04:42:26.705 INFO:tasks.workunit.client.1.smithi104.stdout:9/818: rename d11/d15/d1b/d6d/l7e to d11/d1d/d87/dda/l10d 0 2024-04-26T04:42:26.709 INFO:tasks.workunit.client.1.smithi104.stdout:9/819: mkdir d11/d90/dfd/d10e 0 2024-04-26T04:42:26.711 INFO:tasks.workunit.client.1.smithi104.stdout:8/897: write d0/d33/da6/fa8 [3305829,95483] 0 2024-04-26T04:42:26.712 INFO:tasks.workunit.client.0.smithi090.stdout:1/626: dwrite d0/d4/d7/d49/f5e [4194304,4194304] 0 2024-04-26T04:42:26.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/627: write d0/d4/d7/d49/f6f [1092184,113932] 0 2024-04-26T04:42:26.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/628: chown d0/d4/d7/d49/f5f 1 1 2024-04-26T04:42:26.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/629: stat d0/d4/d7/d17/f2a 0 2024-04-26T04:42:26.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/630: creat d0/d4/d7/d4a/d75/d80/fc6 x:0 0 0 2024-04-26T04:42:26.728 INFO:tasks.workunit.client.0.smithi090.stdout:1/631: creat d0/d4/d7/d4b/fc7 x:0 0 0 2024-04-26T04:42:26.738 INFO:tasks.workunit.client.1.smithi104.stdout:0/966: dwrite d0/d2/d10/d84/fb6 [0,4194304] 0 2024-04-26T04:42:26.738 INFO:tasks.workunit.client.1.smithi104.stdout:8/898: dwrite d0/d33/d35/f3e [0,4194304] 0 2024-04-26T04:42:26.738 INFO:tasks.workunit.client.1.smithi104.stdout:9/820: dwrite d11/d15/d32/d35/d7b/fde [4194304,4194304] 0 2024-04-26T04:42:26.738 INFO:tasks.workunit.client.1.smithi104.stdout:9/821: dread - d11/d15/d98/ff5 zero size 2024-04-26T04:42:26.739 INFO:tasks.workunit.client.1.smithi104.stdout:0/967: mknod d0/d2/d10/d84/c13a 0 2024-04-26T04:42:26.740 INFO:tasks.workunit.client.1.smithi104.stdout:8/899: rename d0/d33/d51/f92 to d0/d1c/d27/d47/d67/d76/d7f/f124 0 2024-04-26T04:42:26.741 INFO:tasks.workunit.client.1.smithi104.stdout:9/822: stat d11/d1d/d71/d94/cc8 0 2024-04-26T04:42:26.741 INFO:tasks.workunit.client.1.smithi104.stdout:9/823: creat d11/d15/d32/d4c/d7c/f10f x:0 0 0 2024-04-26T04:42:26.742 INFO:tasks.workunit.client.1.smithi104.stdout:0/968: mkdir d0/d4/ddb/d13b 0 2024-04-26T04:42:26.743 INFO:tasks.workunit.client.1.smithi104.stdout:8/900: symlink d0/d1c/d27/d47/d67/l125 0 2024-04-26T04:42:26.743 INFO:tasks.workunit.client.1.smithi104.stdout:8/901: readlink d0/d1c/d27/l5a 0 2024-04-26T04:42:26.744 INFO:tasks.workunit.client.1.smithi104.stdout:9/824: mkdir d11/d1d/d34/db3/df1/d110 0 2024-04-26T04:42:26.745 INFO:tasks.workunit.client.1.smithi104.stdout:0/969: symlink d0/d2/d21/d27/l13c 0 2024-04-26T04:42:26.746 INFO:tasks.workunit.client.1.smithi104.stdout:8/902: mkdir d0/d1c/d27/d47/d67/d126 0 2024-04-26T04:42:26.746 INFO:tasks.workunit.client.1.smithi104.stdout:9/825: mknod d11/d15/d32/d4c/c111 0 2024-04-26T04:42:26.747 INFO:tasks.workunit.client.1.smithi104.stdout:0/970: rename d0/d2/d21/d49/d5e/d9c/d53/d5d/d121/f12d to d0/d2/d21/d49/d5e/d9c/d53/d5d/d121/f13d 0 2024-04-26T04:42:26.747 INFO:tasks.workunit.client.1.smithi104.stdout:0/971: chown d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/fe1 0 1 2024-04-26T04:42:26.749 INFO:tasks.workunit.client.1.smithi104.stdout:9/826: symlink d11/d1d/d34/db3/df1/d110/l112 0 2024-04-26T04:42:26.768 INFO:tasks.workunit.client.1.smithi104.stdout:0/972: dwrite d0/d2/d21/d27/f7b [0,4194304] 0 2024-04-26T04:42:26.778 INFO:tasks.workunit.client.1.smithi104.stdout:9/827: dwrite d11/d15/d32/dd6/ded/ffe [0,4194304] 0 2024-04-26T04:42:26.778 INFO:tasks.workunit.client.1.smithi104.stdout:9/828: getdents d11/d15/d1b/d6d/d9c/dbe 0 2024-04-26T04:42:26.811 INFO:tasks.workunit.client.0.smithi090.stdout:5/603: sync 2024-04-26T04:42:26.812 INFO:tasks.workunit.client.0.smithi090.stdout:3/594: sync 2024-04-26T04:42:26.812 INFO:tasks.workunit.client.0.smithi090.stdout:7/688: sync 2024-04-26T04:42:26.812 INFO:tasks.workunit.client.0.smithi090.stdout:7/689: dread - d0/db/df/d4d/d64/f92 zero size 2024-04-26T04:42:26.816 INFO:tasks.workunit.client.1.smithi104.stdout:8/903: rename d0/d1c/d27 to d0/d1c/d59/d85/d127 0 2024-04-26T04:42:26.817 INFO:tasks.workunit.client.1.smithi104.stdout:0/973: unlink d0/d2/d21/d27/c44 0 2024-04-26T04:42:26.819 INFO:tasks.workunit.client.0.smithi090.stdout:3/595: rename d1/d17/d36/d33/d43/db0 to d1/d17/d36/d33/d68/dae/dca 0 2024-04-26T04:42:26.821 INFO:tasks.workunit.client.1.smithi104.stdout:9/829: symlink d11/d15/d32/d35/l113 0 2024-04-26T04:42:26.822 INFO:tasks.workunit.client.0.smithi090.stdout:3/596: creat d1/d17/d61/d9d/da7/fcb x:0 0 0 2024-04-26T04:42:26.824 INFO:tasks.workunit.client.1.smithi104.stdout:8/904: mkdir d0/d33/d35/d6d/da4/d128 0 2024-04-26T04:42:26.824 INFO:tasks.workunit.client.1.smithi104.stdout:8/905: write d0/d1c/d2d/f48 [1987194,129503] 0 2024-04-26T04:42:26.826 INFO:tasks.workunit.client.1.smithi104.stdout:8/906: mkdir d0/d9a/dce/d129 0 2024-04-26T04:42:26.826 INFO:tasks.workunit.client.1.smithi104.stdout:8/907: write d0/d1c/d2d/f48 [3672834,16794] 0 2024-04-26T04:42:26.829 INFO:tasks.workunit.client.1.smithi104.stdout:8/908: creat d0/d9a/dce/d129/f12a x:0 0 0 2024-04-26T04:42:26.830 INFO:tasks.workunit.client.0.smithi090.stdout:5/604: write d4/d1a/d45/f3c [3552426,27750] 0 2024-04-26T04:42:26.843 INFO:tasks.workunit.client.0.smithi090.stdout:7/690: dwrite d0/d1/f61 [0,4194304] 0 2024-04-26T04:42:26.845 INFO:tasks.workunit.client.0.smithi090.stdout:7/691: dread d0/f3c [0,4194304] 0 2024-04-26T04:42:26.857 INFO:tasks.workunit.client.1.smithi104.stdout:7/959: sync 2024-04-26T04:42:26.860 INFO:tasks.workunit.client.1.smithi104.stdout:7/960: rename d2/d4/d32/d3d/d47/fbc to d2/d4/d32/d35/dbe/dda/f135 0 2024-04-26T04:42:26.864 INFO:tasks.workunit.client.1.smithi104.stdout:7/961: dread d2/d4/d1d/fa7 [0,4194304] 0 2024-04-26T04:42:26.864 INFO:tasks.workunit.client.1.smithi104.stdout:7/962: read d2/d4/d32/d35/d37/f6f [2964353,6598] 0 2024-04-26T04:42:26.865 INFO:tasks.workunit.client.1.smithi104.stdout:7/963: mkdir d2/d4/d32/d3d/d41/d136 0 2024-04-26T04:42:26.866 INFO:tasks.workunit.client.0.smithi090.stdout:3/597: dwrite d1/d17/d36/f32 [4194304,4194304] 0 2024-04-26T04:42:26.867 INFO:tasks.workunit.client.0.smithi090.stdout:3/598: write d1/d17/d36/d33/d68/d71/f9e [753974,99478] 0 2024-04-26T04:42:26.868 INFO:tasks.workunit.client.1.smithi104.stdout:7/964: creat d2/d4/d32/d60/dff/f137 x:0 0 0 2024-04-26T04:42:26.868 INFO:tasks.workunit.client.1.smithi104.stdout:0/974: dwrite d0/d2/d10/d3d/db7/fd2 [4194304,4194304] 0 2024-04-26T04:42:26.868 INFO:tasks.workunit.client.1.smithi104.stdout:0/975: stat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80 0 2024-04-26T04:42:26.868 INFO:tasks.workunit.client.1.smithi104.stdout:7/965: mknod d2/d4/d32/d3d/d41/d5f/d6d/c138 0 2024-04-26T04:42:26.870 INFO:tasks.workunit.client.1.smithi104.stdout:7/966: rename d2/d4/d32/d35/d44/f91 to d2/d4/d32/d3d/d41/d5f/d6c/d89/f139 0 2024-04-26T04:42:26.870 INFO:tasks.workunit.client.0.smithi090.stdout:5/605: write d4/d1a/df/d36/f4d [5736211,105596] 0 2024-04-26T04:42:26.870 INFO:tasks.workunit.client.0.smithi090.stdout:5/606: read d4/d1a/d1f/f2b [5401635,56649] 0 2024-04-26T04:42:26.871 INFO:tasks.workunit.client.1.smithi104.stdout:7/967: read d2/d4/d32/d3d/d41/d5f/d6d/fe6 [128534,9627] 0 2024-04-26T04:42:26.871 INFO:tasks.workunit.client.1.smithi104.stdout:7/968: symlink d2/d4/d32/d3d/d41/d5f/d6c/d89/df5/l13a 0 2024-04-26T04:42:26.875 INFO:tasks.workunit.client.1.smithi104.stdout:9/830: dwrite d11/d15/d32/d4c/d54/f77 [0,4194304] 0 2024-04-26T04:42:26.875 INFO:tasks.workunit.client.1.smithi104.stdout:9/831: chown d11/d15/d1b/d6d/f96 10 1 2024-04-26T04:42:26.875 INFO:tasks.workunit.client.1.smithi104.stdout:9/832: chown d11/d15/d1b/d6d/fc0 1 1 2024-04-26T04:42:26.875 INFO:tasks.workunit.client.0.smithi090.stdout:7/692: dwrite d0/d1/d4/d98/d65/db2/fb6 [0,4194304] 0 2024-04-26T04:42:26.876 INFO:tasks.workunit.client.1.smithi104.stdout:9/833: creat d11/d15/da3/f114 x:0 0 0 2024-04-26T04:42:26.876 INFO:tasks.workunit.client.1.smithi104.stdout:9/834: truncate d11/d53/fb6 1831825 0 2024-04-26T04:42:26.877 INFO:tasks.workunit.client.1.smithi104.stdout:8/909: dwrite d0/d9a/dce/d129/f12a [0,4194304] 0 2024-04-26T04:42:26.878 INFO:tasks.workunit.client.0.smithi090.stdout:7/693: link d0/db/de6/cfb d0/cfd 0 2024-04-26T04:42:26.878 INFO:tasks.workunit.client.0.smithi090.stdout:7/694: fdatasync d0/d1/d4/f2e 0 2024-04-26T04:42:26.878 INFO:tasks.workunit.client.0.smithi090.stdout:7/695: stat d0/d1/d4/f10 0 2024-04-26T04:42:26.878 INFO:tasks.workunit.client.0.smithi090.stdout:7/696: chown d0/db/d15/d29/f44 3806 1 2024-04-26T04:42:26.880 INFO:tasks.workunit.client.1.smithi104.stdout:0/976: write d0/d2/d21/d27/d1e/d57/d6c/d85/faf [2374973,24708] 0 2024-04-26T04:42:26.880 INFO:tasks.workunit.client.0.smithi090.stdout:3/599: dread d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:26.882 INFO:tasks.workunit.client.0.smithi090.stdout:7/697: symlink d0/d1/d4/lfe 0 2024-04-26T04:42:26.882 INFO:tasks.workunit.client.0.smithi090.stdout:7/698: write d0/db/d53/f87 [219628,126650] 0 2024-04-26T04:42:26.883 INFO:tasks.workunit.client.0.smithi090.stdout:7/699: symlink d0/db/df/d4d/d64/d66/lff 0 2024-04-26T04:42:26.883 INFO:tasks.workunit.client.0.smithi090.stdout:7/700: stat d0/db/d15/d29/df3 0 2024-04-26T04:42:26.884 INFO:tasks.workunit.client.0.smithi090.stdout:7/701: mkdir d0/db/df/d1c/d100 0 2024-04-26T04:42:26.884 INFO:tasks.workunit.client.0.smithi090.stdout:7/702: write d0/d1/d4/d98/d4a/fbe [3039860,129035] 0 2024-04-26T04:42:26.886 INFO:tasks.workunit.client.0.smithi090.stdout:7/703: rmdir d0/db/d53/db7 39 2024-04-26T04:42:26.887 INFO:tasks.workunit.client.1.smithi104.stdout:7/969: rmdir d2/d4 39 2024-04-26T04:42:26.887 INFO:tasks.workunit.client.1.smithi104.stdout:9/835: dread d11/d1d/d34/d69/f79 [0,4194304] 0 2024-04-26T04:42:26.887 INFO:tasks.workunit.client.1.smithi104.stdout:9/836: truncate d11/d53/f108 831265 0 2024-04-26T04:42:26.887 INFO:tasks.workunit.client.1.smithi104.stdout:9/837: fsync d11/d1d/d34/f3c 0 2024-04-26T04:42:26.888 INFO:tasks.workunit.client.0.smithi090.stdout:5/607: dwrite d4/fe [0,4194304] 0 2024-04-26T04:42:26.889 INFO:tasks.workunit.client.0.smithi090.stdout:3/600: write d1/d17/d36/d33/d68/dae/dca/f91 [4103191,39945] 0 2024-04-26T04:42:26.890 INFO:tasks.workunit.client.1.smithi104.stdout:8/910: creat d0/d1c/d59/d85/d127/d11e/f12b x:0 0 0 2024-04-26T04:42:26.890 INFO:tasks.workunit.client.1.smithi104.stdout:8/911: chown d0/d1c/d59/d85/d127/d47/f100 8741495 1 2024-04-26T04:42:26.890 INFO:tasks.workunit.client.1.smithi104.stdout:8/912: chown d0/d1c/d59/d85/d127/d11e 327351 1 2024-04-26T04:42:26.890 INFO:tasks.workunit.client.1.smithi104.stdout:0/977: mkdir d0/d2/d10/d3d/dc0/df1/d13e 0 2024-04-26T04:42:26.891 INFO:tasks.workunit.client.0.smithi090.stdout:3/601: mknod d1/d17/d61/d9d/ccc 0 2024-04-26T04:42:26.891 INFO:tasks.workunit.client.0.smithi090.stdout:3/602: chown d1/d17/d36/d33/d68/dae/fba 1363282377 1 2024-04-26T04:42:26.891 INFO:tasks.workunit.client.0.smithi090.stdout:3/603: readlink d1/d17/d36/d33/daa/lc6 0 2024-04-26T04:42:26.891 INFO:tasks.workunit.client.0.smithi090.stdout:3/604: link d1/d17/d36/d33/d68/dae/dca/d1b/c96 d1/d17/d61/d9d/da7/ccd 0 2024-04-26T04:42:26.892 INFO:tasks.workunit.client.0.smithi090.stdout:3/605: mkdir d1/d17/d61/dce 0 2024-04-26T04:42:26.892 INFO:tasks.workunit.client.1.smithi104.stdout:7/970: mknod d2/d4/d1d/d121/c13b 0 2024-04-26T04:42:26.893 INFO:tasks.workunit.client.0.smithi090.stdout:3/606: rmdir d1/d17/d36/d33/d68/dae/dbb 0 2024-04-26T04:42:26.894 INFO:tasks.workunit.client.1.smithi104.stdout:9/838: mkdir d11/d15/d32/d35/d115 0 2024-04-26T04:42:26.895 INFO:tasks.workunit.client.1.smithi104.stdout:8/913: creat d0/d1c/d2d/f12c x:0 0 0 2024-04-26T04:42:26.898 INFO:tasks.workunit.client.1.smithi104.stdout:0/978: mkdir d0/d2/d21/d49/d5e/d9c/d53/d5d/d13f 0 2024-04-26T04:42:26.899 INFO:tasks.workunit.client.0.smithi090.stdout:3/607: dread d1/d17/d36/d33/f80 [0,4194304] 0 2024-04-26T04:42:26.900 INFO:tasks.workunit.client.1.smithi104.stdout:9/839: mkdir d11/d15/d1b/d6d/d9c/d116 0 2024-04-26T04:42:26.900 INFO:tasks.workunit.client.0.smithi090.stdout:5/608: dread d4/d1a/f78 [0,4194304] 0 2024-04-26T04:42:26.902 INFO:tasks.workunit.client.0.smithi090.stdout:5/609: rmdir d4/d1a/d2a/d31/db1 0 2024-04-26T04:42:26.906 INFO:tasks.workunit.client.1.smithi104.stdout:9/840: dread d11/d1d/d34/d69/f78 [0,4194304] 0 2024-04-26T04:42:26.908 INFO:tasks.workunit.client.1.smithi104.stdout:9/841: mknod d11/d1d/d71/d94/c117 0 2024-04-26T04:42:26.908 INFO:tasks.workunit.client.1.smithi104.stdout:9/842: stat d11/d15/d32/dd6 0 2024-04-26T04:42:26.910 INFO:tasks.workunit.client.1.smithi104.stdout:9/843: mknod d11/d15/c118 0 2024-04-26T04:42:26.912 INFO:tasks.workunit.client.1.smithi104.stdout:9/844: mkdir d11/d90/dfd/d10e/d119 0 2024-04-26T04:42:26.912 INFO:tasks.workunit.client.1.smithi104.stdout:9/845: chown d11/d1d/d34/ldf 418458 1 2024-04-26T04:42:26.913 INFO:tasks.workunit.client.0.smithi090.stdout:3/608: dread d1/d17/d36/dab/fb7 [0,4194304] 0 2024-04-26T04:42:26.913 INFO:tasks.workunit.client.0.smithi090.stdout:7/704: dwrite d0/d1/d4/d98/d65/db2/fba [0,4194304] 0 2024-04-26T04:42:26.918 INFO:tasks.workunit.client.0.smithi090.stdout:3/609: unlink d1/d17/d36/d2a/d3a/d5d/c7b 0 2024-04-26T04:42:26.918 INFO:tasks.workunit.client.0.smithi090.stdout:3/610: fsync d1/f27 0 2024-04-26T04:42:26.919 INFO:tasks.workunit.client.0.smithi090.stdout:3/611: dread - d1/d17/d36/dab/fc2 zero size 2024-04-26T04:42:26.919 INFO:tasks.workunit.client.0.smithi090.stdout:3/612: fsync d1/d17/d36/d2a/d3a/fbe 0 2024-04-26T04:42:26.919 INFO:tasks.workunit.client.0.smithi090.stdout:7/705: creat d0/d1/d4/d98/d65/f101 x:0 0 0 2024-04-26T04:42:26.919 INFO:tasks.workunit.client.0.smithi090.stdout:3/613: mkdir d1/d17/d36/d33/d68/dae/dca/dcf 0 2024-04-26T04:42:26.922 INFO:tasks.workunit.client.1.smithi104.stdout:9/846: dread d11/d15/d32/d4c/d54/f7a [0,4194304] 0 2024-04-26T04:42:26.924 INFO:tasks.workunit.client.1.smithi104.stdout:8/914: dread d0/d1c/d2d/f48 [0,4194304] 0 2024-04-26T04:42:26.924 INFO:tasks.workunit.client.1.smithi104.stdout:9/847: link d11/d15/d32/le2 d11/d15/d1b/d6d/d9c/dbe/def/l11a 0 2024-04-26T04:42:26.929 INFO:tasks.workunit.client.1.smithi104.stdout:7/971: dwrite d2/d4/d32/d3d/d41/d5f/d6c/d89/fdc [0,4194304] 0 2024-04-26T04:42:26.930 INFO:tasks.workunit.client.0.smithi090.stdout:5/610: dwrite d4/d1a/f17 [0,4194304] 0 2024-04-26T04:42:26.931 INFO:tasks.workunit.client.1.smithi104.stdout:0/979: dwrite d0/d2/d21/d27/d1e/d57/d6c/dd1/d7c/f12a [0,4194304] 0 2024-04-26T04:42:26.932 INFO:tasks.workunit.client.1.smithi104.stdout:7/972: creat d2/d4/d32/d3d/d41/f13c x:0 0 0 2024-04-26T04:42:26.933 INFO:tasks.workunit.client.0.smithi090.stdout:5/611: mkdir d4/d1a/df/d72/dc0 0 2024-04-26T04:42:26.933 INFO:tasks.workunit.client.0.smithi090.stdout:3/614: write d1/d59/dc9/f7c [1392489,594] 0 2024-04-26T04:42:26.933 INFO:tasks.workunit.client.0.smithi090.stdout:3/615: creat d1/d17/d36/d33/d68/d71/fd0 x:0 0 0 2024-04-26T04:42:26.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/612: rename d4/d1a/d45/ca9 to d4/d1a/d2a/cc1 0 2024-04-26T04:42:26.938 INFO:tasks.workunit.client.1.smithi104.stdout:9/848: read d11/d1d/d34/fe1 [5506940,68880] 0 2024-04-26T04:42:26.940 INFO:tasks.workunit.client.1.smithi104.stdout:9/849: mkdir d11/d1d/d87/d11b 0 2024-04-26T04:42:26.940 INFO:tasks.workunit.client.1.smithi104.stdout:9/850: write d11/d15/d32/d4c/d54/f62 [3347125,110777] 0 2024-04-26T04:42:26.942 INFO:tasks.workunit.client.1.smithi104.stdout:9/851: mknod d11/d1d/d34/c11c 0 2024-04-26T04:42:26.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/616: link d1/d17/d36/f6a d1/d17/d36/d2a/d3a/fd1 0 2024-04-26T04:42:26.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/617: chown d1/d9 16180 1 2024-04-26T04:42:26.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/618: creat d1/d17/d36/d33/d68/d76/fd2 x:0 0 0 2024-04-26T04:42:26.944 INFO:tasks.workunit.client.0.smithi090.stdout:5/613: truncate d4/d1a/d2a/d79/d90/d56/f6f 847807 0 2024-04-26T04:42:26.944 INFO:tasks.workunit.client.0.smithi090.stdout:5/614: chown d4/d1a/d1f/d53/d59/f8a 24130400 1 2024-04-26T04:42:26.944 INFO:tasks.workunit.client.0.smithi090.stdout:5/615: truncate d4/d1a/d1f/d53/d59/d5e/fa5 734502 0 2024-04-26T04:42:26.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/706: dwrite d0/db/df/d4d/fc1 [8388608,4194304] 0 2024-04-26T04:42:26.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/707: creat d0/db/df/d4d/d64/f102 x:0 0 0 2024-04-26T04:42:26.948 INFO:tasks.workunit.client.0.smithi090.stdout:3/619: truncate d1/d17/d36/d2a/d3a/fd1 283593 0 2024-04-26T04:42:26.948 INFO:tasks.workunit.client.1.smithi104.stdout:8/915: dwrite d0/d1c/d2d/f48 [4194304,4194304] 0 2024-04-26T04:42:26.948 INFO:tasks.workunit.client.1.smithi104.stdout:8/916: write d0/d1c/fd2 [4300043,38024] 0 2024-04-26T04:42:26.948 INFO:tasks.workunit.client.1.smithi104.stdout:8/917: write d0/d1c/d59/d85/d127/d47/d67/d76/d7f/f124 [55784,16541] 0 2024-04-26T04:42:26.948 INFO:tasks.workunit.client.1.smithi104.stdout:8/918: fdatasync d0/d33/da6/de0/fe2 0 2024-04-26T04:42:26.949 INFO:tasks.workunit.client.1.smithi104.stdout:0/980: write d0/d2/d21/fdf [4041149,101119] 0 2024-04-26T04:42:26.949 INFO:tasks.workunit.client.1.smithi104.stdout:0/981: write d0/d2/d21/d49/f55 [681352,49145] 0 2024-04-26T04:42:26.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/616: mknod d4/d1a/d1f/cc2 0 2024-04-26T04:42:26.953 INFO:tasks.workunit.client.1.smithi104.stdout:8/919: creat d0/d33/f12d x:0 0 0 2024-04-26T04:42:26.953 INFO:tasks.workunit.client.1.smithi104.stdout:8/920: dread - d0/d1c/d59/d85/d127/d47/dd5/f101 zero size 2024-04-26T04:42:26.956 INFO:tasks.workunit.client.0.smithi090.stdout:5/617: link d4/d1a/d45/c25 d4/cc3 0 2024-04-26T04:42:26.958 INFO:tasks.workunit.client.0.smithi090.stdout:5/618: getdents d4/d1a/d2a/d79/d90/d56/d71 0 2024-04-26T04:42:26.958 INFO:tasks.workunit.client.0.smithi090.stdout:5/619: chown d4/d1a 3403751 1 2024-04-26T04:42:26.958 INFO:tasks.workunit.client.0.smithi090.stdout:5/620: readlink d4/d1a/d2a/d79/l92 0 2024-04-26T04:42:26.958 INFO:tasks.workunit.client.0.smithi090.stdout:5/621: chown d4/d1a/d3f/f29 36487 1 2024-04-26T04:42:26.959 INFO:tasks.workunit.client.1.smithi104.stdout:0/982: rename d0/f10d to d0/d2/d21/d27/d1e/d57/d6c/d85/df4/f140 0 2024-04-26T04:42:26.961 INFO:tasks.workunit.client.1.smithi104.stdout:0/983: unlink d0/d2/d21/d49/ffb 0 2024-04-26T04:42:26.961 INFO:tasks.workunit.client.0.smithi090.stdout:5/622: dread d4/d1a/d2a/d79/d90/d56/f8b [0,4194304] 0 2024-04-26T04:42:26.963 INFO:tasks.workunit.client.0.smithi090.stdout:5/623: truncate d4/d1a/df/d36/f84 1013238 0 2024-04-26T04:42:26.963 INFO:tasks.workunit.client.0.smithi090.stdout:5/624: creat d4/d1a/d2a/d79/d90/fc4 x:0 0 0 2024-04-26T04:42:26.968 INFO:tasks.workunit.client.1.smithi104.stdout:8/921: dread d0/d33/d51/fa2 [0,4194304] 0 2024-04-26T04:42:26.969 INFO:tasks.workunit.client.1.smithi104.stdout:7/973: dwrite d2/d4/f33 [0,4194304] 0 2024-04-26T04:42:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:26 smithi090 ceph-mon[26648]: pgmap v255: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 301 MiB/s rd, 367 MiB/s wr, 551 op/s 2024-04-26T04:42:26.976 INFO:tasks.workunit.client.1.smithi104.stdout:9/852: dwrite d11/d15/d1b/f72 [4194304,4194304] 0 2024-04-26T04:42:26.978 INFO:tasks.workunit.client.1.smithi104.stdout:9/853: symlink d11/d90/dfd/l11d 0 2024-04-26T04:42:26.979 INFO:tasks.workunit.client.0.smithi090.stdout:7/708: dwrite d0/db/d53/fa [8388608,4194304] 0 2024-04-26T04:42:26.979 INFO:tasks.workunit.client.0.smithi090.stdout:7/709: stat d0/db/df/d4d/d64/d66/c9e 0 2024-04-26T04:42:26.981 INFO:tasks.workunit.client.1.smithi104.stdout:9/854: symlink d11/d1d/d34/db3/df1/l11e 0 2024-04-26T04:42:26.989 INFO:tasks.workunit.client.0.smithi090.stdout:6/723: sync 2024-04-26T04:42:26.989 INFO:tasks.workunit.client.0.smithi090.stdout:8/816: sync 2024-04-26T04:42:26.989 INFO:tasks.workunit.client.0.smithi090.stdout:0/720: sync 2024-04-26T04:42:26.989 INFO:tasks.workunit.client.0.smithi090.stdout:3/620: dwrite d1/d59/f6d [0,4194304] 0 2024-04-26T04:42:26.989 INFO:tasks.workunit.client.0.smithi090.stdout:3/621: creat d1/d17/d36/d2a/d3a/fd3 x:0 0 0 2024-04-26T04:42:26.989 INFO:tasks.workunit.client.0.smithi090.stdout:2/681: sync 2024-04-26T04:42:26.991 INFO:tasks.workunit.client.0.smithi090.stdout:7/710: dread d0/d1/d4/f6b [0,4194304] 0 2024-04-26T04:42:26.991 INFO:tasks.workunit.client.0.smithi090.stdout:7/711: fsync d0/db/df/d4d/f63 0 2024-04-26T04:42:26.993 INFO:tasks.workunit.client.0.smithi090.stdout:7/712: creat d0/d1/d4/d98/d65/db2/f103 x:0 0 0 2024-04-26T04:42:26.993 INFO:tasks.workunit.client.0.smithi090.stdout:2/682: mkdir db/dc/d6f/de3 0 2024-04-26T04:42:26.994 INFO:tasks.workunit.client.0.smithi090.stdout:2/683: truncate db/d11/d14/d1b/d20/da7/fd2 1036833 0 2024-04-26T04:42:27.000 INFO:tasks.workunit.client.0.smithi090.stdout:6/724: dread d9/d1c/d31/fd1 [0,4194304] 0 2024-04-26T04:42:27.003 INFO:tasks.workunit.client.0.smithi090.stdout:6/725: creat d9/d3c/ffc x:0 0 0 2024-04-26T04:42:27.004 INFO:tasks.workunit.client.0.smithi090.stdout:6/726: symlink d9/d1c/dc5/lfd 0 2024-04-26T04:42:27.005 INFO:tasks.workunit.client.1.smithi104.stdout:0/984: dwrite d0/d2/d21/d49/d5e/d9c/f46 [0,4194304] 0 2024-04-26T04:42:27.005 INFO:tasks.workunit.client.1.smithi104.stdout:0/985: stat d0/d2/d21/d49/d5e/d9c/dbe 0 2024-04-26T04:42:27.008 INFO:tasks.workunit.client.0.smithi090.stdout:8/817: write dc/dd2/d3b/f6d [2226769,112515] 0 2024-04-26T04:42:27.008 INFO:tasks.workunit.client.0.smithi090.stdout:8/818: read dc/dd2/d39/f7c [56700,44710] 0 2024-04-26T04:42:27.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/819: mkdir dc/dd2/d3b/d3d/d111/d11c 0 2024-04-26T04:42:27.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/820: write dc/dd2/d39/f7c [925083,19491] 0 2024-04-26T04:42:27.011 INFO:tasks.workunit.client.0.smithi090.stdout:8/821: write dc/dd2/fd6 [601385,51474] 0 2024-04-26T04:42:27.011 INFO:tasks.workunit.client.0.smithi090.stdout:0/721: write d0/d4b/d5d/d5c/d8a/d45/d53/faf [1725632,58308] 0 2024-04-26T04:42:27.011 INFO:tasks.workunit.client.0.smithi090.stdout:0/722: read - d0/d4b/feb zero size 2024-04-26T04:42:27.011 INFO:tasks.workunit.client.0.smithi090.stdout:0/723: fsync d0/de/f2f 0 2024-04-26T04:42:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:26 smithi104 ceph-mon[32369]: pgmap v255: 65 pgs: 65 active+clean; 2.8 GiB data, 8.1 GiB used, 528 GiB / 536 GiB avail; 301 MiB/s rd, 367 MiB/s wr, 551 op/s 2024-04-26T04:42:27.012 INFO:tasks.workunit.client.0.smithi090.stdout:8/822: creat dc/dd2/d39/d55/f11d x:0 0 0 2024-04-26T04:42:27.013 INFO:tasks.workunit.client.0.smithi090.stdout:0/724: write d0/d4b/fbd [496702,52424] 0 2024-04-26T04:42:27.015 INFO:tasks.workunit.client.0.smithi090.stdout:0/725: mknod d0/d4b/d5d/d5c/d8a/d45/d53/d8e/cf9 0 2024-04-26T04:42:27.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/726: mkdir d0/d4b/d5d/d5c/d8a/d45/d53/dc2/dfa 0 2024-04-26T04:42:27.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/727: chown d0/d4b/d5d/d48/fca 0 1 2024-04-26T04:42:27.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/728: truncate d0/d4b/d5d/d5c/d8a/d7c/f95 344279 0 2024-04-26T04:42:27.017 INFO:tasks.workunit.client.0.smithi090.stdout:0/729: write d0/d4b/d5d/d5c/d8a/d45/d53/dc5/fd6 [1053937,41472] 0 2024-04-26T04:42:27.018 INFO:tasks.workunit.client.0.smithi090.stdout:0/730: dread d0/d4b/db3/fcb [0,4194304] 0 2024-04-26T04:42:27.018 INFO:tasks.workunit.client.0.smithi090.stdout:0/731: chown d0/d4b/d6d/df2/fe6 11710239 1 2024-04-26T04:42:27.019 INFO:tasks.workunit.client.0.smithi090.stdout:0/732: mknod d0/d4b/d5d/d5c/d8a/d45/d53/dc5/cfb 0 2024-04-26T04:42:27.019 INFO:tasks.workunit.client.0.smithi090.stdout:0/733: write d0/d4b/d5d/d5c/d8a/d7c/fc9 [975059,29212] 0 2024-04-26T04:42:27.037 INFO:tasks.workunit.client.0.smithi090.stdout:2/684: dwrite db/d59/f62 [0,4194304] 0 2024-04-26T04:42:27.039 INFO:tasks.workunit.client.0.smithi090.stdout:2/685: mkdir db/dc/d6f/de3/de4 0 2024-04-26T04:42:27.039 INFO:tasks.workunit.client.0.smithi090.stdout:2/686: getdents db/d11/d14/d1b/d20/d4c/d98 0 2024-04-26T04:42:27.041 INFO:tasks.workunit.client.0.smithi090.stdout:2/687: creat db/d11/d14/d25/fe5 x:0 0 0 2024-04-26T04:42:27.042 INFO:tasks.workunit.client.0.smithi090.stdout:2/688: fsync db/d11/d14/d27/f38 0 2024-04-26T04:42:27.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/625: dwrite d4/d1a/d2a/d79/d90/fa8 [0,4194304] 0 2024-04-26T04:42:27.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/626: write d4/d1a/f17 [4226547,79807] 0 2024-04-26T04:42:27.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/627: fsync d4/d1a/df/f28 0 2024-04-26T04:42:27.047 INFO:tasks.workunit.client.0.smithi090.stdout:5/628: getdents d4/d1a/d2a/db5 0 2024-04-26T04:42:27.047 INFO:tasks.workunit.client.0.smithi090.stdout:7/713: dwrite d0/d1/d4/d98/f4c [0,4194304] 0 2024-04-26T04:42:27.048 INFO:tasks.workunit.client.1.smithi104.stdout:8/922: dwrite d0/d33/d35/d6d/def/df7/f117 [0,4194304] 0 2024-04-26T04:42:27.049 INFO:tasks.workunit.client.1.smithi104.stdout:9/855: dwrite d11/d15/f18 [4194304,4194304] 0 2024-04-26T04:42:27.049 INFO:tasks.workunit.client.1.smithi104.stdout:9/856: truncate d11/d15/d32/d4c/d54/f77 4775788 0 2024-04-26T04:42:27.050 INFO:tasks.workunit.client.0.smithi090.stdout:3/622: dwrite d1/d17/d36/d33/d68/dae/dca/d1b/d58/da0/fc8 [0,4194304] 0 2024-04-26T04:42:27.050 INFO:tasks.workunit.client.0.smithi090.stdout:6/727: dwrite d9/d1c/d31/f41 [0,4194304] 0 2024-04-26T04:42:27.052 INFO:tasks.workunit.client.1.smithi104.stdout:9/857: dread d11/d1d/d34/d69/fa7 [0,4194304] 0 2024-04-26T04:42:27.055 INFO:tasks.workunit.client.1.smithi104.stdout:8/923: unlink d0/d1c/d2d/d7a/ldf 0 2024-04-26T04:42:27.056 INFO:tasks.workunit.client.0.smithi090.stdout:4/663: sync 2024-04-26T04:42:27.056 INFO:tasks.workunit.client.0.smithi090.stdout:1/632: sync 2024-04-26T04:42:27.058 INFO:tasks.workunit.client.1.smithi104.stdout:9/858: unlink d11/d15/d32/dd6/l103 0 2024-04-26T04:42:27.058 INFO:tasks.workunit.client.1.smithi104.stdout:9/859: creat d11/d15/d98/f11f x:0 0 0 2024-04-26T04:42:27.058 INFO:tasks.workunit.client.1.smithi104.stdout:8/924: creat d0/d1c/d59/d85/d127/d47/d67/d76/f12e x:0 0 0 2024-04-26T04:42:27.060 INFO:tasks.workunit.client.0.smithi090.stdout:6/728: write d9/d77/db0/db5/dda/d6d/fc6 [2892088,42096] 0 2024-04-26T04:42:27.060 INFO:tasks.workunit.client.0.smithi090.stdout:6/729: creat d9/d1c/d31/d79/ffe x:0 0 0 2024-04-26T04:42:27.063 INFO:tasks.workunit.client.1.smithi104.stdout:9/860: read d11/d1d/d71/fcc [55767,74308] 0 2024-04-26T04:42:27.063 INFO:tasks.workunit.client.1.smithi104.stdout:8/925: truncate d0/d1c/d59/d85/d127/f40 1060985 0 2024-04-26T04:42:27.063 INFO:tasks.workunit.client.1.smithi104.stdout:8/926: creat d0/d50/f12f x:0 0 0 2024-04-26T04:42:27.063 INFO:tasks.workunit.client.1.smithi104.stdout:8/927: write d0/d1c/d59/d85/d127/d47/f123 [199894,13084] 0 2024-04-26T04:42:27.063 INFO:tasks.workunit.client.1.smithi104.stdout:8/928: chown d0/d9a/ce8 15993 1 2024-04-26T04:42:27.063 INFO:tasks.workunit.client.1.smithi104.stdout:8/929: dread - d0/d33/f12d zero size 2024-04-26T04:42:27.063 INFO:tasks.workunit.client.1.smithi104.stdout:9/861: dread d11/d1d/d34/f3c [0,4194304] 0 2024-04-26T04:42:27.064 INFO:tasks.workunit.client.0.smithi090.stdout:5/629: mkdir d4/d1a/d1f/dc5 0 2024-04-26T04:42:27.066 INFO:tasks.workunit.client.1.smithi104.stdout:9/862: rename d11/d15/d1b/d6d/d9c/la8 to d11/d15/d32/dd6/l120 0 2024-04-26T04:42:27.066 INFO:tasks.workunit.client.1.smithi104.stdout:9/863: write d11/d15/d1b/fb5 [947889,119277] 0 2024-04-26T04:42:27.070 INFO:tasks.workunit.client.0.smithi090.stdout:3/623: mknod d1/d17/d36/d33/d68/dae/dca/d1b/cd4 0 2024-04-26T04:42:27.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/734: rmdir d0/d4b 39 2024-04-26T04:42:27.074 INFO:tasks.workunit.client.0.smithi090.stdout:0/735: write d0/de/f71 [1020863,22457] 0 2024-04-26T04:42:27.074 INFO:tasks.workunit.client.0.smithi090.stdout:1/633: write d0/d4/d7/f90 [678047,66215] 0 2024-04-26T04:42:27.076 INFO:tasks.workunit.client.0.smithi090.stdout:4/664: mkdir d3/db/d12/d99/dce 0 2024-04-26T04:42:27.080 INFO:tasks.workunit.client.0.smithi090.stdout:2/689: dwrite db/d11/d14/d27/d2f/d70/d71/f9c [0,4194304] 0 2024-04-26T04:42:27.081 INFO:tasks.workunit.client.0.smithi090.stdout:0/736: mknod d0/d4b/d5d/d5c/d8a/d45/d93/cfc 0 2024-04-26T04:42:27.081 INFO:tasks.workunit.client.0.smithi090.stdout:0/737: fdatasync d0/df/f3c 0 2024-04-26T04:42:27.083 INFO:tasks.workunit.client.0.smithi090.stdout:2/690: creat db/dc/d6f/de3/fe6 x:0 0 0 2024-04-26T04:42:27.083 INFO:tasks.workunit.client.0.smithi090.stdout:2/691: write db/d11/d14/d27/d28/f3d [1893927,88389] 0 2024-04-26T04:42:27.083 INFO:tasks.workunit.client.0.smithi090.stdout:2/692: readlink db/d11/d14/d27/ld4 0 2024-04-26T04:42:27.085 INFO:tasks.workunit.client.1.smithi104.stdout:9/864: dread d11/f17 [0,4194304] 0 2024-04-26T04:42:27.099 INFO:tasks.workunit.client.0.smithi090.stdout:4/665: dread d3/db/d12/d3b/f78 [0,4194304] 0 2024-04-26T04:42:27.100 INFO:tasks.workunit.client.0.smithi090.stdout:4/666: write d3/db/f23 [908048,119329] 0 2024-04-26T04:42:27.105 INFO:tasks.workunit.client.0.smithi090.stdout:4/667: dread d3/d13/f81 [0,4194304] 0 2024-04-26T04:42:27.106 INFO:tasks.workunit.client.0.smithi090.stdout:0/738: creat d0/d4b/d5d/d5c/d8a/d45/d53/dab/ffd x:0 0 0 2024-04-26T04:42:27.107 INFO:tasks.workunit.client.0.smithi090.stdout:2/693: mknod db/d11/d14/d25/d86/db9/ce7 0 2024-04-26T04:42:27.107 INFO:tasks.workunit.client.0.smithi090.stdout:7/714: dwrite d0/db/d53/fa [4194304,4194304] 0 2024-04-26T04:42:27.107 INFO:tasks.workunit.client.0.smithi090.stdout:7/715: creat d0/db/d15/dbc/dd2/f104 x:0 0 0 2024-04-26T04:42:27.113 INFO:tasks.workunit.client.0.smithi090.stdout:9/597: sync 2024-04-26T04:42:27.113 INFO:tasks.workunit.client.0.smithi090.stdout:9/598: dread - db/dd/d44/d82/d85/db4/d99/fc3 zero size 2024-04-26T04:42:27.129 INFO:tasks.workunit.client.0.smithi090.stdout:6/730: dwrite d9/d21/d40/d4c/d6f/fb6 [0,4194304] 0 2024-04-26T04:42:27.130 INFO:tasks.workunit.client.1.smithi104.stdout:8/930: dwrite d0/d1c/d59/d85/d127/f7c [0,4194304] 0 2024-04-26T04:42:27.135 INFO:tasks.workunit.client.0.smithi090.stdout:5/630: dwrite d4/d1a/d2a/d79/d90/d49/f9a [0,4194304] 0 2024-04-26T04:42:27.136 INFO:tasks.workunit.client.0.smithi090.stdout:1/634: dwrite d0/d4/fb2 [0,4194304] 0 2024-04-26T04:42:27.137 INFO:tasks.workunit.client.0.smithi090.stdout:2/694: dwrite db/dc/f8e [4194304,4194304] 0 2024-04-26T04:42:27.138 INFO:tasks.workunit.client.1.smithi104.stdout:9/865: dwrite d11/d15/d1b/d6d/fc4 [0,4194304] 0 2024-04-26T04:42:27.140 INFO:tasks.workunit.client.0.smithi090.stdout:3/624: dwrite d1/d59/fa1 [0,4194304] 0 2024-04-26T04:42:27.142 INFO:tasks.workunit.client.0.smithi090.stdout:7/716: dwrite d0/f3c [0,4194304] 0 2024-04-26T04:42:27.144 INFO:tasks.workunit.client.1.smithi104.stdout:8/931: dread d0/d33/d35/d6d/def/ff6 [0,4194304] 0 2024-04-26T04:42:27.145 INFO:tasks.workunit.client.1.smithi104.stdout:8/932: chown d0/d33/d51/lf3 2646393 1 2024-04-26T04:42:27.146 INFO:tasks.workunit.client.1.smithi104.stdout:8/933: mknod d0/d1c/d59/d85/d127/d47/dd5/c130 0 2024-04-26T04:42:27.148 INFO:tasks.workunit.client.1.smithi104.stdout:8/934: mknod d0/d33/c131 0 2024-04-26T04:42:27.154 INFO:tasks.workunit.client.0.smithi090.stdout:5/631: dread d4/d1a/d2a/d79/d90/d49/f9a [0,4194304] 0 2024-04-26T04:42:27.154 INFO:tasks.workunit.client.0.smithi090.stdout:5/632: stat d4/d1a/df/f8d 0 2024-04-26T04:42:27.155 INFO:tasks.workunit.client.0.smithi090.stdout:5/633: dread d4/d1a/df/d4a/f89 [0,4194304] 0 2024-04-26T04:42:27.156 INFO:tasks.workunit.client.1.smithi104.stdout:0/986: sync 2024-04-26T04:42:27.156 INFO:tasks.workunit.client.1.smithi104.stdout:7/974: sync 2024-04-26T04:42:27.156 INFO:tasks.workunit.client.1.smithi104.stdout:0/987: chown d0/d2/d21 3 1 2024-04-26T04:42:27.157 INFO:tasks.workunit.client.1.smithi104.stdout:0/988: chown d0/d2/d10/d110 147 1 2024-04-26T04:42:27.157 INFO:tasks.workunit.client.1.smithi104.stdout:7/975: fdatasync d2/d4/d32/d3d/f50 0 2024-04-26T04:42:27.157 INFO:tasks.workunit.client.1.smithi104.stdout:7/976: fsync d2/d4/d32/d60/dff/f129 0 2024-04-26T04:42:27.158 INFO:tasks.workunit.client.1.smithi104.stdout:0/989: unlink d0/d2/lca 0 2024-04-26T04:42:27.158 INFO:tasks.workunit.client.1.smithi104.stdout:9/866: dwrite d11/d15/d32/d35/f80 [0,4194304] 0 2024-04-26T04:42:27.158 INFO:tasks.workunit.client.1.smithi104.stdout:9/867: chown d11/fc6 2593181 1 2024-04-26T04:42:27.158 INFO:tasks.workunit.client.1.smithi104.stdout:9/868: getdents d11/d15/d1b/d6d/dd5/df3 0 2024-04-26T04:42:27.159 INFO:tasks.workunit.client.1.smithi104.stdout:8/935: write d0/d1c/d59/d85/d127/d47/d67/d76/d7f/fac [1510118,812] 0 2024-04-26T04:42:27.159 INFO:tasks.workunit.client.1.smithi104.stdout:8/936: fsync d0/d1c/d59/d85/d127/d47/d67/d76/daf/fe9 0 2024-04-26T04:42:27.160 INFO:tasks.workunit.client.1.smithi104.stdout:9/869: link d11/d1d/d87/fd4 d11/d15/d1b/d6d/f121 0 2024-04-26T04:42:27.160 INFO:tasks.workunit.client.1.smithi104.stdout:9/870: chown d11/d1d/d34/c5f 133078941 1 2024-04-26T04:42:27.162 INFO:tasks.workunit.client.1.smithi104.stdout:8/937: link d0/d1c/d59/d85/d127/f86 d0/d33/d35/d113/f132 0 2024-04-26T04:42:27.162 INFO:tasks.workunit.client.1.smithi104.stdout:9/871: truncate d11/d15/d1b/fa4 267167 0 2024-04-26T04:42:27.163 INFO:tasks.workunit.client.1.smithi104.stdout:8/938: mknod d0/d1c/d59/d85/d127/d47/d67/d76/daf/c133 0 2024-04-26T04:42:27.164 INFO:tasks.workunit.client.1.smithi104.stdout:9/872: mknod d11/d1d/d34/db3/de6/c122 0 2024-04-26T04:42:27.164 INFO:tasks.workunit.client.1.smithi104.stdout:9/873: fsync d11/d15/f59 0 2024-04-26T04:42:27.166 INFO:tasks.workunit.client.1.smithi104.stdout:8/939: rename d0/d1c/d59/d85/d127/f90 to d0/d1c/d2d/f134 0 2024-04-26T04:42:27.166 INFO:tasks.workunit.client.1.smithi104.stdout:0/990: write d0/d2/d21/d27/f82 [2673038,128352] 0 2024-04-26T04:42:27.166 INFO:tasks.workunit.client.1.smithi104.stdout:0/991: truncate d0/d2/d21/d27/d1e/d57/d6c/dd1/f34 2609249 0 2024-04-26T04:42:27.167 INFO:tasks.workunit.client.1.smithi104.stdout:9/874: truncate d11/d15/d32/f56 747024 0 2024-04-26T04:42:27.168 INFO:tasks.workunit.client.1.smithi104.stdout:8/940: mknod d0/d1c/d59/d85/d127/d47/c135 0 2024-04-26T04:42:27.168 INFO:tasks.workunit.client.1.smithi104.stdout:8/941: fdatasync d0/d1c/d59/f110 0 2024-04-26T04:42:27.168 INFO:tasks.workunit.client.1.smithi104.stdout:0/992: dread d0/d2/d21/f45 [0,4194304] 0 2024-04-26T04:42:27.171 INFO:tasks.workunit.client.1.smithi104.stdout:0/993: mkdir d0/d2/d10/d3d/dc0/df1/d141 0 2024-04-26T04:42:27.171 INFO:tasks.workunit.client.1.smithi104.stdout:0/994: write d0/d2/d21/d27/d1e/fc8 [3219659,3041] 0 2024-04-26T04:42:27.176 INFO:tasks.workunit.client.0.smithi090.stdout:4/668: mkdir d3/d13/d64/dc1/dcf 0 2024-04-26T04:42:27.177 INFO:tasks.workunit.client.0.smithi090.stdout:4/669: chown d3/db/d12/d99 535720564 1 2024-04-26T04:42:27.181 INFO:tasks.workunit.client.1.smithi104.stdout:8/942: rmdir d0/d1c/d59/d85 39 2024-04-26T04:42:27.182 INFO:tasks.workunit.client.1.smithi104.stdout:0/995: write d0/d2/d10/d3d/db7/fd2 [5294964,54986] 0 2024-04-26T04:42:27.183 INFO:tasks.workunit.client.0.smithi090.stdout:9/599: link db/c10 db/dd/d44/d82/d85/db4/d6a/db6/cd0 0 2024-04-26T04:42:27.183 INFO:tasks.workunit.client.0.smithi090.stdout:9/600: dread - db/dd/d18/d57/f94 zero size 2024-04-26T04:42:27.184 INFO:tasks.workunit.client.1.smithi104.stdout:0/996: creat d0/d2/d21/d49/d5e/d9c/d53/d5d/d80/da1/ded/d11d/f142 x:0 0 0 2024-04-26T04:42:27.185 INFO:tasks.workunit.client.1.smithi104.stdout:0/997: symlink d0/d2/d21/d27/l143 0 2024-04-26T04:42:27.185 INFO:tasks.workunit.client.1.smithi104.stdout:0/998: stat d0/d2/d10/d3d/db7/l116 0 2024-04-26T04:42:27.187 INFO:tasks.workunit.client.0.smithi090.stdout:6/731: getdents d9/d21/d40/d4c/d6f 0 2024-04-26T04:42:27.187 INFO:tasks.workunit.client.0.smithi090.stdout:6/732: fdatasync d9/d77/db0/db5/dda/d6d/d92/f29 0 2024-04-26T04:42:27.188 INFO:tasks.workunit.client.1.smithi104.stdout:0/999: creat d0/d2/f144 x:0 0 0 2024-04-26T04:42:27.189 INFO:tasks.workunit.client.1.smithi104.stdout:9/875: dwrite f4 [0,4194304] 0 2024-04-26T04:42:27.191 INFO:tasks.workunit.client.1.smithi104.stdout:9/876: creat d11/d15/d1b/f123 x:0 0 0 2024-04-26T04:42:27.191 INFO:tasks.workunit.client.1.smithi104.stdout:9/877: chown d11/d1d/d34/db3/df1/d110/l112 0 1 2024-04-26T04:42:27.193 INFO:tasks.workunit.client.1.smithi104.stdout:9/878: mkdir d11/d1d/d71/d124 0 2024-04-26T04:42:27.193 INFO:tasks.workunit.client.1.smithi104.stdout:9/879: fdatasync d11/d1d/f4f 0 2024-04-26T04:42:27.203 INFO:tasks.workunit.client.0.smithi090.stdout:6/733: dwrite d9/d1c/f57 [0,4194304] 0 2024-04-26T04:42:27.204 INFO:tasks.workunit.client.0.smithi090.stdout:6/734: write d9/d1c/dc5/fd0 [80615,22494] 0 2024-04-26T04:42:27.206 INFO:tasks.workunit.client.0.smithi090.stdout:7/717: mkdir d0/d105 0 2024-04-26T04:42:27.207 INFO:tasks.workunit.client.1.smithi104.stdout:9/880: read d11/d15/d32/d4c/d7c/fb4 [1093675,33347] 0 2024-04-26T04:42:27.208 INFO:tasks.workunit.client.0.smithi090.stdout:5/634: mknod d4/d1a/d3f/cc6 0 2024-04-26T04:42:27.208 INFO:tasks.workunit.client.0.smithi090.stdout:5/635: chown d4/d1a/d2a/d79/f85 138281 1 2024-04-26T04:42:27.209 INFO:tasks.workunit.client.0.smithi090.stdout:5/636: fsync d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 0 2024-04-26T04:42:27.212 INFO:tasks.workunit.client.0.smithi090.stdout:9/601: readlink db/dd/l12 0 2024-04-26T04:42:27.215 INFO:tasks.workunit.client.0.smithi090.stdout:5/637: creat d4/d1a/d2a/d79/d90/d56/d80/fc7 x:0 0 0 2024-04-26T04:42:27.223 INFO:tasks.workunit.client.1.smithi104.stdout:9/881: dwrite d11/d1d/f4f [0,4194304] 0 2024-04-26T04:42:27.242 INFO:tasks.workunit.client.1.smithi104.stdout:9/882: dwrite d11/d15/d32/d4c/d54/ddc/fff [0,4194304] 0 2024-04-26T04:42:27.242 INFO:tasks.workunit.client.1.smithi104.stdout:9/883: creat d11/d1d/d87/f125 x:0 0 0 2024-04-26T04:42:27.242 INFO:tasks.workunit.client.1.smithi104.stdout:9/884: readlink d11/d15/d32/d35/d7b/l8d 0 2024-04-26T04:42:27.246 INFO:tasks.workunit.client.1.smithi104.stdout:9/885: mkdir d11/d15/d32/d35/d7b/d126 0 2024-04-26T04:42:27.247 INFO:tasks.workunit.client.0.smithi090.stdout:8/823: sync 2024-04-26T04:42:27.248 INFO:tasks.workunit.client.0.smithi090.stdout:8/824: creat dc/d5a/d5d/d73/f11e x:0 0 0 2024-04-26T04:42:27.252 INFO:tasks.workunit.client.0.smithi090.stdout:8/825: truncate dc/dd2/d3b/d3d/d97/dad/ffa 811053 0 2024-04-26T04:42:27.253 INFO:tasks.workunit.client.0.smithi090.stdout:6/735: unlink d9/d21/d40/d4c/c56 0 2024-04-26T04:42:27.257 INFO:tasks.workunit.client.0.smithi090.stdout:0/739: rename d0/de/cf7 to d0/cfe 0 2024-04-26T04:42:27.261 INFO:tasks.workunit.client.0.smithi090.stdout:2/695: rename db/d11/d14/d27/d2f/d42/l5f to db/d11/d14/d1b/d20/d4c/d98/le8 0 2024-04-26T04:42:27.261 INFO:tasks.workunit.client.0.smithi090.stdout:2/696: readlink db/d11/d14/d25/d41/lb5 0 2024-04-26T04:42:27.264 INFO:tasks.workunit.client.0.smithi090.stdout:6/736: dread d9/d77/db0/db5/dda/d6d/d92/f29 [8388608,4194304] 0 2024-04-26T04:42:27.264 INFO:tasks.workunit.client.0.smithi090.stdout:0/740: unlink d0/d4b/d6d/df2/fe6 0 2024-04-26T04:42:27.266 INFO:tasks.workunit.client.0.smithi090.stdout:3/625: rename d1/d17/d36/dab/fc2 to d1/d17/d61/d9d/fd5 0 2024-04-26T04:42:27.268 INFO:tasks.workunit.client.1.smithi104.stdout:9/886: write d11/f12 [2194152,1781] 0 2024-04-26T04:42:27.268 INFO:tasks.workunit.client.1.smithi104.stdout:9/887: creat d11/d1d/d34/d69/f127 x:0 0 0 2024-04-26T04:42:27.270 INFO:tasks.workunit.client.0.smithi090.stdout:2/697: unlink db/dc/l2c 0 2024-04-26T04:42:27.270 INFO:tasks.workunit.client.0.smithi090.stdout:2/698: write db/d11/d14/d27/d2f/d42/d5c/f4d [754634,50119] 0 2024-04-26T04:42:27.270 INFO:tasks.workunit.client.0.smithi090.stdout:2/699: stat db/dc/f8e 0 2024-04-26T04:42:27.271 INFO:tasks.workunit.client.1.smithi104.stdout:9/888: creat d11/d90/f128 x:0 0 0 2024-04-26T04:42:27.272 INFO:tasks.workunit.client.0.smithi090.stdout:0/741: stat d0/d4b/d5d/l1d 0 2024-04-26T04:42:27.272 INFO:tasks.workunit.client.0.smithi090.stdout:0/742: chown d0/de/lea 3 1 2024-04-26T04:42:27.273 INFO:tasks.workunit.client.1.smithi104.stdout:9/889: mkdir d11/d15/d1b/d6d/d9c/dbe/def/d129 0 2024-04-26T04:42:27.273 INFO:tasks.workunit.client.0.smithi090.stdout:8/826: dwrite dc/dd2/d39/fb9 [0,4194304] 0 2024-04-26T04:42:27.275 INFO:tasks.workunit.client.0.smithi090.stdout:7/718: rename d0/db/df/d4d/d64/df2/df9 to d0/db/df/d4d/daf/dd1/d106 0 2024-04-26T04:42:27.275 INFO:tasks.workunit.client.0.smithi090.stdout:7/719: readlink d0/db/df/d4d/l7f 0 2024-04-26T04:42:27.276 INFO:tasks.workunit.client.1.smithi104.stdout:9/890: truncate d11/d15/d32/d4c/d54/f61 1372509 0 2024-04-26T04:42:27.276 INFO:tasks.workunit.client.1.smithi104.stdout:9/891: truncate d11/d15/d32/d4c/d54/f6e 925396 0 2024-04-26T04:42:27.277 INFO:tasks.workunit.client.1.smithi104.stdout:9/892: unlink d11/d1d/d34/fe1 0 2024-04-26T04:42:27.277 INFO:tasks.workunit.client.1.smithi104.stdout:9/893: chown d11/d15/d32/d4c/ld0 273 1 2024-04-26T04:42:27.278 INFO:tasks.workunit.client.0.smithi090.stdout:2/700: creat db/d11/d14/d25/d2d/d48/fe9 x:0 0 0 2024-04-26T04:42:27.279 INFO:tasks.workunit.client.1.smithi104.stdout:9/894: symlink d11/d90/dfd/d10e/d119/l12a 0 2024-04-26T04:42:27.280 INFO:tasks.workunit.client.0.smithi090.stdout:5/638: creat d4/d1a/d2a/d79/fc8 x:0 0 0 2024-04-26T04:42:27.285 INFO:tasks.workunit.client.0.smithi090.stdout:8/827: mknod dc/dd2/d4f/d58/d95/c11f 0 2024-04-26T04:42:27.286 INFO:tasks.workunit.client.0.smithi090.stdout:7/720: mknod d0/db/df/c107 0 2024-04-26T04:42:27.286 INFO:tasks.workunit.client.0.smithi090.stdout:6/737: dwrite d9/d21/d40/d4c/d6f/d9f/fcd [0,4194304] 0 2024-04-26T04:42:27.288 INFO:tasks.workunit.client.0.smithi090.stdout:9/602: rename db/dd/d44/d82/d85/db4/fc8 to db/dd/d18/d38/fd1 0 2024-04-26T04:42:27.289 INFO:tasks.workunit.client.0.smithi090.stdout:5/639: mknod d4/d1a/cc9 0 2024-04-26T04:42:27.293 INFO:tasks.workunit.client.0.smithi090.stdout:8/828: mknod dc/dd2/d3b/d3d/db8/ddc/c120 0 2024-04-26T04:42:27.297 INFO:tasks.workunit.client.0.smithi090.stdout:7/721: symlink d0/db/df/d4d/l108 0 2024-04-26T04:42:27.298 INFO:tasks.workunit.client.0.smithi090.stdout:6/738: link d9/d77/db0/db5/dda/d6d/d92/cdf d9/d77/db0/db5/dda/cff 0 2024-04-26T04:42:27.299 INFO:tasks.workunit.client.0.smithi090.stdout:2/701: dread db/f10 [0,4194304] 0 2024-04-26T04:42:27.299 INFO:tasks.workunit.client.0.smithi090.stdout:2/702: fsync db/d11/d7f/fa1 0 2024-04-26T04:42:27.299 INFO:tasks.workunit.client.0.smithi090.stdout:0/743: write d0/d4b/fad [3970008,50622] 0 2024-04-26T04:42:27.300 INFO:tasks.workunit.client.0.smithi090.stdout:9/603: rename db/dd/l12 to db/dd/d18/d38/d39/dba/ld2 0 2024-04-26T04:42:27.300 INFO:tasks.workunit.client.0.smithi090.stdout:9/604: fsync db/dd/d18/d38/f90 0 2024-04-26T04:42:27.300 INFO:tasks.workunit.client.0.smithi090.stdout:9/605: write db/dd/d44/f56 [1653952,21676] 0 2024-04-26T04:42:27.300 INFO:tasks.workunit.client.0.smithi090.stdout:2/703: mkdir db/d11/d14/d27/d2f/d70/d71/dea 0 2024-04-26T04:42:27.303 INFO:tasks.workunit.client.0.smithi090.stdout:5/640: rename d4/d1a/d2a/db5 to d4/d1a/d1f/dc5/dca 0 2024-04-26T04:42:27.303 INFO:tasks.workunit.client.0.smithi090.stdout:6/739: write d9/d77/db0/db5/dda/d6d/d92/f29 [10704911,86224] 0 2024-04-26T04:42:27.303 INFO:tasks.workunit.client.0.smithi090.stdout:6/740: fsync d9/d1c/d31/d69/fa4 0 2024-04-26T04:42:27.303 INFO:tasks.workunit.client.0.smithi090.stdout:6/741: write d9/f14 [6276695,64231] 0 2024-04-26T04:42:27.303 INFO:tasks.workunit.client.0.smithi090.stdout:6/742: write d9/d77/db0/db5/dda/d6d/f45 [677837,86743] 0 2024-04-26T04:42:27.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/626: dwrite d1/d17/f4f [0,4194304] 0 2024-04-26T04:42:27.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/627: creat d1/d9/fd6 x:0 0 0 2024-04-26T04:42:27.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/628: fdatasync d1/d59/f6d 0 2024-04-26T04:42:27.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/629: fsync d1/d17/f4f 0 2024-04-26T04:42:27.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/630: chown d1/d17/d36/fa9 23519798 1 2024-04-26T04:42:27.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/631: fsync d1/d17/d36/d33/d68/dae/fba 0 2024-04-26T04:42:27.305 INFO:tasks.workunit.client.1.smithi104.stdout:9/895: dwrite d11/d15/d98/f10c [0,4194304] 0 2024-04-26T04:42:27.305 INFO:tasks.workunit.client.0.smithi090.stdout:5/641: creat d4/d1a/fcb x:0 0 0 2024-04-26T04:42:27.305 INFO:tasks.workunit.client.0.smithi090.stdout:5/642: readlink d4/d1a/d3f/l4b 0 2024-04-26T04:42:27.305 INFO:tasks.workunit.client.0.smithi090.stdout:5/643: read d4/d1a/d2a/d79/d90/db3/f87 [86752,119764] 0 2024-04-26T04:42:27.306 INFO:tasks.workunit.client.0.smithi090.stdout:5/644: readlink d4/d1a/d1f/d53/d59/d5e/da0/lad 0 2024-04-26T04:42:27.308 INFO:tasks.workunit.client.1.smithi104.stdout:9/896: mkdir d11/d90/dfd/d12b 0 2024-04-26T04:42:27.308 INFO:tasks.workunit.client.1.smithi104.stdout:9/897: write d11/d1d/d87/f125 [32025,78682] 0 2024-04-26T04:42:27.309 INFO:tasks.workunit.client.0.smithi090.stdout:8/829: rename dc/dd2/d39/d55/d64/f8d to dc/dd2/d3b/d3d/d111/f121 0 2024-04-26T04:42:27.309 INFO:tasks.workunit.client.0.smithi090.stdout:8/830: fdatasync dc/dd2/d39/d55/f11d 0 2024-04-26T04:42:27.309 INFO:tasks.workunit.client.0.smithi090.stdout:8/831: dread - dc/dd2/d39/d55/d64/ff1 zero size 2024-04-26T04:42:27.309 INFO:tasks.workunit.client.0.smithi090.stdout:8/832: write dc/dd2/d3b/f8b [763513,12560] 0 2024-04-26T04:42:27.310 INFO:tasks.workunit.client.0.smithi090.stdout:8/833: chown dc/dd2/d3b/d3d/d97/fac 1 1 2024-04-26T04:42:27.310 INFO:tasks.workunit.client.1.smithi104.stdout:9/898: symlink d11/d15/d1b/d6d/l12c 0 2024-04-26T04:42:27.310 INFO:tasks.workunit.client.1.smithi104.stdout:9/899: write d11/d15/d98/f11f [656327,41344] 0 2024-04-26T04:42:27.312 INFO:tasks.workunit.client.1.smithi104.stdout:9/900: mkdir d11/d15/d32/dd6/ded/d12d 0 2024-04-26T04:42:27.312 INFO:tasks.workunit.client.1.smithi104.stdout:9/901: write d11/d15/d32/d35/d7b/fa6 [8951809,89549] 0 2024-04-26T04:42:27.312 INFO:tasks.workunit.client.1.smithi104.stdout:9/902: fsync d11/d53/f86 0 2024-04-26T04:42:27.313 INFO:tasks.workunit.client.0.smithi090.stdout:6/743: creat d9/d21/d40/d4c/d6f/f100 x:0 0 0 2024-04-26T04:42:27.313 INFO:tasks.workunit.client.0.smithi090.stdout:0/744: dread d0/d4b/f8c [0,4194304] 0 2024-04-26T04:42:27.318 INFO:tasks.workunit.client.0.smithi090.stdout:3/632: symlink d1/d59/dc9/ld7 0 2024-04-26T04:42:27.325 INFO:tasks.workunit.client.0.smithi090.stdout:9/606: dwrite db/f35 [4194304,4194304] 0 2024-04-26T04:42:27.325 INFO:tasks.workunit.client.0.smithi090.stdout:6/744: mknod d9/d77/c101 0 2024-04-26T04:42:27.330 INFO:tasks.workunit.client.0.smithi090.stdout:7/722: rename d0/d1/d4/d98 to d0/d109 0 2024-04-26T04:42:27.332 INFO:tasks.workunit.client.0.smithi090.stdout:2/704: dwrite db/d11/d14/d27/d2f/d42/d5c/f4f [0,4194304] 0 2024-04-26T04:42:27.333 INFO:tasks.workunit.client.0.smithi090.stdout:9/607: truncate db/dd/d18/d38/f42 45599 0 2024-04-26T04:42:27.333 INFO:tasks.workunit.client.0.smithi090.stdout:9/608: fsync db/d16/f33 0 2024-04-26T04:42:27.336 INFO:tasks.workunit.client.0.smithi090.stdout:9/609: symlink db/dd/d18/d38/d39/d6b/ld3 0 2024-04-26T04:42:27.336 INFO:tasks.workunit.client.0.smithi090.stdout:9/610: chown db/dd/d44 207 1 2024-04-26T04:42:27.337 INFO:tasks.workunit.client.1.smithi104.stdout:9/903: rename d11/d15/d32/d4c/d54 to d11/d15/d1b/d6d/dd5/d12e 0 2024-04-26T04:42:27.338 INFO:tasks.workunit.client.0.smithi090.stdout:6/745: dread d9/d21/d40/d4c/d6f/d9f/dae/fdd [0,4194304] 0 2024-04-26T04:42:27.338 INFO:tasks.workunit.client.0.smithi090.stdout:6/746: chown d9/d1c/d31/d79/fba 892 1 2024-04-26T04:42:27.340 INFO:tasks.workunit.client.1.smithi104.stdout:9/904: link d11/d15/d32/c58 d11/d15/d1b/d6d/d9c/dd7/c12f 0 2024-04-26T04:42:27.341 INFO:tasks.workunit.client.0.smithi090.stdout:6/747: rename d9/d77/db0/db5/dda/d6d/d92/l27 to d9/d1c/d31/d69/dbc/l102 0 2024-04-26T04:42:27.341 INFO:tasks.workunit.client.1.smithi104.stdout:9/905: truncate d11/fc6 21695 0 2024-04-26T04:42:27.341 INFO:tasks.workunit.client.1.smithi104.stdout:9/906: write d11/d1d/d34/d69/fce [4659317,8655] 0 2024-04-26T04:42:27.342 INFO:tasks.workunit.client.1.smithi104.stdout:9/907: chown d11/d15/d1b/d6d/ccb 3 1 2024-04-26T04:42:27.344 INFO:tasks.workunit.client.1.smithi104.stdout:9/908: dread d11/d1d/d34/d69/f78 [0,4194304] 0 2024-04-26T04:42:27.346 INFO:tasks.workunit.client.0.smithi090.stdout:6/748: rmdir d9/d77/db0/db5/dda/d58/dab 0 2024-04-26T04:42:27.348 INFO:tasks.workunit.client.0.smithi090.stdout:7/723: dread d0/d1/f61 [0,4194304] 0 2024-04-26T04:42:27.349 INFO:tasks.workunit.client.0.smithi090.stdout:7/724: mkdir d0/db/df/d4d/d64/df2/d10a 0 2024-04-26T04:42:27.350 INFO:tasks.workunit.client.0.smithi090.stdout:7/725: rmdir d0/d109/d4a 39 2024-04-26T04:42:27.354 INFO:tasks.workunit.client.0.smithi090.stdout:7/726: creat d0/db/df/d4d/d64/df2/d10a/f10b x:0 0 0 2024-04-26T04:42:27.354 INFO:tasks.workunit.client.0.smithi090.stdout:7/727: chown d0/db/d53/db7/d4f/f76 132921 1 2024-04-26T04:42:27.355 INFO:tasks.workunit.client.0.smithi090.stdout:7/728: creat d0/db/df/d4d/daf/dd1/f10c x:0 0 0 2024-04-26T04:42:27.356 INFO:tasks.workunit.client.0.smithi090.stdout:6/749: dread d9/d21/d40/f4b [0,4194304] 0 2024-04-26T04:42:27.357 INFO:tasks.workunit.client.0.smithi090.stdout:6/750: write d9/d77/db0/db5/dda/d6d/f45 [134852,96115] 0 2024-04-26T04:42:27.358 INFO:tasks.workunit.client.0.smithi090.stdout:7/729: truncate d0/db/d15/d29/f18 2714107 0 2024-04-26T04:42:27.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/745: dwrite d0/d4b/d5d/d5c/d8a/f3e [0,4194304] 0 2024-04-26T04:42:27.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/746: chown d0/d4b/d5d/d5c/d8a/d45/fde 150209359 1 2024-04-26T04:42:27.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/747: chown d0/d4b/d5d/d5c/d8a/d45/dcf 913122602 1 2024-04-26T04:42:27.358 INFO:tasks.workunit.client.0.smithi090.stdout:9/611: read db/f84 [8172101,16757] 0 2024-04-26T04:42:27.358 INFO:tasks.workunit.client.0.smithi090.stdout:0/748: stat d0/d4b/d6d/f9a 0 2024-04-26T04:42:27.360 INFO:tasks.workunit.client.0.smithi090.stdout:3/633: dwrite d1/d17/d36/d33/d68/dae/fb3 [0,4194304] 0 2024-04-26T04:42:27.360 INFO:tasks.workunit.client.0.smithi090.stdout:5/645: dwrite d4/d1a/df/f8d [0,4194304] 0 2024-04-26T04:42:27.360 INFO:tasks.workunit.client.0.smithi090.stdout:5/646: truncate d4/d1a/fcb 447011 0 2024-04-26T04:42:27.360 INFO:tasks.workunit.client.0.smithi090.stdout:5/647: chown d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 3353 1 2024-04-26T04:42:27.360 INFO:tasks.workunit.client.0.smithi090.stdout:7/730: creat d0/d1/f10d x:0 0 0 2024-04-26T04:42:27.360 INFO:tasks.workunit.client.0.smithi090.stdout:7/731: truncate d0/db/d53/db7/d8e/fca 84915 0 2024-04-26T04:42:27.362 INFO:tasks.workunit.client.0.smithi090.stdout:0/749: link d0/d4b/d5d/d5c/d8a/d7c/l99 d0/d4b/d5d/d5c/lff 0 2024-04-26T04:42:27.362 INFO:tasks.workunit.client.0.smithi090.stdout:0/750: truncate d0/d4b/d5d/d5c/d8a/d45/d53/dab/ffd 264030 0 2024-04-26T04:42:27.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/834: dwrite dc/dd2/d3b/fbf [0,4194304] 0 2024-04-26T04:42:27.364 INFO:tasks.workunit.client.0.smithi090.stdout:2/705: dwrite db/d11/d14/d1b/d20/fb8 [0,4194304] 0 2024-04-26T04:42:27.364 INFO:tasks.workunit.client.0.smithi090.stdout:2/706: write db/d11/d14/d25/f9a [1185510,56098] 0 2024-04-26T04:42:27.365 INFO:tasks.workunit.client.0.smithi090.stdout:5/648: mknod d4/d1a/d2a/d79/d90/ccc 0 2024-04-26T04:42:27.368 INFO:tasks.workunit.client.0.smithi090.stdout:7/732: mkdir d0/db/df/d4d/d64/db0/dd6/d10e 0 2024-04-26T04:42:27.371 INFO:tasks.workunit.client.0.smithi090.stdout:9/612: mknod db/dd/d18/d38/d39/cd4 0 2024-04-26T04:42:27.371 INFO:tasks.workunit.client.0.smithi090.stdout:9/613: readlink db/dd/d18/d38/d39/d6b/dc0/lbe 0 2024-04-26T04:42:27.374 INFO:tasks.workunit.client.0.smithi090.stdout:0/751: rename d0/d4b/d5d/d5c/d8a/d45/d53/dc5 to d0/d4b/d5d/d48/d100 0 2024-04-26T04:42:27.377 INFO:tasks.workunit.client.0.smithi090.stdout:3/634: dread d1/d9/fb [0,4194304] 0 2024-04-26T04:42:27.390 INFO:tasks.workunit.client.0.smithi090.stdout:7/733: truncate d0/db/f19 7452360 0 2024-04-26T04:42:27.392 INFO:tasks.workunit.client.0.smithi090.stdout:9/614: symlink db/dd/d44/ld5 0 2024-04-26T04:42:27.395 INFO:tasks.workunit.client.0.smithi090.stdout:2/707: rename db/d59/d66/lcf to db/d59/d66/d77/leb 0 2024-04-26T04:42:27.395 INFO:tasks.workunit.client.0.smithi090.stdout:2/708: fdatasync db/d11/d14/d27/d2f/fda 0 2024-04-26T04:42:27.396 INFO:tasks.workunit.client.0.smithi090.stdout:0/752: symlink d0/d4b/db3/dcc/l101 0 2024-04-26T04:42:27.396 INFO:tasks.workunit.client.0.smithi090.stdout:8/835: dwrite dc/dd2/d3b/d3d/d111/f118 [0,4194304] 0 2024-04-26T04:42:27.396 INFO:tasks.workunit.client.0.smithi090.stdout:8/836: readlink dc/dd2/d3b/l5b 0 2024-04-26T04:42:27.399 INFO:tasks.workunit.client.0.smithi090.stdout:3/635: dwrite d1/d17/d36/d33/d68/dae/dca/f14 [0,4194304] 0 2024-04-26T04:42:27.399 INFO:tasks.workunit.client.0.smithi090.stdout:5/649: rename d4/d1a/d2a/d79/d90/db3/d9b/cac to d4/d1a/d2a/d79/d90/ccd 0 2024-04-26T04:42:27.407 INFO:tasks.workunit.client.0.smithi090.stdout:0/753: mknod d0/de/c102 0 2024-04-26T04:42:27.408 INFO:tasks.workunit.client.0.smithi090.stdout:1/635: sync 2024-04-26T04:42:27.411 INFO:tasks.workunit.client.0.smithi090.stdout:1/636: getdents d0/d4/d7/d17 0 2024-04-26T04:42:27.411 INFO:tasks.workunit.client.0.smithi090.stdout:1/637: stat d0/d55/fb9 0 2024-04-26T04:42:27.414 INFO:tasks.workunit.client.0.smithi090.stdout:0/754: rename d0/d4b/d5d/d5c/d8a/d45/d53/d8e to d0/de/d103 0 2024-04-26T04:42:27.414 INFO:tasks.workunit.client.0.smithi090.stdout:0/755: chown d0/d4b/d5d/d5c/d8a/d7c/l8d 127 1 2024-04-26T04:42:27.414 INFO:tasks.workunit.client.0.smithi090.stdout:0/756: chown d0/d4b/d5d/d48/d7d/lbe 0 1 2024-04-26T04:42:27.415 INFO:tasks.workunit.client.0.smithi090.stdout:1/638: mknod d0/d4/d62/d78/cc8 0 2024-04-26T04:42:27.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/670: sync 2024-04-26T04:42:27.416 INFO:tasks.workunit.client.0.smithi090.stdout:0/757: mknod d0/d4b/d5d/d5c/d8a/d45/d53/dab/c104 0 2024-04-26T04:42:27.417 INFO:tasks.workunit.client.0.smithi090.stdout:1/639: symlink d0/d55/dbd/lc9 0 2024-04-26T04:42:27.417 INFO:tasks.workunit.client.1.smithi104.stdout:7/977: sync 2024-04-26T04:42:27.417 INFO:tasks.workunit.client.1.smithi104.stdout:8/943: sync 2024-04-26T04:42:27.417 INFO:tasks.workunit.client.1.smithi104.stdout:7/978: stat d2/d4/d32/d3d/d41 0 2024-04-26T04:42:27.418 INFO:tasks.workunit.client.0.smithi090.stdout:4/671: unlink d3/db/d12/f4d 0 2024-04-26T04:42:27.418 INFO:tasks.workunit.client.0.smithi090.stdout:0/758: mkdir d0/de/d103/d105 0 2024-04-26T04:42:27.419 INFO:tasks.workunit.client.1.smithi104.stdout:7/979: mkdir d2/d4/d1d/d13d 0 2024-04-26T04:42:27.419 INFO:tasks.workunit.client.1.smithi104.stdout:7/980: readlink d2/d4/d32/d3d/d47/db6/dce/ddd/lde 0 2024-04-26T04:42:27.419 INFO:tasks.workunit.client.1.smithi104.stdout:7/981: truncate d2/d4/d5a/da0/fab 6198516 0 2024-04-26T04:42:27.419 INFO:tasks.workunit.client.1.smithi104.stdout:7/982: truncate d2/d4/d32/d35/d44/fb3 4283780 0 2024-04-26T04:42:27.419 INFO:tasks.workunit.client.1.smithi104.stdout:8/944: mknod d0/d1c/d59/d85/d127/d47/c136 0 2024-04-26T04:42:27.420 INFO:tasks.workunit.client.0.smithi090.stdout:1/640: mkdir d0/d4/d7/d4b/dca 0 2024-04-26T04:42:27.420 INFO:tasks.workunit.client.0.smithi090.stdout:4/672: rmdir d3/d13/d6c/dc8 0 2024-04-26T04:42:27.420 INFO:tasks.workunit.client.0.smithi090.stdout:4/673: write d3/db/d12/d20/d2d/d58/d6a/d74/da2/f37 [945700,61913] 0 2024-04-26T04:42:27.421 INFO:tasks.workunit.client.1.smithi104.stdout:7/983: unlink d2/d4/cc1 0 2024-04-26T04:42:27.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/641: creat d0/d4/d7/d4a/d75/fcb x:0 0 0 2024-04-26T04:42:27.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/642: fsync d0/d4/d7/d17/f18 0 2024-04-26T04:42:27.422 INFO:tasks.workunit.client.0.smithi090.stdout:4/674: mknod d3/db/d12/d20/d2d/d58/d6a/d74/da2/cd0 0 2024-04-26T04:42:27.422 INFO:tasks.workunit.client.0.smithi090.stdout:0/759: mkdir d0/d106 0 2024-04-26T04:42:27.423 INFO:tasks.workunit.client.1.smithi104.stdout:7/984: mknod d2/d4/d32/d3d/d41/d5f/c13e 0 2024-04-26T04:42:27.423 INFO:tasks.workunit.client.1.smithi104.stdout:7/985: stat d2/ff1 0 2024-04-26T04:42:27.425 INFO:tasks.workunit.client.0.smithi090.stdout:9/615: dwrite db/dd/d44/d82/d85/db4/d6a/d70/f96 [0,4194304] 0 2024-04-26T04:42:27.425 INFO:tasks.workunit.client.0.smithi090.stdout:9/616: write db/dd/d44/d82/d85/db4/d58/f62 [239391,83753] 0 2024-04-26T04:42:27.425 INFO:tasks.workunit.client.0.smithi090.stdout:9/617: fsync db/d16/fc2 0 2024-04-26T04:42:27.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/650: dwrite d4/d1a/d1f/d53/d59/f8a [0,4194304] 0 2024-04-26T04:42:27.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/651: fdatasync d4/d1a/df/f8d 0 2024-04-26T04:42:27.427 INFO:tasks.workunit.client.0.smithi090.stdout:5/652: write d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 [511674,85991] 0 2024-04-26T04:42:27.427 INFO:tasks.workunit.client.0.smithi090.stdout:9/618: symlink db/dd/d18/ld6 0 2024-04-26T04:42:27.428 INFO:tasks.workunit.client.1.smithi104.stdout:7/986: creat d2/d4/d32/d3d/d41/d7e/de0/f13f x:0 0 0 2024-04-26T04:42:27.428 INFO:tasks.workunit.client.1.smithi104.stdout:7/987: write d2/d4/d32/f76 [2227344,54772] 0 2024-04-26T04:42:27.430 INFO:tasks.workunit.client.1.smithi104.stdout:7/988: link d2/d4/d32/d3d/d41/d7e/l9b d2/d4/d32/d35/d116/l140 0 2024-04-26T04:42:27.431 INFO:tasks.workunit.client.0.smithi090.stdout:1/643: dread d0/d4/f20 [0,4194304] 0 2024-04-26T04:42:27.431 INFO:tasks.workunit.client.0.smithi090.stdout:7/734: dwrite d0/d1/d4/d3e/fcf [4194304,4194304] 0 2024-04-26T04:42:27.431 INFO:tasks.workunit.client.0.smithi090.stdout:7/735: dread - d0/d109/d65/fc7 zero size 2024-04-26T04:42:27.432 INFO:tasks.workunit.client.0.smithi090.stdout:5/653: mknod d4/d1a/cce 0 2024-04-26T04:42:27.436 INFO:tasks.workunit.client.0.smithi090.stdout:9/619: mkdir db/da3/da8/dd7 0 2024-04-26T04:42:27.437 INFO:tasks.workunit.client.0.smithi090.stdout:1/644: mkdir d0/d4/d62/d78/dcc 0 2024-04-26T04:42:27.438 INFO:tasks.workunit.client.0.smithi090.stdout:1/645: fsync d0/d55/f63 0 2024-04-26T04:42:27.441 INFO:tasks.workunit.client.1.smithi104.stdout:7/989: dread d2/fb8 [0,4194304] 0 2024-04-26T04:42:27.442 INFO:tasks.workunit.client.0.smithi090.stdout:7/736: unlink d0/db/d53/f87 0 2024-04-26T04:42:27.442 INFO:tasks.workunit.client.0.smithi090.stdout:1/646: stat d0/d4/d7/l37 0 2024-04-26T04:42:27.443 INFO:tasks.workunit.client.0.smithi090.stdout:1/647: fsync d0/d4/d7/d17/d22/d31/f34 0 2024-04-26T04:42:27.443 INFO:tasks.workunit.client.1.smithi104.stdout:7/990: mknod d2/d4/d32/d3d/d47/db1/dfa/c141 0 2024-04-26T04:42:27.444 INFO:tasks.workunit.client.1.smithi104.stdout:7/991: mknod d2/d4/d1d/d121/d10c/c142 0 2024-04-26T04:42:27.444 INFO:tasks.workunit.client.1.smithi104.stdout:7/992: stat d2/d4/d32/d35/d37/f86 0 2024-04-26T04:42:27.444 INFO:tasks.workunit.client.1.smithi104.stdout:7/993: fsync d2/d4/f13 0 2024-04-26T04:42:27.444 INFO:tasks.workunit.client.1.smithi104.stdout:7/994: readlink d2/d4/d32/d35/dbe/l11a 0 2024-04-26T04:42:27.444 INFO:tasks.workunit.client.1.smithi104.stdout:7/995: write d2/d4/d32/d3d/d41/d7e/de0/d127/faf [12702466,87278] 0 2024-04-26T04:42:27.444 INFO:tasks.workunit.client.0.smithi090.stdout:9/620: dread db/dd/d44/f5e [0,4194304] 0 2024-04-26T04:42:27.445 INFO:tasks.workunit.client.0.smithi090.stdout:1/648: rename d0/d4/d43/caa to d0/d4/d7/d4a/d75/ccd 0 2024-04-26T04:42:27.446 INFO:tasks.workunit.client.0.smithi090.stdout:1/649: truncate d0/d4/d7/d4a/da8/fad 487182 0 2024-04-26T04:42:27.446 INFO:tasks.workunit.client.0.smithi090.stdout:7/737: getdents d0/db/d15/dbc/dd2 0 2024-04-26T04:42:27.450 INFO:tasks.workunit.client.0.smithi090.stdout:9/621: link db/dd/d44/d82/d85/db4/l8b db/dd/d44/d82/d85/db4/d6a/db6/ld8 0 2024-04-26T04:42:27.450 INFO:tasks.workunit.client.0.smithi090.stdout:9/622: truncate db/d16/f88 198742 0 2024-04-26T04:42:27.450 INFO:tasks.workunit.client.0.smithi090.stdout:1/650: mknod d0/d55/dbd/cce 0 2024-04-26T04:42:27.450 INFO:tasks.workunit.client.0.smithi090.stdout:7/738: getdents d0/db/df/d4d/daf/dd1 0 2024-04-26T04:42:27.450 INFO:tasks.workunit.client.0.smithi090.stdout:7/739: stat d0/d1/da3/lce 0 2024-04-26T04:42:27.451 INFO:tasks.workunit.client.0.smithi090.stdout:9/623: dread db/f87 [0,4194304] 0 2024-04-26T04:42:27.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/740: rmdir d0/db/df/d1c/d100 0 2024-04-26T04:42:27.453 INFO:tasks.workunit.client.0.smithi090.stdout:7/741: fdatasync d0/db/df/d1c/fa6 0 2024-04-26T04:42:27.453 INFO:tasks.workunit.client.0.smithi090.stdout:9/624: mknod db/dd/d44/d82/d85/db4/d6a/d70/cd9 0 2024-04-26T04:42:27.453 INFO:tasks.workunit.client.0.smithi090.stdout:7/742: rmdir d0/db/df/d4d/daf/dd1 39 2024-04-26T04:42:27.458 INFO:tasks.workunit.client.1.smithi104.stdout:7/996: write d2/d4/d32/d3d/d41/d7e/de0/d127/faf [6234708,6701] 0 2024-04-26T04:42:27.459 INFO:tasks.workunit.client.1.smithi104.stdout:7/997: creat d2/d4/d32/d3d/d41/d5f/d6c/d89/d9d/dd8/f143 x:0 0 0 2024-04-26T04:42:27.460 INFO:tasks.workunit.client.0.smithi090.stdout:7/743: dread d0/d1/d4/d3e/fcf [0,4194304] 0 2024-04-26T04:42:27.460 INFO:tasks.workunit.client.1.smithi104.stdout:7/998: mkdir d2/d4/d32/d3d/d41/d5f/d6c/d144 0 2024-04-26T04:42:27.461 INFO:tasks.workunit.client.1.smithi104.stdout:7/999: unlink d2/d4/l2d 0 2024-04-26T04:42:27.466 INFO:tasks.workunit.client.0.smithi090.stdout:9/625: dread db/dd/d44/f2c [0,4194304] 0 2024-04-26T04:42:27.466 INFO:tasks.workunit.client.1.smithi104.stdout:8/945: dwrite d0/d1c/d59/d85/d127/d47/db1/fc9 [4194304,4194304] 0 2024-04-26T04:42:27.467 INFO:tasks.workunit.client.0.smithi090.stdout:4/675: dwrite d3/db/d12/d20/d24/f52 [0,4194304] 0 2024-04-26T04:42:27.467 INFO:tasks.workunit.client.0.smithi090.stdout:4/676: creat d3/d13/d64/fd1 x:0 0 0 2024-04-26T04:42:27.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/654: dwrite d4/d1a/df/d4a/f55 [0,4194304] 0 2024-04-26T04:42:27.467 INFO:tasks.workunit.client.0.smithi090.stdout:5/655: creat d4/d1a/d2a/d79/fcf x:0 0 0 2024-04-26T04:42:27.469 INFO:tasks.workunit.client.0.smithi090.stdout:9/626: link db/d5f/ca7 db/da3/da8/cda 0 2024-04-26T04:42:27.473 INFO:tasks.workunit.client.0.smithi090.stdout:4/677: creat d3/db/d12/d20/d45/fd2 x:0 0 0 2024-04-26T04:42:27.479 INFO:tasks.workunit.client.0.smithi090.stdout:5/656: link d4/d1a/d1f/d53/f76 d4/d1a/d2a/fd0 0 2024-04-26T04:42:27.481 INFO:tasks.workunit.client.0.smithi090.stdout:9/627: link db/dd/d18/d57/fbb db/dd/d18/d57/dcc/fdb 0 2024-04-26T04:42:27.482 INFO:tasks.workunit.client.0.smithi090.stdout:4/678: mknod d3/db/d12/d20/d2d/d58/d6a/cd3 0 2024-04-26T04:42:27.482 INFO:tasks.workunit.client.0.smithi090.stdout:4/679: chown d3/d13/d6c/d7e/f94 24286 1 2024-04-26T04:42:27.483 INFO:tasks.workunit.client.0.smithi090.stdout:1/651: getdents d0/d4/d7/d4a/d75 0 2024-04-26T04:42:27.483 INFO:tasks.workunit.client.1.smithi104.stdout:8/946: dwrite d0/d50/f12f [0,4194304] 0 2024-04-26T04:42:27.488 INFO:tasks.workunit.client.1.smithi104.stdout:8/947: symlink d0/d33/l137 0 2024-04-26T04:42:27.488 INFO:tasks.workunit.client.1.smithi104.stdout:8/948: write d0/d50/ff9 [748569,110190] 0 2024-04-26T04:42:27.490 INFO:tasks.workunit.client.1.smithi104.stdout:8/949: creat d0/d1c/d59/d85/d127/d11e/f138 x:0 0 0 2024-04-26T04:42:27.491 INFO:tasks.workunit.client.1.smithi104.stdout:8/950: chown d0/d1c/d59/d85/d127/f66 971 1 2024-04-26T04:42:27.494 INFO:tasks.workunit.client.0.smithi090.stdout:4/680: rename d3/db/d12/d20/d45 to d3/db/d12/d20/d24/d43/dd4 0 2024-04-26T04:42:27.506 INFO:tasks.workunit.client.1.smithi104.stdout:8/951: dread d0/d1c/d59/ff1 [0,4194304] 0 2024-04-26T04:42:27.507 INFO:tasks.workunit.client.1.smithi104.stdout:8/952: dread - d0/d33/d35/d6d/da4/f10f zero size 2024-04-26T04:42:27.508 INFO:tasks.workunit.client.1.smithi104.stdout:8/953: symlink d0/d33/d35/d6d/da4/l139 0 2024-04-26T04:42:27.508 INFO:tasks.workunit.client.1.smithi104.stdout:8/954: readlink d0/l15 0 2024-04-26T04:42:27.508 INFO:tasks.workunit.client.1.smithi104.stdout:8/955: chown d0/d33/d35/d6d/def/ff6 512935 1 2024-04-26T04:42:27.508 INFO:tasks.workunit.client.1.smithi104.stdout:8/956: chown d0/l15 4593 1 2024-04-26T04:42:27.509 INFO:tasks.workunit.client.0.smithi090.stdout:4/681: dread d3/f27 [0,4194304] 0 2024-04-26T04:42:27.509 INFO:tasks.workunit.client.0.smithi090.stdout:4/682: stat f1 0 2024-04-26T04:42:27.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/657: dwrite d4/d1a/df/f28 [4194304,4194304] 0 2024-04-26T04:42:27.509 INFO:tasks.workunit.client.0.smithi090.stdout:5/658: readlink d4/d1a/d2a/d79/d90/d49/l5f 0 2024-04-26T04:42:27.510 INFO:tasks.workunit.client.0.smithi090.stdout:4/683: symlink d3/d13/d6c/d8c/ld5 0 2024-04-26T04:42:27.510 INFO:tasks.workunit.client.0.smithi090.stdout:4/684: chown d3/db/d12/d20/d2d/d58/d6a/d74/da2 60262 1 2024-04-26T04:42:27.511 INFO:tasks.workunit.client.0.smithi090.stdout:4/685: mknod d3/db/d12/d20/d24/da9/cd6 0 2024-04-26T04:42:27.513 INFO:tasks.workunit.client.0.smithi090.stdout:4/686: mknod d3/d13/d6c/d7e/cd7 0 2024-04-26T04:42:27.515 INFO:tasks.workunit.client.0.smithi090.stdout:4/687: mkdir d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/dd8 0 2024-04-26T04:42:27.516 INFO:tasks.workunit.client.0.smithi090.stdout:9/628: dwrite db/f1b [4194304,4194304] 0 2024-04-26T04:42:27.517 INFO:tasks.workunit.client.0.smithi090.stdout:1/652: dwrite d0/d4/d7/d49/f6f [4194304,4194304] 0 2024-04-26T04:42:27.518 INFO:tasks.workunit.client.0.smithi090.stdout:4/688: dread d3/db/d12/d20/d2d/d2e/f77 [0,4194304] 0 2024-04-26T04:42:27.518 INFO:tasks.workunit.client.0.smithi090.stdout:5/659: write d4/d1a/d45/f74 [1117064,93200] 0 2024-04-26T04:42:27.520 INFO:tasks.workunit.client.0.smithi090.stdout:9/629: write db/f47 [553088,65028] 0 2024-04-26T04:42:27.527 INFO:tasks.workunit.client.0.smithi090.stdout:9/630: rename db/dd/d44/f34 to db/fdc 0 2024-04-26T04:42:27.529 INFO:tasks.workunit.client.0.smithi090.stdout:9/631: unlink db/dd/d18/d57/f68 0 2024-04-26T04:42:27.529 INFO:tasks.workunit.client.0.smithi090.stdout:9/632: truncate db/dd/d44/d82/d85/db4/d58/f97 498070 0 2024-04-26T04:42:27.529 INFO:tasks.workunit.client.0.smithi090.stdout:9/633: truncate db/dd/d18/d57/f94 127390 0 2024-04-26T04:42:27.529 INFO:tasks.workunit.client.0.smithi090.stdout:9/634: readlink db/d16/l71 0 2024-04-26T04:42:27.531 INFO:tasks.workunit.client.0.smithi090.stdout:4/689: truncate d3/db/d12/d20/d2d/d58/f63 3805359 0 2024-04-26T04:42:27.533 INFO:tasks.workunit.client.0.smithi090.stdout:4/690: unlink d3/db/d12/d20/d24/d43/d72/f7c 0 2024-04-26T04:42:27.533 INFO:tasks.workunit.client.0.smithi090.stdout:4/691: fdatasync d3/db/d12/d20/d24/d43/dd4/d53/f6e 0 2024-04-26T04:42:27.534 INFO:tasks.workunit.client.0.smithi090.stdout:4/692: mknod d3/d13/d1d/d50/d51/cd9 0 2024-04-26T04:42:27.534 INFO:tasks.workunit.client.0.smithi090.stdout:4/693: chown d3/db/d12/d3b/c7b 5 1 2024-04-26T04:42:27.536 INFO:tasks.workunit.client.1.smithi104.stdout:8/957: dwrite d0/f9 [0,4194304] 0 2024-04-26T04:42:27.572 INFO:tasks.workunit.client.0.smithi090.stdout:5/660: dwrite d4/d1a/fcb [0,4194304] 0 2024-04-26T04:42:27.576 INFO:tasks.workunit.client.0.smithi090.stdout:5/661: mknod d4/cd1 0 2024-04-26T04:42:27.576 INFO:tasks.workunit.client.0.smithi090.stdout:5/662: symlink d4/d1a/d2a/d31/ld2 0 2024-04-26T04:42:27.577 INFO:tasks.workunit.client.0.smithi090.stdout:1/653: dwrite d0/d4/d7/d49/fa9 [0,4194304] 0 2024-04-26T04:42:27.577 INFO:tasks.workunit.client.0.smithi090.stdout:9/635: dwrite db/dd/d18/d38/d39/d3f/f76 [0,4194304] 0 2024-04-26T04:42:27.578 INFO:tasks.workunit.client.0.smithi090.stdout:9/636: chown db/c10 0 1 2024-04-26T04:42:27.580 INFO:tasks.workunit.client.1.smithi104.stdout:8/958: dwrite d0/d1c/d59/d85/d127/d11e/f12b [0,4194304] 0 2024-04-26T04:42:27.581 INFO:tasks.workunit.client.0.smithi090.stdout:4/694: dwrite d3/d13/d64/dae/da5/fca [0,4194304] 0 2024-04-26T04:42:27.581 INFO:tasks.workunit.client.0.smithi090.stdout:4/695: fsync d3/db/d12/d20/d24/d43/dd4/fb7 0 2024-04-26T04:42:27.581 INFO:tasks.workunit.client.0.smithi090.stdout:1/654: rename d0/d4/d7/d17/d22/f66 to d0/d4/d62/d78/fcf 0 2024-04-26T04:42:27.581 INFO:tasks.workunit.client.0.smithi090.stdout:1/655: chown d0/d4/d7/d17/d22/l64 339146 1 2024-04-26T04:42:27.583 INFO:tasks.workunit.client.0.smithi090.stdout:4/696: mkdir d3/db/d12/d20/d2d/d2e/dda 0 2024-04-26T04:42:27.584 INFO:tasks.workunit.client.1.smithi104.stdout:9/909: sync 2024-04-26T04:42:27.587 INFO:tasks.workunit.client.1.smithi104.stdout:9/910: symlink d11/d15/d32/l130 0 2024-04-26T04:42:27.601 INFO:tasks.workunit.client.1.smithi104.stdout:9/911: write d11/d15/d32/d35/f80 [1418241,79088] 0 2024-04-26T04:42:27.601 INFO:tasks.workunit.client.1.smithi104.stdout:9/912: chown l0 15546220 1 2024-04-26T04:42:27.601 INFO:tasks.workunit.client.1.smithi104.stdout:9/913: chown d11/d1d/d34/db3/df1 2 1 2024-04-26T04:42:27.601 INFO:tasks.workunit.client.1.smithi104.stdout:9/914: truncate d11/d15/da3/f114 507925 0 2024-04-26T04:42:27.621 INFO:tasks.workunit.client.0.smithi090.stdout:1/656: dwrite d0/d4/d7/d49/f6f [0,4194304] 0 2024-04-26T04:42:27.648 INFO:tasks.workunit.client.1.smithi104.stdout:8/959: dwrite d0/d1c/d59/d85/d127/d47/d67/d76/daf/feb [4194304,4194304] 0 2024-04-26T04:42:27.649 INFO:tasks.workunit.client.0.smithi090.stdout:4/697: dwrite d3/db/d12/d20/d24/d43/dd4/d53/f6e [4194304,4194304] 0 2024-04-26T04:42:27.649 INFO:tasks.workunit.client.0.smithi090.stdout:4/698: symlink d3/db/d12/d20/d24/d43/ldb 0 2024-04-26T04:42:27.650 INFO:tasks.workunit.client.0.smithi090.stdout:1/657: dwrite d0/f94 [0,4194304] 0 2024-04-26T04:42:27.650 INFO:tasks.workunit.client.1.smithi104.stdout:9/915: dwrite f4 [4194304,4194304] 0 2024-04-26T04:42:27.651 INFO:tasks.workunit.client.1.smithi104.stdout:8/960: rename d0/d1c/d59/d85/d127/d47/d67/d76/d7f/cf2 to d0/d33/d35/d6d/da4/c13a 0 2024-04-26T04:42:27.651 INFO:tasks.workunit.client.1.smithi104.stdout:8/961: fsync d0/d33/d35/ffd 0 2024-04-26T04:42:27.651 INFO:tasks.workunit.client.0.smithi090.stdout:1/658: write d0/d4/d7/f90 [668713,20834] 0 2024-04-26T04:42:27.652 INFO:tasks.workunit.client.1.smithi104.stdout:8/962: truncate d0/d1c/d2d/f48 7840771 0 2024-04-26T04:42:27.654 INFO:tasks.workunit.client.1.smithi104.stdout:8/963: mkdir d0/d1c/d13b 0 2024-04-26T04:42:27.655 INFO:tasks.workunit.client.0.smithi090.stdout:1/659: mknod d0/d4/d7/cd0 0 2024-04-26T04:42:27.655 INFO:tasks.workunit.client.0.smithi090.stdout:1/660: creat d0/d55/fd1 x:0 0 0 2024-04-26T04:42:27.659 INFO:tasks.workunit.client.0.smithi090.stdout:1/661: getdents d0/d4/d7/d4a/d75/d80 0 2024-04-26T04:42:27.659 INFO:tasks.workunit.client.0.smithi090.stdout:1/662: chown d0/d4/d62/d78/f8f 220 1 2024-04-26T04:42:27.661 INFO:tasks.workunit.client.0.smithi090.stdout:1/663: link d0/d4/d7/f2c d0/d55/dbd/fd2 0 2024-04-26T04:42:27.663 INFO:tasks.workunit.client.0.smithi090.stdout:1/664: mknod d0/d4/d7/d17/d22/d93/db8/cd3 0 2024-04-26T04:42:27.663 INFO:tasks.workunit.client.0.smithi090.stdout:1/665: write d0/d4/d7/d4a/f77 [1041917,75797] 0 2024-04-26T04:42:27.663 INFO:tasks.workunit.client.0.smithi090.stdout:1/666: dread - d0/fc3 zero size 2024-04-26T04:42:27.663 INFO:tasks.workunit.client.0.smithi090.stdout:1/667: truncate d0/d55/fb9 304110 0 2024-04-26T04:42:27.663 INFO:tasks.workunit.client.1.smithi104.stdout:9/916: dread d11/d15/d1b/d6d/dd5/d12e/ddc/fff [0,4194304] 0 2024-04-26T04:42:27.664 INFO:tasks.workunit.client.0.smithi090.stdout:1/668: creat d0/d4/d7/d4b/fd4 x:0 0 0 2024-04-26T04:42:27.664 INFO:tasks.workunit.client.0.smithi090.stdout:1/669: write d0/d4/d7/d17/f18 [1741177,89622] 0 2024-04-26T04:42:27.664 INFO:tasks.workunit.client.1.smithi104.stdout:9/917: mkdir d11/d1d/d34/d69/d104/d131 0 2024-04-26T04:42:27.665 INFO:tasks.workunit.client.1.smithi104.stdout:9/918: creat d11/d15/d1b/d6d/d9c/f132 x:0 0 0 2024-04-26T04:42:27.666 INFO:tasks.workunit.client.0.smithi090.stdout:1/670: symlink d0/d4/d7/d17/d22/d93/ld5 0 2024-04-26T04:42:27.667 INFO:tasks.workunit.client.0.smithi090.stdout:1/671: creat d0/d4/d7/d4a/d75/fd6 x:0 0 0 2024-04-26T04:42:27.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/672: getdents d0/d4/d7/d4a 0 2024-04-26T04:42:27.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/673: write d0/d4/d7/d4b/f4c [4713641,47308] 0 2024-04-26T04:42:27.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/674: stat d0/d4/d7/l99 0 2024-04-26T04:42:27.670 INFO:tasks.workunit.client.0.smithi090.stdout:1/675: mknod d0/d4/d7/d4a/d75/d80/d58/d97/cd7 0 2024-04-26T04:42:27.670 INFO:tasks.workunit.client.0.smithi090.stdout:1/676: stat d0/d4/f91 0 2024-04-26T04:42:27.670 INFO:tasks.workunit.client.0.smithi090.stdout:1/677: dread - d0/d4/d7/d4a/d75/d80/fc6 zero size 2024-04-26T04:42:27.671 INFO:tasks.workunit.client.0.smithi090.stdout:1/678: rmdir d0/d4/d7/d4b/d53 39 2024-04-26T04:42:27.671 INFO:tasks.workunit.client.0.smithi090.stdout:1/679: getdents d0/d4/d62/d78/dcc 0 2024-04-26T04:42:27.673 INFO:tasks.workunit.client.0.smithi090.stdout:6/751: sync 2024-04-26T04:42:27.673 INFO:tasks.workunit.client.0.smithi090.stdout:1/680: rename d0/d4/d7/f2c to d0/fd8 0 2024-04-26T04:42:27.675 INFO:tasks.workunit.client.0.smithi090.stdout:1/681: symlink d0/d4/d7/d4a/d75/d80/ld9 0 2024-04-26T04:42:27.679 INFO:tasks.workunit.client.1.smithi104.stdout:9/919: write d11/d15/d32/f83 [467318,126414] 0 2024-04-26T04:42:27.684 INFO:tasks.workunit.client.0.smithi090.stdout:1/682: write d0/d55/f67 [2022085,101057] 0 2024-04-26T04:42:27.721 INFO:tasks.workunit.client.0.smithi090.stdout:4/699: dwrite d3/db/d12/d20/d24/d43/dd4/f96 [4194304,4194304] 0 2024-04-26T04:42:27.722 INFO:tasks.workunit.client.1.smithi104.stdout:8/964: dwrite d0/d1c/d59/d85/d127/d47/d67/d76/d7f/fd8 [0,4194304] 0 2024-04-26T04:42:27.723 INFO:tasks.workunit.client.0.smithi090.stdout:6/752: dwrite d9/d3c/ffc [0,4194304] 0 2024-04-26T04:42:27.724 INFO:tasks.workunit.client.1.smithi104.stdout:8/965: mkdir d0/d1c/d2d/d7a/d112/d13c 0 2024-04-26T04:42:27.724 INFO:tasks.workunit.client.1.smithi104.stdout:9/920: dwrite d11/d15/d98/f11f [0,4194304] 0 2024-04-26T04:42:27.725 INFO:tasks.workunit.client.0.smithi090.stdout:4/700: symlink d3/db/d12/d99/dce/ldc 0 2024-04-26T04:42:27.726 INFO:tasks.workunit.client.1.smithi104.stdout:9/921: creat d11/d15/d32/d4c/d8a/f133 x:0 0 0 2024-04-26T04:42:27.729 INFO:tasks.workunit.client.1.smithi104.stdout:9/922: creat d11/d1d/d87/d11b/f134 x:0 0 0 2024-04-26T04:42:27.737 INFO:tasks.workunit.client.0.smithi090.stdout:1/683: rename d0/d4 to d0/d55/dda 0 2024-04-26T04:42:27.740 INFO:tasks.workunit.client.1.smithi104.stdout:9/923: read d11/d1d/d34/db3/fee [3505816,94213] 0 2024-04-26T04:42:27.741 INFO:tasks.workunit.client.1.smithi104.stdout:9/924: link d11/f20 d11/d1d/d34/d69/d104/f135 0 2024-04-26T04:42:27.743 INFO:tasks.workunit.client.1.smithi104.stdout:9/925: rename d11/d53/l5a to d11/d15/d32/dd6/l136 0 2024-04-26T04:42:27.743 INFO:tasks.workunit.client.1.smithi104.stdout:9/926: write d11/d53/f108 [1637276,11667] 0 2024-04-26T04:42:27.743 INFO:tasks.workunit.client.1.smithi104.stdout:9/927: symlink d11/d53/l137 0 2024-04-26T04:42:27.745 INFO:tasks.workunit.client.1.smithi104.stdout:9/928: symlink d11/l138 0 2024-04-26T04:42:27.745 INFO:tasks.workunit.client.1.smithi104.stdout:9/929: chown d11/d1d/d34/db3 3 1 2024-04-26T04:42:27.745 INFO:tasks.workunit.client.1.smithi104.stdout:9/930: read d11/d15/d32/d4c/d7c/fb4 [1037147,41292] 0 2024-04-26T04:42:27.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/684: read d0/d55/dda/fa4 [886405,30761] 0 2024-04-26T04:42:27.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/685: dread - d0/d55/dda/d7/d4b/d53/f92 zero size 2024-04-26T04:42:27.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/686: chown d0/d55/dda/d43/f9f 161152 1 2024-04-26T04:42:27.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/687: truncate d0/d55/fd1 133872 0 2024-04-26T04:42:27.760 INFO:tasks.workunit.client.1.smithi104.stdout:8/966: dwrite d0/d33/d35/d6d/f105 [0,4194304] 0 2024-04-26T04:42:27.769 INFO:tasks.workunit.client.0.smithi090.stdout:6/753: dwrite d9/d77/db0/db5/dda/d6d/f98 [0,4194304] 0 2024-04-26T04:42:27.776 INFO:tasks.workunit.client.0.smithi090.stdout:4/701: dwrite d3/db/d12/d20/f41 [0,4194304] 0 2024-04-26T04:42:27.776 INFO:tasks.workunit.client.0.smithi090.stdout:4/702: dread - d3/d13/f9f zero size 2024-04-26T04:42:27.784 INFO:tasks.workunit.client.0.smithi090.stdout:6/754: write d9/d77/db0/db5/dda/d6d/d66/f75 [665762,67982] 0 2024-04-26T04:42:27.784 INFO:tasks.workunit.client.0.smithi090.stdout:6/755: write d9/d21/d3b/f46 [1746417,27142] 0 2024-04-26T04:42:27.784 INFO:tasks.workunit.client.0.smithi090.stdout:4/703: dread f1 [0,4194304] 0 2024-04-26T04:42:27.785 INFO:tasks.workunit.client.0.smithi090.stdout:6/756: truncate d9/d21/d40/f74 2598758 0 2024-04-26T04:42:27.786 INFO:tasks.workunit.client.0.smithi090.stdout:4/704: truncate d3/f6d 16829 0 2024-04-26T04:42:27.786 INFO:tasks.workunit.client.0.smithi090.stdout:6/757: creat d9/d77/db0/f103 x:0 0 0 2024-04-26T04:42:27.787 INFO:tasks.workunit.client.0.smithi090.stdout:4/705: mkdir d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/ddd 0 2024-04-26T04:42:27.788 INFO:tasks.workunit.client.0.smithi090.stdout:6/758: creat d9/d1c/dc5/f104 x:0 0 0 2024-04-26T04:42:27.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/706: mkdir d3/db/d12/d20/d2d/dbe/dde 0 2024-04-26T04:42:27.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/707: readlink d3/db/d12/d20/l31 0 2024-04-26T04:42:27.790 INFO:tasks.workunit.client.0.smithi090.stdout:6/759: creat d9/d21/f105 x:0 0 0 2024-04-26T04:42:27.790 INFO:tasks.workunit.client.0.smithi090.stdout:6/760: fdatasync d9/d77/f9d 0 2024-04-26T04:42:27.790 INFO:tasks.workunit.client.1.smithi104.stdout:9/931: dwrite d11/d1d/d34/d69/fce [4194304,4194304] 0 2024-04-26T04:42:27.790 INFO:tasks.workunit.client.0.smithi090.stdout:4/708: mknod d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/cdf 0 2024-04-26T04:42:27.791 INFO:tasks.workunit.client.1.smithi104.stdout:9/932: dread d11/d15/f6a [0,4194304] 0 2024-04-26T04:42:27.792 INFO:tasks.workunit.client.0.smithi090.stdout:4/709: write d3/db/d12/d20/d2d/d58/d6a/d74/da2/f37 [432914,46388] 0 2024-04-26T04:42:27.792 INFO:tasks.workunit.client.0.smithi090.stdout:4/710: chown d3/db/d12/d20/c25 26400015 1 2024-04-26T04:42:27.793 INFO:tasks.workunit.client.1.smithi104.stdout:9/933: rmdir d11/d15/d1b/d6d/d9c/dbe/def 39 2024-04-26T04:42:27.793 INFO:tasks.workunit.client.1.smithi104.stdout:8/967: dwrite d0/d1c/d59/d85/d127/d47/d67/d76/fea [0,4194304] 0 2024-04-26T04:42:27.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/688: dwrite d0/d55/dda/d7/d17/d22/f52 [0,4194304] 0 2024-04-26T04:42:27.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/689: dread - d0/d55/dda/d7/d17/f79 zero size 2024-04-26T04:42:27.794 INFO:tasks.workunit.client.0.smithi090.stdout:1/690: fsync d0/d55/dda/d62/d78/fcf 0 2024-04-26T04:42:27.794 INFO:tasks.workunit.client.0.smithi090.stdout:4/711: mknod d3/d13/d64/dae/dc9/ce0 0 2024-04-26T04:42:27.794 INFO:tasks.workunit.client.0.smithi090.stdout:4/712: creat d3/db/d12/d20/d2d/d2e/fe1 x:0 0 0 2024-04-26T04:42:27.795 INFO:tasks.workunit.client.1.smithi104.stdout:9/934: symlink d11/d1d/d34/db3/df1/l139 0 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:8/837: sync 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:3/636: sync 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:0/760: sync 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:2/709: sync 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:3/637: stat d1/d17/d61 0 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:0/761: truncate d0/de/f89 849200 0 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:0/762: chown d0/d4b/d5d/d48/d100 203911643 1 2024-04-26T04:42:27.798 INFO:tasks.workunit.client.0.smithi090.stdout:8/838: creat dc/dd2/d4f/da6/dd8/f122 x:0 0 0 2024-04-26T04:42:27.799 INFO:tasks.workunit.client.1.smithi104.stdout:8/968: mknod d0/d1c/d59/d85/d127/d47/d67/d76/c13d 0 2024-04-26T04:42:27.801 INFO:tasks.workunit.client.1.smithi104.stdout:9/935: link d11/c8c d11/d15/d1b/d6d/d9c/dfb/d102/c13a 0 2024-04-26T04:42:27.801 INFO:tasks.workunit.client.1.smithi104.stdout:9/936: fsync d11/d1d/d34/f41 0 2024-04-26T04:42:27.804 INFO:tasks.workunit.client.1.smithi104.stdout:9/937: dread d11/fc6 [0,4194304] 0 2024-04-26T04:42:27.804 INFO:tasks.workunit.client.0.smithi090.stdout:2/710: rename db/d11/d14/d1b/l35 to db/d59/d66/d69/lec 0 2024-04-26T04:42:27.805 INFO:tasks.workunit.client.0.smithi090.stdout:2/711: write db/dc/f8e [3325847,3981] 0 2024-04-26T04:42:27.805 INFO:tasks.workunit.client.1.smithi104.stdout:9/938: mkdir d11/d1d/d71/d94/d13b 0 2024-04-26T04:42:27.808 INFO:tasks.workunit.client.1.smithi104.stdout:9/939: link d11/d90/dfd/l11d d11/d1d/d34/db3/df1/d110/l13c 0 2024-04-26T04:42:27.808 INFO:tasks.workunit.client.1.smithi104.stdout:9/940: dread - d11/d15/d32/dd6/ff4 zero size 2024-04-26T04:42:27.808 INFO:tasks.workunit.client.0.smithi090.stdout:3/638: link d1/d9/f12 d1/d17/d61/d9d/da7/fd8 0 2024-04-26T04:42:27.809 INFO:tasks.workunit.client.0.smithi090.stdout:3/639: dread - d1/d17/d61/d9d/da7/fcb zero size 2024-04-26T04:42:27.809 INFO:tasks.workunit.client.0.smithi090.stdout:3/640: readlink d1/d17/d36/d2a/d3a/d5d/l93 0 2024-04-26T04:42:27.809 INFO:tasks.workunit.client.0.smithi090.stdout:8/839: symlink dc/dd2/d4f/l123 0 2024-04-26T04:42:27.810 INFO:tasks.workunit.client.1.smithi104.stdout:9/941: symlink d11/d1d/d87/dda/l13d 0 2024-04-26T04:42:27.811 INFO:tasks.workunit.client.0.smithi090.stdout:4/713: dread d3/d13/f61 [0,4194304] 0 2024-04-26T04:42:27.812 INFO:tasks.workunit.client.0.smithi090.stdout:2/712: symlink db/dc/led 0 2024-04-26T04:42:27.813 INFO:tasks.workunit.client.0.smithi090.stdout:2/713: stat db/d11/d14/d25/d65 0 2024-04-26T04:42:27.813 INFO:tasks.workunit.client.0.smithi090.stdout:2/714: fdatasync db/d11/d14/d27/d2f/d42/d5c/fb3 0 2024-04-26T04:42:27.813 INFO:tasks.workunit.client.0.smithi090.stdout:2/715: creat db/d11/d14/fee x:0 0 0 2024-04-26T04:42:27.813 INFO:tasks.workunit.client.0.smithi090.stdout:2/716: fdatasync db/d59/f78 0 2024-04-26T04:42:27.813 INFO:tasks.workunit.client.0.smithi090.stdout:6/761: dwrite d9/d21/f105 [0,4194304] 0 2024-04-26T04:42:27.813 INFO:tasks.workunit.client.1.smithi104.stdout:9/942: unlink d11/d53/d84/fcf 0 2024-04-26T04:42:27.813 INFO:tasks.workunit.client.1.smithi104.stdout:8/969: write d0/d50/ffe [2250343,37136] 0 2024-04-26T04:42:27.814 INFO:tasks.workunit.client.0.smithi090.stdout:8/840: symlink dc/dd2/d3b/d3d/l124 0 2024-04-26T04:42:27.816 INFO:tasks.workunit.client.0.smithi090.stdout:4/714: symlink d3/db/d12/d99/dce/le2 0 2024-04-26T04:42:27.820 INFO:tasks.workunit.client.0.smithi090.stdout:3/641: write d1/d17/f1f [1715880,46799] 0 2024-04-26T04:42:27.820 INFO:tasks.workunit.client.0.smithi090.stdout:3/642: fsync d1/d9/fd6 0 2024-04-26T04:42:27.820 INFO:tasks.workunit.client.0.smithi090.stdout:3/643: creat d1/d17/d61/d9d/da7/fd9 x:0 0 0 2024-04-26T04:42:27.822 INFO:tasks.workunit.client.0.smithi090.stdout:2/717: read db/d59/f62 [396145,102104] 0 2024-04-26T04:42:27.824 INFO:tasks.workunit.client.0.smithi090.stdout:4/715: creat d3/d13/d1d/d50/fe3 x:0 0 0 2024-04-26T04:42:27.827 INFO:tasks.workunit.client.0.smithi090.stdout:7/744: sync 2024-04-26T04:42:27.827 INFO:tasks.workunit.client.0.smithi090.stdout:9/637: sync 2024-04-26T04:42:27.827 INFO:tasks.workunit.client.0.smithi090.stdout:5/663: sync 2024-04-26T04:42:27.827 INFO:tasks.workunit.client.0.smithi090.stdout:5/664: chown d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 4 1 2024-04-26T04:42:27.828 INFO:tasks.workunit.client.1.smithi104.stdout:9/943: write d11/d15/f59 [2001590,49964] 0 2024-04-26T04:42:27.828 INFO:tasks.workunit.client.1.smithi104.stdout:9/944: chown d11/d1d/d34/db3/lbf 179265 1 2024-04-26T04:42:27.828 INFO:tasks.workunit.client.1.smithi104.stdout:9/945: stat d11/d15/da3/f114 0 2024-04-26T04:42:27.828 INFO:tasks.workunit.client.0.smithi090.stdout:6/762: rmdir d9/d77/db0/db5 39 2024-04-26T04:42:27.828 INFO:tasks.workunit.client.0.smithi090.stdout:6/763: readlink d9/d77/db0/db5/dda/d58/l65 0 2024-04-26T04:42:27.828 INFO:tasks.workunit.client.0.smithi090.stdout:6/764: stat d9/d1c/d31/d69/db7/dea 0 2024-04-26T04:42:27.830 INFO:tasks.workunit.client.0.smithi090.stdout:8/841: rename dc/d5a to dc/dd2/d39/d125 0 2024-04-26T04:42:27.832 INFO:tasks.workunit.client.0.smithi090.stdout:5/665: dread d4/d1a/d2a/d79/d90/d49/f82 [0,4194304] 0 2024-04-26T04:42:27.832 INFO:tasks.workunit.client.0.smithi090.stdout:4/716: rename d3/db/d12/d20/d2d/la8 to d3/db/d12/d20/d24/d43/le4 0 2024-04-26T04:42:27.833 INFO:tasks.workunit.client.0.smithi090.stdout:4/717: getdents d3/db/d12/d20/d2d/d2e/dda 0 2024-04-26T04:42:27.834 INFO:tasks.workunit.client.0.smithi090.stdout:8/842: link dc/dd2/d39/d125/d5d/d73/dce/ff2 dc/dd2/d3b/d3d/f126 0 2024-04-26T04:42:27.834 INFO:tasks.workunit.client.0.smithi090.stdout:8/843: stat dc/l78 0 2024-04-26T04:42:27.835 INFO:tasks.workunit.client.0.smithi090.stdout:5/666: symlink d4/d1a/d2a/d79/d90/db3/ld3 0 2024-04-26T04:42:27.836 INFO:tasks.workunit.client.0.smithi090.stdout:8/844: unlink dc/d1f/l48 0 2024-04-26T04:42:27.838 INFO:tasks.workunit.client.0.smithi090.stdout:8/845: creat dc/dd2/d4f/da6/dfc/f127 x:0 0 0 2024-04-26T04:42:27.841 INFO:tasks.workunit.client.0.smithi090.stdout:3/644: read d1/d17/d36/d33/d68/dae/dca/f14 [1997753,128957] 0 2024-04-26T04:42:27.843 INFO:tasks.workunit.client.0.smithi090.stdout:3/645: symlink d1/d17/d36/d2a/dc7/lda 0 2024-04-26T04:42:27.845 INFO:tasks.workunit.client.0.smithi090.stdout:7/745: dread d0/db/d53/f2d [0,4194304] 0 2024-04-26T04:42:27.845 INFO:tasks.workunit.client.1.smithi104.stdout:9/946: dread f6 [0,4194304] 0 2024-04-26T04:42:27.846 INFO:tasks.workunit.client.1.smithi104.stdout:9/947: dread - d11/d15/d1b/f123 zero size 2024-04-26T04:42:27.846 INFO:tasks.workunit.client.0.smithi090.stdout:9/638: read db/dd/d18/f3a [1918180,40344] 0 2024-04-26T04:42:27.846 INFO:tasks.workunit.client.0.smithi090.stdout:3/646: mknod d1/d17/d36/d2a/d3a/d5d/d82/cdb 0 2024-04-26T04:42:27.848 INFO:tasks.workunit.client.0.smithi090.stdout:9/639: rmdir db/dd/d44/d82/d85/db4/d6a/db6 39 2024-04-26T04:42:27.849 INFO:tasks.workunit.client.1.smithi104.stdout:9/948: dread d11/d1d/d87/f125 [0,4194304] 0 2024-04-26T04:42:27.850 INFO:tasks.workunit.client.0.smithi090.stdout:3/647: unlink d1/d17/d36/d33/f89 0 2024-04-26T04:42:27.850 INFO:tasks.workunit.client.0.smithi090.stdout:3/648: write d1/d17/d36/d33/d42/f63 [1678830,10661] 0 2024-04-26T04:42:27.850 INFO:tasks.workunit.client.0.smithi090.stdout:3/649: fdatasync d1/d17/d61/d9d/da7/fd8 0 2024-04-26T04:42:27.853 INFO:tasks.workunit.client.0.smithi090.stdout:9/640: rename db/d16/l6d to db/ldd 0 2024-04-26T04:42:27.857 INFO:tasks.workunit.client.0.smithi090.stdout:9/641: mkdir db/dd/d44/d82/d85/db4/dde 0 2024-04-26T04:42:27.858 INFO:tasks.workunit.client.0.smithi090.stdout:9/642: mknod db/dd/d18/d38/cdf 0 2024-04-26T04:42:27.861 INFO:tasks.workunit.client.0.smithi090.stdout:9/643: rename db/dd/d18/d38/d39/d6b/dc0/cce to db/dd/d18/d38/d39/dba/ce0 0 2024-04-26T04:42:27.866 INFO:tasks.workunit.client.0.smithi090.stdout:0/763: dwrite d0/d4b/d6d/f9a [0,4194304] 0 2024-04-26T04:42:27.866 INFO:tasks.workunit.client.0.smithi090.stdout:0/764: write d0/d4b/d5d/d5c/d8a/d7c/f95 [12775,11007] 0 2024-04-26T04:42:27.866 INFO:tasks.workunit.client.0.smithi090.stdout:0/765: dread - d0/d4b/d5d/d48/fa5 zero size 2024-04-26T04:42:27.877 INFO:tasks.workunit.client.0.smithi090.stdout:3/650: write d1/d59/d5e/f62 [772792,47906] 0 2024-04-26T04:42:27.878 INFO:tasks.workunit.client.0.smithi090.stdout:3/651: link d1/d17/d36/d33/daa/lc6 d1/d17/d36/d2a/d3a/ldc 0 2024-04-26T04:42:27.878 INFO:tasks.workunit.client.0.smithi090.stdout:3/652: readlink d1/d17/d36/d33/la2 0 2024-04-26T04:42:27.879 INFO:tasks.workunit.client.0.smithi090.stdout:3/653: creat d1/d17/d36/d2a/d3a/d5d/fdd x:0 0 0 2024-04-26T04:42:27.879 INFO:tasks.workunit.client.0.smithi090.stdout:3/654: readlink d1/d17/d36/d33/d68/dae/dca/d2c/l77 0 2024-04-26T04:42:27.881 INFO:tasks.workunit.client.0.smithi090.stdout:0/766: dread d0/de/f2f [0,4194304] 0 2024-04-26T04:42:27.882 INFO:tasks.workunit.client.0.smithi090.stdout:0/767: mkdir d0/d4b/d5d/d5c/d8a/d45/d107 0 2024-04-26T04:42:27.883 INFO:tasks.workunit.client.0.smithi090.stdout:0/768: creat d0/d4b/d5d/d48/dc1/f108 x:0 0 0 2024-04-26T04:42:27.907 INFO:tasks.workunit.client.0.smithi090.stdout:2/718: dwrite db/d11/d14/d63/fd5 [0,4194304] 0 2024-04-26T04:42:27.908 INFO:tasks.workunit.client.0.smithi090.stdout:2/719: link db/d59/d66/d77/lcd db/d11/d14/d25/d86/db9/lef 0 2024-04-26T04:42:27.909 INFO:tasks.workunit.client.0.smithi090.stdout:2/720: symlink db/d11/d14/d1b/d20/lf0 0 2024-04-26T04:42:27.911 INFO:tasks.workunit.client.0.smithi090.stdout:2/721: write db/d11/d14/d27/f38 [473326,56670] 0 2024-04-26T04:42:27.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/722: write db/d11/d14/d27/d2f/d42/d5c/f4f [1119497,3759] 0 2024-04-26T04:42:27.917 INFO:tasks.workunit.client.0.smithi090.stdout:2/723: chown db/d11/d14/d1b/d20/d4c/d98 7 1 2024-04-26T04:42:27.918 INFO:tasks.workunit.client.0.smithi090.stdout:2/724: stat db/d11/d14/d1b/d20/c23 0 2024-04-26T04:42:27.919 INFO:tasks.workunit.client.0.smithi090.stdout:2/725: link db/d11/d14/d63/c6b db/cf1 0 2024-04-26T04:42:27.920 INFO:tasks.workunit.client.0.smithi090.stdout:2/726: stat db/d59/d66/d77/lcd 0 2024-04-26T04:42:27.920 INFO:tasks.workunit.client.0.smithi090.stdout:2/727: dread - db/d11/d14/d1b/f6d zero size 2024-04-26T04:42:27.922 INFO:tasks.workunit.client.0.smithi090.stdout:1/691: dwrite d0/d55/dda/d7/d69/f71 [0,4194304] 0 2024-04-26T04:42:27.922 INFO:tasks.workunit.client.0.smithi090.stdout:1/692: fsync d0/d55/dda/d62/d78/fcf 0 2024-04-26T04:42:27.922 INFO:tasks.workunit.client.0.smithi090.stdout:1/693: write d0/d55/dda/f26 [1330005,92021] 0 2024-04-26T04:42:27.922 INFO:tasks.workunit.client.0.smithi090.stdout:7/746: dwrite d0/f45 [0,4194304] 0 2024-04-26T04:42:27.923 INFO:tasks.workunit.client.0.smithi090.stdout:6/765: dwrite d9/d1c/dc5/f104 [0,4194304] 0 2024-04-26T04:42:27.923 INFO:tasks.workunit.client.0.smithi090.stdout:7/747: truncate d0/db/d53/db7/f3a 4788186 0 2024-04-26T04:42:27.923 INFO:tasks.workunit.client.1.smithi104.stdout:8/970: dwrite d0/d33/f2c [0,4194304] 0 2024-04-26T04:42:27.925 INFO:tasks.workunit.client.0.smithi090.stdout:4/718: dwrite d3/d13/d1d/d50/fe3 [0,4194304] 0 2024-04-26T04:42:27.925 INFO:tasks.workunit.client.0.smithi090.stdout:4/719: fsync d3/f48 0 2024-04-26T04:42:27.925 INFO:tasks.workunit.client.0.smithi090.stdout:4/720: chown d3/db/d12/d20/d2d/dbe 78585861 1 2024-04-26T04:42:27.929 INFO:tasks.workunit.client.1.smithi104.stdout:8/971: dread d0/d1c/d59/d85/d127/d47/f1d [0,4194304] 0 2024-04-26T04:42:27.929 INFO:tasks.workunit.client.1.smithi104.stdout:8/972: chown d0/l8b 0 1 2024-04-26T04:42:27.929 INFO:tasks.workunit.client.0.smithi090.stdout:6/766: mknod d9/d77/d9a/dc8/c106 0 2024-04-26T04:42:27.936 INFO:tasks.workunit.client.1.smithi104.stdout:8/973: rename d0/d33/da6/c104 to d0/d33/da6/c13e 0 2024-04-26T04:42:27.936 INFO:tasks.workunit.client.1.smithi104.stdout:8/974: creat d0/d1c/d2d/d7a/d112/f13f x:0 0 0 2024-04-26T04:42:27.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/767: mknod d9/d77/dc0/c107 0 2024-04-26T04:42:27.936 INFO:tasks.workunit.client.0.smithi090.stdout:6/768: chown d9/d3c/c71 12354737 1 2024-04-26T04:42:27.938 INFO:tasks.workunit.client.1.smithi104.stdout:8/975: symlink d0/d33/d35/d6d/da4/d128/l140 0 2024-04-26T04:42:27.940 INFO:tasks.workunit.client.1.smithi104.stdout:8/976: creat d0/d33/d35/d6d/da4/d128/f141 x:0 0 0 2024-04-26T04:42:27.943 INFO:tasks.workunit.client.1.smithi104.stdout:8/977: rename d0/d1c/d2d/f48 to d0/d33/d35/d6d/f142 0 2024-04-26T04:42:27.945 INFO:tasks.workunit.client.1.smithi104.stdout:8/978: truncate d0/d33/f7b 723663 0 2024-04-26T04:42:27.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/748: dread d0/db/df/d1c/d23/f82 [4194304,4194304] 0 2024-04-26T04:42:27.945 INFO:tasks.workunit.client.0.smithi090.stdout:7/749: truncate d0/db/df/d1c/fa6 1260657 0 2024-04-26T04:42:27.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/750: chown d0/db/df/d4d/f63 678 1 2024-04-26T04:42:27.947 INFO:tasks.workunit.client.1.smithi104.stdout:8/979: link d0/d1c/d59/d85/d127/d47/d67/d76/d7f/f124 d0/d1c/d2d/f143 0 2024-04-26T04:42:27.947 INFO:tasks.workunit.client.1.smithi104.stdout:8/980: read - d0/d33/da6/de0/f10d zero size 2024-04-26T04:42:27.948 INFO:tasks.workunit.client.0.smithi090.stdout:7/751: rename d0/db/df/d1c/d23/f82 to d0/db/d53/db7/d4f/f10f 0 2024-04-26T04:42:27.948 INFO:tasks.workunit.client.0.smithi090.stdout:7/752: write d0/db/d53/f56 [5205809,7230] 0 2024-04-26T04:42:27.950 INFO:tasks.workunit.client.0.smithi090.stdout:7/753: rmdir d0/db/d15/d29 39 2024-04-26T04:42:27.961 INFO:tasks.workunit.client.1.smithi104.stdout:8/981: dread d0/d1c/d59/d85/d127/d47/d67/d76/fea [0,4194304] 0 2024-04-26T04:42:27.961 INFO:tasks.workunit.client.1.smithi104.stdout:8/982: chown d0/d1c/d59/d85/d127/d47/d67/d76/daf/fe9 45754 1 2024-04-26T04:42:27.962 INFO:tasks.workunit.client.1.smithi104.stdout:8/983: creat d0/d1c/d2d/d7a/d112/f144 x:0 0 0 2024-04-26T04:42:27.962 INFO:tasks.workunit.client.1.smithi104.stdout:8/984: readlink d0/d1c/d59/d85/d127/d47/db1/le1 0 2024-04-26T04:42:27.963 INFO:tasks.workunit.client.1.smithi104.stdout:8/985: rename d0/d33/d35/d6d/f105 to d0/d50/f145 0 2024-04-26T04:42:27.964 INFO:tasks.workunit.client.1.smithi104.stdout:8/986: symlink d0/d9a/dce/d129/l146 0 2024-04-26T04:42:27.967 INFO:tasks.workunit.client.0.smithi090.stdout:9/644: dwrite db/dd/d44/f2c [0,4194304] 0 2024-04-26T04:42:27.967 INFO:tasks.workunit.client.1.smithi104.stdout:8/987: link d0/d1c/d59/d85/d127/cda d0/d1c/d59/d85/d127/d47/d67/d76/d7f/c147 0 2024-04-26T04:42:27.967 INFO:tasks.workunit.client.1.smithi104.stdout:8/988: chown d0/d1c/d59/f110 67519983 1 2024-04-26T04:42:27.967 INFO:tasks.workunit.client.1.smithi104.stdout:8/989: truncate d0/d1c/d59/d85/d127/f122 102626 0 2024-04-26T04:42:27.968 INFO:tasks.workunit.client.1.smithi104.stdout:8/990: readlink d0/d33/d35/l3d 0 2024-04-26T04:42:27.968 INFO:tasks.workunit.client.1.smithi104.stdout:8/991: creat d0/d33/d35/d6d/def/f148 x:0 0 0 2024-04-26T04:42:27.968 INFO:tasks.workunit.client.1.smithi104.stdout:8/992: write d0/d1c/d59/d85/d127/d11e/f138 [88094,62800] 0 2024-04-26T04:42:27.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/645: rename db/dd/d44/d82/d85/db4/d6a/d70/l75 to db/dd/d44/d82/d85/db4/d58/le1 0 2024-04-26T04:42:27.971 INFO:tasks.workunit.client.0.smithi090.stdout:2/728: dwrite db/d11/d14/d25/d2d/f36 [0,4194304] 0 2024-04-26T04:42:27.980 INFO:tasks.workunit.client.0.smithi090.stdout:1/694: dwrite d0/d55/dda/d7/d4b/f4c [0,4194304] 0 2024-04-26T04:42:27.995 INFO:tasks.workunit.client.0.smithi090.stdout:5/667: dwrite d4/d1a/df/d4a/f7a [0,4194304] 0 2024-04-26T04:42:27.996 INFO:tasks.workunit.client.0.smithi090.stdout:5/668: dread d4/d1a/d2a/d79/d90/d49/f82 [0,4194304] 0 2024-04-26T04:42:27.996 INFO:tasks.workunit.client.0.smithi090.stdout:5/669: write d4/d1a/df/d36/fbf [49986,35395] 0 2024-04-26T04:42:28.004 INFO:tasks.workunit.client.0.smithi090.stdout:2/729: dwrite db/d59/f62 [0,4194304] 0 2024-04-26T04:42:28.005 INFO:tasks.workunit.client.0.smithi090.stdout:2/730: write db/d11/d14/d1b/f51 [1970176,96987] 0 2024-04-26T04:42:28.006 INFO:tasks.workunit.client.0.smithi090.stdout:2/731: symlink db/d11/d14/d25/d2d/d48/lf2 0 2024-04-26T04:42:28.007 INFO:tasks.workunit.client.0.smithi090.stdout:2/732: mknod db/d11/d14/d25/d65/d95/cf3 0 2024-04-26T04:42:28.008 INFO:tasks.workunit.client.0.smithi090.stdout:2/733: mkdir db/d11/d14/d27/d2f/d42/df4 0 2024-04-26T04:42:28.008 INFO:tasks.workunit.client.0.smithi090.stdout:2/734: chown db/d11/d14/d27/db0 46948 1 2024-04-26T04:42:28.009 INFO:tasks.workunit.client.0.smithi090.stdout:2/735: creat db/d11/d14/d27/d2f/d70/d71/dea/ff5 x:0 0 0 2024-04-26T04:42:28.009 INFO:tasks.workunit.client.0.smithi090.stdout:2/736: write db/d11/d14/d27/d28/f61 [1236054,50904] 0 2024-04-26T04:42:28.009 INFO:tasks.workunit.client.0.smithi090.stdout:2/737: creat db/d11/d14/d27/d2f/ff6 x:0 0 0 2024-04-26T04:42:28.024 INFO:tasks.workunit.client.0.smithi090.stdout:2/738: dread db/dc/f1f [0,4194304] 0 2024-04-26T04:42:28.025 INFO:tasks.workunit.client.0.smithi090.stdout:2/739: mknod db/d11/d14/d25/d2d/d48/d5b/cf7 0 2024-04-26T04:42:28.029 INFO:tasks.workunit.client.0.smithi090.stdout:2/740: rmdir db/d11/d14/d25/d2d/d48/d81 0 2024-04-26T04:42:28.029 INFO:tasks.workunit.client.0.smithi090.stdout:2/741: write db/d11/d14/d27/d2f/f32 [404314,77379] 0 2024-04-26T04:42:28.029 INFO:tasks.workunit.client.0.smithi090.stdout:4/721: dwrite d3/f48 [0,4194304] 0 2024-04-26T04:42:28.030 INFO:tasks.workunit.client.1.smithi104.stdout:9/949: dwrite d11/d15/d1b/fb5 [0,4194304] 0 2024-04-26T04:42:28.030 INFO:tasks.workunit.client.0.smithi090.stdout:6/769: dwrite d9/d77/db0/db5/dda/d6d/d66/ffa [0,4194304] 0 2024-04-26T04:42:28.030 INFO:tasks.workunit.client.0.smithi090.stdout:1/695: dwrite d0/d55/dda/d7/f33 [0,4194304] 0 2024-04-26T04:42:28.030 INFO:tasks.workunit.client.0.smithi090.stdout:6/770: read d9/f5c [19422,55194] 0 2024-04-26T04:42:28.031 INFO:tasks.workunit.client.0.smithi090.stdout:0/769: dwrite d0/d4b/d5d/fd9 [0,4194304] 0 2024-04-26T04:42:28.032 INFO:tasks.workunit.client.1.smithi104.stdout:9/950: getdents d11/d15/d1b/d6d/dd5/d12e 0 2024-04-26T04:42:28.032 INFO:tasks.workunit.client.0.smithi090.stdout:2/742: creat db/d11/d14/d1b/d20/ff8 x:0 0 0 2024-04-26T04:42:28.037 INFO:tasks.workunit.client.1.smithi104.stdout:9/951: unlink d11/d1d/d34/c45 0 2024-04-26T04:42:28.039 INFO:tasks.workunit.client.0.smithi090.stdout:4/722: dread d3/d13/f87 [0,4194304] 0 2024-04-26T04:42:28.064 INFO:tasks.workunit.client.0.smithi090.stdout:1/696: creat d0/d55/dda/d7/d17/d22/d93/fdb x:0 0 0 2024-04-26T04:42:28.066 INFO:tasks.workunit.client.0.smithi090.stdout:6/771: creat d9/d77/db0/db5/dda/d58/f108 x:0 0 0 2024-04-26T04:42:28.069 INFO:tasks.workunit.client.0.smithi090.stdout:0/770: creat d0/d4b/d5d/d5c/d8a/d45/d93/f109 x:0 0 0 2024-04-26T04:42:28.070 INFO:tasks.workunit.client.0.smithi090.stdout:9/646: rename db/d16 to db/dd/d18/d38/d39/d6b/de2 0 2024-04-26T04:42:28.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/772: link d9/d77/db0/db5/dda/d6d/d92/l2e d9/d77/d9a/de9/l109 0 2024-04-26T04:42:28.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/773: fdatasync d9/d77/db0/db5/fcf 0 2024-04-26T04:42:28.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/771: mknod d0/de/d103/d105/c10a 0 2024-04-26T04:42:28.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/772: fsync d0/d4b/db3/fed 0 2024-04-26T04:42:28.074 INFO:tasks.workunit.client.0.smithi090.stdout:5/670: rename d4/d1a/df/d36/d8e/lbb to d4/ld4 0 2024-04-26T04:42:28.075 INFO:tasks.workunit.client.0.smithi090.stdout:6/774: symlink d9/d21/d40/d4c/d6f/d9f/dae/l10a 0 2024-04-26T04:42:28.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/743: rename db/d11/d14/d63/c6b to db/d11/d14/d1b/d20/da7/cf9 0 2024-04-26T04:42:28.079 INFO:tasks.workunit.client.0.smithi090.stdout:5/671: symlink d4/d1a/d2a/d79/d90/d56/d80/db2/ld5 0 2024-04-26T04:42:28.079 INFO:tasks.workunit.client.0.smithi090.stdout:5/672: write d4/d1a/d2a/d79/d90/d56/f75 [5139848,57482] 0 2024-04-26T04:42:28.079 INFO:tasks.workunit.client.0.smithi090.stdout:5/673: chown d4/d1a/d2a/d79/d90/d56/d71/l7b 226356 1 2024-04-26T04:42:28.079 INFO:tasks.workunit.client.0.smithi090.stdout:5/674: stat d4/d1a/df/d36 0 2024-04-26T04:42:28.080 INFO:tasks.workunit.client.1.smithi104.stdout:9/952: dwrite d11/d15/d1b/d6d/fc0 [0,4194304] 0 2024-04-26T04:42:28.081 INFO:tasks.workunit.client.0.smithi090.stdout:0/773: mknod d0/de/c10b 0 2024-04-26T04:42:28.081 INFO:tasks.workunit.client.0.smithi090.stdout:0/774: chown d0/d4b/d5d/d48/d100/ff6 93787321 1 2024-04-26T04:42:28.081 INFO:tasks.workunit.client.0.smithi090.stdout:0/775: write d0/d4b/fbd [1097443,29010] 0 2024-04-26T04:42:28.082 INFO:tasks.workunit.client.0.smithi090.stdout:2/744: rmdir db/d11/d14/d27/d2f/d70/d71 39 2024-04-26T04:42:28.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/697: dread d0/d55/dbd/fd2 [0,4194304] 0 2024-04-26T04:42:28.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/698: fsync d0/f12 0 2024-04-26T04:42:28.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/699: chown d0/d55/dda/d7/d49/la1 32 1 2024-04-26T04:42:28.082 INFO:tasks.workunit.client.0.smithi090.stdout:4/723: dwrite d3/db/d12/d20/d2d/d58/d6a/d74/da2/f37 [0,4194304] 0 2024-04-26T04:42:28.082 INFO:tasks.workunit.client.0.smithi090.stdout:1/700: chown d0/d55/dda/d7/d49/la1 5119582 1 2024-04-26T04:42:28.083 INFO:tasks.workunit.client.1.smithi104.stdout:9/953: creat d11/d15/d1b/d6d/d9c/dbe/def/f13e x:0 0 0 2024-04-26T04:42:28.084 INFO:tasks.workunit.client.1.smithi104.stdout:9/954: rename d11/d1d/d34/f41 to d11/d1d/d34/db3/de6/f13f 0 2024-04-26T04:42:28.085 INFO:tasks.workunit.client.0.smithi090.stdout:5/675: creat d4/fd6 x:0 0 0 2024-04-26T04:42:28.086 INFO:tasks.workunit.client.1.smithi104.stdout:9/955: rename d11/d15/d32/d35/d7b/fde to d11/d15/d32/d35/f140 0 2024-04-26T04:42:28.086 INFO:tasks.workunit.client.0.smithi090.stdout:5/676: dread d4/d1a/d2a/f83 [0,4194304] 0 2024-04-26T04:42:28.086 INFO:tasks.workunit.client.0.smithi090.stdout:5/677: write d4/d1a/d1f/f3a [1498168,110442] 0 2024-04-26T04:42:28.087 INFO:tasks.workunit.client.1.smithi104.stdout:9/956: symlink d11/d53/l141 0 2024-04-26T04:42:28.090 INFO:tasks.workunit.client.0.smithi090.stdout:1/701: creat d0/d55/dbd/fdc x:0 0 0 2024-04-26T04:42:28.091 INFO:tasks.workunit.client.0.smithi090.stdout:1/702: dread d0/d55/f83 [0,4194304] 0 2024-04-26T04:42:28.091 INFO:tasks.workunit.client.0.smithi090.stdout:1/703: readlink d0/d55/dda/l41 0 2024-04-26T04:42:28.092 INFO:tasks.workunit.client.1.smithi104.stdout:9/957: write d11/d15/d1b/d6d/dd5/d12e/ddc/fff [1136037,64336] 0 2024-04-26T04:42:28.093 INFO:tasks.workunit.client.1.smithi104.stdout:9/958: getdents d11/d1d/d71/d94 0 2024-04-26T04:42:28.093 INFO:tasks.workunit.client.1.smithi104.stdout:9/959: chown d11/d1d/d87/d11b 0 1 2024-04-26T04:42:28.094 INFO:tasks.workunit.client.0.smithi090.stdout:1/704: rmdir d0/d55/dda/d7/d4b/d53/d7b/d85 0 2024-04-26T04:42:28.095 INFO:tasks.workunit.client.0.smithi090.stdout:1/705: creat d0/d55/dda/d7/d17/d22/d93/db0/fdd x:0 0 0 2024-04-26T04:42:28.097 INFO:tasks.workunit.client.1.smithi104.stdout:9/960: rmdir d11/d15/d1b/d6d/dd5/df3 0 2024-04-26T04:42:28.099 INFO:tasks.workunit.client.0.smithi090.stdout:2/745: dread db/d11/d14/d25/f49 [0,4194304] 0 2024-04-26T04:42:28.100 INFO:tasks.workunit.client.0.smithi090.stdout:0/776: write d0/d4b/d5d/d5c/d8a/d45/f9b [815085,120966] 0 2024-04-26T04:42:28.101 INFO:tasks.workunit.client.1.smithi104.stdout:8/993: dwrite d0/f1e [0,4194304] 0 2024-04-26T04:42:28.101 INFO:tasks.workunit.client.1.smithi104.stdout:8/994: write d0/d33/d35/d6d/f106 [628230,16723] 0 2024-04-26T04:42:28.102 INFO:tasks.workunit.client.0.smithi090.stdout:2/746: rmdir db/d11/d14/d27/d28 39 2024-04-26T04:42:28.102 INFO:tasks.workunit.client.0.smithi090.stdout:2/747: rename db/d11 to db/d11/d14/d25/d65/dfa 22 2024-04-26T04:42:28.105 INFO:tasks.workunit.client.0.smithi090.stdout:5/678: dread d4/d1a/d2a/fd0 [4194304,4194304] 0 2024-04-26T04:42:28.105 INFO:tasks.workunit.client.0.smithi090.stdout:2/748: rename db/d59/d66/d69/fa6 to db/d11/d14/d27/d2f/d70/d8b/ffb 0 2024-04-26T04:42:28.110 INFO:tasks.workunit.client.0.smithi090.stdout:5/679: rename d4/d1a/d1f/l24 to d4/d1a/d2a/d79/d90/db3/ld7 0 2024-04-26T04:42:28.111 INFO:tasks.workunit.client.0.smithi090.stdout:5/680: read d4/f88 [2581171,31520] 0 2024-04-26T04:42:28.111 INFO:tasks.workunit.client.0.smithi090.stdout:5/681: dread - d4/d1a/d2a/d79/fcf zero size 2024-04-26T04:42:28.111 INFO:tasks.workunit.client.0.smithi090.stdout:5/682: read - d4/d1a/df/fa7 zero size 2024-04-26T04:42:28.111 INFO:tasks.workunit.client.1.smithi104.stdout:9/961: write d11/d15/d1b/f60 [6141646,115504] 0 2024-04-26T04:42:28.112 INFO:tasks.workunit.client.1.smithi104.stdout:8/995: dread d0/d1c/d59/d85/d127/d47/d67/d76/d7f/fac [0,4194304] 0 2024-04-26T04:42:28.112 INFO:tasks.workunit.client.0.smithi090.stdout:5/683: creat d4/d1a/d2a/d79/d90/d94/fd8 x:0 0 0 2024-04-26T04:42:28.113 INFO:tasks.workunit.client.1.smithi104.stdout:9/962: creat d11/d15/d1b/d6d/d9c/dfb/d102/f142 x:0 0 0 2024-04-26T04:42:28.113 INFO:tasks.workunit.client.1.smithi104.stdout:9/963: truncate d11/d15/d32/d4c/d7c/f10f 203694 0 2024-04-26T04:42:28.114 INFO:tasks.workunit.client.1.smithi104.stdout:8/996: mknod d0/d1c/d2d/d7a/d112/c149 0 2024-04-26T04:42:28.118 INFO:tasks.workunit.client.1.smithi104.stdout:8/997: read d0/d1c/d59/d85/d127/d47/d67/d76/fbf [98295,46402] 0 2024-04-26T04:42:28.120 INFO:tasks.workunit.client.1.smithi104.stdout:8/998: mkdir d0/d33/d35/d6d/da4/d128/d14a 0 2024-04-26T04:42:28.120 INFO:tasks.workunit.client.1.smithi104.stdout:8/999: chown d0/d1c/d59/d85/d127/d47/f100 42912 1 2024-04-26T04:42:28.121 INFO:tasks.workunit.client.0.smithi090.stdout:2/749: dread db/dc/f19 [0,4194304] 0 2024-04-26T04:42:28.134 INFO:tasks.workunit.client.0.smithi090.stdout:8/846: sync 2024-04-26T04:42:28.134 INFO:tasks.workunit.client.0.smithi090.stdout:7/754: sync 2024-04-26T04:42:28.134 INFO:tasks.workunit.client.0.smithi090.stdout:3/655: sync 2024-04-26T04:42:28.134 INFO:tasks.workunit.client.0.smithi090.stdout:6/775: dwrite d9/f5c [0,4194304] 0 2024-04-26T04:42:28.136 INFO:tasks.workunit.client.0.smithi090.stdout:9/647: dwrite db/dd/d18/d38/d39/d6b/de2/fa9 [0,4194304] 0 2024-04-26T04:42:28.137 INFO:tasks.workunit.client.0.smithi090.stdout:8/847: creat dc/dd2/d39/d125/d5d/d73/f128 x:0 0 0 2024-04-26T04:42:28.140 INFO:tasks.workunit.client.0.smithi090.stdout:3/656: rmdir d1/d17/d36/d33/d43 39 2024-04-26T04:42:28.142 INFO:tasks.workunit.client.0.smithi090.stdout:4/724: dwrite d3/d13/d1d/d50/fe3 [0,4194304] 0 2024-04-26T04:42:28.142 INFO:tasks.workunit.client.0.smithi090.stdout:1/706: dwrite d0/d55/dda/d7/d4a/d75/d80/fa2 [0,4194304] 0 2024-04-26T04:42:28.143 INFO:tasks.workunit.client.0.smithi090.stdout:3/657: unlink d1/d17/d36/d33/d68/dae/dca/d2c/d97/la6 0 2024-04-26T04:42:28.144 INFO:tasks.workunit.client.0.smithi090.stdout:1/707: write d0/d55/fd1 [69009,66757] 0 2024-04-26T04:42:28.147 INFO:tasks.workunit.client.0.smithi090.stdout:4/725: link d3/db/l16 d3/db/d12/d20/d2d/d58/d6a/d74/le5 0 2024-04-26T04:42:28.147 INFO:tasks.workunit.client.0.smithi090.stdout:0/777: dwrite d0/d4b/d5d/d48/f56 [0,4194304] 0 2024-04-26T04:42:28.152 INFO:tasks.workunit.client.0.smithi090.stdout:4/726: dread d3/db/d12/d20/d2d/d58/d6a/f8b [0,4194304] 0 2024-04-26T04:42:28.152 INFO:tasks.workunit.client.0.smithi090.stdout:4/727: fdatasync d3/db/d12/d20/d2d/d2e/f77 0 2024-04-26T04:42:28.152 INFO:tasks.workunit.client.0.smithi090.stdout:2/750: dread db/d11/d14/d27/d2f/f32 [0,4194304] 0 2024-04-26T04:42:28.152 INFO:tasks.workunit.client.0.smithi090.stdout:2/751: fdatasync db/d11/d14/d27/fab 0 2024-04-26T04:42:28.152 INFO:tasks.workunit.client.0.smithi090.stdout:3/658: mknod d1/d17/d36/d2a/dc7/cde 0 2024-04-26T04:42:28.153 INFO:tasks.workunit.client.0.smithi090.stdout:8/848: dread dc/dd2/d39/fb9 [0,4194304] 0 2024-04-26T04:42:28.154 INFO:tasks.workunit.client.0.smithi090.stdout:1/708: rename d0/d55/dbd/cce to d0/d55/dda/d7/d4a/d75/cde 0 2024-04-26T04:42:28.161 INFO:tasks.workunit.client.0.smithi090.stdout:2/752: symlink db/d11/d14/d25/d2d/d48/lfc 0 2024-04-26T04:42:28.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/778: dread d0/d4b/d5d/d48/fca [0,4194304] 0 2024-04-26T04:42:28.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/779: truncate d0/d4b/db3/fed 1350735 0 2024-04-26T04:42:28.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/780: write d0/d4b/d6d/df2/fda [472422,36757] 0 2024-04-26T04:42:28.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/781: dread - d0/de/f90 zero size 2024-04-26T04:42:28.162 INFO:tasks.workunit.client.0.smithi090.stdout:8/849: mknod dc/dd2/d39/d55/d64/c129 0 2024-04-26T04:42:28.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/753: symlink db/d11/d14/lfd 0 2024-04-26T04:42:28.163 INFO:tasks.workunit.client.0.smithi090.stdout:2/754: write db/d11/d14/d25/d2d/fe2 [989878,62559] 0 2024-04-26T04:42:28.164 INFO:tasks.workunit.client.0.smithi090.stdout:0/782: mknod d0/d4b/d5d/d5c/d8a/d45/d93/c10c 0 2024-04-26T04:42:28.164 INFO:tasks.workunit.client.0.smithi090.stdout:4/728: dread d3/db/d12/d20/d24/d43/dd4/fb7 [0,4194304] 0 2024-04-26T04:42:28.165 INFO:tasks.workunit.client.0.smithi090.stdout:8/850: creat dc/dd2/d3b/f12a x:0 0 0 2024-04-26T04:42:28.165 INFO:tasks.workunit.client.0.smithi090.stdout:8/851: chown dc/d71/f105 1663652493 1 2024-04-26T04:42:28.165 INFO:tasks.workunit.client.0.smithi090.stdout:8/852: creat dc/d34/d4d/f12b x:0 0 0 2024-04-26T04:42:28.168 INFO:tasks.workunit.client.0.smithi090.stdout:8/853: unlink dc/d71/f94 0 2024-04-26T04:42:28.170 INFO:tasks.workunit.client.0.smithi090.stdout:0/783: rename d0/l50 to d0/d4b/l10d 0 2024-04-26T04:42:28.171 INFO:tasks.workunit.client.0.smithi090.stdout:0/784: mknod d0/d4b/d5d/c10e 0 2024-04-26T04:42:28.171 INFO:tasks.workunit.client.0.smithi090.stdout:0/785: creat d0/d4b/d5d/d5c/d8a/d45/d93/f10f x:0 0 0 2024-04-26T04:42:28.171 INFO:tasks.workunit.client.0.smithi090.stdout:0/786: fdatasync d0/d4b/d5d/d5c/d8a/d7c/fc9 0 2024-04-26T04:42:28.178 INFO:tasks.workunit.client.0.smithi090.stdout:2/755: read db/d11/d14/d27/d2f/d70/d71/f83 [2929480,116288] 0 2024-04-26T04:42:28.178 INFO:tasks.workunit.client.0.smithi090.stdout:2/756: chown db/d11/d14/d63/fd5 4156683 1 2024-04-26T04:42:28.179 INFO:tasks.workunit.client.0.smithi090.stdout:4/729: dread d3/db/d12/ff [4194304,4194304] 0 2024-04-26T04:42:28.181 INFO:tasks.workunit.client.0.smithi090.stdout:2/757: rename db/d11/d14/d27/d2f/d70 to db/d11/d14/d27/d28/dd1/dfe 0 2024-04-26T04:42:28.182 INFO:tasks.workunit.client.0.smithi090.stdout:0/787: dread d0/d4b/d5d/d5c/fa8 [0,4194304] 0 2024-04-26T04:42:28.183 INFO:tasks.workunit.client.0.smithi090.stdout:2/758: mkdir db/d11/d14/d1b/d20/d4c/d98/db4/dff 0 2024-04-26T04:42:28.183 INFO:tasks.workunit.client.0.smithi090.stdout:2/759: readlink db/d59/d66/l85 0 2024-04-26T04:42:28.184 INFO:tasks.workunit.client.0.smithi090.stdout:0/788: creat d0/d4b/d5d/d5c/d8a/d45/dcf/f110 x:0 0 0 2024-04-26T04:42:28.184 INFO:tasks.workunit.client.0.smithi090.stdout:0/789: write d0/df/fd4 [1709717,89197] 0 2024-04-26T04:42:28.187 INFO:tasks.workunit.client.0.smithi090.stdout:2/760: link db/d11/d14/d25/d41/fba db/d59/d66/d77/f100 0 2024-04-26T04:42:28.187 INFO:tasks.workunit.client.0.smithi090.stdout:2/761: chown db/d11/d14/d1b/d20/da7/cf9 142 1 2024-04-26T04:42:28.188 INFO:tasks.workunit.client.0.smithi090.stdout:0/790: creat d0/d4b/d5d/d5c/f111 x:0 0 0 2024-04-26T04:42:28.191 INFO:tasks.workunit.client.0.smithi090.stdout:2/762: mknod db/d11/d14/d1b/d20/da7/c101 0 2024-04-26T04:42:28.204 INFO:tasks.workunit.client.0.smithi090.stdout:2/763: dread db/d11/d14/fc0 [0,4194304] 0 2024-04-26T04:42:28.205 INFO:tasks.workunit.client.0.smithi090.stdout:2/764: chown db/d11/d14/d27/d28/dd1/dfe/ca2 2 1 2024-04-26T04:42:28.222 INFO:tasks.workunit.client.0.smithi090.stdout:6/776: dwrite d9/d77/db0/db5/dda/d6d/fc6 [0,4194304] 0 2024-04-26T04:42:28.224 INFO:tasks.workunit.client.0.smithi090.stdout:6/777: creat d9/d1c/d31/f10b x:0 0 0 2024-04-26T04:42:28.224 INFO:tasks.workunit.client.0.smithi090.stdout:6/778: chown d9/d21/d40/f4b 1084 1 2024-04-26T04:42:28.224 INFO:tasks.workunit.client.0.smithi090.stdout:6/779: rmdir d9/d77/dc0 39 2024-04-26T04:42:28.225 INFO:tasks.workunit.client.0.smithi090.stdout:6/780: truncate d9/d77/f9d 2663385 0 2024-04-26T04:42:28.227 INFO:tasks.workunit.client.0.smithi090.stdout:6/781: rename d9/d21/l6a to d9/d3c/db4/ddb/l10c 0 2024-04-26T04:42:28.227 INFO:tasks.workunit.client.0.smithi090.stdout:6/782: creat d9/d77/d9a/dc8/f10d x:0 0 0 2024-04-26T04:42:28.227 INFO:tasks.workunit.client.0.smithi090.stdout:6/783: write d9/d1c/d31/d69/ff2 [248013,117136] 0 2024-04-26T04:42:28.230 INFO:tasks.workunit.client.0.smithi090.stdout:6/784: rmdir d9/d21/d40/d4c/d6f/d9f 39 2024-04-26T04:42:28.231 INFO:tasks.workunit.client.0.smithi090.stdout:9/648: dwrite db/dd/d44/d82/d85/faf [0,4194304] 0 2024-04-26T04:42:28.231 INFO:tasks.workunit.client.0.smithi090.stdout:1/709: dwrite d0/d55/dbd/fd2 [0,4194304] 0 2024-04-26T04:42:28.232 INFO:tasks.workunit.client.0.smithi090.stdout:6/785: truncate d9/d21/d40/d4c/f90 1070716 0 2024-04-26T04:42:28.232 INFO:tasks.workunit.client.0.smithi090.stdout:8/854: dwrite dc/dd2/fd6 [0,4194304] 0 2024-04-26T04:42:28.232 INFO:tasks.workunit.client.0.smithi090.stdout:4/730: dwrite d3/f2c [0,4194304] 0 2024-04-26T04:42:28.233 INFO:tasks.workunit.client.0.smithi090.stdout:4/731: dread - d3/d13/d1d/d50/d51/fcd zero size 2024-04-26T04:42:28.233 INFO:tasks.workunit.client.0.smithi090.stdout:8/855: write dc/dd2/d3b/f12a [203836,63317] 0 2024-04-26T04:42:28.233 INFO:tasks.workunit.client.0.smithi090.stdout:8/856: fsync dc/dd2/d3b/d3d/d111/f121 0 2024-04-26T04:42:28.233 INFO:tasks.workunit.client.0.smithi090.stdout:0/791: dwrite d0/d4b/d5d/f27 [0,4194304] 0 2024-04-26T04:42:28.233 INFO:tasks.workunit.client.0.smithi090.stdout:1/710: mknod d0/d55/dda/d7/d4a/d75/cdf 0 2024-04-26T04:42:28.235 INFO:tasks.workunit.client.0.smithi090.stdout:2/765: dwrite db/d11/d14/d1b/d20/fb8 [0,4194304] 0 2024-04-26T04:42:28.236 INFO:tasks.workunit.client.0.smithi090.stdout:6/786: rename d9/d1c/d31/d69/dbc/cc4 to d9/d21/d40/d4c/c10e 0 2024-04-26T04:42:28.241 INFO:tasks.workunit.client.0.smithi090.stdout:4/732: symlink d3/db/d12/d3b/le6 0 2024-04-26T04:42:28.246 INFO:tasks.workunit.client.0.smithi090.stdout:8/857: symlink dc/dd2/d4f/d58/d95/d106/l12c 0 2024-04-26T04:42:28.252 INFO:tasks.workunit.client.0.smithi090.stdout:2/766: link db/d11/d14/d1b/cc1 db/d11/d14/d1b/d20/d4c/d98/c102 0 2024-04-26T04:42:28.253 INFO:tasks.workunit.client.0.smithi090.stdout:8/858: dread dc/dd2/d3b/d3d/d97/dad/ffa [0,4194304] 0 2024-04-26T04:42:28.253 INFO:tasks.workunit.client.0.smithi090.stdout:8/859: stat dc/dd2/d3b/d3d/db8/le0 0 2024-04-26T04:42:28.262 INFO:tasks.workunit.client.0.smithi090.stdout:2/767: creat db/d11/d14/d25/d86/f103 x:0 0 0 2024-04-26T04:42:28.263 INFO:tasks.workunit.client.0.smithi090.stdout:0/792: rename d0/d4b/d5d/d5c/d8a/lb8 to d0/d4b/l112 0 2024-04-26T04:42:28.264 INFO:tasks.workunit.client.0.smithi090.stdout:6/787: mknod d9/d77/db0/db5/dda/d6d/d92/c10f 0 2024-04-26T04:42:28.265 INFO:tasks.workunit.client.0.smithi090.stdout:8/860: dread dc/dd2/d4f/d58/d95/f60 [0,4194304] 0 2024-04-26T04:42:28.266 INFO:tasks.workunit.client.0.smithi090.stdout:1/711: rename d0/d55/dda/d7/fa0 to d0/d55/dda/d7/d4a/fe0 0 2024-04-26T04:42:28.267 INFO:tasks.workunit.client.0.smithi090.stdout:8/861: mknod dc/dd2/d4f/d58/d95/d106/d119/c12d 0 2024-04-26T04:42:28.268 INFO:tasks.workunit.client.0.smithi090.stdout:6/788: truncate d9/d21/d40/f74 2569747 0 2024-04-26T04:42:28.270 INFO:tasks.workunit.client.0.smithi090.stdout:8/862: creat dc/dd2/d3b/d3d/f12e x:0 0 0 2024-04-26T04:42:28.273 INFO:tasks.workunit.client.0.smithi090.stdout:9/649: dwrite db/dd/d44/d82/d85/db4/d99/fcf [0,4194304] 0 2024-04-26T04:42:28.273 INFO:tasks.workunit.client.0.smithi090.stdout:9/650: truncate db/dd/d44/d82/d85/db4/d58/f97 810914 0 2024-04-26T04:42:28.274 INFO:tasks.workunit.client.0.smithi090.stdout:4/733: dwrite d3/db/d12/d20/d24/d43/dd4/d53/f6e [4194304,4194304] 0 2024-04-26T04:42:28.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/734: read - d3/db/d12/d20/d24/da9/fbc zero size 2024-04-26T04:42:28.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/735: creat d3/db/d12/d20/d2d/d2e/fe7 x:0 0 0 2024-04-26T04:42:28.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/736: stat d3/db/d12/d20/d24/d43/d72 0 2024-04-26T04:42:28.275 INFO:tasks.workunit.client.0.smithi090.stdout:4/737: chown d3/db/d12/d20/d2d/dbe/dde 3632 1 2024-04-26T04:42:28.276 INFO:tasks.workunit.client.0.smithi090.stdout:9/651: getdents db/dd/d18/d38/d39 0 2024-04-26T04:42:28.278 INFO:tasks.workunit.client.0.smithi090.stdout:9/652: truncate db/dd/d44/d82/d85/db4/d6a/d70/f96 461178 0 2024-04-26T04:42:28.279 INFO:tasks.workunit.client.0.smithi090.stdout:9/653: dread db/dd/d18/d38/f9b [0,4194304] 0 2024-04-26T04:42:28.281 INFO:tasks.workunit.client.0.smithi090.stdout:9/654: link db/dd/d18/d57/cca db/dd/d18/ce3 0 2024-04-26T04:42:28.282 INFO:tasks.workunit.client.0.smithi090.stdout:9/655: link db/dd/d18/d38/f79 db/dd/d44/d82/fe4 0 2024-04-26T04:42:28.283 INFO:tasks.workunit.client.0.smithi090.stdout:1/712: dread d0/d55/dda/d7/d4b/d53/f6a [0,4194304] 0 2024-04-26T04:42:28.283 INFO:tasks.workunit.client.0.smithi090.stdout:9/656: mknod db/dd/d44/d82/d85/db4/d99/db2/dcd/ce5 0 2024-04-26T04:42:28.284 INFO:tasks.workunit.client.0.smithi090.stdout:1/713: symlink d0/d55/dda/d7/d17/d22/d31/le1 0 2024-04-26T04:42:28.284 INFO:tasks.workunit.client.0.smithi090.stdout:9/657: symlink db/dd/d18/le6 0 2024-04-26T04:42:28.285 INFO:tasks.workunit.client.0.smithi090.stdout:1/714: creat d0/d55/dda/d7/d4a/d75/d80/d58/fe2 x:0 0 0 2024-04-26T04:42:28.286 INFO:tasks.workunit.client.0.smithi090.stdout:1/715: rename d0/d55/dda/d7/d49/la1 to d0/d55/dda/d7/d49/le3 0 2024-04-26T04:42:28.288 INFO:tasks.workunit.client.0.smithi090.stdout:1/716: rename d0/d55/dda/l32 to d0/d55/dda/d7/d4a/d75/d80/d58/d97/le4 0 2024-04-26T04:42:28.289 INFO:tasks.workunit.client.0.smithi090.stdout:1/717: symlink d0/d55/dda/d7/d4a/da8/le5 0 2024-04-26T04:42:28.289 INFO:tasks.workunit.client.0.smithi090.stdout:1/718: dread - d0/d55/dda/d7/d4b/f5b zero size 2024-04-26T04:42:28.289 INFO:tasks.workunit.client.0.smithi090.stdout:1/719: mkdir d0/d55/dda/d7/d4b/d53/de6 0 2024-04-26T04:42:28.289 INFO:tasks.workunit.client.0.smithi090.stdout:1/720: chown d0/d55/dda/d7/d49/l88 46453088 1 2024-04-26T04:42:28.290 INFO:tasks.workunit.client.0.smithi090.stdout:1/721: fsync d0/d55/dda/d62/d78/f8f 0 2024-04-26T04:42:28.290 INFO:tasks.workunit.client.0.smithi090.stdout:1/722: rmdir d0/d55/dda/d7/d49 39 2024-04-26T04:42:28.291 INFO:tasks.workunit.client.0.smithi090.stdout:1/723: link d0/d55/dda/d62/d78/fcf d0/d55/dda/d7/d69/fe7 0 2024-04-26T04:42:28.294 INFO:tasks.workunit.client.0.smithi090.stdout:4/738: write d3/db/f19 [483911,80819] 0 2024-04-26T04:42:28.298 INFO:tasks.workunit.client.0.smithi090.stdout:1/724: dread d0/d55/dda/d7/d17/f29 [0,4194304] 0 2024-04-26T04:42:28.309 INFO:tasks.workunit.client.0.smithi090.stdout:4/739: read d3/d13/d64/dae/da5/fca [2791203,41537] 0 2024-04-26T04:42:28.310 INFO:tasks.workunit.client.0.smithi090.stdout:4/740: mkdir d3/d13/d1d/d50/d51/de8 0 2024-04-26T04:42:28.311 INFO:tasks.workunit.client.0.smithi090.stdout:4/741: symlink d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/le9 0 2024-04-26T04:42:28.312 INFO:tasks.workunit.client.0.smithi090.stdout:4/742: symlink d3/db/d12/d99/lea 0 2024-04-26T04:42:28.312 INFO:tasks.workunit.client.0.smithi090.stdout:4/743: write d3/db/d12/d20/d2d/d58/d6a/d74/d85/fb0 [1025159,26266] 0 2024-04-26T04:42:28.313 INFO:tasks.workunit.client.0.smithi090.stdout:1/725: dread d0/d55/dda/d7/f9c [0,4194304] 0 2024-04-26T04:42:28.314 INFO:tasks.workunit.client.0.smithi090.stdout:1/726: dread - d0/d55/dda/d43/fc5 zero size 2024-04-26T04:42:28.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/727: creat d0/d55/dda/d7/d4b/fe8 x:0 0 0 2024-04-26T04:42:28.323 INFO:tasks.workunit.client.0.smithi090.stdout:0/793: dwrite d0/d4b/d5d/d48/d100/fd6 [0,4194304] 0 2024-04-26T04:42:28.323 INFO:tasks.workunit.client.0.smithi090.stdout:0/794: write d0/d4b/d5d/d5c/d8a/d7c/fdb [470370,45959] 0 2024-04-26T04:42:28.323 INFO:tasks.workunit.client.0.smithi090.stdout:4/744: dread d3/db/d12/d20/d24/d43/d72/f9b [0,4194304] 0 2024-04-26T04:42:28.328 INFO:tasks.workunit.client.0.smithi090.stdout:4/745: symlink d3/d13/d6c/leb 0 2024-04-26T04:42:28.331 INFO:tasks.workunit.client.0.smithi090.stdout:4/746: getdents d3/d13/d64/dae 0 2024-04-26T04:42:28.331 INFO:tasks.workunit.client.0.smithi090.stdout:4/747: truncate d3/f7d 4570274 0 2024-04-26T04:42:28.333 INFO:tasks.workunit.client.0.smithi090.stdout:4/748: rename d3/db/d12/d20/d2d/l5e to d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/ddd/lec 0 2024-04-26T04:42:28.335 INFO:tasks.workunit.client.0.smithi090.stdout:4/749: creat d3/db/d12/fed x:0 0 0 2024-04-26T04:42:28.336 INFO:tasks.workunit.client.0.smithi090.stdout:4/750: unlink d3/d13/d6c/d7e/cd7 0 2024-04-26T04:42:28.337 INFO:tasks.workunit.client.0.smithi090.stdout:4/751: fsync d3/db/d12/d20/d24/d43/dd4/d53/f6e 0 2024-04-26T04:42:28.337 INFO:tasks.workunit.client.0.smithi090.stdout:4/752: fsync d3/d13/f61 0 2024-04-26T04:42:28.337 INFO:tasks.workunit.client.0.smithi090.stdout:4/753: chown d3/d13/d6c/leb 156949048 1 2024-04-26T04:42:28.337 INFO:tasks.workunit.client.0.smithi090.stdout:4/754: write d3/d13/f9f [53479,123320] 0 2024-04-26T04:42:28.338 INFO:tasks.workunit.client.0.smithi090.stdout:4/755: write f1 [86625,85466] 0 2024-04-26T04:42:28.343 INFO:tasks.workunit.client.0.smithi090.stdout:4/756: rename d3/d13/d64 to d3/db/d60/dee 0 2024-04-26T04:42:28.343 INFO:tasks.workunit.client.0.smithi090.stdout:4/757: chown d3/db/d12/d20/d24/d43/dd4 61 1 2024-04-26T04:42:28.345 INFO:tasks.workunit.client.0.smithi090.stdout:4/758: mknod d3/db/d12/d20/d24/da9/cef 0 2024-04-26T04:42:28.346 INFO:tasks.workunit.client.0.smithi090.stdout:4/759: mknod d3/d13/cf0 0 2024-04-26T04:42:28.351 INFO:tasks.workunit.client.0.smithi090.stdout:2/768: dwrite db/f84 [0,4194304] 0 2024-04-26T04:42:28.351 INFO:tasks.workunit.client.0.smithi090.stdout:8/863: dwrite dc/dd2/d4f/d58/d95/fa7 [0,4194304] 0 2024-04-26T04:42:28.352 INFO:tasks.workunit.client.0.smithi090.stdout:1/728: dwrite d0/d55/dda/d7/d4a/faf [0,4194304] 0 2024-04-26T04:42:28.352 INFO:tasks.workunit.client.0.smithi090.stdout:0/795: dwrite d0/d4b/d5d/d5c/d8a/d45/f59 [0,4194304] 0 2024-04-26T04:42:28.352 INFO:tasks.workunit.client.0.smithi090.stdout:0/796: dread - d0/d4b/d5d/d48/d7d/fb2 zero size 2024-04-26T04:42:28.352 INFO:tasks.workunit.client.0.smithi090.stdout:6/789: dwrite d9/d3c/f64 [0,4194304] 0 2024-04-26T04:42:28.353 INFO:tasks.workunit.client.0.smithi090.stdout:1/729: dread d0/d55/dda/d7/d17/f29 [0,4194304] 0 2024-04-26T04:42:28.362 INFO:tasks.workunit.client.0.smithi090.stdout:8/864: unlink dc/dd2/d4f/da6/dd8/cf9 0 2024-04-26T04:42:28.363 INFO:tasks.workunit.client.0.smithi090.stdout:2/769: write db/d11/d14/d27/d28/dd1/dfe/d8b/ffb [1936144,80850] 0 2024-04-26T04:42:28.370 INFO:tasks.workunit.client.0.smithi090.stdout:0/797: mknod d0/d4b/d5d/d48/d100/c113 0 2024-04-26T04:42:28.372 INFO:tasks.workunit.client.0.smithi090.stdout:0/798: write d0/d4b/d5d/d5c/d8a/f44 [748314,38020] 0 2024-04-26T04:42:28.372 INFO:tasks.workunit.client.0.smithi090.stdout:2/770: unlink db/d11/d14/d27/d28/dd1/dfe/d71/lb7 0 2024-04-26T04:42:28.372 INFO:tasks.workunit.client.0.smithi090.stdout:2/771: creat db/d11/d7f/f104 x:0 0 0 2024-04-26T04:42:28.374 INFO:tasks.workunit.client.0.smithi090.stdout:2/772: stat db/d11/l46 0 2024-04-26T04:42:28.374 INFO:tasks.workunit.client.0.smithi090.stdout:2/773: write db/d11/d14/d27/d2f/d42/fc8 [712101,17033] 0 2024-04-26T04:42:28.374 INFO:tasks.workunit.client.0.smithi090.stdout:8/865: read dc/dd2/d39/fb9 [2433199,65581] 0 2024-04-26T04:42:28.374 INFO:tasks.workunit.client.0.smithi090.stdout:8/866: fdatasync dc/dd2/d4f/f9f 0 2024-04-26T04:42:28.375 INFO:tasks.workunit.client.0.smithi090.stdout:2/774: mknod db/d11/d14/d27/d28/dd1/ddf/c105 0 2024-04-26T04:42:28.375 INFO:tasks.workunit.client.0.smithi090.stdout:2/775: chown db/d11/d14/d1b/d20/d4c/d98/lcb 2420 1 2024-04-26T04:42:28.376 INFO:tasks.workunit.client.0.smithi090.stdout:8/867: rename dc/d34/fff to dc/dd2/d39/d99/f12f 0 2024-04-26T04:42:28.377 INFO:tasks.workunit.client.0.smithi090.stdout:8/868: creat dc/dd2/f130 x:0 0 0 2024-04-26T04:42:28.377 INFO:tasks.workunit.client.0.smithi090.stdout:8/869: dread - dc/dd2/d3b/d3d/f126 zero size 2024-04-26T04:42:28.377 INFO:tasks.workunit.client.0.smithi090.stdout:8/870: chown dc/ff 32451 1 2024-04-26T04:42:28.378 INFO:tasks.workunit.client.0.smithi090.stdout:0/799: dread d0/d4b/d5d/d5c/d8a/d45/d93/faa [0,4194304] 0 2024-04-26T04:42:28.381 INFO:tasks.workunit.client.0.smithi090.stdout:0/800: link d0/d4b/d5d/d5c/d8a/d45/d93/faa d0/d4b/f114 0 2024-04-26T04:42:28.383 INFO:tasks.workunit.client.0.smithi090.stdout:0/801: mkdir d0/d4b/d5d/d5c/d8a/d45/d53/dab/d115 0 2024-04-26T04:42:28.384 INFO:tasks.workunit.client.0.smithi090.stdout:0/802: mknod d0/d4b/d5d/d5c/d8a/d45/d53/c116 0 2024-04-26T04:42:28.384 INFO:tasks.workunit.client.0.smithi090.stdout:0/803: stat d0/d4b/d5d/d5c/d8a/d45/d53/dab/fc8 0 2024-04-26T04:42:28.384 INFO:tasks.workunit.client.0.smithi090.stdout:0/804: fdatasync d0/d4b/d5d/fe7 0 2024-04-26T04:42:28.388 INFO:tasks.workunit.client.0.smithi090.stdout:4/760: dwrite d3/db/d12/f4c [4194304,4194304] 0 2024-04-26T04:42:28.388 INFO:tasks.workunit.client.0.smithi090.stdout:4/761: read d3/db/d12/d20/d2d/d58/d6a/d74/da2/f5a [2784212,80621] 0 2024-04-26T04:42:28.391 INFO:tasks.workunit.client.0.smithi090.stdout:4/762: mknod d3/d13/d1d/cf1 0 2024-04-26T04:42:28.402 INFO:tasks.workunit.client.0.smithi090.stdout:5/684: sync 2024-04-26T04:42:28.402 INFO:tasks.workunit.client.0.smithi090.stdout:3/659: sync 2024-04-26T04:42:28.402 INFO:tasks.workunit.client.0.smithi090.stdout:7/755: sync 2024-04-26T04:42:28.404 INFO:tasks.workunit.client.0.smithi090.stdout:1/730: dwrite d0/d55/dda/d7/d4b/fe8 [0,4194304] 0 2024-04-26T04:42:28.407 INFO:tasks.workunit.client.0.smithi090.stdout:1/731: mkdir d0/d55/dda/d7/d17/d22/d93/db8/de9 0 2024-04-26T04:42:28.409 INFO:tasks.workunit.client.0.smithi090.stdout:1/732: rename d0/d55/dda/d7/d4a/d75/d80/d58/d97/fb7 to d0/d55/dda/d7/d17/fea 0 2024-04-26T04:42:28.410 INFO:tasks.workunit.client.0.smithi090.stdout:1/733: symlink d0/d55/dda/d7/d17/d22/d93/db0/leb 0 2024-04-26T04:42:28.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/734: dread d0/d55/dda/d7/d4a/fe0 [0,4194304] 0 2024-04-26T04:42:28.420 INFO:tasks.workunit.client.0.smithi090.stdout:1/735: creat d0/d55/dda/d7/d17/d22/d93/db8/de9/fec x:0 0 0 2024-04-26T04:42:28.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/736: write d0/d55/dda/d7/f90 [1685822,25544] 0 2024-04-26T04:42:28.421 INFO:tasks.workunit.client.0.smithi090.stdout:1/737: creat d0/d55/dda/d7/d4a/d75/d80/d58/fed x:0 0 0 2024-04-26T04:42:28.422 INFO:tasks.workunit.client.0.smithi090.stdout:5/685: dread d4/d1a/d2a/d79/d90/d49/f9a [0,4194304] 0 2024-04-26T04:42:28.422 INFO:tasks.workunit.client.0.smithi090.stdout:5/686: truncate d4/d1a/df/d4a/fa4 992135 0 2024-04-26T04:42:28.422 INFO:tasks.workunit.client.0.smithi090.stdout:5/687: stat d4/d1a/d45/db8 0 2024-04-26T04:42:28.422 INFO:tasks.workunit.client.0.smithi090.stdout:5/688: getdents d4/d1a/d2a/d79/d90/d56/d71/d91/d86 0 2024-04-26T04:42:28.423 INFO:tasks.workunit.client.0.smithi090.stdout:1/738: rename d0/d55/dda/d7/d4b/d53/c8a to d0/d55/dda/d7/d4b/cee 0 2024-04-26T04:42:28.424 INFO:tasks.workunit.client.0.smithi090.stdout:1/739: write d0/d55/dda/d7/d17/f3c [1472798,32863] 0 2024-04-26T04:42:28.425 INFO:tasks.workunit.client.0.smithi090.stdout:6/790: dwrite d9/d21/d3b/d86/fdc [0,4194304] 0 2024-04-26T04:42:28.425 INFO:tasks.workunit.client.0.smithi090.stdout:7/756: dread d0/d1/f38 [8388608,4194304] 0 2024-04-26T04:42:28.425 INFO:tasks.workunit.client.0.smithi090.stdout:7/757: fdatasync d0/db/d53/db7/d4f/f10f 0 2024-04-26T04:42:28.426 INFO:tasks.workunit.client.0.smithi090.stdout:1/740: symlink d0/d55/dda/d62/d78/dcc/lef 0 2024-04-26T04:42:28.431 INFO:tasks.workunit.client.0.smithi090.stdout:6/791: rename d9/d21/d3b/c81 to d9/d77/db0/db5/dca/dd8/c110 0 2024-04-26T04:42:28.433 INFO:tasks.workunit.client.0.smithi090.stdout:1/741: creat d0/d55/dda/d7/d4b/dca/ff0 x:0 0 0 2024-04-26T04:42:28.433 INFO:tasks.workunit.client.0.smithi090.stdout:1/742: write d0/d55/f63 [30848,116639] 0 2024-04-26T04:42:28.434 INFO:tasks.workunit.client.0.smithi090.stdout:1/743: readlink d0/d55/dda/d7/d4b/d53/l60 0 2024-04-26T04:42:28.434 INFO:tasks.workunit.client.0.smithi090.stdout:1/744: creat d0/d55/ff1 x:0 0 0 2024-04-26T04:42:28.434 INFO:tasks.workunit.client.0.smithi090.stdout:1/745: fsync d0/d55/dda/d7/d4a/fe0 0 2024-04-26T04:42:28.436 INFO:tasks.workunit.client.0.smithi090.stdout:1/746: creat d0/d55/dda/ff2 x:0 0 0 2024-04-26T04:42:28.438 INFO:tasks.workunit.client.0.smithi090.stdout:1/747: symlink d0/d55/dda/d7/d4b/lf3 0 2024-04-26T04:42:28.439 INFO:tasks.workunit.client.0.smithi090.stdout:1/748: unlink d0/c19 0 2024-04-26T04:42:28.440 INFO:tasks.workunit.client.0.smithi090.stdout:1/749: chown d0/d55/dda/d7/d69/f71 459 1 2024-04-26T04:42:28.457 INFO:tasks.workunit.client.0.smithi090.stdout:2/776: dwrite db/d11/d14/d25/d41/fba [4194304,4194304] 0 2024-04-26T04:42:28.458 INFO:tasks.workunit.client.0.smithi090.stdout:2/777: symlink db/d11/d14/d27/d28/dd1/dfe/l106 0 2024-04-26T04:42:28.459 INFO:tasks.workunit.client.0.smithi090.stdout:2/778: mkdir db/d11/d14/d25/d2d/d48/d5b/d107 0 2024-04-26T04:42:28.460 INFO:tasks.workunit.client.0.smithi090.stdout:2/779: write db/d11/d14/d25/d65/d95/faf [772471,28956] 0 2024-04-26T04:42:28.492 INFO:tasks.workunit.client.0.smithi090.stdout:5/689: dwrite d4/d1a/d1f/f3d [0,4194304] 0 2024-04-26T04:42:28.492 INFO:tasks.workunit.client.0.smithi090.stdout:0/805: dwrite d0/d4b/d5d/d48/dc1/f108 [0,4194304] 0 2024-04-26T04:42:28.493 INFO:tasks.workunit.client.0.smithi090.stdout:6/792: dwrite d9/d1c/f2b [0,4194304] 0 2024-04-26T04:42:28.494 INFO:tasks.workunit.client.0.smithi090.stdout:3/660: dwrite d1/d17/d61/d9d/fa5 [0,4194304] 0 2024-04-26T04:42:28.494 INFO:tasks.workunit.client.0.smithi090.stdout:7/758: dwrite d0/db/df/d4d/fc1 [8388608,4194304] 0 2024-04-26T04:42:28.494 INFO:tasks.workunit.client.0.smithi090.stdout:4/763: dwrite d3/db/d12/d20/d2d/d58/d6a/d74/da2/f37 [0,4194304] 0 2024-04-26T04:42:28.494 INFO:tasks.workunit.client.0.smithi090.stdout:4/764: fdatasync d3/db/d12/d3b/f82 0 2024-04-26T04:42:28.494 INFO:tasks.workunit.client.0.smithi090.stdout:4/765: write d3/d13/f87 [4805721,34603] 0 2024-04-26T04:42:28.494 INFO:tasks.workunit.client.0.smithi090.stdout:4/766: truncate d3/db/d60/dee/f6b 726409 0 2024-04-26T04:42:28.494 INFO:tasks.workunit.client.0.smithi090.stdout:8/871: dwrite dc/d34/d4d/f9e [0,4194304] 0 2024-04-26T04:42:28.497 INFO:tasks.workunit.client.0.smithi090.stdout:1/750: dwrite d0/d55/dda/d7/d17/d22/d93/fbb [0,4194304] 0 2024-04-26T04:42:28.497 INFO:tasks.workunit.client.0.smithi090.stdout:1/751: chown d0/d55/dda/d7/d17/d22/d31/cae 2638 1 2024-04-26T04:42:28.497 INFO:tasks.workunit.client.0.smithi090.stdout:0/806: mkdir d0/d4b/d5d/d5c/d117 0 2024-04-26T04:42:28.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/759: mknod d0/d109/d4a/c110 0 2024-04-26T04:42:28.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/760: stat d0/d109/d65/dbb/fc9 0 2024-04-26T04:42:28.503 INFO:tasks.workunit.client.0.smithi090.stdout:7/761: fsync d0/db/d15/dbc/ff6 0 2024-04-26T04:42:28.503 INFO:tasks.workunit.client.1.smithi104.stdout:9/964: sync 2024-04-26T04:42:28.504 INFO:tasks.workunit.client.1.smithi104.stdout:9/965: mkdir d11/d15/d1b/d143 0 2024-04-26T04:42:28.504 INFO:tasks.workunit.client.1.smithi104.stdout:9/966: fdatasync d11/d15/d1b/d6d/d9c/dfb/d102/f142 0 2024-04-26T04:42:28.504 INFO:tasks.workunit.client.1.smithi104.stdout:9/967: chown d11/d15/d98/cab 259866736 1 2024-04-26T04:42:28.515 INFO:tasks.workunit.client.0.smithi090.stdout:1/752: unlink d0/d55/dda/d7/d4a/d75/fd6 0 2024-04-26T04:42:28.516 INFO:tasks.workunit.client.0.smithi090.stdout:0/807: truncate d0/f7a 172051 0 2024-04-26T04:42:28.517 INFO:tasks.workunit.client.0.smithi090.stdout:6/793: getdents d9/d21/d40/d4c 0 2024-04-26T04:42:28.517 INFO:tasks.workunit.client.0.smithi090.stdout:6/794: dread - d9/d77/f7d zero size 2024-04-26T04:42:28.517 INFO:tasks.workunit.client.0.smithi090.stdout:6/795: fdatasync d9/d1c/f47 0 2024-04-26T04:42:28.518 INFO:tasks.workunit.client.0.smithi090.stdout:4/767: dread d3/f2c [0,4194304] 0 2024-04-26T04:42:28.523 INFO:tasks.workunit.client.0.smithi090.stdout:7/762: mkdir d0/db/df/d1c/d23/d8d/d111 0 2024-04-26T04:42:28.524 INFO:tasks.workunit.client.0.smithi090.stdout:1/753: symlink d0/d55/dda/d62/lf4 0 2024-04-26T04:42:28.526 INFO:tasks.workunit.client.0.smithi090.stdout:0/808: rmdir d0/d4b/d5d/d5c/dc4 39 2024-04-26T04:42:28.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/768: mknod d3/db/d12/d20/d24/d43/dd4/cf2 0 2024-04-26T04:42:28.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/769: write d3/d13/f81 [678505,104163] 0 2024-04-26T04:42:28.528 INFO:tasks.workunit.client.0.smithi090.stdout:4/770: creat d3/db/d12/d20/d24/da9/ff3 x:0 0 0 2024-04-26T04:42:28.531 INFO:tasks.workunit.client.0.smithi090.stdout:7/763: creat d0/db/df/d4d/d64/db0/dd6/f112 x:0 0 0 2024-04-26T04:42:28.531 INFO:tasks.workunit.client.0.smithi090.stdout:7/764: read d0/db/d53/db7/d4f/f76 [1107853,123830] 0 2024-04-26T04:42:28.531 INFO:tasks.workunit.client.0.smithi090.stdout:7/765: readlink d0/d1/d4/d3e/ld4 0 2024-04-26T04:42:28.536 INFO:tasks.workunit.client.0.smithi090.stdout:9/658: sync 2024-04-26T04:42:28.536 INFO:tasks.workunit.client.0.smithi090.stdout:4/771: truncate d3/db/d12/d20/d24/d43/d72/f80 1634789 0 2024-04-26T04:42:28.536 INFO:tasks.workunit.client.0.smithi090.stdout:4/772: dread - d3/db/d60/dee/fd1 zero size 2024-04-26T04:42:28.536 INFO:tasks.workunit.client.0.smithi090.stdout:7/766: rmdir d0/d109/d65 39 2024-04-26T04:42:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:6/796: dread d9/d77/db0/db5/dda/d6d/d92/f28 [0,4194304] 0 2024-04-26T04:42:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:6/797: chown d9/l63 4669365 1 2024-04-26T04:42:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:7/767: dread d0/db/d53/db7/d8e/fca [0,4194304] 0 2024-04-26T04:42:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:7/768: readlink d0/d1/da3/lb4 0 2024-04-26T04:42:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:7/769: fdatasync d0/db/d15/d29/f18 0 2024-04-26T04:42:28.538 INFO:tasks.workunit.client.0.smithi090.stdout:9/659: mkdir db/dd/d44/d82/dbd/de7 0 2024-04-26T04:42:28.539 INFO:tasks.workunit.client.0.smithi090.stdout:4/773: creat d3/db/d12/d20/d24/ff4 x:0 0 0 2024-04-26T04:42:28.541 INFO:tasks.workunit.client.0.smithi090.stdout:7/770: symlink d0/db/df/d4d/daf/dd1/d106/l113 0 2024-04-26T04:42:28.542 INFO:tasks.workunit.client.0.smithi090.stdout:4/774: mkdir d3/db/d12/d20/d2d/d58/d6a/da3/df5 0 2024-04-26T04:42:28.543 INFO:tasks.workunit.client.0.smithi090.stdout:7/771: chown d0/db/d53/db7/c6f 44 1 2024-04-26T04:42:28.544 INFO:tasks.workunit.client.0.smithi090.stdout:7/772: mkdir d0/d109/d114 0 2024-04-26T04:42:28.546 INFO:tasks.workunit.client.0.smithi090.stdout:7/773: creat d0/db/d53/f115 x:0 0 0 2024-04-26T04:42:28.547 INFO:tasks.workunit.client.0.smithi090.stdout:0/809: getdents d0/d4b/d5d/d5c 0 2024-04-26T04:42:28.549 INFO:tasks.workunit.client.0.smithi090.stdout:7/774: unlink d0/d109/d65/fc7 0 2024-04-26T04:42:28.551 INFO:tasks.workunit.client.0.smithi090.stdout:4/775: write d3/db/d12/d20/d24/d43/dd4/fb7 [3526563,100758] 0 2024-04-26T04:42:28.552 INFO:tasks.workunit.client.0.smithi090.stdout:4/776: symlink d3/db/d12/d20/d2d/d58/d6a/da3/df5/lf6 0 2024-04-26T04:42:28.553 INFO:tasks.workunit.client.0.smithi090.stdout:5/690: dwrite d4/d1a/d2a/d79/d90/d56/f8b [0,4194304] 0 2024-04-26T04:42:28.553 INFO:tasks.workunit.client.0.smithi090.stdout:6/798: read d9/d1c/f2b [2706911,10523] 0 2024-04-26T04:42:28.553 INFO:tasks.workunit.client.0.smithi090.stdout:6/799: creat d9/d1c/dc5/f111 x:0 0 0 2024-04-26T04:42:28.556 INFO:tasks.workunit.client.0.smithi090.stdout:5/691: symlink d4/d1a/d2a/d79/d90/d56/d80/ld9 0 2024-04-26T04:42:28.557 INFO:tasks.workunit.client.0.smithi090.stdout:5/692: symlink d4/d1a/d2a/d31/lda 0 2024-04-26T04:42:28.558 INFO:tasks.workunit.client.0.smithi090.stdout:7/775: dread d0/d1/d4/f2e [0,4194304] 0 2024-04-26T04:42:28.560 INFO:tasks.workunit.client.0.smithi090.stdout:7/776: rmdir d0/d109/d114 0 2024-04-26T04:42:28.562 INFO:tasks.workunit.client.0.smithi090.stdout:7/777: mkdir d0/d109/d65/db2/d116 0 2024-04-26T04:42:28.564 INFO:tasks.workunit.client.0.smithi090.stdout:7/778: creat d0/db/df/d4d/d64/f117 x:0 0 0 2024-04-26T04:42:28.564 INFO:tasks.workunit.client.0.smithi090.stdout:7/779: truncate d0/db/df/d4d/d64/df2/d10a/f10b 226990 0 2024-04-26T04:42:28.570 INFO:tasks.workunit.client.0.smithi090.stdout:3/661: dwrite d1/d17/d36/d33/f80 [0,4194304] 0 2024-04-26T04:42:28.571 INFO:tasks.workunit.client.0.smithi090.stdout:4/777: write d3/db/d12/f4c [3100799,13709] 0 2024-04-26T04:42:28.571 INFO:tasks.workunit.client.0.smithi090.stdout:4/778: write d3/db/d60/dee/dae/da5/fca [4791030,76452] 0 2024-04-26T04:42:28.574 INFO:tasks.workunit.client.0.smithi090.stdout:4/779: rename d3/db/d12/d20/d24/d43/dd4/fd2 to d3/db/d12/d20/d24/ff7 0 2024-04-26T04:42:28.576 INFO:tasks.workunit.client.0.smithi090.stdout:7/780: dread d0/db/d15/d29/f18 [0,4194304] 0 2024-04-26T04:42:28.576 INFO:tasks.workunit.client.0.smithi090.stdout:7/781: chown d0/db/df/d4d/daf 35 1 2024-04-26T04:42:28.577 INFO:tasks.workunit.client.0.smithi090.stdout:7/782: symlink d0/d109/d65/l118 0 2024-04-26T04:42:28.577 INFO:tasks.workunit.client.0.smithi090.stdout:7/783: readlink d0/db/de6/le8 0 2024-04-26T04:42:28.580 INFO:tasks.workunit.client.0.smithi090.stdout:7/784: rename d0/db/d15/d29/df3 to d0/d1/d4/d3e/d119 0 2024-04-26T04:42:28.580 INFO:tasks.workunit.client.0.smithi090.stdout:7/785: read - d0/db/d15/dbc/ff6 zero size 2024-04-26T04:42:28.580 INFO:tasks.workunit.client.0.smithi090.stdout:7/786: write d0/db/df/d1c/d23/d8d/fdd [132384,6775] 0 2024-04-26T04:42:28.586 INFO:tasks.workunit.client.0.smithi090.stdout:4/780: dread d3/db/d12/f10 [0,4194304] 0 2024-04-26T04:42:28.586 INFO:tasks.workunit.client.0.smithi090.stdout:4/781: write f0 [149910,18293] 0 2024-04-26T04:42:28.586 INFO:tasks.workunit.client.0.smithi090.stdout:4/782: chown d3/db/d12/d20/d24/d43/dd4/cf2 31 1 2024-04-26T04:42:28.588 INFO:tasks.workunit.client.0.smithi090.stdout:3/662: dread d1/f3c [0,4194304] 0 2024-04-26T04:42:28.593 INFO:tasks.workunit.client.0.smithi090.stdout:7/787: dread d0/d1/f61 [0,4194304] 0 2024-04-26T04:42:28.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/872: dwrite dc/dd2/d39/d125/ff5 [0,4194304] 0 2024-04-26T04:42:28.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/873: dread - dc/dd2/d3b/d3d/f126 zero size 2024-04-26T04:42:28.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/874: creat dc/d34/f131 x:0 0 0 2024-04-26T04:42:28.611 INFO:tasks.workunit.client.0.smithi090.stdout:8/875: write dc/f74 [1789103,81798] 0 2024-04-26T04:42:28.625 INFO:tasks.workunit.client.0.smithi090.stdout:8/876: write dc/dd2/d39/d55/f70 [1564923,107632] 0 2024-04-26T04:42:28.626 INFO:tasks.workunit.client.0.smithi090.stdout:8/877: readlink dc/d1f/l2a 0 2024-04-26T04:42:28.636 INFO:tasks.workunit.client.1.smithi104.stdout:9/968: dwrite d11/d15/f59 [4194304,4194304] 0 2024-04-26T04:42:28.636 INFO:tasks.workunit.client.1.smithi104.stdout:9/969: creat d11/d1d/d87/f144 x:0 0 0 2024-04-26T04:42:28.638 INFO:tasks.workunit.client.1.smithi104.stdout:9/970: creat d11/d15/d32/dd6/ded/d12d/f145 x:0 0 0 2024-04-26T04:42:28.642 INFO:tasks.workunit.client.0.smithi090.stdout:5/693: dwrite d4/d1a/d2a/d79/d90/f96 [0,4194304] 0 2024-04-26T04:42:28.642 INFO:tasks.workunit.client.0.smithi090.stdout:5/694: creat d4/d1a/d2a/d79/d90/d94/fdb x:0 0 0 2024-04-26T04:42:28.647 INFO:tasks.workunit.client.0.smithi090.stdout:5/695: mknod d4/d1a/d2a/d79/d90/db3/cdc 0 2024-04-26T04:42:28.647 INFO:tasks.workunit.client.0.smithi090.stdout:5/696: creat d4/fdd x:0 0 0 2024-04-26T04:42:28.647 INFO:tasks.workunit.client.0.smithi090.stdout:5/697: creat d4/d1a/df/d4a/fde x:0 0 0 2024-04-26T04:42:28.651 INFO:tasks.workunit.client.0.smithi090.stdout:5/698: creat d4/d1a/df/d72/dc0/fdf x:0 0 0 2024-04-26T04:42:28.652 INFO:tasks.workunit.client.0.smithi090.stdout:6/800: dwrite d9/d1c/f80 [0,4194304] 0 2024-04-26T04:42:28.653 INFO:tasks.workunit.client.0.smithi090.stdout:1/754: dwrite d0/d55/dda/d7/d49/f5f [0,4194304] 0 2024-04-26T04:42:28.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/660: dwrite db/dd/d18/d38/d39/d6b/dc0/fad [0,4194304] 0 2024-04-26T04:42:28.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/661: chown db/dd/d44/d82/d85/db4/d58/l78 30426734 1 2024-04-26T04:42:28.655 INFO:tasks.workunit.client.0.smithi090.stdout:3/663: dwrite d1/d5c/f99 [0,4194304] 0 2024-04-26T04:42:28.656 INFO:tasks.workunit.client.0.smithi090.stdout:3/664: read d1/d17/d36/d33/d68/d71/f9e [589812,93758] 0 2024-04-26T04:42:28.657 INFO:tasks.workunit.client.0.smithi090.stdout:6/801: mknod d9/d77/db0/db5/dda/d6d/c112 0 2024-04-26T04:42:28.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/699: write d4/d1a/f17 [1618405,78008] 0 2024-04-26T04:42:28.668 INFO:tasks.workunit.client.0.smithi090.stdout:1/755: dread d0/d55/dda/fb2 [0,4194304] 0 2024-04-26T04:42:28.668 INFO:tasks.workunit.client.0.smithi090.stdout:9/662: link db/dd/d18/d57/lbc db/dd/d18/d38/le8 0 2024-04-26T04:42:28.670 INFO:tasks.workunit.client.0.smithi090.stdout:3/665: symlink d1/d17/d36/d33/d68/dae/dca/d1b/d58/db5/ldf 0 2024-04-26T04:42:28.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/878: dwrite dc/dd2/d4f/da6/fd5 [0,4194304] 0 2024-04-26T04:42:28.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/879: creat dc/dd2/d3b/d3d/d111/f132 x:0 0 0 2024-04-26T04:42:28.672 INFO:tasks.workunit.client.0.smithi090.stdout:8/880: readlink dc/d1f/ld3 0 2024-04-26T04:42:28.672 INFO:tasks.workunit.client.0.smithi090.stdout:6/802: link d9/d77/db0/db5/dda/d6d/leb d9/d77/db0/db5/dda/d58/l113 0 2024-04-26T04:42:28.672 INFO:tasks.workunit.client.0.smithi090.stdout:7/788: dwrite d0/db/fa1 [0,4194304] 0 2024-04-26T04:42:28.674 INFO:tasks.workunit.client.0.smithi090.stdout:1/756: rename d0/d55/dda/d7/d4a/d75/fcb to d0/d55/dda/d7/d4a/ff5 0 2024-04-26T04:42:28.674 INFO:tasks.workunit.client.0.smithi090.stdout:1/757: fdatasync d0/d55/dda/d7/d17/d22/d31/f6d 0 2024-04-26T04:42:28.677 INFO:tasks.workunit.client.0.smithi090.stdout:5/700: dread d4/d1a/d2a/d79/d90/d49/f9a [0,4194304] 0 2024-04-26T04:42:28.679 INFO:tasks.workunit.client.1.smithi104.stdout:9/971: dwrite d11/d15/d32/d4c/d8a/f133 [0,4194304] 0 2024-04-26T04:42:28.679 INFO:tasks.workunit.client.1.smithi104.stdout:9/972: rename d11/d15/d1b/f72 to d11/d90/f146 0 2024-04-26T04:42:28.679 INFO:tasks.workunit.client.1.smithi104.stdout:9/973: link d11/lb0 d11/d1d/d71/d94/l147 0 2024-04-26T04:42:28.679 INFO:tasks.workunit.client.0.smithi090.stdout:3/666: symlink d1/d17/d36/d2a/d3a/d5d/le0 0 2024-04-26T04:42:28.680 INFO:tasks.workunit.client.0.smithi090.stdout:3/667: chown d1/d17/d36/d33/d68/dae/dca/d2c/fb8 86506599 1 2024-04-26T04:42:28.680 INFO:tasks.workunit.client.0.smithi090.stdout:3/668: chown d1/d17/d36/d33/d68/dae/fb3 100 1 2024-04-26T04:42:28.680 INFO:tasks.workunit.client.0.smithi090.stdout:1/758: dread d0/d55/dda/d7/d17/f29 [0,4194304] 0 2024-04-26T04:42:28.681 INFO:tasks.workunit.client.1.smithi104.stdout:9/974: mknod d11/d15/d1b/d6d/c148 0 2024-04-26T04:42:28.681 INFO:tasks.workunit.client.1.smithi104.stdout:9/975: symlink d11/d15/d1b/d6d/d9c/d116/l149 0 2024-04-26T04:42:28.681 INFO:tasks.workunit.client.1.smithi104.stdout:9/976: truncate d11/d15/d1b/f10a 1197492 0 2024-04-26T04:42:28.684 INFO:tasks.workunit.client.0.smithi090.stdout:6/803: dread d9/d3c/ffc [0,4194304] 0 2024-04-26T04:42:28.685 INFO:tasks.workunit.client.0.smithi090.stdout:8/881: unlink dc/d34/d4d/f12b 0 2024-04-26T04:42:28.690 INFO:tasks.workunit.client.0.smithi090.stdout:4/783: link d3/db/d12/d20/d24/ff7 d3/db/d12/d20/d2d/d2e/ff8 0 2024-04-26T04:42:28.691 INFO:tasks.workunit.client.1.smithi104.stdout:9/977: dread d11/f16 [4194304,4194304] 0 2024-04-26T04:42:28.691 INFO:tasks.workunit.client.1.smithi104.stdout:9/978: creat d11/d90/dfd/f14a x:0 0 0 2024-04-26T04:42:28.693 INFO:tasks.workunit.client.0.smithi090.stdout:9/663: rename db/dd/d18/d57/dcc/fdb to db/dd/d44/d82/d85/db4/d99/db2/fe9 0 2024-04-26T04:42:28.694 INFO:tasks.workunit.client.1.smithi104.stdout:9/979: rename d11/d15/da3 to d11/d15/d32/d14b 0 2024-04-26T04:42:28.695 INFO:tasks.workunit.client.1.smithi104.stdout:9/980: dread d11/d1d/d34/f3c [0,4194304] 0 2024-04-26T04:42:28.695 INFO:tasks.workunit.client.0.smithi090.stdout:3/669: creat d1/d17/d36/d2a/d3a/d5d/d7e/fe1 x:0 0 0 2024-04-26T04:42:28.695 INFO:tasks.workunit.client.0.smithi090.stdout:3/670: chown d1/d17/d61/d9d/fd5 3555 1 2024-04-26T04:42:28.695 INFO:tasks.workunit.client.0.smithi090.stdout:3/671: chown d1/d59/dc9/d7d/ca3 450498023 1 2024-04-26T04:42:28.697 INFO:tasks.workunit.client.0.smithi090.stdout:1/759: unlink d0/d55/dda/d7/d17/d22/d93/db8/de9/fec 0 2024-04-26T04:42:28.699 INFO:tasks.workunit.client.0.smithi090.stdout:6/804: truncate d9/d21/d3b/d86/fdc 3184972 0 2024-04-26T04:42:28.700 INFO:tasks.workunit.client.1.smithi104.stdout:9/981: mkdir d11/d15/d1b/d6d/d14c 0 2024-04-26T04:42:28.700 INFO:tasks.workunit.client.0.smithi090.stdout:8/882: creat dc/dd2/d4f/da6/db1/ddd/f133 x:0 0 0 2024-04-26T04:42:28.702 INFO:tasks.workunit.client.1.smithi104.stdout:9/982: write d11/f20 [302310,71238] 0 2024-04-26T04:42:28.702 INFO:tasks.workunit.client.1.smithi104.stdout:9/983: chown d11/d1d/d71/d94/c117 4 1 2024-04-26T04:42:28.702 INFO:tasks.workunit.client.1.smithi104.stdout:9/984: stat d11/lb0 0 2024-04-26T04:42:28.702 INFO:tasks.workunit.client.1.smithi104.stdout:9/985: fsync d11/d15/d1b/d6d/dd5/d12e/f7a 0 2024-04-26T04:42:28.703 INFO:tasks.workunit.client.1.smithi104.stdout:9/986: truncate d11/d15/d32/d35/f80 3445463 0 2024-04-26T04:42:28.703 INFO:tasks.workunit.client.0.smithi090.stdout:7/789: rename d0/d1/d4/ccc to d0/db/df/d4d/c11a 0 2024-04-26T04:42:28.705 INFO:tasks.workunit.client.0.smithi090.stdout:9/664: link db/dd/d18/f43 db/da3/da8/fea 0 2024-04-26T04:42:28.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/883: unlink dc/c32 0 2024-04-26T04:42:28.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/884: chown dc/dd2/d3b/d3d/db8/le0 23632687 1 2024-04-26T04:42:28.707 INFO:tasks.workunit.client.0.smithi090.stdout:8/885: fdatasync dc/dd2/d3b/d3d/f126 0 2024-04-26T04:42:28.710 INFO:tasks.workunit.client.0.smithi090.stdout:7/790: link d0/db/df/d4d/d64/l89 d0/db/df/d4d/daf/dd1/l11b 0 2024-04-26T04:42:28.710 INFO:tasks.workunit.client.0.smithi090.stdout:4/784: dread d3/db/d12/d20/d2d/d58/d6a/d74/da2/f5a [0,4194304] 0 2024-04-26T04:42:28.710 INFO:tasks.workunit.client.0.smithi090.stdout:7/791: write d0/db/d53/d5e/fa9 [1045101,95162] 0 2024-04-26T04:42:28.710 INFO:tasks.workunit.client.0.smithi090.stdout:7/792: dread - d0/db/d15/dbc/ff6 zero size 2024-04-26T04:42:28.710 INFO:tasks.workunit.client.0.smithi090.stdout:8/886: mknod dc/dd2/d4f/da6/dfc/c134 0 2024-04-26T04:42:28.713 INFO:tasks.workunit.client.0.smithi090.stdout:4/785: symlink d3/db/d12/d3b/lf9 0 2024-04-26T04:42:28.715 INFO:tasks.workunit.client.0.smithi090.stdout:7/793: mkdir d0/d109/d65/d11c 0 2024-04-26T04:42:28.717 INFO:tasks.workunit.client.0.smithi090.stdout:1/760: dread d0/d55/dda/d7/d17/d22/d31/f34 [0,4194304] 0 2024-04-26T04:42:28.717 INFO:tasks.workunit.client.0.smithi090.stdout:1/761: creat d0/d55/dda/d7/d17/d22/d93/ff6 x:0 0 0 2024-04-26T04:42:28.718 INFO:tasks.workunit.client.0.smithi090.stdout:5/701: dwrite d4/d1a/d2a/d79/d90/fa8 [0,4194304] 0 2024-04-26T04:42:28.718 INFO:tasks.workunit.client.0.smithi090.stdout:5/702: chown d4/d1a/d1f/d53/d59 623041 1 2024-04-26T04:42:28.719 INFO:tasks.workunit.client.0.smithi090.stdout:7/794: rename d0/db/d53/db7/c6f to d0/c11d 0 2024-04-26T04:42:28.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/703: symlink d4/le0 0 2024-04-26T04:42:28.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/704: read - d4/d1a/fba zero size 2024-04-26T04:42:28.723 INFO:tasks.workunit.client.0.smithi090.stdout:1/762: getdents d0/d55/dda/d7/d17/d22/d93/db0 0 2024-04-26T04:42:28.726 INFO:tasks.workunit.client.0.smithi090.stdout:5/705: creat d4/d1a/d1f/fe1 x:0 0 0 2024-04-26T04:42:28.727 INFO:tasks.workunit.client.0.smithi090.stdout:5/706: chown d4/d1a/f78 0 1 2024-04-26T04:42:28.728 INFO:tasks.workunit.client.0.smithi090.stdout:5/707: link d4/d1a/d45/f74 d4/d1a/df/d36/fe2 0 2024-04-26T04:42:28.743 INFO:tasks.workunit.client.0.smithi090.stdout:1/763: read d0/d55/dda/d7/d17/d22/f52 [2120412,81580] 0 2024-04-26T04:42:28.757 INFO:tasks.workunit.client.0.smithi090.stdout:3/672: dwrite d1/d9/fa [0,4194304] 0 2024-04-26T04:42:28.766 INFO:tasks.workunit.client.0.smithi090.stdout:3/673: dread d1/d9/f20 [0,4194304] 0 2024-04-26T04:42:28.767 INFO:tasks.workunit.client.0.smithi090.stdout:3/674: creat d1/d17/d36/d33/d43/fe2 x:0 0 0 2024-04-26T04:42:28.768 INFO:tasks.workunit.client.0.smithi090.stdout:3/675: creat d1/d17/d36/d2a/fe3 x:0 0 0 2024-04-26T04:42:28.779 INFO:tasks.workunit.client.0.smithi090.stdout:2/780: sync 2024-04-26T04:42:28.779 INFO:tasks.workunit.client.0.smithi090.stdout:6/805: dwrite d9/d77/db0/db5/dda/d6d/d92/f29 [0,4194304] 0 2024-04-26T04:42:28.780 INFO:tasks.workunit.client.0.smithi090.stdout:2/781: mkdir db/dc/d6f/dd8/d108 0 2024-04-26T04:42:28.780 INFO:tasks.workunit.client.0.smithi090.stdout:2/782: write db/d11/d14/d1b/d20/d4c/fae [366485,26667] 0 2024-04-26T04:42:28.781 INFO:tasks.workunit.client.0.smithi090.stdout:6/806: creat d9/d77/db0/db5/dca/dd8/f114 x:0 0 0 2024-04-26T04:42:28.781 INFO:tasks.workunit.client.0.smithi090.stdout:2/783: truncate db/d59/f62 1491035 0 2024-04-26T04:42:28.782 INFO:tasks.workunit.client.0.smithi090.stdout:6/807: mknod d9/d1c/d31/d79/c115 0 2024-04-26T04:42:28.785 INFO:tasks.workunit.client.0.smithi090.stdout:2/784: write db/d59/f78 [143284,130983] 0 2024-04-26T04:42:28.786 INFO:tasks.workunit.client.0.smithi090.stdout:0/810: sync 2024-04-26T04:42:28.786 INFO:tasks.workunit.client.0.smithi090.stdout:0/811: dread - d0/d4b/d5d/d5c/d8a/d45/d93/f10f zero size 2024-04-26T04:42:28.788 INFO:tasks.workunit.client.0.smithi090.stdout:9/665: dwrite db/f35 [0,4194304] 0 2024-04-26T04:42:28.788 INFO:tasks.workunit.client.0.smithi090.stdout:4/786: dwrite d3/f9 [0,4194304] 0 2024-04-26T04:42:28.788 INFO:tasks.workunit.client.0.smithi090.stdout:5/708: dwrite d4/d1a/d2a/d79/f85 [0,4194304] 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/787: getdents d3/db/d12/d20/d2d/dbe/dde 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:4/788: write d3/d13/f9f [488632,27285] 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:5/709: write d4/d1a/f17 [457906,90319] 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:5/710: stat d4/l9d 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:5/711: fdatasync d4/d1a/d45/f74 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:7/795: dwrite d0/db/d15/d29/f18 [0,4194304] 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:7/796: truncate d0/d1/d4/d3e/dd3/de2/fea 686781 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:7/797: getdents d0/d109/d65/db2/d116 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:1/764: dwrite d0/d55/dda/d7/d4a/d75/d80/d58/fbf [0,4194304] 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:2/785: getdents db/d11/d14/d27/d28/dd1 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:3/676: dwrite d1/d17/d36/d2a/d3a/d5d/d7e/fe1 [0,4194304] 0 2024-04-26T04:42:28.789 INFO:tasks.workunit.client.0.smithi090.stdout:3/677: truncate d1/d17/d36/d2a/d3a/fd3 821048 0 2024-04-26T04:42:28.790 INFO:tasks.workunit.client.0.smithi090.stdout:0/812: symlink d0/d4b/d5d/d48/d7d/l118 0 2024-04-26T04:42:28.790 INFO:tasks.workunit.client.0.smithi090.stdout:0/813: stat d0/d4b/d5d/d5c/d8a/d45/d53/dc2/cd8 0 2024-04-26T04:42:28.792 INFO:tasks.workunit.client.0.smithi090.stdout:9/666: creat db/feb x:0 0 0 2024-04-26T04:42:28.793 INFO:tasks.workunit.client.1.smithi104.stdout:9/987: truncate d11/d15/d32/d35/f80 3166144 0 2024-04-26T04:42:28.793 INFO:tasks.workunit.client.0.smithi090.stdout:6/808: dread d9/d77/db0/db5/dda/d6d/d92/f25 [0,4194304] 0 2024-04-26T04:42:28.795 INFO:tasks.workunit.client.1.smithi104.stdout:9/988: creat d11/d1d/d34/db3/df1/f14d x:0 0 0 2024-04-26T04:42:28.797 INFO:tasks.workunit.client.0.smithi090.stdout:7/798: mknod d0/db/d53/d5e/c11e 0 2024-04-26T04:42:28.798 INFO:tasks.workunit.client.0.smithi090.stdout:7/799: write d0/db/df/d1c/fa6 [941769,114610] 0 2024-04-26T04:42:28.798 INFO:tasks.workunit.client.0.smithi090.stdout:5/712: dread d4/d1a/df/d4a/f55 [0,4194304] 0 2024-04-26T04:42:28.799 INFO:tasks.workunit.client.0.smithi090.stdout:2/786: write db/d11/d14/d25/d41/fba [4419651,86830] 0 2024-04-26T04:42:28.805 INFO:tasks.workunit.client.1.smithi104.stdout:9/989: dread d11/d1d/f49 [0,4194304] 0 2024-04-26T04:42:28.805 INFO:tasks.workunit.client.1.smithi104.stdout:9/990: write d11/fd3 [87314,75359] 0 2024-04-26T04:42:28.807 INFO:tasks.workunit.client.1.smithi104.stdout:9/991: mkdir d11/d15/d1b/d143/d14e 0 2024-04-26T04:42:28.809 INFO:tasks.workunit.client.1.smithi104.stdout:9/992: truncate d11/d15/d98/f11f 2176935 0 2024-04-26T04:42:28.809 INFO:tasks.workunit.client.1.smithi104.stdout:9/993: getdents d11/d15/d1b/d6d/dd5/d12e 0 2024-04-26T04:42:28.811 INFO:tasks.workunit.client.0.smithi090.stdout:1/765: rmdir d0/d55/dda/d7/d4b/d53/de6 0 2024-04-26T04:42:28.811 INFO:tasks.workunit.client.0.smithi090.stdout:1/766: write d0/d55/dda/d7/f90 [2360805,14552] 0 2024-04-26T04:42:28.811 INFO:tasks.workunit.client.0.smithi090.stdout:1/767: write d0/d55/dda/d7/d17/d22/d93/ff6 [464816,5689] 0 2024-04-26T04:42:28.812 INFO:tasks.workunit.client.0.smithi090.stdout:7/800: dread d0/d1/d4/f10 [0,4194304] 0 2024-04-26T04:42:28.812 INFO:tasks.workunit.client.0.smithi090.stdout:7/801: chown d0/db/d15/dbc 355233 1 2024-04-26T04:42:28.814 INFO:tasks.workunit.client.0.smithi090.stdout:3/678: mkdir d1/d17/d36/d33/d68/dae/dca/d2c/d97/de4 0 2024-04-26T04:42:28.814 INFO:tasks.workunit.client.0.smithi090.stdout:3/679: readlink d1/d17/d36/d33/d43/l69 0 2024-04-26T04:42:28.816 INFO:tasks.workunit.client.0.smithi090.stdout:9/667: rename db/dd/d18/d38/cdf to db/dd/d18/d38/d39/d3f/cec 0 2024-04-26T04:42:28.817 INFO:tasks.workunit.client.0.smithi090.stdout:6/809: mknod d9/d1c/c116 0 2024-04-26T04:42:28.817 INFO:tasks.workunit.client.0.smithi090.stdout:6/810: fsync d9/d1c/d31/d69/fa4 0 2024-04-26T04:42:28.817 INFO:tasks.workunit.client.0.smithi090.stdout:6/811: chown d9/d21/f48 41350298 1 2024-04-26T04:42:28.817 INFO:tasks.workunit.client.0.smithi090.stdout:0/814: dwrite d0/d4b/d5d/d48/fa5 [0,4194304] 0 2024-04-26T04:42:28.817 INFO:tasks.workunit.client.0.smithi090.stdout:4/789: getdents d3/d13/d1d/d50 0 2024-04-26T04:42:28.821 INFO:tasks.workunit.client.0.smithi090.stdout:5/713: mkdir d4/d1a/df/d36/d8e/de3 0 2024-04-26T04:42:28.825 INFO:tasks.workunit.client.0.smithi090.stdout:7/802: symlink d0/db/d15/dbc/l11f 0 2024-04-26T04:42:28.825 INFO:tasks.workunit.client.0.smithi090.stdout:2/787: dwrite db/d11/d14/d25/d65/f9d [4194304,4194304] 0 2024-04-26T04:42:28.825 INFO:tasks.workunit.client.0.smithi090.stdout:2/788: readlink db/dc/l1a 0 2024-04-26T04:42:28.826 INFO:tasks.workunit.client.0.smithi090.stdout:3/680: symlink d1/d17/d61/d9d/le5 0 2024-04-26T04:42:28.827 INFO:tasks.workunit.client.0.smithi090.stdout:9/668: symlink db/dd/d18/d38/d39/d6b/led 0 2024-04-26T04:42:28.829 INFO:tasks.workunit.client.0.smithi090.stdout:6/812: mkdir d9/d1c/d31/d69/db7/d117 0 2024-04-26T04:42:28.829 INFO:tasks.workunit.client.0.smithi090.stdout:6/813: fsync d9/d77/db0/db5/dda/d6d/f45 0 2024-04-26T04:42:28.829 INFO:tasks.workunit.client.0.smithi090.stdout:6/814: creat d9/d77/db0/db5/dda/d58/f118 x:0 0 0 2024-04-26T04:42:28.832 INFO:tasks.workunit.client.0.smithi090.stdout:0/815: mkdir d0/d4b/db3/dcc/d119 0 2024-04-26T04:42:28.832 INFO:tasks.workunit.client.0.smithi090.stdout:0/816: creat d0/d4b/d5d/d48/dc1/f11a x:0 0 0 2024-04-26T04:42:28.833 INFO:tasks.workunit.client.0.smithi090.stdout:0/817: write d0/d4b/d5d/d5c/d8a/d45/d53/fa3 [1627170,88513] 0 2024-04-26T04:42:28.835 INFO:tasks.workunit.client.0.smithi090.stdout:7/803: rename d0/d1/d4/d3e/d119 to d0/d109/d65/db2/d120 0 2024-04-26T04:42:28.836 INFO:tasks.workunit.client.0.smithi090.stdout:1/768: dwrite d0/d55/dda/d7/d69/f71 [0,4194304] 0 2024-04-26T04:42:28.836 INFO:tasks.workunit.client.0.smithi090.stdout:1/769: fsync d0/d55/dda/d7/d49/f5e 0 2024-04-26T04:42:28.836 INFO:tasks.workunit.client.0.smithi090.stdout:1/770: read d0/d55/dda/d7/d17/f3d [6543,24224] 0 2024-04-26T04:42:28.836 INFO:tasks.workunit.client.0.smithi090.stdout:9/669: symlink db/dd/d44/d82/d85/db4/d99/lee 0 2024-04-26T04:42:28.839 INFO:tasks.workunit.client.0.smithi090.stdout:6/815: mknod d9/d77/d9a/c119 0 2024-04-26T04:42:28.841 INFO:tasks.workunit.client.1.smithi104.stdout:9/994: dwrite d11/d15/d32/d4c/d7c/fb4 [0,4194304] 0 2024-04-26T04:42:28.841 INFO:tasks.workunit.client.0.smithi090.stdout:0/818: chown d0/d4b/d5d/l4c 15 1 2024-04-26T04:42:28.842 INFO:tasks.workunit.client.1.smithi104.stdout:9/995: mkdir d11/d15/d32/d14f 0 2024-04-26T04:42:28.842 INFO:tasks.workunit.client.1.smithi104.stdout:9/996: chown d11/l1c 773277 1 2024-04-26T04:42:28.844 INFO:tasks.workunit.client.0.smithi090.stdout:7/804: getdents d0/db/d15/d29 0 2024-04-26T04:42:28.844 INFO:tasks.workunit.client.0.smithi090.stdout:5/714: dwrite d4/d1a/df/d36/f54 [4194304,4194304] 0 2024-04-26T04:42:28.845 INFO:tasks.workunit.client.0.smithi090.stdout:7/805: write d0/db/d53/db7/f9b [232014,100971] 0 2024-04-26T04:42:28.847 INFO:tasks.workunit.client.0.smithi090.stdout:2/789: rename db/d11/d14/d1b/d20/ld6 to db/d11/d14/d1b/d20/d4c/d98/db4/dff/l109 0 2024-04-26T04:42:28.848 INFO:tasks.workunit.client.0.smithi090.stdout:2/790: readlink db/d11/d14/d63/l7e 0 2024-04-26T04:42:28.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/806: dread d0/db/d53/db7/d4f/f76 [0,4194304] 0 2024-04-26T04:42:28.848 INFO:tasks.workunit.client.0.smithi090.stdout:7/807: chown d0/d1/d4/lac 11982 1 2024-04-26T04:42:28.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/670: creat db/dd/d44/d82/dbd/fef x:0 0 0 2024-04-26T04:42:28.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/671: stat db/dd/d44/d82/dbd/fef 0 2024-04-26T04:42:28.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/672: readlink db/dd/d18/d95/lc7 0 2024-04-26T04:42:28.856 INFO:tasks.workunit.client.0.smithi090.stdout:1/771: dwrite d0/f86 [4194304,4194304] 0 2024-04-26T04:42:28.856 INFO:tasks.workunit.client.0.smithi090.stdout:6/816: unlink d9/l5e 0 2024-04-26T04:42:28.857 INFO:tasks.workunit.client.0.smithi090.stdout:0/819: truncate d0/d4b/d5d/d5c/d8a/f3e 3286853 0 2024-04-26T04:42:28.862 INFO:tasks.workunit.client.0.smithi090.stdout:5/715: link d4/d1a/d1f/d53/d59/d5e/fa5 d4/d1a/df/d4a/fe4 0 2024-04-26T04:42:28.867 INFO:tasks.workunit.client.0.smithi090.stdout:3/681: rename d1/d17/d36/d33/daa/lc6 to d1/d17/d36/d2a/d3a/d5d/d7e/le6 0 2024-04-26T04:42:28.868 INFO:tasks.workunit.client.0.smithi090.stdout:2/791: creat db/d11/d14/d25/d41/f10a x:0 0 0 2024-04-26T04:42:28.870 INFO:tasks.workunit.client.0.smithi090.stdout:7/808: creat d0/d105/f121 x:0 0 0 2024-04-26T04:42:28.873 INFO:tasks.workunit.client.0.smithi090.stdout:9/673: symlink db/dd/d44/d82/d85/db4/dde/lf0 0 2024-04-26T04:42:28.877 INFO:tasks.workunit.client.0.smithi090.stdout:3/682: unlink d1/d17/d36/d2a/d3a/d5d/cc5 0 2024-04-26T04:42:28.878 INFO:tasks.workunit.client.0.smithi090.stdout:8/887: sync 2024-04-26T04:42:28.879 INFO:tasks.workunit.client.0.smithi090.stdout:7/809: unlink d0/db/d53/d5e/cdb 0 2024-04-26T04:42:28.879 INFO:tasks.workunit.client.0.smithi090.stdout:7/810: write d0/db/df/d4d/d64/d66/f7c [60999,39031] 0 2024-04-26T04:42:28.881 INFO:tasks.workunit.client.0.smithi090.stdout:9/674: creat db/dd/d18/d38/d39/dba/ff1 x:0 0 0 2024-04-26T04:42:28.882 INFO:tasks.workunit.client.0.smithi090.stdout:6/817: mknod d9/d77/db0/db5/dda/d6d/d92/c11a 0 2024-04-26T04:42:28.886 INFO:tasks.workunit.client.0.smithi090.stdout:1/772: write d0/d55/dda/d7/f30 [12186,116573] 0 2024-04-26T04:42:28.887 INFO:tasks.workunit.client.0.smithi090.stdout:8/888: rename dc/dd2/d4f/d58/d95/fb3 to dc/dd2/d4f/db7/f135 0 2024-04-26T04:42:28.887 INFO:tasks.workunit.client.0.smithi090.stdout:8/889: truncate dc/dd2/d4f/da6/db1/ddd/f133 569585 0 2024-04-26T04:42:28.887 INFO:tasks.workunit.client.0.smithi090.stdout:8/890: write dc/dd2/d39/d125/f110 [111541,34774] 0 2024-04-26T04:42:28.888 INFO:tasks.workunit.client.0.smithi090.stdout:8/891: write dc/d71/f105 [898875,16178] 0 2024-04-26T04:42:28.890 INFO:tasks.workunit.client.0.smithi090.stdout:3/683: mkdir d1/d17/d61/de7 0 2024-04-26T04:42:28.891 INFO:tasks.workunit.client.0.smithi090.stdout:1/773: getdents d0/d55/dda/d7/d4a/d75/d80 0 2024-04-26T04:42:28.893 INFO:tasks.workunit.client.0.smithi090.stdout:6/818: mknod d9/d77/db0/db5/dda/d6d/d92/c11b 0 2024-04-26T04:42:28.893 INFO:tasks.workunit.client.0.smithi090.stdout:6/819: readlink d9/d21/d40/d4c/l96 0 2024-04-26T04:42:28.893 INFO:tasks.workunit.client.0.smithi090.stdout:6/820: write d9/d1c/d31/d79/ffb [58669,78720] 0 2024-04-26T04:42:28.894 INFO:tasks.workunit.client.0.smithi090.stdout:3/684: symlink d1/d17/d36/d33/d68/dae/dca/d2c/le8 0 2024-04-26T04:42:28.894 INFO:tasks.workunit.client.0.smithi090.stdout:3/685: getdents d1/d17/d36/d33/d68/dae/dca/dcf 0 2024-04-26T04:42:28.895 INFO:tasks.workunit.client.0.smithi090.stdout:5/716: dwrite d4/d1a/fba [0,4194304] 0 2024-04-26T04:42:28.895 INFO:tasks.workunit.client.0.smithi090.stdout:0/820: dwrite d0/d4b/d5d/d5c/d8a/f17 [0,4194304] 0 2024-04-26T04:42:28.895 INFO:tasks.workunit.client.0.smithi090.stdout:2/792: dwrite db/d11/d14/d1b/d20/fe0 [0,4194304] 0 2024-04-26T04:42:28.895 INFO:tasks.workunit.client.0.smithi090.stdout:9/675: dread db/dd/d44/f2c [4194304,4194304] 0 2024-04-26T04:42:28.895 INFO:tasks.workunit.client.0.smithi090.stdout:1/774: creat d0/d55/dda/d7/d49/ff7 x:0 0 0 2024-04-26T04:42:28.897 INFO:tasks.workunit.client.0.smithi090.stdout:3/686: rename d1/d17/d36/d33/l41 to d1/d17/d61/de7/le9 0 2024-04-26T04:42:28.897 INFO:tasks.workunit.client.0.smithi090.stdout:3/687: write d1/d17/d36/f65 [4824919,79939] 0 2024-04-26T04:42:28.898 INFO:tasks.workunit.client.0.smithi090.stdout:8/892: dread dc/ff [0,4194304] 0 2024-04-26T04:42:28.900 INFO:tasks.workunit.client.0.smithi090.stdout:9/676: creat db/dd/d18/d57/dcc/ff2 x:0 0 0 2024-04-26T04:42:28.901 INFO:tasks.workunit.client.0.smithi090.stdout:1/775: link d0/d55/dda/d7/d4a/d75/d80/d58/fe2 d0/d55/dbd/ff8 0 2024-04-26T04:42:28.902 INFO:tasks.workunit.client.0.smithi090.stdout:7/811: dwrite d0/d109/d65/dbb/fc9 [0,4194304] 0 2024-04-26T04:42:28.902 INFO:tasks.workunit.client.0.smithi090.stdout:7/812: chown d0/d109/d4a/l9c 164970 1 2024-04-26T04:42:28.902 INFO:tasks.workunit.client.0.smithi090.stdout:7/813: creat d0/db/df/d4d/d64/db0/dd6/f122 x:0 0 0 2024-04-26T04:42:28.902 INFO:tasks.workunit.client.0.smithi090.stdout:7/814: fsync d0/db/df/d4d/d64/f102 0 2024-04-26T04:42:28.903 INFO:tasks.workunit.client.0.smithi090.stdout:0/821: rename d0/d4b/d5d/d5c/d8a/d45/d93/f10f to d0/d4b/d5d/d48/f11b 0 2024-04-26T04:42:28.905 INFO:tasks.workunit.client.0.smithi090.stdout:3/688: symlink d1/d17/d36/d33/d68/dae/dca/d1b/lea 0 2024-04-26T04:42:28.905 INFO:tasks.workunit.client.0.smithi090.stdout:3/689: write d1/d17/d61/d9d/fd5 [397343,44501] 0 2024-04-26T04:42:28.907 INFO:tasks.workunit.client.0.smithi090.stdout:8/893: creat dc/f136 x:0 0 0 2024-04-26T04:42:28.907 INFO:tasks.workunit.client.0.smithi090.stdout:8/894: getdents dc/dd2/d3b/d3d/d111/d11c 0 2024-04-26T04:42:28.908 INFO:tasks.workunit.client.0.smithi090.stdout:9/677: creat db/dd/d18/d38/d39/ff3 x:0 0 0 2024-04-26T04:42:28.909 INFO:tasks.workunit.client.0.smithi090.stdout:5/717: dread d4/d1a/d1f/f3a [0,4194304] 0 2024-04-26T04:42:28.910 INFO:tasks.workunit.client.0.smithi090.stdout:2/793: dread db/d59/d66/fdb [0,4194304] 0 2024-04-26T04:42:28.911 INFO:tasks.workunit.client.0.smithi090.stdout:1/776: write d0/d55/dda/fb2 [2582088,47983] 0 2024-04-26T04:42:28.913 INFO:tasks.workunit.client.0.smithi090.stdout:7/815: rename d0/d109/d65/db2/fb6 to d0/d109/d65/f123 0 2024-04-26T04:42:28.915 INFO:tasks.workunit.client.0.smithi090.stdout:8/895: mkdir dc/dd2/d137 0 2024-04-26T04:42:28.916 INFO:tasks.workunit.client.0.smithi090.stdout:9/678: mknod db/dd/d18/d57/cf4 0 2024-04-26T04:42:28.919 INFO:tasks.workunit.client.0.smithi090.stdout:5/718: unlink d4/d1a/d2a/d79/d90/d94/fd8 0 2024-04-26T04:42:28.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/794: dread db/d11/d14/d27/d28/f3d [0,4194304] 0 2024-04-26T04:42:28.923 INFO:tasks.workunit.client.0.smithi090.stdout:1/777: truncate d0/d55/dda/d7/d4b/d53/f6a 545103 0 2024-04-26T04:42:28.924 INFO:tasks.workunit.client.0.smithi090.stdout:7/816: mknod d0/d109/d65/d11c/c124 0 2024-04-26T04:42:28.926 INFO:tasks.workunit.client.0.smithi090.stdout:8/896: mknod dc/dd2/d4f/da6/db1/ddd/c138 0 2024-04-26T04:42:28.928 INFO:tasks.workunit.client.0.smithi090.stdout:9/679: creat db/dd/d44/d82/d85/db4/d99/db2/ff5 x:0 0 0 2024-04-26T04:42:28.930 INFO:tasks.workunit.client.0.smithi090.stdout:7/817: write d0/d1/d4/d3e/fcf [2236632,67151] 0 2024-04-26T04:42:28.932 INFO:tasks.workunit.client.0.smithi090.stdout:5/719: mkdir d4/d1a/de5 0 2024-04-26T04:42:28.933 INFO:tasks.workunit.client.0.smithi090.stdout:5/720: readlink d4/d1a/d2a/d31/ld2 0 2024-04-26T04:42:28.934 INFO:tasks.workunit.client.0.smithi090.stdout:1/778: stat d0/d55/dda/d7/c7a 0 2024-04-26T04:42:28.934 INFO:tasks.workunit.client.0.smithi090.stdout:1/779: write d0/d55/dda/d7/d4a/fe0 [66853,113816] 0 2024-04-26T04:42:28.935 INFO:tasks.workunit.client.0.smithi090.stdout:7/818: dread d0/db/df/d1c/d23/f26 [0,4194304] 0 2024-04-26T04:42:28.937 INFO:tasks.workunit.client.0.smithi090.stdout:5/721: rmdir d4/d1a/d1f/d53/d59/d5e/da0 39 2024-04-26T04:42:28.938 INFO:tasks.workunit.client.0.smithi090.stdout:7/819: write d0/db/d53/f17 [519514,40467] 0 2024-04-26T04:42:28.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/722: symlink d4/d1a/d1f/d53/le6 0 2024-04-26T04:42:28.940 INFO:tasks.workunit.client.0.smithi090.stdout:8/897: dread dc/dd2/d39/d55/d64/f7a [4194304,4194304] 0 2024-04-26T04:42:28.942 INFO:tasks.workunit.client.0.smithi090.stdout:9/680: write f9 [4702952,15300] 0 2024-04-26T04:42:28.942 INFO:tasks.workunit.client.0.smithi090.stdout:0/822: dwrite d0/de/f78 [0,4194304] 0 2024-04-26T04:42:28.943 INFO:tasks.workunit.client.0.smithi090.stdout:1/780: dread d0/f86 [0,4194304] 0 2024-04-26T04:42:28.943 INFO:tasks.workunit.client.0.smithi090.stdout:3/690: dwrite d1/d17/d36/d33/d68/d76/f8e [0,4194304] 0 2024-04-26T04:42:28.943 INFO:tasks.workunit.client.0.smithi090.stdout:8/898: rename dc/dd2/d39/d55/d64/f7a to dc/dd2/d4f/f139 0 2024-04-26T04:42:28.946 INFO:tasks.workunit.client.0.smithi090.stdout:9/681: mkdir db/dd/d44/d82/d85/df6 0 2024-04-26T04:42:28.946 INFO:tasks.workunit.client.0.smithi090.stdout:9/682: fsync db/dd/d44/d82/d85/db4/d6a/d70/f96 0 2024-04-26T04:42:28.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/820: dread d0/d1/d4/f10 [0,4194304] 0 2024-04-26T04:42:28.946 INFO:tasks.workunit.client.0.smithi090.stdout:7/821: creat d0/db/df/d4d/d7b/db3/f125 x:0 0 0 2024-04-26T04:42:28.948 INFO:tasks.workunit.client.0.smithi090.stdout:5/723: dread d4/d1a/d2a/d79/d90/d49/f9a [0,4194304] 0 2024-04-26T04:42:28.949 INFO:tasks.workunit.client.0.smithi090.stdout:2/795: dwrite db/d11/d14/d27/d28/dd1/dfe/d8b/ffb [0,4194304] 0 2024-04-26T04:42:28.951 INFO:tasks.workunit.client.0.smithi090.stdout:8/899: rename dc/dd2/d39/c61 to dc/dd2/d4f/da6/c13a 0 2024-04-26T04:42:28.952 INFO:tasks.workunit.client.0.smithi090.stdout:3/691: write d1/d17/d36/dab/fb7 [3778953,31836] 0 2024-04-26T04:42:28.953 INFO:tasks.workunit.client.0.smithi090.stdout:5/724: symlink d4/d1a/d2a/le7 0 2024-04-26T04:42:28.953 INFO:tasks.workunit.client.0.smithi090.stdout:5/725: write d4/d1a/d1f/f3a [1440671,121022] 0 2024-04-26T04:42:28.955 INFO:tasks.workunit.client.0.smithi090.stdout:2/796: creat db/d11/d14/d27/d28/dd1/dfe/d8b/f10b x:0 0 0 2024-04-26T04:42:28.957 INFO:tasks.workunit.client.0.smithi090.stdout:8/900: link dc/dd2/d4f/d58/d95/f10b dc/dd2/d3b/d3d/db8/f13b 0 2024-04-26T04:42:28.957 INFO:tasks.workunit.client.0.smithi090.stdout:8/901: readlink dc/dd2/d4f/da6/dd8/lc1 0 2024-04-26T04:42:28.958 INFO:tasks.workunit.client.0.smithi090.stdout:3/692: creat d1/d17/d36/d33/d68/dae/dca/dcf/feb x:0 0 0 2024-04-26T04:42:28.958 INFO:tasks.workunit.client.0.smithi090.stdout:3/693: chown d1/cd 2105241 1 2024-04-26T04:42:28.959 INFO:tasks.workunit.client.0.smithi090.stdout:5/726: rmdir d4/d1a/d45 39 2024-04-26T04:42:28.960 INFO:tasks.workunit.client.0.smithi090.stdout:7/822: dwrite d0/d109/f4c [0,4194304] 0 2024-04-26T04:42:28.960 INFO:tasks.workunit.client.0.smithi090.stdout:2/797: creat db/d11/d14/d27/d28/dd1/dfe/f10c x:0 0 0 2024-04-26T04:42:28.960 INFO:tasks.workunit.client.0.smithi090.stdout:5/727: dread d4/d1a/df/d36/f5c [0,4194304] 0 2024-04-26T04:42:28.961 INFO:tasks.workunit.client.0.smithi090.stdout:8/902: symlink dc/dd2/d4f/d58/d95/d106/d119/l13c 0 2024-04-26T04:42:28.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/694: dread d1/d9/f38 [0,4194304] 0 2024-04-26T04:42:28.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/695: creat d1/d17/d61/d9d/fec x:0 0 0 2024-04-26T04:42:28.966 INFO:tasks.workunit.client.0.smithi090.stdout:3/696: write d1/d17/f25 [2262142,71529] 0 2024-04-26T04:42:28.967 INFO:tasks.workunit.client.0.smithi090.stdout:1/781: read d0/d55/dda/d7/d4b/d53/f6a [259399,4159] 0 2024-04-26T04:42:28.968 INFO:tasks.workunit.client.0.smithi090.stdout:2/798: rename db/dc/led to db/d11/d14/d27/d28/dd1/ddf/l10d 0 2024-04-26T04:42:28.969 INFO:tasks.workunit.client.0.smithi090.stdout:5/728: creat d4/d1a/df/d36/d8e/fe8 x:0 0 0 2024-04-26T04:42:28.970 INFO:tasks.workunit.client.0.smithi090.stdout:8/903: truncate dc/ff 1807937 0 2024-04-26T04:42:28.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:28 smithi090 ceph-mon[26648]: pgmap v256: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 302 MiB/s rd, 380 MiB/s wr, 549 op/s 2024-04-26T04:42:28.971 INFO:tasks.workunit.client.0.smithi090.stdout:7/823: mkdir d0/db/d53/db7/d126 0 2024-04-26T04:42:28.971 INFO:tasks.workunit.client.0.smithi090.stdout:7/824: truncate d0/db/d15/dbc/ff6 383872 0 2024-04-26T04:42:28.971 INFO:tasks.workunit.client.0.smithi090.stdout:7/825: stat d0/db/df/d4d/fc1 0 2024-04-26T04:42:28.976 INFO:tasks.workunit.client.0.smithi090.stdout:3/697: symlink d1/d17/d36/d33/led 0 2024-04-26T04:42:28.976 INFO:tasks.workunit.client.0.smithi090.stdout:3/698: truncate d1/d17/d36/d33/d68/dae/dca/d2c/fb8 397493 0 2024-04-26T04:42:28.979 INFO:tasks.workunit.client.0.smithi090.stdout:5/729: creat d4/d1a/d2a/d79/d90/d56/d71/d91/fe9 x:0 0 0 2024-04-26T04:42:28.979 INFO:tasks.workunit.client.0.smithi090.stdout:5/730: write d4/d1a/d2a/d79/d90/d49/f7d [1277970,6466] 0 2024-04-26T04:42:28.982 INFO:tasks.workunit.client.0.smithi090.stdout:8/904: link dc/d1f/l2d dc/dd2/d4f/da6/db1/l13d 0 2024-04-26T04:42:28.982 INFO:tasks.workunit.client.0.smithi090.stdout:7/826: getdents d0/db/d53/db7/d8e 0 2024-04-26T04:42:28.982 INFO:tasks.workunit.client.0.smithi090.stdout:2/799: dread db/d11/d14/d1b/d20/fb8 [0,4194304] 0 2024-04-26T04:42:28.982 INFO:tasks.workunit.client.0.smithi090.stdout:7/827: fdatasync d0/db/d15/d29/f6e 0 2024-04-26T04:42:28.983 INFO:tasks.workunit.client.0.smithi090.stdout:5/731: getdents d4/d1a/d2a/d79/d90/d56 0 2024-04-26T04:42:28.983 INFO:tasks.workunit.client.0.smithi090.stdout:5/732: truncate d4/d1a/df/fa7 970999 0 2024-04-26T04:42:28.986 INFO:tasks.workunit.client.0.smithi090.stdout:8/905: rename dc/dd2/le4 to dc/dd2/d39/d125/d5d/d73/l13e 0 2024-04-26T04:42:28.987 INFO:tasks.workunit.client.0.smithi090.stdout:2/800: rmdir db/d11/d14/d1b/d20/d4c 39 2024-04-26T04:42:28.989 INFO:tasks.workunit.client.0.smithi090.stdout:5/733: mkdir d4/d1a/d2a/d79/d90/d56/d71/dea 0 2024-04-26T04:42:28.989 INFO:tasks.workunit.client.0.smithi090.stdout:5/734: write d4/d1a/df/f8d [2178541,87956] 0 2024-04-26T04:42:28.989 INFO:tasks.workunit.client.0.smithi090.stdout:1/782: dwrite d0/d55/dda/d43/f9f [0,4194304] 0 2024-04-26T04:42:28.992 INFO:tasks.workunit.client.0.smithi090.stdout:2/801: symlink db/d11/d14/d27/d28/l10e 0 2024-04-26T04:42:28.992 INFO:tasks.workunit.client.0.smithi090.stdout:2/802: readlink db/d11/d14/d25/d65/d95/l99 0 2024-04-26T04:42:28.992 INFO:tasks.workunit.client.0.smithi090.stdout:2/803: write db/d11/d14/d27/d28/f3d [4724563,23996] 0 2024-04-26T04:42:28.992 INFO:tasks.workunit.client.0.smithi090.stdout:2/804: stat db/d11/f17 0 2024-04-26T04:42:28.993 INFO:tasks.workunit.client.0.smithi090.stdout:3/699: dwrite d1/d17/d36/d33/d68/d76/fd2 [0,4194304] 0 2024-04-26T04:42:28.994 INFO:tasks.workunit.client.0.smithi090.stdout:5/735: mkdir d4/d1a/d2a/d79/d90/d56/d71/deb 0 2024-04-26T04:42:28.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/805: mknod db/d11/d14/d27/d28/c10f 0 2024-04-26T04:42:28.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/806: write db/d11/d14/d1b/f51 [3756829,61505] 0 2024-04-26T04:42:28.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/807: dread - db/d11/d14/d27/d28/fa3 zero size 2024-04-26T04:42:28.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/808: write db/dc/f1f [3096264,75441] 0 2024-04-26T04:42:28.997 INFO:tasks.workunit.client.0.smithi090.stdout:2/809: stat db/d11/d14/d25/d2d 0 2024-04-26T04:42:29.000 INFO:tasks.workunit.client.0.smithi090.stdout:3/700: mkdir d1/d17/d36/dee 0 2024-04-26T04:42:29.000 INFO:tasks.workunit.client.0.smithi090.stdout:3/701: chown d1/d17/d36/d33/d68/dae 348954507 1 2024-04-26T04:42:29.002 INFO:tasks.workunit.client.0.smithi090.stdout:7/828: dwrite d0/db/df/d4d/d64/f92 [0,4194304] 0 2024-04-26T04:42:29.002 INFO:tasks.workunit.client.0.smithi090.stdout:4/790: sync 2024-04-26T04:42:29.004 INFO:tasks.workunit.client.0.smithi090.stdout:2/810: dread db/d11/d7f/fa1 [0,4194304] 0 2024-04-26T04:42:29.007 INFO:tasks.workunit.client.0.smithi090.stdout:8/906: write dc/ff [1720173,19284] 0 2024-04-26T04:42:29.010 INFO:tasks.workunit.client.0.smithi090.stdout:5/736: getdents d4/d1a 0 2024-04-26T04:42:29.010 INFO:tasks.workunit.client.0.smithi090.stdout:5/737: dread - d4/d1a/d2a/d79/d90/d94/fdb zero size 2024-04-26T04:42:29.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:28 smithi104 ceph-mon[32369]: pgmap v256: 65 pgs: 65 active+clean; 3.1 GiB data, 9.0 GiB used, 527 GiB / 536 GiB avail; 302 MiB/s rd, 380 MiB/s wr, 549 op/s 2024-04-26T04:42:29.013 INFO:tasks.workunit.client.0.smithi090.stdout:7/829: creat d0/d1/d4/d3e/dd3/f127 x:0 0 0 2024-04-26T04:42:29.014 INFO:tasks.workunit.client.0.smithi090.stdout:4/791: creat d3/ffa x:0 0 0 2024-04-26T04:42:29.015 INFO:tasks.workunit.client.0.smithi090.stdout:3/702: rmdir d1/d59/d5e 39 2024-04-26T04:42:29.016 INFO:tasks.workunit.client.0.smithi090.stdout:2/811: mknod db/d11/d14/d1b/c110 0 2024-04-26T04:42:29.018 INFO:tasks.workunit.client.0.smithi090.stdout:2/812: read db/d11/d14/d25/f3e [299113,63105] 0 2024-04-26T04:42:29.018 INFO:tasks.workunit.client.0.smithi090.stdout:2/813: write db/dc/d6f/de3/fe6 [900856,80007] 0 2024-04-26T04:42:29.018 INFO:tasks.workunit.client.0.smithi090.stdout:7/830: dread d0/db/d53/db7/d4f/f10f [4194304,4194304] 0 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/907: link dc/d1f/f35 dc/dd2/d3b/d3d/db8/f13f 0 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/908: write dc/dd2/d3b/f85 [1176244,117355] 0 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/909: chown dc/dd2/d39/d125/d5d/l9b 2083974349 1 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/910: creat dc/dd2/d4f/da6/dd8/f140 x:0 0 0 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/911: chown dc/d34/c11b 6810586 1 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/912: chown dc/dd2/d3b/d3d/f9c 364426891 1 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:8/913: dread - dc/f136 zero size 2024-04-26T04:42:29.020 INFO:tasks.workunit.client.0.smithi090.stdout:5/738: mknod d4/d1a/d2a/d79/d90/d56/d71/d91/cec 0 2024-04-26T04:42:29.023 INFO:tasks.workunit.client.0.smithi090.stdout:3/703: symlink d1/lef 0 2024-04-26T04:42:29.024 INFO:tasks.workunit.client.0.smithi090.stdout:7/831: mknod d0/d105/c128 0 2024-04-26T04:42:29.028 INFO:tasks.workunit.client.0.smithi090.stdout:8/914: mknod dc/dd2/d137/c141 0 2024-04-26T04:42:29.028 INFO:tasks.workunit.client.0.smithi090.stdout:8/915: readlink dc/dd2/d4f/d58/d95/l102 0 2024-04-26T04:42:29.029 INFO:tasks.workunit.client.0.smithi090.stdout:5/739: symlink d4/d1a/d3f/led 0 2024-04-26T04:42:29.029 INFO:tasks.workunit.client.0.smithi090.stdout:2/814: read db/d11/d14/d27/d2f/d42/fdd [1039002,103700] 0 2024-04-26T04:42:29.029 INFO:tasks.workunit.client.0.smithi090.stdout:2/815: chown db/d11/d14/d25/l31 1557 1 2024-04-26T04:42:29.030 INFO:tasks.workunit.client.0.smithi090.stdout:4/792: dwrite d3/db/d12/d20/d2d/d58/d6a/d74/fb6 [0,4194304] 0 2024-04-26T04:42:29.031 INFO:tasks.workunit.client.0.smithi090.stdout:3/704: unlink d1/d17/d36/d33/d42/l4a 0 2024-04-26T04:42:29.033 INFO:tasks.workunit.client.0.smithi090.stdout:8/916: link dc/dd2/d4f/f9f dc/dd2/d4f/d58/d95/d106/f142 0 2024-04-26T04:42:29.036 INFO:tasks.workunit.client.0.smithi090.stdout:4/793: mkdir d3/db/d12/d3b/dfb 0 2024-04-26T04:42:29.036 INFO:tasks.workunit.client.0.smithi090.stdout:4/794: stat d3/db/d12/d20/d24/d43/ldb 0 2024-04-26T04:42:29.037 INFO:tasks.workunit.client.0.smithi090.stdout:3/705: creat d1/d59/ff0 x:0 0 0 2024-04-26T04:42:29.037 INFO:tasks.workunit.client.0.smithi090.stdout:3/706: stat d1/d59 0 2024-04-26T04:42:29.038 INFO:tasks.workunit.client.0.smithi090.stdout:3/707: fdatasync d1/d17/d61/d9d/fec 0 2024-04-26T04:42:29.039 INFO:tasks.workunit.client.0.smithi090.stdout:2/816: write db/d11/d14/fca [2305219,71174] 0 2024-04-26T04:42:29.040 INFO:tasks.workunit.client.0.smithi090.stdout:8/917: rename dc/dd2/d4f/f96 to dc/dd2/d39/d125/d5d/d73/f143 0 2024-04-26T04:42:29.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/832: dwrite d0/db/df/d1c/d23/f26 [0,4194304] 0 2024-04-26T04:42:29.041 INFO:tasks.workunit.client.0.smithi090.stdout:5/740: write d4/d1a/df/d4a/f7a [2253230,22119] 0 2024-04-26T04:42:29.041 INFO:tasks.workunit.client.0.smithi090.stdout:7/833: getdents d0/db/d15/d29 0 2024-04-26T04:42:29.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/708: mknod d1/d5c/cf1 0 2024-04-26T04:42:29.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/709: write d1/d59/f64 [4933073,62139] 0 2024-04-26T04:42:29.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/710: creat d1/d5c/ff2 x:0 0 0 2024-04-26T04:42:29.044 INFO:tasks.workunit.client.0.smithi090.stdout:2/817: creat db/d59/d66/d69/f111 x:0 0 0 2024-04-26T04:42:29.045 INFO:tasks.workunit.client.0.smithi090.stdout:8/918: mknod dc/dd2/dd9/c144 0 2024-04-26T04:42:29.050 INFO:tasks.workunit.client.0.smithi090.stdout:6/821: sync 2024-04-26T04:42:29.051 INFO:tasks.workunit.client.0.smithi090.stdout:3/711: symlink d1/d17/d61/d9d/lf3 0 2024-04-26T04:42:29.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/712: dread d1/d17/d36/d33/d68/dae/dca/d2c/f5f [0,4194304] 0 2024-04-26T04:42:29.056 INFO:tasks.workunit.client.0.smithi090.stdout:6/822: rmdir d9/d21/d3b/d86 39 2024-04-26T04:42:29.059 INFO:tasks.workunit.client.0.smithi090.stdout:5/741: dwrite d4/dab/fbd [0,4194304] 0 2024-04-26T04:42:29.059 INFO:tasks.workunit.client.0.smithi090.stdout:6/823: creat d9/d1c/d31/d69/f11c x:0 0 0 2024-04-26T04:42:29.059 INFO:tasks.workunit.client.0.smithi090.stdout:7/834: dread d0/d109/d65/f123 [0,4194304] 0 2024-04-26T04:42:29.062 INFO:tasks.workunit.client.0.smithi090.stdout:5/742: creat d4/d1a/d1f/dc5/dca/fee x:0 0 0 2024-04-26T04:42:29.063 INFO:tasks.workunit.client.0.smithi090.stdout:5/743: read d4/d1a/d1f/f2b [3627399,19477] 0 2024-04-26T04:42:29.065 INFO:tasks.workunit.client.0.smithi090.stdout:6/824: unlink d9/d21/c42 0 2024-04-26T04:42:29.065 INFO:tasks.workunit.client.0.smithi090.stdout:6/825: write d9/d21/d40/f4b [2233776,68248] 0 2024-04-26T04:42:29.066 INFO:tasks.workunit.client.0.smithi090.stdout:7/835: creat d0/d109/d65/dbb/dc0/f129 x:0 0 0 2024-04-26T04:42:29.066 INFO:tasks.workunit.client.0.smithi090.stdout:7/836: fsync d0/d109/d4a/fbe 0 2024-04-26T04:42:29.066 INFO:tasks.workunit.client.0.smithi090.stdout:7/837: creat d0/db/df/d1c/d23/d8d/f12a x:0 0 0 2024-04-26T04:42:29.066 INFO:tasks.workunit.client.0.smithi090.stdout:6/826: mknod d9/d1c/da2/c11d 0 2024-04-26T04:42:29.075 INFO:tasks.workunit.client.0.smithi090.stdout:8/919: dwrite dc/dd2/d39/d55/d64/fb0 [0,4194304] 0 2024-04-26T04:42:29.075 INFO:tasks.workunit.client.0.smithi090.stdout:8/920: write dc/dd2/d3b/f12a [443894,123386] 0 2024-04-26T04:42:29.075 INFO:tasks.workunit.client.0.smithi090.stdout:8/921: chown dc/dd2/d39/d99/laf 233808 1 2024-04-26T04:42:29.075 INFO:tasks.workunit.client.0.smithi090.stdout:8/922: chown dc/dd2/dd9 948529 1 2024-04-26T04:42:29.086 INFO:tasks.workunit.client.0.smithi090.stdout:3/713: dwrite d1/d17/d36/f6a [0,4194304] 0 2024-04-26T04:42:29.089 INFO:tasks.workunit.client.0.smithi090.stdout:3/714: truncate d1/d17/f2f 1274945 0 2024-04-26T04:42:29.093 INFO:tasks.workunit.client.0.smithi090.stdout:5/744: dwrite d4/d1a/d3f/f29 [0,4194304] 0 2024-04-26T04:42:29.093 INFO:tasks.workunit.client.0.smithi090.stdout:5/745: truncate d4/fdd 337177 0 2024-04-26T04:42:29.096 INFO:tasks.workunit.client.0.smithi090.stdout:3/715: dread d1/d17/d36/f24 [0,4194304] 0 2024-04-26T04:42:29.101 INFO:tasks.workunit.client.0.smithi090.stdout:7/838: dwrite d0/d1/d4/d3e/dd3/f127 [0,4194304] 0 2024-04-26T04:42:29.101 INFO:tasks.workunit.client.0.smithi090.stdout:7/839: creat d0/db/df/d4d/d64/d66/f12b x:0 0 0 2024-04-26T04:42:29.101 INFO:tasks.workunit.client.0.smithi090.stdout:7/840: truncate d0/db/df/d4d/d64/db0/dd6/f112 641095 0 2024-04-26T04:42:29.101 INFO:tasks.workunit.client.0.smithi090.stdout:8/923: dwrite dc/dd2/d39/d125/ff5 [0,4194304] 0 2024-04-26T04:42:29.103 INFO:tasks.workunit.client.0.smithi090.stdout:6/827: dwrite d9/d77/d9a/ff8 [0,4194304] 0 2024-04-26T04:42:29.106 INFO:tasks.workunit.client.0.smithi090.stdout:7/841: creat d0/db/df/d1c/d23/d8d/d111/f12c x:0 0 0 2024-04-26T04:42:29.107 INFO:tasks.workunit.client.0.smithi090.stdout:7/842: dread - d0/d105/f121 zero size 2024-04-26T04:42:29.107 INFO:tasks.workunit.client.0.smithi090.stdout:7/843: chown d0/db/d53/db7/f9b 450380 1 2024-04-26T04:42:29.107 INFO:tasks.workunit.client.0.smithi090.stdout:7/844: readlink d0/db/d53/db7/l72 0 2024-04-26T04:42:29.112 INFO:tasks.workunit.client.0.smithi090.stdout:8/924: dread dc/f74 [0,4194304] 0 2024-04-26T04:42:29.113 INFO:tasks.workunit.client.0.smithi090.stdout:7/845: link d0/d105/c128 d0/db/d53/c12d 0 2024-04-26T04:42:29.115 INFO:tasks.workunit.client.0.smithi090.stdout:5/746: dwrite d4/d1a/d45/db8/f9e [0,4194304] 0 2024-04-26T04:42:29.116 INFO:tasks.workunit.client.0.smithi090.stdout:7/846: creat d0/d109/d65/d11c/f12e x:0 0 0 2024-04-26T04:42:29.116 INFO:tasks.workunit.client.0.smithi090.stdout:7/847: chown d0/db/df/d4d/c11a 9871663 1 2024-04-26T04:42:29.116 INFO:tasks.workunit.client.0.smithi090.stdout:7/848: creat d0/db/df/d1c/d23/d8d/d111/f12f x:0 0 0 2024-04-26T04:42:29.116 INFO:tasks.workunit.client.0.smithi090.stdout:7/849: fdatasync d0/db/f19 0 2024-04-26T04:42:29.116 INFO:tasks.workunit.client.0.smithi090.stdout:7/850: fdatasync d0/d109/d65/db2/fcb 0 2024-04-26T04:42:29.119 INFO:tasks.workunit.client.0.smithi090.stdout:7/851: symlink d0/db/d15/l130 0 2024-04-26T04:42:29.133 INFO:tasks.workunit.client.0.smithi090.stdout:5/747: dread d4/d1a/d45/db8/f9e [0,4194304] 0 2024-04-26T04:42:29.134 INFO:tasks.workunit.client.0.smithi090.stdout:5/748: write d4/d1a/d45/f97 [88739,109975] 0 2024-04-26T04:42:29.134 INFO:tasks.workunit.client.0.smithi090.stdout:5/749: fsync d4/d1a/df/d36/f5c 0 2024-04-26T04:42:29.134 INFO:tasks.workunit.client.0.smithi090.stdout:5/750: dread d4/d1a/df/d4a/fe4 [0,4194304] 0 2024-04-26T04:42:29.138 INFO:tasks.workunit.client.0.smithi090.stdout:3/716: dwrite d1/d17/f25 [0,4194304] 0 2024-04-26T04:42:29.138 INFO:tasks.workunit.client.0.smithi090.stdout:3/717: creat d1/d17/d61/d9d/da7/ff4 x:0 0 0 2024-04-26T04:42:29.138 INFO:tasks.workunit.client.0.smithi090.stdout:3/718: chown d1/d59 22862 1 2024-04-26T04:42:29.138 INFO:tasks.workunit.client.0.smithi090.stdout:3/719: fsync d1/d17/d36/f6a 0 2024-04-26T04:42:29.140 INFO:tasks.workunit.client.0.smithi090.stdout:8/925: dwrite dc/dd2/d39/d125/d5d/d73/f143 [0,4194304] 0 2024-04-26T04:42:29.140 INFO:tasks.workunit.client.0.smithi090.stdout:6/828: dwrite d9/d1c/dc5/fd0 [0,4194304] 0 2024-04-26T04:42:29.141 INFO:tasks.workunit.client.0.smithi090.stdout:3/720: link d1/d17/d36/d33/d68/dae/dca/c15 d1/d59/d5e/cf5 0 2024-04-26T04:42:29.141 INFO:tasks.workunit.client.0.smithi090.stdout:3/721: readlink d1/d17/l9c 0 2024-04-26T04:42:29.147 INFO:tasks.workunit.client.0.smithi090.stdout:8/926: mknod dc/dd2/d39/d125/d5d/d73/dce/c145 0 2024-04-26T04:42:29.147 INFO:tasks.workunit.client.0.smithi090.stdout:8/927: chown dc/dd2/d3b/d3d/d111/f132 29752 1 2024-04-26T04:42:29.148 INFO:tasks.workunit.client.0.smithi090.stdout:6/829: mkdir d9/d3c/db4/d11e 0 2024-04-26T04:42:29.151 INFO:tasks.workunit.client.0.smithi090.stdout:8/928: symlink dc/l146 0 2024-04-26T04:42:29.152 INFO:tasks.workunit.client.0.smithi090.stdout:8/929: stat dc/dd2/d3b/l4b 0 2024-04-26T04:42:29.152 INFO:tasks.workunit.client.0.smithi090.stdout:8/930: dread - dc/dd2/d4f/db7/f135 zero size 2024-04-26T04:42:29.156 INFO:tasks.workunit.client.0.smithi090.stdout:7/852: dwrite d0/f8 [8388608,4194304] 0 2024-04-26T04:42:29.156 INFO:tasks.workunit.client.0.smithi090.stdout:3/722: write d1/d17/d36/d2a/d3a/f53 [636150,90633] 0 2024-04-26T04:42:29.156 INFO:tasks.workunit.client.0.smithi090.stdout:7/853: chown d0/db/df/d4d 0 1 2024-04-26T04:42:29.157 INFO:tasks.workunit.client.0.smithi090.stdout:7/854: fsync d0/d109/d4a/fbe 0 2024-04-26T04:42:29.157 INFO:tasks.workunit.client.0.smithi090.stdout:7/855: write d0/d109/d65/dbb/dc0/fdf [323665,94160] 0 2024-04-26T04:42:29.158 INFO:tasks.workunit.client.0.smithi090.stdout:3/723: truncate d1/d59/d5e/f62 2750877 0 2024-04-26T04:42:29.161 INFO:tasks.workunit.client.0.smithi090.stdout:3/724: mkdir d1/d17/d36/d33/d68/dae/dca/d2c/d97/de4/df6 0 2024-04-26T04:42:29.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/856: dread d0/db/df/d4d/f5a [0,4194304] 0 2024-04-26T04:42:29.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/857: write d0/d105/f121 [877825,76449] 0 2024-04-26T04:42:29.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/858: readlink d0/db/df/d4d/d64/d66/lff 0 2024-04-26T04:42:29.164 INFO:tasks.workunit.client.0.smithi090.stdout:7/859: write d0/db/fd9 [7236997,106450] 0 2024-04-26T04:42:29.164 INFO:tasks.workunit.client.0.smithi090.stdout:5/751: dwrite d4/d1a/d1f/d53/f76 [8388608,4194304] 0 2024-04-26T04:42:29.164 INFO:tasks.workunit.client.0.smithi090.stdout:6/830: dread d9/d77/db0/db5/fbe [0,4194304] 0 2024-04-26T04:42:29.165 INFO:tasks.workunit.client.0.smithi090.stdout:6/831: chown d9/f5c 15775725 1 2024-04-26T04:42:29.166 INFO:tasks.workunit.client.0.smithi090.stdout:3/725: creat d1/d17/d36/dee/ff7 x:0 0 0 2024-04-26T04:42:29.166 INFO:tasks.workunit.client.0.smithi090.stdout:7/860: rmdir d0 39 2024-04-26T04:42:29.167 INFO:tasks.workunit.client.0.smithi090.stdout:5/752: mknod d4/d1a/d2a/d79/d90/d56/d80/db2/cef 0 2024-04-26T04:42:29.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/726: mkdir d1/d17/d36/d2a/d3a/d5d/d82/df8 0 2024-04-26T04:42:29.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/727: creat d1/d17/d36/d33/d68/dae/dca/dcf/ff9 x:0 0 0 2024-04-26T04:42:29.170 INFO:tasks.workunit.client.0.smithi090.stdout:3/728: chown d1/d17/d36/d33/d68/dae/dca/d2c 3028720 1 2024-04-26T04:42:29.171 INFO:tasks.workunit.client.0.smithi090.stdout:7/861: getdents d0/db/d53/db7/d4f 0 2024-04-26T04:42:29.172 INFO:tasks.workunit.client.0.smithi090.stdout:8/931: dwrite dc/dd2/d3b/f85 [0,4194304] 0 2024-04-26T04:42:29.172 INFO:tasks.workunit.client.0.smithi090.stdout:9/683: sync 2024-04-26T04:42:29.173 INFO:tasks.workunit.client.0.smithi090.stdout:0/823: sync 2024-04-26T04:42:29.173 INFO:tasks.workunit.client.0.smithi090.stdout:9/684: stat db/dd/d18/d38/d39/d6b/de2/c49 0 2024-04-26T04:42:29.176 INFO:tasks.workunit.client.1.smithi104.stdout:9/997: sync 2024-04-26T04:42:29.177 INFO:tasks.workunit.client.1.smithi104.stdout:9/998: mkdir d11/d1d/d34/d69/d104/d150 0 2024-04-26T04:42:29.178 INFO:tasks.workunit.client.1.smithi104.stdout:9/999: fsync d11/d15/d32/f93 0 2024-04-26T04:42:29.178 INFO:tasks.workunit.client.0.smithi090.stdout:9/685: link db/dd/d18/f2a db/dd/d44/d82/d85/df6/ff7 0 2024-04-26T04:42:29.180 INFO:tasks.workunit.client.0.smithi090.stdout:0/824: creat d0/d4b/d5d/d5c/d8a/d64/f11c x:0 0 0 2024-04-26T04:42:29.180 INFO:tasks.workunit.client.0.smithi090.stdout:0/825: read - d0/fe9 zero size 2024-04-26T04:42:29.180 INFO:tasks.workunit.client.0.smithi090.stdout:0/826: stat d0/d4b/d5d/d5c/d8a/f17 0 2024-04-26T04:42:29.180 INFO:tasks.workunit.client.0.smithi090.stdout:0/827: dread - d0/d4b/ff8 zero size 2024-04-26T04:42:29.180 INFO:tasks.workunit.client.0.smithi090.stdout:0/828: dread - d0/d4b/d5d/d5c/d8a/ff1 zero size 2024-04-26T04:42:29.180 INFO:tasks.workunit.client.0.smithi090.stdout:0/829: fdatasync d0/df/f77 0 2024-04-26T04:42:29.183 INFO:tasks.workunit.client.0.smithi090.stdout:8/932: write dc/dd2/d39/d55/d64/f44 [2785581,70006] 0 2024-04-26T04:42:29.183 INFO:tasks.workunit.client.0.smithi090.stdout:8/933: truncate dc/dd2/d3b/f12a 882495 0 2024-04-26T04:42:29.183 INFO:tasks.workunit.client.1.smithi104.stderr:+ rm -rf -- ./tmp.3rns7cWTtV 2024-04-26T04:42:29.184 INFO:tasks.workunit.client.0.smithi090.stdout:0/830: rename d0/de/d103/lae to d0/d4b/d5d/d48/dc1/l11d 0 2024-04-26T04:42:29.186 INFO:tasks.workunit.client.0.smithi090.stdout:6/832: dwrite d9/d77/db0/db5/fbe [0,4194304] 0 2024-04-26T04:42:29.186 INFO:tasks.workunit.client.0.smithi090.stdout:6/833: fdatasync d9/d21/f105 0 2024-04-26T04:42:29.186 INFO:tasks.workunit.client.0.smithi090.stdout:8/934: link dc/dd2/d39/d125/ff5 dc/dd2/d4f/d58/f147 0 2024-04-26T04:42:29.189 INFO:tasks.workunit.client.0.smithi090.stdout:0/831: unlink d0/d4b/d5d/d5c/d8a/d45/d53/dab/fc8 0 2024-04-26T04:42:29.191 INFO:tasks.workunit.client.0.smithi090.stdout:6/834: creat d9/d21/d40/d4c/f11f x:0 0 0 2024-04-26T04:42:29.192 INFO:tasks.workunit.client.0.smithi090.stdout:8/935: getdents dc/dd2/d3b/d3d/db8 0 2024-04-26T04:42:29.194 INFO:tasks.workunit.client.0.smithi090.stdout:0/832: creat d0/d4b/f11e x:0 0 0 2024-04-26T04:42:29.194 INFO:tasks.workunit.client.0.smithi090.stdout:0/833: fdatasync d0/d4b/d5d/d5c/d8a/f17 0 2024-04-26T04:42:29.195 INFO:tasks.workunit.client.0.smithi090.stdout:6/835: rmdir d9/d3c/db4 39 2024-04-26T04:42:29.196 INFO:tasks.workunit.client.0.smithi090.stdout:8/936: symlink dc/dd2/d4f/da6/db1/ddd/l148 0 2024-04-26T04:42:29.198 INFO:tasks.workunit.client.0.smithi090.stdout:0/834: link d0/fdc d0/d106/f11f 0 2024-04-26T04:42:29.199 INFO:tasks.workunit.client.0.smithi090.stdout:6/836: creat d9/d77/db0/db5/dda/d6d/d92/f120 x:0 0 0 2024-04-26T04:42:29.199 INFO:tasks.workunit.client.0.smithi090.stdout:6/837: creat d9/d77/db0/f121 x:0 0 0 2024-04-26T04:42:29.200 INFO:tasks.workunit.client.0.smithi090.stdout:5/753: truncate d4/f1e 2060100 0 2024-04-26T04:42:29.200 INFO:tasks.workunit.client.0.smithi090.stdout:8/937: mknod dc/dd2/d3b/d3d/d97/c149 0 2024-04-26T04:42:29.200 INFO:tasks.workunit.client.0.smithi090.stdout:8/938: creat dc/dd2/d39/d55/d64/f14a x:0 0 0 2024-04-26T04:42:29.201 INFO:tasks.workunit.client.0.smithi090.stdout:5/754: dread d4/d1a/d2a/f83 [0,4194304] 0 2024-04-26T04:42:29.201 INFO:tasks.workunit.client.0.smithi090.stdout:3/729: dwrite d1/d9/fb [0,4194304] 0 2024-04-26T04:42:29.202 INFO:tasks.workunit.client.0.smithi090.stdout:0/835: mkdir d0/d4b/d5d/d5c/d117/d120 0 2024-04-26T04:42:29.203 INFO:tasks.workunit.client.0.smithi090.stdout:6/838: mkdir d9/d1c/d31/d122 0 2024-04-26T04:42:29.206 INFO:tasks.workunit.client.0.smithi090.stdout:8/939: rename dc/dd2/d3b/d3d/d97/fac to dc/dd2/d3b/d3d/d111/d11c/f14b 0 2024-04-26T04:42:29.206 INFO:tasks.workunit.client.0.smithi090.stdout:3/730: symlink d1/d17/d36/d33/daa/lfa 0 2024-04-26T04:42:29.207 INFO:tasks.workunit.client.0.smithi090.stdout:6/839: symlink d9/d3c/db4/d11e/l123 0 2024-04-26T04:42:29.207 INFO:tasks.workunit.client.0.smithi090.stdout:6/840: write d9/d77/f7d [223026,7516] 0 2024-04-26T04:42:29.207 INFO:tasks.workunit.client.0.smithi090.stdout:6/841: write d9/d1c/dc5/dd9/fed [825220,42667] 0 2024-04-26T04:42:29.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/731: creat d1/d17/d36/d33/d42/ffb x:0 0 0 2024-04-26T04:42:29.209 INFO:tasks.workunit.client.0.smithi090.stdout:3/732: fdatasync d1/d17/f2f 0 2024-04-26T04:42:29.210 INFO:tasks.workunit.client.0.smithi090.stdout:9/686: dwrite db/dd/d18/d38/d39/d3f/f76 [4194304,4194304] 0 2024-04-26T04:42:29.210 INFO:tasks.workunit.client.0.smithi090.stdout:9/687: readlink db/dd/d44/d82/d85/db4/dde/lf0 0 2024-04-26T04:42:29.210 INFO:tasks.workunit.client.0.smithi090.stdout:9/688: write db/dd/d44/d82/d85/db4/d6a/fa0 [580175,109717] 0 2024-04-26T04:42:29.210 INFO:tasks.workunit.client.0.smithi090.stdout:9/689: fsync db/dd/d18/d95/fb9 0 2024-04-26T04:42:29.210 INFO:tasks.workunit.client.0.smithi090.stdout:7/862: dwrite d0/db/d53/f56 [0,4194304] 0 2024-04-26T04:42:29.210 INFO:tasks.workunit.client.0.smithi090.stdout:6/842: symlink d9/d1c/dc5/dd9/l124 0 2024-04-26T04:42:29.210 INFO:tasks.workunit.client.0.smithi090.stdout:6/843: creat d9/d77/db0/db5/dca/dd8/f125 x:0 0 0 2024-04-26T04:42:29.211 INFO:tasks.workunit.client.0.smithi090.stdout:3/733: mknod d1/d17/d36/d33/d68/dae/dca/cfc 0 2024-04-26T04:42:29.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/863: creat d0/db/d15/d29/f131 x:0 0 0 2024-04-26T04:42:29.216 INFO:tasks.workunit.client.0.smithi090.stdout:8/940: write dc/dd2/d4f/f139 [2863512,78933] 0 2024-04-26T04:42:29.217 INFO:tasks.workunit.client.0.smithi090.stdout:8/941: chown dc/dd2/d3b/d3d/f54 31646 1 2024-04-26T04:42:29.217 INFO:tasks.workunit.client.0.smithi090.stdout:6/844: link d9/d77/db0/db5/dca/cd7 d9/d1c/d31/d69/c126 0 2024-04-26T04:42:29.218 INFO:tasks.workunit.client.0.smithi090.stdout:3/734: link d1/l4 d1/d59/lfd 0 2024-04-26T04:42:29.220 INFO:tasks.workunit.client.0.smithi090.stdout:9/690: link db/dd/c74 db/dd/d44/d82/d85/db4/d6a/db6/cf8 0 2024-04-26T04:42:29.221 INFO:tasks.workunit.client.0.smithi090.stdout:8/942: rmdir dc/dd2/d4f 39 2024-04-26T04:42:29.224 INFO:tasks.workunit.client.0.smithi090.stdout:6/845: mknod d9/d77/db0/db5/dca/c127 0 2024-04-26T04:42:29.228 INFO:tasks.workunit.client.0.smithi090.stdout:3/735: unlink d1/d9/f20 0 2024-04-26T04:42:29.228 INFO:tasks.workunit.client.0.smithi090.stdout:9/691: creat db/dd/d44/d82/ff9 x:0 0 0 2024-04-26T04:42:29.228 INFO:tasks.workunit.client.0.smithi090.stdout:5/755: dwrite d4/d1a/d1f/d53/d59/d5e/fa5 [0,4194304] 0 2024-04-26T04:42:29.229 INFO:tasks.workunit.client.0.smithi090.stdout:7/864: read d0/d1/d4/fc5 [817784,78570] 0 2024-04-26T04:42:29.229 INFO:tasks.workunit.client.0.smithi090.stdout:0/836: dwrite d0/de/f36 [0,4194304] 0 2024-04-26T04:42:29.229 INFO:tasks.workunit.client.0.smithi090.stdout:6/846: rename d9/d3c/db4/d11e/l123 to d9/d21/d3b/l128 0 2024-04-26T04:42:29.231 INFO:tasks.workunit.client.0.smithi090.stdout:3/736: mknod d1/d17/d36/d33/d68/dae/dca/d2c/d78/cfe 0 2024-04-26T04:42:29.231 INFO:tasks.workunit.client.0.smithi090.stdout:1/783: sync 2024-04-26T04:42:29.232 INFO:tasks.workunit.client.0.smithi090.stdout:1/784: fsync d0/d55/dda/d7/d17/d22/d31/fc1 0 2024-04-26T04:42:29.232 INFO:tasks.workunit.client.0.smithi090.stdout:9/692: dread db/dd/d44/d82/d85/db4/d6a/fa0 [0,4194304] 0 2024-04-26T04:42:29.239 INFO:tasks.workunit.client.0.smithi090.stdout:6/847: truncate d9/d1c/f80 510927 0 2024-04-26T04:42:29.240 INFO:tasks.workunit.client.0.smithi090.stdout:9/693: creat db/dd/d44/ffa x:0 0 0 2024-04-26T04:42:29.240 INFO:tasks.workunit.client.0.smithi090.stdout:9/694: truncate db/dd/d44/d82/d85/db4/d6a/fb5 332786 0 2024-04-26T04:42:29.242 INFO:tasks.workunit.client.0.smithi090.stdout:3/737: dread d1/d17/f25 [0,4194304] 0 2024-04-26T04:42:29.242 INFO:tasks.workunit.client.0.smithi090.stdout:6/848: truncate d9/d1c/f2b 629284 0 2024-04-26T04:42:29.244 INFO:tasks.workunit.client.0.smithi090.stdout:5/756: rmdir d4/d1a/d2a/d79/d90/d56/d71/d91/d86 0 2024-04-26T04:42:29.244 INFO:tasks.workunit.client.0.smithi090.stdout:5/757: fdatasync d4/d1a/df/d72/dc0/fdf 0 2024-04-26T04:42:29.245 INFO:tasks.workunit.client.0.smithi090.stdout:3/738: mkdir d1/d17/d36/d33/daa/dff 0 2024-04-26T04:42:29.246 INFO:tasks.workunit.client.0.smithi090.stdout:5/758: rmdir d4/d1a/d2a/d79/d90/d49 39 2024-04-26T04:42:29.253 INFO:tasks.workunit.client.0.smithi090.stdout:3/739: rmdir d1/d5c 39 2024-04-26T04:42:29.253 INFO:tasks.workunit.client.0.smithi090.stdout:3/740: chown d1/d17/d36/d33/d68/dae/dca/f91 121298735 1 2024-04-26T04:42:29.255 INFO:tasks.workunit.client.0.smithi090.stdout:3/741: mknod d1/d17/d36/d2a/d3a/d5d/d82/c100 0 2024-04-26T04:42:29.255 INFO:tasks.workunit.client.0.smithi090.stdout:9/695: dread db/dd/d18/d57/f4d [0,4194304] 0 2024-04-26T04:42:29.256 INFO:tasks.workunit.client.0.smithi090.stdout:9/696: mkdir db/dd/d44/dfb 0 2024-04-26T04:42:29.257 INFO:tasks.workunit.client.0.smithi090.stdout:6/849: dread d9/d21/d3b/f51 [0,4194304] 0 2024-04-26T04:42:29.257 INFO:tasks.workunit.client.0.smithi090.stdout:9/697: mknod db/dd/d44/d82/d85/db4/dde/cfc 0 2024-04-26T04:42:29.258 INFO:tasks.workunit.client.0.smithi090.stdout:5/759: read d4/d1a/df/d4a/fe4 [826839,73343] 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/760: symlink d4/d1a/d2a/d79/d90/d56/d71/lf0 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/761: write d4/d1a/df/d4a/fe4 [4067812,92341] 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/762: creat d4/d1a/d1f/dc5/dca/ff1 x:0 0 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/763: write d4/d1a/df/fa7 [1995619,63682] 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:5/764: write d4/d1a/df/d4a/fde [117754,90494] 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:7/865: dwrite d0/db/d53/f56 [0,4194304] 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:7/866: truncate d0/db/df/d1c/d23/d8d/d111/f12f 170111 0 2024-04-26T04:42:29.262 INFO:tasks.workunit.client.0.smithi090.stdout:1/785: dwrite d0/d55/dda/d7/d17/d22/d31/f6d [0,4194304] 0 2024-04-26T04:42:29.263 INFO:tasks.workunit.client.0.smithi090.stdout:9/698: getdents db/dd/d18/d57 0 2024-04-26T04:42:29.263 INFO:tasks.workunit.client.0.smithi090.stdout:0/837: dwrite d0/d4b/d5d/d5c/d8a/d45/f66 [0,4194304] 0 2024-04-26T04:42:29.263 INFO:tasks.workunit.client.0.smithi090.stdout:0/838: chown d0/d4b/d5d/d5c/d8a/d64/ca4 2006632 1 2024-04-26T04:42:29.269 INFO:tasks.workunit.client.0.smithi090.stdout:5/765: mkdir d4/d1a/d1f/dc5/dca/df2 0 2024-04-26T04:42:29.270 INFO:tasks.workunit.client.0.smithi090.stdout:5/766: write d4/d1a/f19 [697589,115016] 0 2024-04-26T04:42:29.270 INFO:tasks.workunit.client.0.smithi090.stdout:7/867: mknod d0/d1/d4/d3e/dd3/de2/c132 0 2024-04-26T04:42:29.276 INFO:tasks.workunit.client.0.smithi090.stdout:6/850: dwrite d9/d21/d40/d4c/d6f/d9f/dae/fdd [4194304,4194304] 0 2024-04-26T04:42:29.277 INFO:tasks.workunit.client.0.smithi090.stdout:1/786: creat d0/d55/dda/d7/d17/d22/ff9 x:0 0 0 2024-04-26T04:42:29.279 INFO:tasks.workunit.client.0.smithi090.stdout:9/699: mknod db/dd/d44/d82/d85/db4/dde/cfd 0 2024-04-26T04:42:29.286 INFO:tasks.workunit.client.0.smithi090.stdout:0/839: creat d0/d4b/d5d/d5c/d8a/d45/d107/f121 x:0 0 0 2024-04-26T04:42:29.289 INFO:tasks.workunit.client.0.smithi090.stdout:5/767: creat d4/d1a/d1f/d53/ff3 x:0 0 0 2024-04-26T04:42:29.290 INFO:tasks.workunit.client.0.smithi090.stdout:5/768: creat d4/d1a/df/d72/dc0/ff4 x:0 0 0 2024-04-26T04:42:29.290 INFO:tasks.workunit.client.0.smithi090.stdout:5/769: write d4/d1a/d2a/d79/d90/d56/d71/d91/fe9 [420125,58200] 0 2024-04-26T04:42:29.292 INFO:tasks.workunit.client.0.smithi090.stdout:7/868: mkdir d0/db/df/d1c/d23/d8d/d133 0 2024-04-26T04:42:29.297 INFO:tasks.workunit.client.0.smithi090.stdout:1/787: creat d0/d55/dda/d7/d17/d22/ffa x:0 0 0 2024-04-26T04:42:29.297 INFO:tasks.workunit.client.0.smithi090.stdout:1/788: fsync d0/d55/dda/d7/d17/d22/f89 0 2024-04-26T04:42:29.297 INFO:tasks.workunit.client.0.smithi090.stdout:9/700: creat db/dd/d44/d82/d85/db4/d58/ffe x:0 0 0 2024-04-26T04:42:29.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/701: dread db/dd/d18/d38/d39/d6b/de2/f88 [0,4194304] 0 2024-04-26T04:42:29.298 INFO:tasks.workunit.client.0.smithi090.stdout:9/702: truncate db/dd/d44/d82/d85/db4/d99/db2/fe9 61209 0 2024-04-26T04:42:29.299 INFO:tasks.workunit.client.0.smithi090.stdout:9/703: chown db/dd/d44/c80 0 1 2024-04-26T04:42:29.299 INFO:tasks.workunit.client.0.smithi090.stdout:0/840: mkdir d0/d4b/db3/d122 0 2024-04-26T04:42:29.300 INFO:tasks.workunit.client.0.smithi090.stdout:6/851: dwrite d9/d77/db0/db5/dda/d58/fee [0,4194304] 0 2024-04-26T04:42:29.300 INFO:tasks.workunit.client.0.smithi090.stdout:1/789: mknod d0/cfb 0 2024-04-26T04:42:29.301 INFO:tasks.workunit.client.0.smithi090.stdout:1/790: write d0/d55/fb9 [317664,105112] 0 2024-04-26T04:42:29.304 INFO:tasks.workunit.client.0.smithi090.stdout:0/841: rmdir d0/d4b/d5d/d48/dc1 39 2024-04-26T04:42:29.305 INFO:tasks.workunit.client.0.smithi090.stdout:7/869: dread d0/d109/d65/f123 [4194304,4194304] 0 2024-04-26T04:42:29.305 INFO:tasks.workunit.client.0.smithi090.stdout:1/791: dread d0/d55/dda/d7/d4a/da8/fad [0,4194304] 0 2024-04-26T04:42:29.305 INFO:tasks.workunit.client.0.smithi090.stdout:1/792: dread - d0/d55/dda/d43/fc5 zero size 2024-04-26T04:42:29.305 INFO:tasks.workunit.client.0.smithi090.stdout:1/793: write d0/d55/fd1 [362372,122236] 0 2024-04-26T04:42:29.305 INFO:tasks.workunit.client.0.smithi090.stdout:1/794: chown d0/d55/dda/d7/d17/d22/d93/fdb 89645 1 2024-04-26T04:42:29.306 INFO:tasks.workunit.client.0.smithi090.stdout:7/870: mknod d0/d1/c134 0 2024-04-26T04:42:29.306 INFO:tasks.workunit.client.0.smithi090.stdout:7/871: chown d0/db/df/d1c/d23/d8d/d111/f12f 8291785 1 2024-04-26T04:42:29.308 INFO:tasks.workunit.client.0.smithi090.stdout:6/852: dread d9/d77/db0/db5/dda/d58/fee [4194304,4194304] 0 2024-04-26T04:42:29.313 INFO:tasks.workunit.client.0.smithi090.stdout:7/872: mkdir d0/db/de6/d135 0 2024-04-26T04:42:29.315 INFO:tasks.workunit.client.0.smithi090.stdout:6/853: rename d9/d21/d40/d4c/d6f/d9f/fcd to d9/d77/db0/db5/f129 0 2024-04-26T04:42:29.317 INFO:tasks.workunit.client.0.smithi090.stdout:7/873: creat d0/db/de6/f136 x:0 0 0 2024-04-26T04:42:29.319 INFO:tasks.workunit.client.0.smithi090.stdout:6/854: link d9/d1c/d31/fd1 d9/d77/db0/db5/dda/d6d/f12a 0 2024-04-26T04:42:29.321 INFO:tasks.workunit.client.0.smithi090.stdout:6/855: mknod d9/d77/db0/db5/dca/dd8/c12b 0 2024-04-26T04:42:29.322 INFO:tasks.workunit.client.0.smithi090.stdout:4/795: sync 2024-04-26T04:42:29.322 INFO:tasks.workunit.client.0.smithi090.stdout:2/818: sync 2024-04-26T04:42:29.322 INFO:tasks.workunit.client.0.smithi090.stdout:2/819: readlink db/d11/d14/d27/ld4 0 2024-04-26T04:42:29.322 INFO:tasks.workunit.client.0.smithi090.stdout:2/820: dread - db/d11/d14/d25/d2d/d48/fbf zero size 2024-04-26T04:42:29.322 INFO:tasks.workunit.client.0.smithi090.stdout:0/842: dwrite d0/d4b/fad [0,4194304] 0 2024-04-26T04:42:29.323 INFO:tasks.workunit.client.0.smithi090.stdout:4/796: mkdir d3/db/d60/dee/dfc 0 2024-04-26T04:42:29.325 INFO:tasks.workunit.client.0.smithi090.stdout:2/821: mkdir db/d11/d14/d25/d65/d95/d112 0 2024-04-26T04:42:29.329 INFO:tasks.workunit.client.0.smithi090.stdout:1/795: fsync d0/d55/dda/d7/d17/d22/ffa 0 2024-04-26T04:42:29.331 INFO:tasks.workunit.client.0.smithi090.stdout:1/796: mkdir d0/d55/dda/d7/d17/d22/d93/db0/dfc 0 2024-04-26T04:42:29.333 INFO:tasks.workunit.client.0.smithi090.stdout:0/843: dread d0/d4b/d5d/d5c/d8a/f3e [0,4194304] 0 2024-04-26T04:42:29.333 INFO:tasks.workunit.client.0.smithi090.stdout:0/844: read - d0/d4b/d5d/d5c/f111 zero size 2024-04-26T04:42:29.333 INFO:tasks.workunit.client.0.smithi090.stdout:0/845: stat d0/df/l1a 0 2024-04-26T04:42:29.333 INFO:tasks.workunit.client.0.smithi090.stdout:1/797: mknod d0/d55/dda/d7/cfd 0 2024-04-26T04:42:29.334 INFO:tasks.workunit.client.0.smithi090.stdout:0/846: mkdir d0/d4b/d5d/d5c/d117/d120/d123 0 2024-04-26T04:42:29.334 INFO:tasks.workunit.client.0.smithi090.stdout:0/847: fsync d0/d4b/db3/dcc/fd1 0 2024-04-26T04:42:29.334 INFO:tasks.workunit.client.0.smithi090.stdout:1/798: symlink d0/d55/dda/d7/d4b/lfe 0 2024-04-26T04:42:29.335 INFO:tasks.workunit.client.0.smithi090.stdout:0/848: symlink d0/d4b/db3/d122/l124 0 2024-04-26T04:42:29.337 INFO:tasks.workunit.client.0.smithi090.stdout:2/822: dread db/d11/d14/d27/d28/f56 [0,4194304] 0 2024-04-26T04:42:29.341 INFO:tasks.workunit.client.0.smithi090.stdout:7/874: dwrite d0/d1/f61 [4194304,4194304] 0 2024-04-26T04:42:29.341 INFO:tasks.workunit.client.0.smithi090.stdout:7/875: write d0/db/df/d4d/d64/db0/dd6/f122 [468683,72924] 0 2024-04-26T04:42:29.342 INFO:tasks.workunit.client.0.smithi090.stdout:7/876: chown d0/db/d53/l86 1 1 2024-04-26T04:42:29.352 INFO:tasks.workunit.client.0.smithi090.stdout:7/877: dread d0/db/d53/f16 [0,4194304] 0 2024-04-26T04:42:29.354 INFO:tasks.workunit.client.0.smithi090.stdout:7/878: unlink d0/d109/d4a/le1 0 2024-04-26T04:42:29.354 INFO:tasks.workunit.client.0.smithi090.stdout:7/879: read - d0/d109/d65/db2/f103 zero size 2024-04-26T04:42:29.358 INFO:tasks.workunit.client.0.smithi090.stdout:1/799: dwrite d0/d55/dda/d7/d4b/fc7 [0,4194304] 0 2024-04-26T04:42:29.362 INFO:tasks.workunit.client.0.smithi090.stdout:2/823: dwrite db/d11/d14/d1b/d20/fe0 [0,4194304] 0 2024-04-26T04:42:29.390 INFO:tasks.workunit.client.0.smithi090.stdout:7/880: dwrite d0/db/d53/db7/d4f/ffa [0,4194304] 0 2024-04-26T04:42:29.392 INFO:tasks.workunit.client.0.smithi090.stdout:7/881: stat d0/d1/da3/la5 0 2024-04-26T04:42:29.392 INFO:tasks.workunit.client.0.smithi090.stdout:7/882: write d0/db/df/d1c/d23/d8d/d111/f12c [878091,16125] 0 2024-04-26T04:42:29.396 INFO:tasks.workunit.client.0.smithi090.stdout:2/824: dwrite db/dc/f1f [0,4194304] 0 2024-04-26T04:42:29.396 INFO:tasks.workunit.client.0.smithi090.stdout:2/825: stat db/d11/d14/d25/f4e 0 2024-04-26T04:42:29.397 INFO:tasks.workunit.client.0.smithi090.stdout:2/826: mknod db/d11/d14/d1b/d20/da7/c113 0 2024-04-26T04:42:29.398 INFO:tasks.workunit.client.0.smithi090.stdout:1/800: dwrite d0/d55/dda/d43/f54 [0,4194304] 0 2024-04-26T04:42:29.400 INFO:tasks.workunit.client.0.smithi090.stdout:2/827: link db/d59/d66/d69/lec db/l114 0 2024-04-26T04:42:29.401 INFO:tasks.workunit.client.0.smithi090.stdout:2/828: write db/d11/d14/d1b/d20/d4c/fae [331880,79501] 0 2024-04-26T04:42:29.404 INFO:tasks.workunit.client.0.smithi090.stdout:2/829: link db/d11/d14/d27/d28/dd1/dfe/d71/dea/ff5 db/d11/d14/d1b/d20/f115 0 2024-04-26T04:42:29.405 INFO:tasks.workunit.client.0.smithi090.stdout:2/830: unlink db/d59/d66/l85 0 2024-04-26T04:42:29.410 INFO:tasks.workunit.client.0.smithi090.stdout:2/831: dread db/d11/d14/d27/fab [0,4194304] 0 2024-04-26T04:42:29.410 INFO:tasks.workunit.client.0.smithi090.stdout:7/883: dwrite d0/d1/f61 [0,4194304] 0 2024-04-26T04:42:29.412 INFO:tasks.workunit.client.0.smithi090.stdout:2/832: write db/d11/d14/d25/f9a [826514,25888] 0 2024-04-26T04:42:29.417 INFO:tasks.workunit.client.0.smithi090.stdout:7/884: write d0/d1/d4/d3e/fcf [6222447,107359] 0 2024-04-26T04:42:29.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/885: symlink d0/db/df/d1c/l137 0 2024-04-26T04:42:29.419 INFO:tasks.workunit.client.0.smithi090.stdout:7/886: chown d0/c55 9438 1 2024-04-26T04:42:29.420 INFO:tasks.workunit.client.0.smithi090.stdout:7/887: truncate d0/d1/d4/d3e/dd3/de2/fea 124883 0 2024-04-26T04:42:29.421 INFO:tasks.workunit.client.0.smithi090.stdout:7/888: mknod d0/db/df/d4d/daf/dd1/d106/c138 0 2024-04-26T04:42:29.422 INFO:tasks.workunit.client.0.smithi090.stdout:7/889: symlink d0/db/d53/db7/d8e/l139 0 2024-04-26T04:42:29.423 INFO:tasks.workunit.client.0.smithi090.stdout:7/890: rmdir d0/d109/db1 39 2024-04-26T04:42:29.439 INFO:tasks.workunit.client.0.smithi090.stdout:7/891: dwrite d0/db/df/d1c/d23/d8d/fdd [0,4194304] 0 2024-04-26T04:42:29.443 INFO:tasks.workunit.client.0.smithi090.stdout:7/892: write d0/db/df/f70 [5403243,28166] 0 2024-04-26T04:42:29.445 INFO:tasks.workunit.client.0.smithi090.stdout:7/893: mkdir d0/db/d53/d5e/d13a 0 2024-04-26T04:42:29.445 INFO:tasks.workunit.client.0.smithi090.stdout:7/894: chown d0/db/df/d4d/d64/d66/f7c 1082 1 2024-04-26T04:42:29.445 INFO:tasks.workunit.client.0.smithi090.stdout:7/895: write d0/db/df/d4d/d64/df2/d10a/f10b [479842,60893] 0 2024-04-26T04:42:29.446 INFO:tasks.workunit.client.0.smithi090.stdout:7/896: creat d0/d1/d4/d3e/dd3/de2/f13b x:0 0 0 2024-04-26T04:42:29.447 INFO:tasks.workunit.client.0.smithi090.stdout:7/897: rmdir d0/db/df/d4d/daf/dd1 39 2024-04-26T04:42:29.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/898: mkdir d0/db/df/d4d/daf/dd1/d13c 0 2024-04-26T04:42:29.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/899: rename d0/d109/d65/d11c/c124 to d0/db/d53/db7/d126/c13d 0 2024-04-26T04:42:29.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/900: dread - d0/db/df/d4d/d64/d66/f12b zero size 2024-04-26T04:42:29.450 INFO:tasks.workunit.client.0.smithi090.stdout:7/901: mknod d0/db/df/d4d/d64/db0/dd6/d10e/c13e 0 2024-04-26T04:42:29.451 INFO:tasks.workunit.client.0.smithi090.stdout:7/902: truncate d0/db/df/d1c/d23/d8d/fdd 3086779 0 2024-04-26T04:42:29.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/903: symlink d0/db/df/d4d/l13f 0 2024-04-26T04:42:29.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/904: write d0/d109/d65/f101 [617761,59964] 0 2024-04-26T04:42:29.461 INFO:tasks.workunit.client.0.smithi090.stdout:7/905: dread d0/d1/d4/f2e [0,4194304] 0 2024-04-26T04:42:29.520 INFO:tasks.workunit.client.0.smithi090.stdout:8/943: sync 2024-04-26T04:42:29.520 INFO:tasks.workunit.client.0.smithi090.stdout:3/742: sync 2024-04-26T04:42:29.529 INFO:tasks.workunit.client.0.smithi090.stdout:3/743: dread d1/d59/f6d [0,4194304] 0 2024-04-26T04:42:29.530 INFO:tasks.workunit.client.0.smithi090.stdout:3/744: dread d1/d5c/f6e [0,4194304] 0 2024-04-26T04:42:29.531 INFO:tasks.workunit.client.0.smithi090.stdout:3/745: fsync d1/d17/f1f 0 2024-04-26T04:42:29.531 INFO:tasks.workunit.client.0.smithi090.stdout:3/746: fsync d1/d59/f6d 0 2024-04-26T04:42:29.532 INFO:tasks.workunit.client.0.smithi090.stdout:3/747: mkdir d1/d17/d61/dce/d101 0 2024-04-26T04:42:29.532 INFO:tasks.workunit.client.0.smithi090.stdout:3/748: getdents d1/d17/d61/dce/d101 0 2024-04-26T04:42:29.534 INFO:tasks.workunit.client.0.smithi090.stdout:3/749: link d1/d17/d36/d33/d68/dae/dca/d1b/c7f d1/d17/d61/dce/c102 0 2024-04-26T04:42:29.535 INFO:tasks.workunit.client.0.smithi090.stdout:3/750: creat d1/d17/d36/d2a/d3a/f103 x:0 0 0 2024-04-26T04:42:29.535 INFO:tasks.workunit.client.0.smithi090.stdout:3/751: mknod d1/d17/d36/d33/d68/dae/dca/d1b/d58/db5/c104 0 2024-04-26T04:42:29.536 INFO:tasks.workunit.client.0.smithi090.stdout:3/752: fsync d1/d17/d36/d33/d42/f63 0 2024-04-26T04:42:29.537 INFO:tasks.workunit.client.0.smithi090.stdout:3/753: mkdir d1/d17/d105 0 2024-04-26T04:42:29.539 INFO:tasks.workunit.client.0.smithi090.stdout:3/754: rmdir d1/d17/d36/d33/d68/dae/dca/d2c/d97 39 2024-04-26T04:42:29.539 INFO:tasks.workunit.client.0.smithi090.stdout:3/755: truncate d1/d17/d36/d33/d68/dae/dca/dcf/ff9 162538 0 2024-04-26T04:42:29.541 INFO:tasks.workunit.client.0.smithi090.stdout:8/944: dwrite dc/dd2/d3b/d3d/f5e [0,4194304] 0 2024-04-26T04:42:29.549 INFO:tasks.workunit.client.0.smithi090.stdout:3/756: rmdir d1/d17/d36 39 2024-04-26T04:42:29.552 INFO:tasks.workunit.client.0.smithi090.stdout:3/757: rename d1/d17/d36/d2a/d3a/d5d to d1/d17/d36/d33/d68/dae/dca/dcf/d106 0 2024-04-26T04:42:29.553 INFO:tasks.workunit.client.0.smithi090.stdout:3/758: mknod d1/d17/d36/d2a/c107 0 2024-04-26T04:42:29.553 INFO:tasks.workunit.client.0.smithi090.stdout:3/759: write d1/d59/dc9/d7d/fbf [507258,28840] 0 2024-04-26T04:42:29.559 INFO:tasks.workunit.client.0.smithi090.stdout:8/945: dwrite dc/dd2/d4f/f7f [0,4194304] 0 2024-04-26T04:42:29.559 INFO:tasks.workunit.client.0.smithi090.stdout:8/946: readlink dc/d1f/l2a 0 2024-04-26T04:42:29.561 INFO:tasks.workunit.client.0.smithi090.stdout:8/947: creat dc/dd2/d3b/d3d/db8/f14c x:0 0 0 2024-04-26T04:42:29.561 INFO:tasks.workunit.client.0.smithi090.stdout:8/948: chown dc/dd2/d39/d55/d64/lb4 158792 1 2024-04-26T04:42:29.561 INFO:tasks.workunit.client.0.smithi090.stdout:8/949: creat dc/dd2/d4f/db7/f14d x:0 0 0 2024-04-26T04:42:29.563 INFO:tasks.workunit.client.0.smithi090.stdout:8/950: rename dc/d1f/f67 to dc/dd2/d4f/da6/dfc/f14e 0 2024-04-26T04:42:29.563 INFO:tasks.workunit.client.0.smithi090.stdout:8/951: fdatasync dc/dd2/d39/d125/d5d/d73/dce/f100 0 2024-04-26T04:42:29.564 INFO:tasks.workunit.client.0.smithi090.stdout:8/952: getdents dc/dd2/d4f 0 2024-04-26T04:42:29.564 INFO:tasks.workunit.client.0.smithi090.stdout:8/953: fsync dc/dd2/d4f/f9f 0 2024-04-26T04:42:29.564 INFO:tasks.workunit.client.0.smithi090.stdout:8/954: stat dc/dd2/d3b/d3d/d97/dee/c11a 0 2024-04-26T04:42:29.565 INFO:tasks.workunit.client.0.smithi090.stdout:8/955: mkdir dc/dd2/d4f/da6/dd0/d14f 0 2024-04-26T04:42:29.568 INFO:tasks.workunit.client.0.smithi090.stdout:3/760: dwrite d1/d17/d36/d2a/d3a/fd1 [4194304,4194304] 0 2024-04-26T04:42:29.568 INFO:tasks.workunit.client.0.smithi090.stdout:3/761: chown d1/d9/fd6 0 1 2024-04-26T04:42:29.579 INFO:tasks.workunit.client.0.smithi090.stdout:3/762: dread d1/d17/d36/d33/d68/dae/dca/f21 [0,4194304] 0 2024-04-26T04:42:29.579 INFO:tasks.workunit.client.0.smithi090.stdout:3/763: readlink d1/d17/d36/d33/d68/dae/dca/dcf/d106/le0 0 2024-04-26T04:42:29.583 INFO:tasks.workunit.client.0.smithi090.stdout:8/956: dwrite dc/dd2/d4f/d58/f6f [0,4194304] 0 2024-04-26T04:42:29.583 INFO:tasks.workunit.client.0.smithi090.stdout:8/957: write dc/dd2/d3b/d3d/f54 [627674,116410] 0 2024-04-26T04:42:29.583 INFO:tasks.workunit.client.0.smithi090.stdout:8/958: readlink dc/dd2/d3b/l5b 0 2024-04-26T04:42:29.584 INFO:tasks.workunit.client.0.smithi090.stdout:8/959: mknod dc/dd2/d3b/d3d/db8/c150 0 2024-04-26T04:42:29.584 INFO:tasks.workunit.client.0.smithi090.stdout:8/960: creat dc/dd2/d4f/da6/dd8/f151 x:0 0 0 2024-04-26T04:42:29.589 INFO:tasks.workunit.client.0.smithi090.stdout:8/961: dread dc/dd2/d39/d125/d5d/f69 [0,4194304] 0 2024-04-26T04:42:29.589 INFO:tasks.workunit.client.0.smithi090.stdout:8/962: creat dc/dd2/d4f/da6/dfc/f152 x:0 0 0 2024-04-26T04:42:29.592 INFO:tasks.workunit.client.0.smithi090.stdout:8/963: unlink dc/dd2/d3b/d3d/d97/ca3 0 2024-04-26T04:42:29.597 INFO:tasks.workunit.client.0.smithi090.stdout:3/764: dwrite d1/d59/d5e/f62 [0,4194304] 0 2024-04-26T04:42:29.597 INFO:tasks.workunit.client.0.smithi090.stdout:3/765: dread - d1/d17/d36/d33/d68/dae/dca/dcf/feb zero size 2024-04-26T04:42:29.599 INFO:tasks.workunit.client.0.smithi090.stdout:3/766: mknod d1/d5c/c108 0 2024-04-26T04:42:29.599 INFO:tasks.workunit.client.0.smithi090.stdout:3/767: write d1/d59/f64 [6908557,61045] 0 2024-04-26T04:42:29.609 INFO:tasks.workunit.client.0.smithi090.stdout:8/964: dwrite dc/dd2/d39/d55/d64/f3e [4194304,4194304] 0 2024-04-26T04:42:29.620 INFO:tasks.workunit.client.0.smithi090.stdout:8/965: dread dc/dd2/d39/fb9 [0,4194304] 0 2024-04-26T04:42:29.637 INFO:tasks.workunit.client.0.smithi090.stdout:8/966: dwrite dc/dd2/d39/d55/f6c [4194304,4194304] 0 2024-04-26T04:42:29.637 INFO:tasks.workunit.client.0.smithi090.stdout:8/967: dread - dc/dd2/d4f/da6/dd8/f122 zero size 2024-04-26T04:42:29.637 INFO:tasks.workunit.client.0.smithi090.stdout:8/968: chown dc/d1f/ld3 24739 1 2024-04-26T04:42:29.638 INFO:tasks.workunit.client.0.smithi090.stdout:8/969: chown dc/dd2/d39/d125/d5d/f69 370 1 2024-04-26T04:42:29.653 INFO:tasks.workunit.client.0.smithi090.stdout:6/856: sync 2024-04-26T04:42:29.653 INFO:tasks.workunit.client.0.smithi090.stdout:5/770: sync 2024-04-26T04:42:29.653 INFO:tasks.workunit.client.0.smithi090.stdout:9/704: sync 2024-04-26T04:42:29.653 INFO:tasks.workunit.client.0.smithi090.stdout:5/771: fsync d4/dab/fbd 0 2024-04-26T04:42:29.654 INFO:tasks.workunit.client.0.smithi090.stdout:9/705: unlink db/dd/d18/d38/d39/c8f 0 2024-04-26T04:42:29.655 INFO:tasks.workunit.client.0.smithi090.stdout:6/857: rmdir d9/d77/db0/db5/dda/d6d/d66 39 2024-04-26T04:42:29.656 INFO:tasks.workunit.client.0.smithi090.stdout:5/772: mknod d4/d1a/d2a/d79/d90/d56/d71/d91/cf5 0 2024-04-26T04:42:29.656 INFO:tasks.workunit.client.0.smithi090.stdout:5/773: read - d4/d1a/d2a/d79/fcf zero size 2024-04-26T04:42:29.656 INFO:tasks.workunit.client.0.smithi090.stdout:5/774: dread - d4/d1a/df/d72/dc0/ff4 zero size 2024-04-26T04:42:29.656 INFO:tasks.workunit.client.0.smithi090.stdout:5/775: chown d4/d1a/d2a/d79/d90/d56/d80/c93 4358 1 2024-04-26T04:42:29.656 INFO:tasks.workunit.client.0.smithi090.stdout:8/970: dwrite dc/dd2/d4f/f7f [0,4194304] 0 2024-04-26T04:42:29.657 INFO:tasks.workunit.client.0.smithi090.stdout:9/706: rmdir db/dd/d44/d82/dbd/de7 0 2024-04-26T04:42:29.658 INFO:tasks.workunit.client.0.smithi090.stdout:6/858: creat d9/d21/d40/f12c x:0 0 0 2024-04-26T04:42:29.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/776: rmdir d4/d1a/d1f/dc5/dca/df2 0 2024-04-26T04:42:29.659 INFO:tasks.workunit.client.0.smithi090.stdout:9/707: mkdir db/dd/d18/d38/d39/d3f/dff 0 2024-04-26T04:42:29.661 INFO:tasks.workunit.client.0.smithi090.stdout:6/859: truncate d9/f24 1403661 0 2024-04-26T04:42:29.663 INFO:tasks.workunit.client.0.smithi090.stdout:5/777: rename d4/d1a/d3f/f65 to d4/d1a/d45/db8/ff6 0 2024-04-26T04:42:29.663 INFO:tasks.workunit.client.0.smithi090.stdout:5/778: chown d4/d1a/d2a/f41 279820 1 2024-04-26T04:42:29.663 INFO:tasks.workunit.client.0.smithi090.stdout:5/779: write d4/d1a/d45/f32 [276987,39484] 0 2024-04-26T04:42:29.663 INFO:tasks.workunit.client.0.smithi090.stdout:5/780: stat d4/d1a/df/f6d 0 2024-04-26T04:42:29.663 INFO:tasks.workunit.client.0.smithi090.stdout:9/708: mknod db/dd/d18/c100 0 2024-04-26T04:42:29.663 INFO:tasks.workunit.client.0.smithi090.stdout:9/709: dread - db/dd/d44/d82/d85/db4/d58/ffe zero size 2024-04-26T04:42:29.665 INFO:tasks.workunit.client.0.smithi090.stdout:8/971: write dc/dd2/d39/d125/fc8 [2321489,86394] 0 2024-04-26T04:42:29.665 INFO:tasks.workunit.client.0.smithi090.stdout:5/781: truncate d4/f88 3525021 0 2024-04-26T04:42:29.665 INFO:tasks.workunit.client.0.smithi090.stdout:5/782: write d4/d1a/d2a/d79/d90/db3/f87 [693786,102499] 0 2024-04-26T04:42:29.666 INFO:tasks.workunit.client.0.smithi090.stdout:8/972: read dc/dd2/d39/d125/f62 [20971,58509] 0 2024-04-26T04:42:29.666 INFO:tasks.workunit.client.0.smithi090.stdout:6/860: symlink d9/d77/db0/db5/dda/d6d/d92/l12d 0 2024-04-26T04:42:29.666 INFO:tasks.workunit.client.0.smithi090.stdout:6/861: truncate d9/d77/db0/f103 331326 0 2024-04-26T04:42:29.666 INFO:tasks.workunit.client.0.smithi090.stdout:6/862: creat d9/d3c/f12e x:0 0 0 2024-04-26T04:42:29.666 INFO:tasks.workunit.client.0.smithi090.stdout:9/710: write db/dd/d18/d38/d39/d6b/de2/f9a [1505578,107958] 0 2024-04-26T04:42:29.667 INFO:tasks.workunit.client.0.smithi090.stdout:5/783: mknod d4/d1a/d3f/cf7 0 2024-04-26T04:42:29.667 INFO:tasks.workunit.client.0.smithi090.stdout:5/784: chown d4/d1a/d1f/d53/d59/d5e/da0/cbc 3 1 2024-04-26T04:42:29.668 INFO:tasks.workunit.client.0.smithi090.stdout:8/973: unlink dc/d1f/l2d 0 2024-04-26T04:42:29.671 INFO:tasks.workunit.client.0.smithi090.stdout:9/711: symlink db/dd/d44/d82/d85/df6/l101 0 2024-04-26T04:42:29.671 INFO:tasks.workunit.client.0.smithi090.stdout:9/712: fsync db/dd/d18/f2a 0 2024-04-26T04:42:29.671 INFO:tasks.workunit.client.0.smithi090.stdout:9/713: chown db/dd/d44/d82/d85/db4/d6a/lc1 187302 1 2024-04-26T04:42:29.671 INFO:tasks.workunit.client.0.smithi090.stdout:9/714: creat db/dd/d18/d57/dcc/f102 x:0 0 0 2024-04-26T04:42:29.673 INFO:tasks.workunit.client.0.smithi090.stdout:5/785: creat d4/d1a/d2a/d79/d90/d56/d71/deb/ff8 x:0 0 0 2024-04-26T04:42:29.673 INFO:tasks.workunit.client.0.smithi090.stdout:6/863: dread d9/d1c/d31/f3a [0,4194304] 0 2024-04-26T04:42:29.679 INFO:tasks.workunit.client.0.smithi090.stdout:6/864: creat d9/d1c/d31/d69/db7/d117/f12f x:0 0 0 2024-04-26T04:42:29.682 INFO:tasks.workunit.client.0.smithi090.stdout:9/715: read f2 [3125358,62065] 0 2024-04-26T04:42:29.684 INFO:tasks.workunit.client.0.smithi090.stdout:6/865: mkdir d9/d21/d40/d130 0 2024-04-26T04:42:29.685 INFO:tasks.workunit.client.0.smithi090.stdout:9/716: mknod db/dd/d18/d38/d39/d6b/c103 0 2024-04-26T04:42:29.685 INFO:tasks.workunit.client.0.smithi090.stdout:9/717: write db/dd/d18/d57/dcc/f102 [355310,88694] 0 2024-04-26T04:42:29.687 INFO:tasks.workunit.client.0.smithi090.stdout:9/718: rename db/dd/d18/d38/d39/d6b/dc0/lb1 to db/dd/d18/d38/d39/d6b/l104 0 2024-04-26T04:42:29.688 INFO:tasks.workunit.client.0.smithi090.stdout:9/719: rename db/dd/d18/d38/d39/d6b/de2/f33 to db/dd/d18/d38/d39/dba/f105 0 2024-04-26T04:42:29.689 INFO:tasks.workunit.client.0.smithi090.stdout:9/720: creat db/d5f/f106 x:0 0 0 2024-04-26T04:42:29.691 INFO:tasks.workunit.client.0.smithi090.stdout:5/786: dwrite d4/d1a/d1f/d53/ff3 [0,4194304] 0 2024-04-26T04:42:29.694 INFO:tasks.workunit.client.0.smithi090.stdout:6/866: dread d9/d21/d3b/d86/fdc [0,4194304] 0 2024-04-26T04:42:29.695 INFO:tasks.workunit.client.0.smithi090.stdout:5/787: unlink d4/d1a/d2a/d79/d90/d56/d71/fa1 0 2024-04-26T04:42:29.696 INFO:tasks.workunit.client.0.smithi090.stdout:5/788: truncate d4/d1a/d1f/d53/d59/d5e/fa5 1443122 0 2024-04-26T04:42:29.698 INFO:tasks.workunit.client.0.smithi090.stdout:9/721: dread db/dd/d44/d82/d85/db4/f4a [0,4194304] 0 2024-04-26T04:42:29.698 INFO:tasks.workunit.client.0.smithi090.stdout:9/722: chown db/dd/d18/d38/d39/d6b/de2/fc2 3 1 2024-04-26T04:42:29.701 INFO:tasks.workunit.client.0.smithi090.stdout:9/723: dread db/dd/d44/d82/d85/db4/f4a [0,4194304] 0 2024-04-26T04:42:29.702 INFO:tasks.workunit.client.0.smithi090.stdout:8/974: truncate dc/dd2/d3b/fbf 2127975 0 2024-04-26T04:42:29.703 INFO:tasks.workunit.client.0.smithi090.stdout:9/724: mkdir db/dd/d44/d82/d85/db4/d99/db2/dcd/d107 0 2024-04-26T04:42:29.704 INFO:tasks.workunit.client.0.smithi090.stdout:5/789: dread d4/d1a/d2a/d79/d90/f96 [0,4194304] 0 2024-04-26T04:42:29.704 INFO:tasks.workunit.client.0.smithi090.stdout:5/790: write d4/d1a/df/f22 [3952415,57800] 0 2024-04-26T04:42:29.705 INFO:tasks.workunit.client.0.smithi090.stdout:8/975: link dc/dd2/d39/f81 dc/dd2/d3b/d3d/db8/f153 0 2024-04-26T04:42:29.708 INFO:tasks.workunit.client.0.smithi090.stdout:9/725: getdents db/dd/d44 0 2024-04-26T04:42:29.710 INFO:tasks.workunit.client.0.smithi090.stdout:9/726: mkdir db/dd/d18/d38/d108 0 2024-04-26T04:42:29.711 INFO:tasks.workunit.client.0.smithi090.stdout:9/727: mkdir db/dd/d44/d82/dbd/d109 0 2024-04-26T04:42:29.712 INFO:tasks.workunit.client.0.smithi090.stdout:9/728: link db/dd/d44/d82/d85/db4/d58/f6c db/dd/d44/d82/d85/db4/d58/f10a 0 2024-04-26T04:42:29.713 INFO:tasks.workunit.client.0.smithi090.stdout:6/867: dwrite d9/d77/db0/db5/dda/d58/fee [12582912,4194304] 0 2024-04-26T04:42:29.714 INFO:tasks.workunit.client.0.smithi090.stdout:6/868: rmdir d9/d21/d40/dc7 39 2024-04-26T04:42:29.722 INFO:tasks.workunit.client.0.smithi090.stdout:8/976: write dc/d34/f93 [1373899,45944] 0 2024-04-26T04:42:29.722 INFO:tasks.workunit.client.0.smithi090.stdout:5/791: dread d4/d1a/d45/f32 [4194304,4194304] 0 2024-04-26T04:42:29.726 INFO:tasks.workunit.client.0.smithi090.stdout:5/792: symlink d4/lf9 0 2024-04-26T04:42:29.726 INFO:tasks.workunit.client.0.smithi090.stdout:5/793: mknod d4/d1a/cfa 0 2024-04-26T04:42:29.726 INFO:tasks.workunit.client.0.smithi090.stdout:5/794: truncate d4/d1a/df/d4a/fde 467794 0 2024-04-26T04:42:29.734 INFO:tasks.workunit.client.0.smithi090.stdout:9/729: dwrite db/dd/d44/d82/d85/db4/d99/fcf [0,4194304] 0 2024-04-26T04:42:29.734 INFO:tasks.workunit.client.0.smithi090.stdout:9/730: readlink db/dd/l24 0 2024-04-26T04:42:29.739 INFO:tasks.workunit.client.0.smithi090.stdout:9/731: dread db/dd/d44/d82/d85/db4/d99/db2/fe9 [0,4194304] 0 2024-04-26T04:42:29.740 INFO:tasks.workunit.client.0.smithi090.stdout:9/732: link db/dd/d18/d38/f79 db/d5f/f10b 0 2024-04-26T04:42:29.742 INFO:tasks.workunit.client.0.smithi090.stdout:9/733: rename db/f1b to db/f10c 0 2024-04-26T04:42:29.743 INFO:tasks.workunit.client.0.smithi090.stdout:9/734: write db/dd/d18/d38/d39/dba/f105 [1009258,13804] 0 2024-04-26T04:42:29.760 INFO:tasks.workunit.client.0.smithi090.stdout:9/735: write db/dd/d18/d38/d39/d6b/de2/f5a [2652327,110474] 0 2024-04-26T04:42:29.760 INFO:tasks.workunit.client.0.smithi090.stdout:9/736: truncate db/dd/d44/d82/ff9 413232 0 2024-04-26T04:42:29.760 INFO:tasks.workunit.client.0.smithi090.stdout:9/737: readlink db/dd/d18/d38/d39/d6b/l73 0 2024-04-26T04:42:29.761 INFO:tasks.workunit.client.0.smithi090.stdout:6/869: dwrite d9/f68 [0,4194304] 0 2024-04-26T04:42:29.761 INFO:tasks.workunit.client.0.smithi090.stdout:8/977: dwrite dc/f74 [0,4194304] 0 2024-04-26T04:42:29.761 INFO:tasks.workunit.client.0.smithi090.stdout:5/795: dwrite d4/d1a/d2a/d79/d90/d56/f8b [0,4194304] 0 2024-04-26T04:42:29.761 INFO:tasks.workunit.client.0.smithi090.stdout:5/796: readlink d4/d1a/d2a/d79/d90/d56/d80/ld9 0 2024-04-26T04:42:29.763 INFO:tasks.workunit.client.0.smithi090.stdout:5/797: dread d4/d1a/d1f/d53/d59/d5e/fa5 [0,4194304] 0 2024-04-26T04:42:29.764 INFO:tasks.workunit.client.0.smithi090.stdout:6/870: mkdir d9/d77/db0/db5/dca/dd8/d131 0 2024-04-26T04:42:29.765 INFO:tasks.workunit.client.0.smithi090.stdout:8/978: mkdir dc/dd2/d4f/db7/d154 0 2024-04-26T04:42:29.774 INFO:tasks.workunit.client.0.smithi090.stdout:9/738: dwrite db/dd/d18/d38/d39/d6b/de2/f9a [0,4194304] 0 2024-04-26T04:42:29.774 INFO:tasks.workunit.client.0.smithi090.stdout:9/739: write db/dd/d18/d38/d39/dba/f105 [1330871,6985] 0 2024-04-26T04:42:29.776 INFO:tasks.workunit.client.0.smithi090.stdout:5/798: link d4/d1a/fba d4/d1a/d2a/d79/ffb 0 2024-04-26T04:42:29.776 INFO:tasks.workunit.client.0.smithi090.stdout:5/799: dread - d4/d1a/df/d36/d8e/fe8 zero size 2024-04-26T04:42:29.776 INFO:tasks.workunit.client.0.smithi090.stdout:5/800: fsync d4/d1a/df/f8d 0 2024-04-26T04:42:29.777 INFO:tasks.workunit.client.0.smithi090.stdout:6/871: symlink d9/d1c/d31/d79/l132 0 2024-04-26T04:42:29.777 INFO:tasks.workunit.client.0.smithi090.stdout:6/872: chown d9/d21/d40/f74 312992 1 2024-04-26T04:42:29.779 INFO:tasks.workunit.client.0.smithi090.stdout:8/979: link dc/dd2/d39/d125/d5d/d73/dce/c145 dc/d71/c155 0 2024-04-26T04:42:29.780 INFO:tasks.workunit.client.0.smithi090.stdout:9/740: dread db/dd/d18/f43 [0,4194304] 0 2024-04-26T04:42:29.780 INFO:tasks.workunit.client.0.smithi090.stdout:9/741: creat db/dd/d44/d82/f10d x:0 0 0 2024-04-26T04:42:29.781 INFO:tasks.workunit.client.0.smithi090.stdout:8/980: truncate dc/dd2/d3b/d3d/d111/f121 4299482 0 2024-04-26T04:42:29.784 INFO:tasks.workunit.client.0.smithi090.stdout:5/801: rmdir d4/d1a/d2a/d79/d90/d56/d80 39 2024-04-26T04:42:29.785 INFO:tasks.workunit.client.0.smithi090.stdout:5/802: chown d4/d1a/df/f22 2062449453 1 2024-04-26T04:42:29.785 INFO:tasks.workunit.client.0.smithi090.stdout:8/981: symlink dc/dd2/d4f/db7/d154/l156 0 2024-04-26T04:42:29.785 INFO:tasks.workunit.client.0.smithi090.stdout:8/982: readlink dc/dd2/d4f/da6/lfb 0 2024-04-26T04:42:29.786 INFO:tasks.workunit.client.0.smithi090.stdout:5/803: symlink d4/d1a/df/d36/d8e/lfc 0 2024-04-26T04:42:29.788 INFO:tasks.workunit.client.0.smithi090.stdout:8/983: symlink dc/dd2/d39/d125/l157 0 2024-04-26T04:42:29.792 INFO:tasks.workunit.client.0.smithi090.stdout:5/804: dread d4/d1a/df/f22 [0,4194304] 0 2024-04-26T04:42:29.792 INFO:tasks.workunit.client.0.smithi090.stdout:5/805: write d4/d1a/d2a/d79/d90/d49/f82 [874773,89945] 0 2024-04-26T04:42:29.797 INFO:tasks.workunit.client.0.smithi090.stdout:5/806: unlink d4/d1a/d2a/d79/d90/d49/l70 0 2024-04-26T04:42:29.799 INFO:tasks.workunit.client.0.smithi090.stdout:8/984: rename dc/dd2/d3b to dc/dd2/d4f/da6/d158 0 2024-04-26T04:42:29.799 INFO:tasks.workunit.client.0.smithi090.stdout:8/985: dread - dc/dd2/d4f/da6/dd8/f103 zero size 2024-04-26T04:42:29.800 INFO:tasks.workunit.client.0.smithi090.stdout:8/986: mkdir dc/dd2/d4f/da6/d158/d3d/d159 0 2024-04-26T04:42:29.801 INFO:tasks.workunit.client.0.smithi090.stdout:8/987: write dc/d1f/f35 [712910,11339] 0 2024-04-26T04:42:29.804 INFO:tasks.workunit.client.0.smithi090.stdout:4/797: sync 2024-04-26T04:42:29.805 INFO:tasks.workunit.client.0.smithi090.stdout:5/807: dread d4/d1a/f19 [0,4194304] 0 2024-04-26T04:42:29.805 INFO:tasks.workunit.client.0.smithi090.stdout:5/808: getdents d4/d1a/df/d36/d8e/de3 0 2024-04-26T04:42:29.808 INFO:tasks.workunit.client.0.smithi090.stdout:5/809: creat d4/d1a/df/d36/d8e/de3/ffd x:0 0 0 2024-04-26T04:42:29.808 INFO:tasks.workunit.client.0.smithi090.stdout:5/810: read d4/d1a/d2a/d31/f34 [911106,95158] 0 2024-04-26T04:42:29.809 INFO:tasks.workunit.client.0.smithi090.stdout:5/811: symlink d4/d1a/df/d72/dc0/lfe 0 2024-04-26T04:42:29.830 INFO:tasks.workunit.client.0.smithi090.stdout:4/798: dwrite d3/db/d12/f5b [0,4194304] 0 2024-04-26T04:42:29.831 INFO:tasks.workunit.client.0.smithi090.stdout:8/988: dwrite dc/dd2/d39/d55/d64/f3e [0,4194304] 0 2024-04-26T04:42:29.831 INFO:tasks.workunit.client.0.smithi090.stdout:5/812: dwrite d4/d1a/d1f/f3d [4194304,4194304] 0 2024-04-26T04:42:29.831 INFO:tasks.workunit.client.0.smithi090.stdout:5/813: chown d4/c1b 10972 1 2024-04-26T04:42:29.834 INFO:tasks.workunit.client.0.smithi090.stdout:4/799: unlink d3/db/d12/d20/d2d/d58/d6a/d74/fb6 0 2024-04-26T04:42:29.834 INFO:tasks.workunit.client.0.smithi090.stdout:8/989: rename dc/dd2/d4f/d58/d95/d106/l12c to dc/d34/l15a 0 2024-04-26T04:42:29.836 INFO:tasks.workunit.client.0.smithi090.stdout:4/800: creat d3/d13/d6c/ffd x:0 0 0 2024-04-26T04:42:29.839 INFO:tasks.workunit.client.0.smithi090.stdout:8/990: symlink dc/dd2/d39/d125/d5d/d73/l15b 0 2024-04-26T04:42:29.840 INFO:tasks.workunit.client.0.smithi090.stdout:8/991: mkdir dc/dd2/dd9/d15c 0 2024-04-26T04:42:29.841 INFO:tasks.workunit.client.0.smithi090.stdout:8/992: truncate dc/dd2/d4f/da6/dd8/fa4 1453156 0 2024-04-26T04:42:29.843 INFO:tasks.workunit.client.0.smithi090.stdout:5/814: dread d4/d1a/d1f/d53/ff3 [0,4194304] 0 2024-04-26T04:42:29.843 INFO:tasks.workunit.client.0.smithi090.stdout:8/993: link dc/dd2/d4f/da6/db1/ddd/f133 dc/dd2/d4f/da6/db1/ddd/f15d 0 2024-04-26T04:42:29.845 INFO:tasks.workunit.client.0.smithi090.stdout:8/994: mknod dc/dd2/d4f/da6/d158/d3d/db8/ddc/c15e 0 2024-04-26T04:42:29.845 INFO:tasks.workunit.client.0.smithi090.stdout:8/995: write dc/dd2/d39/fb9 [4109627,65175] 0 2024-04-26T04:42:29.845 INFO:tasks.workunit.client.0.smithi090.stdout:8/996: creat dc/f15f x:0 0 0 2024-04-26T04:42:29.866 INFO:tasks.workunit.client.0.smithi090.stdout:4/801: dwrite d3/f2c [0,4194304] 0 2024-04-26T04:42:29.866 INFO:tasks.workunit.client.0.smithi090.stdout:5/815: dwrite d4/d1a/d2a/d79/d90/fa8 [0,4194304] 0 2024-04-26T04:42:29.871 INFO:tasks.workunit.client.0.smithi090.stdout:4/802: mknod d3/db/d12/d20/d2d/d58/cfe 0 2024-04-26T04:42:29.872 INFO:tasks.workunit.client.0.smithi090.stdout:5/816: truncate d4/d1a/df/d36/f84 1256937 0 2024-04-26T04:42:29.873 INFO:tasks.workunit.client.0.smithi090.stdout:4/803: truncate d3/db/d12/d20/d2d/d58/d6a/f71 2824057 0 2024-04-26T04:42:29.875 INFO:tasks.workunit.client.0.smithi090.stdout:5/817: rename d4/lf9 to d4/d1a/d2a/d79/lff 0 2024-04-26T04:42:29.877 INFO:tasks.workunit.client.0.smithi090.stdout:5/818: dread d4/d1a/d45/f97 [0,4194304] 0 2024-04-26T04:42:29.877 INFO:tasks.workunit.client.0.smithi090.stdout:5/819: readlink d4/d1a/d1f/d53/d59/d5e/da0/lae 0 2024-04-26T04:42:29.878 INFO:tasks.workunit.client.0.smithi090.stdout:4/804: creat d3/db/d12/d99/fff x:0 0 0 2024-04-26T04:42:29.878 INFO:tasks.workunit.client.0.smithi090.stdout:4/805: chown d3/db/d12/f9a 45130206 1 2024-04-26T04:42:29.878 INFO:tasks.workunit.client.0.smithi090.stdout:4/806: chown d3/db/d12/d20/d2d/d58/d6a/cd3 330 1 2024-04-26T04:42:29.878 INFO:tasks.workunit.client.0.smithi090.stdout:5/820: mknod d4/d1a/d3f/c100 0 2024-04-26T04:42:29.879 INFO:tasks.workunit.client.0.smithi090.stdout:5/821: dread - d4/fd6 zero size 2024-04-26T04:42:29.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/807: creat d3/db/d12/d20/d2d/d2e/dda/f100 x:0 0 0 2024-04-26T04:42:29.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/808: chown d3/db/d12/d20/d2d/d2e/dda/f100 29664 1 2024-04-26T04:42:29.880 INFO:tasks.workunit.client.0.smithi090.stdout:4/809: write d3/db/d12/d20/d2d/d58/f63 [4755790,118892] 0 2024-04-26T04:42:29.881 INFO:tasks.workunit.client.0.smithi090.stdout:5/822: creat d4/d1a/df/d72/f101 x:0 0 0 2024-04-26T04:42:29.882 INFO:tasks.workunit.client.0.smithi090.stdout:4/810: mknod d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/c101 0 2024-04-26T04:42:29.883 INFO:tasks.workunit.client.0.smithi090.stdout:5/823: mknod d4/d1a/d2a/d79/c102 0 2024-04-26T04:42:29.884 INFO:tasks.workunit.client.0.smithi090.stdout:5/824: creat d4/d1a/d2a/d79/d90/f103 x:0 0 0 2024-04-26T04:42:29.891 INFO:tasks.workunit.client.0.smithi090.stdout:5/825: dread d4/d1a/f17 [0,4194304] 0 2024-04-26T04:42:29.892 INFO:tasks.workunit.client.0.smithi090.stdout:5/826: write d4/d1a/df/d4a/f89 [284462,45523] 0 2024-04-26T04:42:29.894 INFO:tasks.workunit.client.0.smithi090.stdout:5/827: rename d4/d1a/d2a/d79/d90/db3/la2 to d4/d1a/d1f/dc5/l104 0 2024-04-26T04:42:29.894 INFO:tasks.workunit.client.0.smithi090.stdout:5/828: mkdir d4/d1a/d2a/d79/d90/d49/d105 0 2024-04-26T04:42:29.899 INFO:tasks.workunit.client.0.smithi090.stdout:5/829: dread d4/d1a/d1f/d53/ff3 [0,4194304] 0 2024-04-26T04:42:29.900 INFO:tasks.workunit.client.0.smithi090.stdout:5/830: truncate d4/d1a/df/d72/f77 2625632 0 2024-04-26T04:42:29.902 INFO:tasks.workunit.client.0.smithi090.stdout:5/831: rename d4/d1a/d45/f74 to d4/d1a/d2a/d79/d90/d49/d105/f106 0 2024-04-26T04:42:29.905 INFO:tasks.workunit.client.0.smithi090.stdout:5/832: dread d4/d1a/d2a/d79/d90/d56/f6f [0,4194304] 0 2024-04-26T04:42:29.905 INFO:tasks.workunit.client.0.smithi090.stdout:5/833: creat d4/d1a/d2a/d79/d90/db3/d9b/f107 x:0 0 0 2024-04-26T04:42:29.913 INFO:tasks.workunit.client.0.smithi090.stdout:5/834: dread d4/d1a/d2a/d79/d90/fa8 [0,4194304] 0 2024-04-26T04:42:29.915 INFO:tasks.workunit.client.0.smithi090.stdout:5/835: mkdir d4/d1a/d1f/dc5/dca/d108 0 2024-04-26T04:42:29.921 INFO:tasks.workunit.client.0.smithi090.stdout:5/836: dread d4/d1a/df/d36/f84 [0,4194304] 0 2024-04-26T04:42:29.921 INFO:tasks.workunit.client.0.smithi090.stdout:5/837: read d4/d1a/d1f/d53/f76 [1072324,16460] 0 2024-04-26T04:42:29.938 INFO:tasks.workunit.client.0.smithi090.stdout:0/849: sync 2024-04-26T04:42:29.938 INFO:tasks.workunit.client.0.smithi090.stdout:1/801: sync 2024-04-26T04:42:29.938 INFO:tasks.workunit.client.0.smithi090.stdout:0/850: chown d0/de/f71 89708 1 2024-04-26T04:42:29.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/838: dwrite d4/d1a/f19 [0,4194304] 0 2024-04-26T04:42:29.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/839: write d4/d1a/d2a/d79/fc8 [404288,109424] 0 2024-04-26T04:42:29.938 INFO:tasks.workunit.client.0.smithi090.stdout:5/840: write d4/d1a/df/d36/f84 [1067916,27436] 0 2024-04-26T04:42:29.939 INFO:tasks.workunit.client.0.smithi090.stdout:1/802: truncate d0/f94 1615618 0 2024-04-26T04:42:29.940 INFO:tasks.workunit.client.0.smithi090.stdout:0/851: symlink d0/de/d103/l125 0 2024-04-26T04:42:29.941 INFO:tasks.workunit.client.0.smithi090.stdout:1/803: creat d0/d55/dda/d7/d4a/d75/fff x:0 0 0 2024-04-26T04:42:29.941 INFO:tasks.workunit.client.0.smithi090.stdout:1/804: chown d0/d55/dda/d62/l72 445702 1 2024-04-26T04:42:29.942 INFO:tasks.workunit.client.0.smithi090.stdout:0/852: unlink d0/df/c37 0 2024-04-26T04:42:29.942 INFO:tasks.workunit.client.0.smithi090.stdout:0/853: write d0/d4b/d5d/d5c/d8a/f3e [3939089,10010] 0 2024-04-26T04:42:29.943 INFO:tasks.workunit.client.0.smithi090.stdout:1/805: symlink d0/d55/dda/d62/l100 0 2024-04-26T04:42:29.944 INFO:tasks.workunit.client.0.smithi090.stdout:0/854: symlink d0/d4b/d6d/l126 0 2024-04-26T04:42:29.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/855: unlink d0/d4b/d5d/c51 0 2024-04-26T04:42:29.949 INFO:tasks.workunit.client.0.smithi090.stdout:0/856: chown d0/d4b/d5d/d5c/d8a/d45/d53/faf 290 1 2024-04-26T04:42:29.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/841: dread d4/d1a/df/d36/fe2 [0,4194304] 0 2024-04-26T04:42:29.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/842: stat d4/d1a/df/d36/f5c 0 2024-04-26T04:42:29.950 INFO:tasks.workunit.client.0.smithi090.stdout:5/843: creat d4/d1a/df/d36/d8e/f109 x:0 0 0 2024-04-26T04:42:29.951 INFO:tasks.workunit.client.0.smithi090.stdout:0/857: mknod d0/d4b/d5d/d48/dc1/c127 0 2024-04-26T04:42:29.951 INFO:tasks.workunit.client.0.smithi090.stdout:0/858: chown d0/d4b/fbd 1180 1 2024-04-26T04:42:29.953 INFO:tasks.workunit.client.0.smithi090.stdout:0/859: mknod d0/d4b/d5d/d48/dc1/c128 0 2024-04-26T04:42:29.955 INFO:tasks.workunit.client.0.smithi090.stdout:0/860: rename d0/d4b/d5d/d48/fca to d0/d4b/d5d/d5c/d8a/d64/f129 0 2024-04-26T04:42:29.957 INFO:tasks.workunit.client.0.smithi090.stdout:0/861: creat d0/d4b/d5d/d5c/d8a/d45/d53/f12a x:0 0 0 2024-04-26T04:42:29.957 INFO:tasks.workunit.client.0.smithi090.stdout:0/862: dread - d0/d4b/d5d/d5c/f9c zero size 2024-04-26T04:42:29.957 INFO:tasks.workunit.client.0.smithi090.stdout:0/863: stat d0/de/f90 0 2024-04-26T04:42:29.961 INFO:tasks.workunit.client.0.smithi090.stdout:1/806: dwrite d0/d55/dda/d7/d69/f71 [0,4194304] 0 2024-04-26T04:42:29.967 INFO:tasks.workunit.client.0.smithi090.stdout:5/844: dwrite d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 [0,4194304] 0 2024-04-26T04:42:29.978 INFO:tasks.workunit.client.0.smithi090.stdout:0/864: write d0/de/f89 [458944,24070] 0 2024-04-26T04:42:29.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/865: creat d0/d4b/d5d/d5c/d117/d120/d123/f12b x:0 0 0 2024-04-26T04:42:29.979 INFO:tasks.workunit.client.0.smithi090.stdout:0/866: dread - d0/d4b/d5d/d5c/f91 zero size 2024-04-26T04:42:29.980 INFO:tasks.workunit.client.0.smithi090.stdout:0/867: mkdir d0/d4b/d5d/d5c/d8a/d45/dcf/d12c 0 2024-04-26T04:42:29.980 INFO:tasks.workunit.client.0.smithi090.stdout:0/868: chown d0/d4b/d5d/f6a 60635408 1 2024-04-26T04:42:29.981 INFO:tasks.workunit.client.0.smithi090.stdout:1/807: dwrite d0/d55/dda/d7/d4a/d75/fff [0,4194304] 0 2024-04-26T04:42:29.981 INFO:tasks.workunit.client.0.smithi090.stdout:1/808: fsync d0/d55/dda/d43/f54 0 2024-04-26T04:42:29.981 INFO:tasks.workunit.client.0.smithi090.stdout:1/809: creat d0/d55/dda/d7/d4a/d75/f101 x:0 0 0 2024-04-26T04:42:29.982 INFO:tasks.workunit.client.0.smithi090.stdout:0/869: write d0/df/fd4 [1324486,115618] 0 2024-04-26T04:42:29.982 INFO:tasks.workunit.client.0.smithi090.stdout:0/870: write d0/fdc [1359315,85202] 0 2024-04-26T04:42:29.982 INFO:tasks.workunit.client.0.smithi090.stdout:0/871: fsync d0/de/f2c 0 2024-04-26T04:42:29.983 INFO:tasks.workunit.client.0.smithi090.stdout:0/872: write d0/d4b/d5d/d5c/d8a/d45/d93/fd5 [4689652,45193] 0 2024-04-26T04:42:29.983 INFO:tasks.workunit.client.0.smithi090.stdout:0/873: truncate d0/d4b/d5d/d5c/d117/d120/d123/f12b 1013511 0 2024-04-26T04:42:29.984 INFO:tasks.workunit.client.0.smithi090.stdout:5/845: dwrite d4/d1a/d2a/d79/d90/d56/f75 [4194304,4194304] 0 2024-04-26T04:42:29.984 INFO:tasks.workunit.client.0.smithi090.stdout:1/810: mkdir d0/d55/dda/d7/d4a/d102 0 2024-04-26T04:42:29.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/874: stat d0/d4b/d5d/d5c/d8a/d64/c75 0 2024-04-26T04:42:29.986 INFO:tasks.workunit.client.0.smithi090.stdout:0/875: readlink d0/df/l1a 0 2024-04-26T04:42:29.988 INFO:tasks.workunit.client.0.smithi090.stdout:5/846: mkdir d4/d1a/d2a/d79/d10a 0 2024-04-26T04:42:29.992 INFO:tasks.workunit.client.0.smithi090.stdout:0/876: mkdir d0/de/d12d 0 2024-04-26T04:42:29.993 INFO:tasks.workunit.client.0.smithi090.stdout:5/847: rmdir d4/d1a/d1f/d53/d59/d5e 39 2024-04-26T04:42:29.994 INFO:tasks.workunit.client.0.smithi090.stdout:1/811: read d0/d55/dda/d7/d4a/d75/fff [1464771,121198] 0 2024-04-26T04:42:29.996 INFO:tasks.workunit.client.0.smithi090.stdout:0/877: creat d0/d106/f12e x:0 0 0 2024-04-26T04:42:29.996 INFO:tasks.workunit.client.0.smithi090.stdout:0/878: truncate d0/d4b/d5d/d5c/d8a/d45/d53/fba 355878 0 2024-04-26T04:42:29.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/812: rename d0/d55/dda/d7/d17/d22/f74 to d0/d55/dda/d7/d4a/d75/d80/d58/d97/f103 0 2024-04-26T04:42:29.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/813: fsync d0/d55/dda/d7/d17/d22/f89 0 2024-04-26T04:42:29.999 INFO:tasks.workunit.client.0.smithi090.stdout:1/814: chown d0/d55/dda/d7/d4a 0 1 2024-04-26T04:42:30.001 INFO:tasks.workunit.client.0.smithi090.stdout:0/879: mkdir d0/d4b/d6d/d12f 0 2024-04-26T04:42:30.001 INFO:tasks.workunit.client.0.smithi090.stdout:0/880: write d0/d4b/d5d/d5c/d8a/f44 [1767766,43773] 0 2024-04-26T04:42:30.004 INFO:tasks.workunit.client.0.smithi090.stdout:0/881: mkdir d0/d4b/d5d/d5c/d8a/d45/dcf/d12c/d130 0 2024-04-26T04:42:30.006 INFO:tasks.workunit.client.0.smithi090.stdout:0/882: symlink d0/d4b/l131 0 2024-04-26T04:42:30.007 INFO:tasks.workunit.client.0.smithi090.stdout:0/883: chown d0/d4b/db3/d122/l124 29325 1 2024-04-26T04:42:30.007 INFO:tasks.workunit.client.0.smithi090.stdout:0/884: chown d0/df/fd4 8055 1 2024-04-26T04:42:30.007 INFO:tasks.workunit.client.0.smithi090.stdout:0/885: truncate d0/d4b/d5d/d5c/f111 629302 0 2024-04-26T04:42:30.007 INFO:tasks.workunit.client.0.smithi090.stdout:0/886: write d0/d4b/ff8 [9752,65099] 0 2024-04-26T04:42:30.008 INFO:tasks.workunit.client.0.smithi090.stdout:0/887: link d0/d4b/d5d/l1f d0/de/d103/d105/l132 0 2024-04-26T04:42:30.009 INFO:tasks.workunit.client.0.smithi090.stdout:0/888: dread d0/d4b/d5d/d5c/f111 [0,4194304] 0 2024-04-26T04:42:30.010 INFO:tasks.workunit.client.0.smithi090.stdout:0/889: write d0/d4b/d5d/d5c/d8a/d45/d53/f12a [225504,279] 0 2024-04-26T04:42:30.010 INFO:tasks.workunit.client.0.smithi090.stdout:0/890: write d0/d4b/d5d/d5c/d8a/ff1 [93713,59582] 0 2024-04-26T04:42:30.010 INFO:tasks.workunit.client.0.smithi090.stdout:1/815: dread d0/d55/dda/d7/d4b/f4c [0,4194304] 0 2024-04-26T04:42:30.011 INFO:tasks.workunit.client.0.smithi090.stdout:0/891: symlink d0/d4b/d5d/d5c/d8a/d7c/l133 0 2024-04-26T04:42:30.011 INFO:tasks.workunit.client.0.smithi090.stdout:5/848: dwrite d4/d1a/d2a/fd0 [0,4194304] 0 2024-04-26T04:42:30.012 INFO:tasks.workunit.client.0.smithi090.stdout:1/816: mkdir d0/d55/dda/d7/d17/d22/d93/db0/dfc/d104 0 2024-04-26T04:42:30.014 INFO:tasks.workunit.client.0.smithi090.stdout:5/849: mkdir d4/d1a/d2a/d10b 0 2024-04-26T04:42:30.014 INFO:tasks.workunit.client.0.smithi090.stdout:0/892: dread d0/d4b/d5d/d5c/d8a/d64/f88 [0,4194304] 0 2024-04-26T04:42:30.019 INFO:tasks.workunit.client.0.smithi090.stdout:1/817: creat d0/d55/dda/d7/d4a/f105 x:0 0 0 2024-04-26T04:42:30.021 INFO:tasks.workunit.client.0.smithi090.stdout:0/893: getdents d0/d4b/d5d/d5c/d117/d120/d123 0 2024-04-26T04:42:30.022 INFO:tasks.workunit.client.0.smithi090.stdout:5/850: dread d4/d1a/df/f28 [4194304,4194304] 0 2024-04-26T04:42:30.022 INFO:tasks.workunit.client.0.smithi090.stdout:5/851: stat d4/cd1 0 2024-04-26T04:42:30.022 INFO:tasks.workunit.client.0.smithi090.stdout:5/852: dread - d4/d1a/d2a/d79/d90/f103 zero size 2024-04-26T04:42:30.023 INFO:tasks.workunit.client.0.smithi090.stdout:0/894: mkdir d0/de/d12d/d134 0 2024-04-26T04:42:30.023 INFO:tasks.workunit.client.0.smithi090.stdout:0/895: chown d0/de/d103/c9f 3990 1 2024-04-26T04:42:30.025 INFO:tasks.workunit.client.0.smithi090.stdout:0/896: dread d0/d4b/d5d/d5c/d8a/ff1 [0,4194304] 0 2024-04-26T04:42:30.027 INFO:tasks.workunit.client.0.smithi090.stdout:0/897: mknod d0/d4b/db3/dcc/c135 0 2024-04-26T04:42:30.028 INFO:tasks.workunit.client.0.smithi090.stdout:0/898: truncate d0/d4b/d5d/d5c/d8a/d45/d53/fa3 568546 0 2024-04-26T04:42:30.028 INFO:tasks.workunit.client.0.smithi090.stdout:0/899: truncate d0/d4b/d5d/fb1 733278 0 2024-04-26T04:42:30.030 INFO:tasks.workunit.client.0.smithi090.stdout:0/900: mknod d0/d4b/d5d/d5c/d8a/d64/c136 0 2024-04-26T04:42:30.030 INFO:tasks.workunit.client.0.smithi090.stdout:0/901: fdatasync d0/d4b/d5d/d48/dc1/f11a 0 2024-04-26T04:42:30.030 INFO:tasks.workunit.client.0.smithi090.stdout:0/902: stat d0/d4b/d5d/d48/d100/c113 0 2024-04-26T04:42:30.030 INFO:tasks.workunit.client.0.smithi090.stdout:0/903: fdatasync d0/d4b/d5d/d5c/d8a/d45/d53/fba 0 2024-04-26T04:42:30.030 INFO:tasks.workunit.client.0.smithi090.stdout:5/853: dread d4/d1a/d45/db8/f9e [0,4194304] 0 2024-04-26T04:42:30.030 INFO:tasks.workunit.client.0.smithi090.stdout:5/854: creat d4/d1a/d2a/d79/d90/d56/d71/deb/f10c x:0 0 0 2024-04-26T04:42:30.033 INFO:tasks.workunit.client.0.smithi090.stdout:5/855: rename d4/d1a/d3f/l2e to d4/d1a/d2a/d79/d90/d56/d80/l10d 0 2024-04-26T04:42:30.064 INFO:tasks.workunit.client.0.smithi090.stdout:5/856: dwrite d4/d1a/df/d4a/f7a [0,4194304] 0 2024-04-26T04:42:30.064 INFO:tasks.workunit.client.0.smithi090.stdout:0/904: dwrite d0/d4b/d5d/d5c/d8a/d45/d93/faa [0,4194304] 0 2024-04-26T04:42:30.064 INFO:tasks.workunit.client.0.smithi090.stdout:0/905: readlink d0/d4b/l131 0 2024-04-26T04:42:30.068 INFO:tasks.workunit.client.0.smithi090.stdout:0/906: mkdir d0/d4b/d5d/d48/d137 0 2024-04-26T04:42:30.068 INFO:tasks.workunit.client.0.smithi090.stdout:0/907: write d0/d4b/d5d/d5c/d8a/d7c/f95 [122988,46580] 0 2024-04-26T04:42:30.068 INFO:tasks.workunit.client.0.smithi090.stdout:0/908: chown d0/d4b/d5d/d5c/d8a/f3e 441 1 2024-04-26T04:42:30.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/857: rmdir d4/d1a/d1f/d53 39 2024-04-26T04:42:30.069 INFO:tasks.workunit.client.0.smithi090.stdout:5/858: write d4/d1a/d2a/f83 [964299,129975] 0 2024-04-26T04:42:30.070 INFO:tasks.workunit.client.0.smithi090.stdout:0/909: symlink d0/d4b/d5d/d48/d7d/l138 0 2024-04-26T04:42:30.070 INFO:tasks.workunit.client.0.smithi090.stdout:0/910: chown d0/d4b/d5d/d5c/d8a/d45/d53/l5b 108419 1 2024-04-26T04:42:30.071 INFO:tasks.workunit.client.0.smithi090.stdout:0/911: write d0/d4b/f114 [4950404,119056] 0 2024-04-26T04:42:30.072 INFO:tasks.workunit.client.0.smithi090.stdout:5/859: creat d4/d1a/d1f/dc5/f10e x:0 0 0 2024-04-26T04:42:30.073 INFO:tasks.workunit.client.0.smithi090.stdout:0/912: rmdir d0/d4b/db3/dcc/d119 0 2024-04-26T04:42:30.074 INFO:tasks.workunit.client.0.smithi090.stdout:5/860: rmdir d4/d1a/d2a/d79/d90/d49 39 2024-04-26T04:42:30.074 INFO:tasks.workunit.client.0.smithi090.stdout:5/861: dread - d4/d1a/d2a/d79/d90/d56/d71/deb/ff8 zero size 2024-04-26T04:42:30.076 INFO:tasks.workunit.client.0.smithi090.stdout:2/833: sync 2024-04-26T04:42:30.077 INFO:tasks.workunit.client.0.smithi090.stdout:2/834: link db/d11/d14/d27/d2f/d42/d5c/l97 db/dc/l116 0 2024-04-26T04:42:30.079 INFO:tasks.workunit.client.0.smithi090.stdout:2/835: mknod db/d11/d14/d25/d2d/d48/c117 0 2024-04-26T04:42:30.083 INFO:tasks.workunit.client.0.smithi090.stdout:2/836: symlink db/d11/d14/d27/d28/dd1/ddf/l118 0 2024-04-26T04:42:30.084 INFO:tasks.workunit.client.0.smithi090.stdout:2/837: mkdir db/d11/d14/d1b/d119 0 2024-04-26T04:42:30.102 INFO:tasks.workunit.client.0.smithi090.stdout:5/862: dwrite d4/d1a/df/d72/f101 [0,4194304] 0 2024-04-26T04:42:30.102 INFO:tasks.workunit.client.0.smithi090.stdout:5/863: fsync d4/d1a/df/d4a/f89 0 2024-04-26T04:42:30.102 INFO:tasks.workunit.client.0.smithi090.stdout:0/913: dwrite d0/df/f6b [0,4194304] 0 2024-04-26T04:42:30.102 INFO:tasks.workunit.client.0.smithi090.stdout:5/864: creat d4/d1a/d1f/dc5/dca/f10f x:0 0 0 2024-04-26T04:42:30.103 INFO:tasks.workunit.client.0.smithi090.stdout:2/838: dwrite db/d11/d14/d1b/f43 [0,4194304] 0 2024-04-26T04:42:30.103 INFO:tasks.workunit.client.0.smithi090.stdout:2/839: read - db/d11/d14/d27/d2f/ff6 zero size 2024-04-26T04:42:30.103 INFO:tasks.workunit.client.0.smithi090.stdout:2/840: fsync db/d11/d14/d25/f9a 0 2024-04-26T04:42:30.104 INFO:tasks.workunit.client.0.smithi090.stdout:0/914: rename d0/df/le4 to d0/d4b/d6d/d12f/l139 0 2024-04-26T04:42:30.106 INFO:tasks.workunit.client.0.smithi090.stdout:5/865: symlink d4/d1a/df/d36/d8e/de3/l110 0 2024-04-26T04:42:30.106 INFO:tasks.workunit.client.0.smithi090.stdout:5/866: chown d4/d1a/d2a/d79/d90/fc4 160 1 2024-04-26T04:42:30.113 INFO:tasks.workunit.client.0.smithi090.stdout:5/867: rename d4/d1a/df/d72/f101 to d4/d1a/d3f/f111 0 2024-04-26T04:42:30.121 INFO:tasks.workunit.client.0.smithi090.stdout:2/841: dwrite db/d11/d14/d27/d28/dd1/dfe/f10c [0,4194304] 0 2024-04-26T04:42:30.124 INFO:tasks.workunit.client.0.smithi090.stdout:2/842: mkdir db/d11/d14/d63/d11a 0 2024-04-26T04:42:30.125 INFO:tasks.workunit.client.0.smithi090.stdout:2/843: rename db/d11/d14/d1b/d20/lf0 to db/d11/d14/d27/d28/dd1/dfe/d71/l11b 0 2024-04-26T04:42:30.131 INFO:tasks.workunit.client.0.smithi090.stdout:0/915: dwrite d0/df/f67 [0,4194304] 0 2024-04-26T04:42:30.133 INFO:tasks.workunit.client.0.smithi090.stdout:0/916: dread d0/de/f89 [0,4194304] 0 2024-04-26T04:42:30.133 INFO:tasks.workunit.client.0.smithi090.stdout:5/868: dwrite d4/d1a/f78 [4194304,4194304] 0 2024-04-26T04:42:30.133 INFO:tasks.workunit.client.0.smithi090.stdout:5/869: chown d4/d1a/d2a/d79/d90/d56/d80/cb0 745 1 2024-04-26T04:42:30.137 INFO:tasks.workunit.client.0.smithi090.stdout:5/870: dread d4/d1a/d2a/d79/d90/f96 [0,4194304] 0 2024-04-26T04:42:30.139 INFO:tasks.workunit.client.0.smithi090.stdout:5/871: mkdir d4/d1a/d2a/d79/d90/d112 0 2024-04-26T04:42:30.139 INFO:tasks.workunit.client.0.smithi090.stdout:5/872: write d4/d1a/df/d72/dc0/ff4 [318020,42459] 0 2024-04-26T04:42:30.141 INFO:tasks.workunit.client.0.smithi090.stdout:2/844: dread db/d11/d14/fc0 [0,4194304] 0 2024-04-26T04:42:30.141 INFO:tasks.workunit.client.0.smithi090.stdout:2/845: truncate db/d11/d14/d27/d2f/d42/fd9 876441 0 2024-04-26T04:42:30.156 INFO:tasks.workunit.client.0.smithi090.stdout:0/917: dwrite d0/d4b/d6d/f9a [0,4194304] 0 2024-04-26T04:42:30.158 INFO:tasks.workunit.client.0.smithi090.stdout:0/918: link d0/d4b/d5d/d48/d100/c113 d0/d4b/d6d/d12f/c13a 0 2024-04-26T04:42:30.158 INFO:tasks.workunit.client.0.smithi090.stdout:0/919: mkdir d0/d4b/d5d/d5c/d8a/d45/dcf/d12c/d13b 0 2024-04-26T04:42:30.160 INFO:tasks.workunit.client.0.smithi090.stdout:0/920: getdents d0/d4b/d5d/d5c/d8a/d45/d53 0 2024-04-26T04:42:30.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/921: rename d0/d4b/d5d/d5c/d8a/f3e to d0/df/f13c 0 2024-04-26T04:42:30.162 INFO:tasks.workunit.client.0.smithi090.stdout:0/922: read - d0/d4b/d5d/f6a zero size 2024-04-26T04:42:30.164 INFO:tasks.workunit.client.0.smithi090.stdout:5/873: dwrite d4/d1a/d2a/d79/d90/d49/f7d [0,4194304] 0 2024-04-26T04:42:30.164 INFO:tasks.workunit.client.0.smithi090.stdout:0/923: mkdir d0/d4b/d6d/d13d 0 2024-04-26T04:42:30.164 INFO:tasks.workunit.client.0.smithi090.stdout:2/846: dwrite db/d11/d14/d1b/d20/fb8 [0,4194304] 0 2024-04-26T04:42:30.165 INFO:tasks.workunit.client.0.smithi090.stdout:2/847: fsync db/d11/d14/d25/d86/fc3 0 2024-04-26T04:42:30.167 INFO:tasks.workunit.client.0.smithi090.stdout:5/874: link d4/d1a/d45/f97 d4/d1a/d2a/d79/d10a/f113 0 2024-04-26T04:42:30.169 INFO:tasks.workunit.client.0.smithi090.stdout:0/924: rename d0/df/l74 to d0/d4b/db3/dcc/l13e 0 2024-04-26T04:42:30.184 INFO:tasks.workunit.client.0.smithi090.stdout:0/925: creat d0/de/d103/d105/f13f x:0 0 0 2024-04-26T04:42:30.184 INFO:tasks.workunit.client.0.smithi090.stdout:5/875: dwrite d4/d1a/d2a/fd0 [0,4194304] 0 2024-04-26T04:42:30.185 INFO:tasks.workunit.client.0.smithi090.stdout:2/848: getdents db/d11/d14/d25/d86 0 2024-04-26T04:42:30.185 INFO:tasks.workunit.client.0.smithi090.stdout:2/849: fdatasync db/d11/d14/d1b/d20/ff8 0 2024-04-26T04:42:30.185 INFO:tasks.workunit.client.0.smithi090.stdout:2/850: creat db/d11/d14/d27/d28/dd1/dfe/d8b/f11c x:0 0 0 2024-04-26T04:42:30.187 INFO:tasks.workunit.client.0.smithi090.stdout:0/926: rmdir d0/d4b/d5d/d48/d137 0 2024-04-26T04:42:30.187 INFO:tasks.workunit.client.0.smithi090.stdout:0/927: dread - d0/d4b/d5d/d48/f11b zero size 2024-04-26T04:42:30.187 INFO:tasks.workunit.client.0.smithi090.stdout:0/928: creat d0/d4b/d5d/d5c/d8a/d45/d107/f140 x:0 0 0 2024-04-26T04:42:30.187 INFO:tasks.workunit.client.0.smithi090.stdout:0/929: fdatasync d0/d4b/d5d/f6a 0 2024-04-26T04:42:30.189 INFO:tasks.workunit.client.0.smithi090.stdout:5/876: link d4/d1a/d2a/d79/d90/ccd d4/d1a/d1f/c114 0 2024-04-26T04:42:30.194 INFO:tasks.workunit.client.0.smithi090.stdout:2/851: write db/dc/f3c [4758846,93181] 0 2024-04-26T04:42:30.199 INFO:tasks.workunit.client.0.smithi090.stdout:5/877: dread d4/fe [0,4194304] 0 2024-04-26T04:42:30.199 INFO:tasks.workunit.client.0.smithi090.stdout:5/878: creat d4/d1a/d1f/f115 x:0 0 0 2024-04-26T04:42:30.199 INFO:tasks.workunit.client.0.smithi090.stdout:5/879: readlink d4/d1a/d2a/d79/d90/db3/ld3 0 2024-04-26T04:42:30.199 INFO:tasks.workunit.client.0.smithi090.stdout:5/880: fdatasync d4/d1a/df/fa7 0 2024-04-26T04:42:30.199 INFO:tasks.workunit.client.0.smithi090.stdout:5/881: chown d4/d1a/df 0 1 2024-04-26T04:42:30.214 INFO:tasks.workunit.client.0.smithi090.stdout:7/906: sync 2024-04-26T04:42:30.214 INFO:tasks.workunit.client.0.smithi090.stdout:3/768: sync 2024-04-26T04:42:30.216 INFO:tasks.workunit.client.0.smithi090.stdout:7/907: getdents d0/d109/d65/d11c 0 2024-04-26T04:42:30.216 INFO:tasks.workunit.client.0.smithi090.stdout:7/908: stat d0/db/d15/d29/l8a 0 2024-04-26T04:42:30.218 INFO:tasks.workunit.client.0.smithi090.stdout:7/909: write d0/db/d53/d5e/fa9 [596702,55522] 0 2024-04-26T04:42:30.218 INFO:tasks.workunit.client.0.smithi090.stdout:7/910: read d0/db/d15/d29/f14 [2173062,106107] 0 2024-04-26T04:42:30.218 INFO:tasks.workunit.client.0.smithi090.stdout:3/769: dread d1/d9/f38 [0,4194304] 0 2024-04-26T04:42:30.219 INFO:tasks.workunit.client.0.smithi090.stdout:2/852: dwrite db/d11/d14/fee [0,4194304] 0 2024-04-26T04:42:30.219 INFO:tasks.workunit.client.0.smithi090.stdout:7/911: getdents d0/d109/d65 0 2024-04-26T04:42:30.219 INFO:tasks.workunit.client.0.smithi090.stdout:7/912: fdatasync d0/db/d53/db7/d4f/f10f 0 2024-04-26T04:42:30.219 INFO:tasks.workunit.client.0.smithi090.stdout:7/913: chown d0/d1/da3/cf7 137 1 2024-04-26T04:42:30.220 INFO:tasks.workunit.client.0.smithi090.stdout:3/770: truncate d1/d17/d36/d33/d43/fe2 392577 0 2024-04-26T04:42:30.222 INFO:tasks.workunit.client.0.smithi090.stdout:5/882: dwrite d4/d1a/d1f/dc5/dca/ff1 [0,4194304] 0 2024-04-26T04:42:30.222 INFO:tasks.workunit.client.0.smithi090.stdout:2/853: symlink db/dc/d6f/dd8/l11d 0 2024-04-26T04:42:30.223 INFO:tasks.workunit.client.0.smithi090.stdout:3/771: mknod d1/d17/d36/d33/d68/dae/dca/dcf/d106/c109 0 2024-04-26T04:42:30.227 INFO:tasks.workunit.client.0.smithi090.stdout:5/883: creat d4/d1a/d2a/d79/d90/d112/f116 x:0 0 0 2024-04-26T04:42:30.229 INFO:tasks.workunit.client.0.smithi090.stdout:5/884: link d4/d1a/d3f/l4b d4/d1a/d2a/d79/d90/l117 0 2024-04-26T04:42:30.229 INFO:tasks.workunit.client.0.smithi090.stdout:5/885: stat d4/d1a/df/d36/d8e/de3/ffd 0 2024-04-26T04:42:30.232 INFO:tasks.workunit.client.0.smithi090.stdout:2/854: read db/d11/d14/d25/d2d/f36 [2998645,43975] 0 2024-04-26T04:42:30.232 INFO:tasks.workunit.client.0.smithi090.stdout:2/855: fsync db/d11/d14/d25/fe5 0 2024-04-26T04:42:30.232 INFO:tasks.workunit.client.0.smithi090.stdout:5/886: dread d4/d1a/df/d4a/fa4 [0,4194304] 0 2024-04-26T04:42:30.233 INFO:tasks.workunit.client.0.smithi090.stdout:5/887: mkdir d4/d1a/d2a/d79/d90/d118 0 2024-04-26T04:42:30.235 INFO:tasks.workunit.client.0.smithi090.stdout:3/772: write d1/d9/fa [4022630,9435] 0 2024-04-26T04:42:30.236 INFO:tasks.workunit.client.0.smithi090.stdout:7/914: dwrite d0/d1/d4/d3e/dd3/de2/fea [0,4194304] 0 2024-04-26T04:42:30.236 INFO:tasks.workunit.client.0.smithi090.stdout:7/915: fsync d0/d109/d65/dbb/dc0/fdf 0 2024-04-26T04:42:30.237 INFO:tasks.workunit.client.0.smithi090.stdout:2/856: read db/d11/d14/fc0 [931764,88398] 0 2024-04-26T04:42:30.237 INFO:tasks.workunit.client.0.smithi090.stdout:2/857: creat db/d11/d14/d25/f11e x:0 0 0 2024-04-26T04:42:30.246 INFO:tasks.workunit.client.0.smithi090.stdout:3/773: dread d1/d17/f2f [0,4194304] 0 2024-04-26T04:42:30.247 INFO:tasks.workunit.client.0.smithi090.stdout:2/858: dread db/dc/f3c [0,4194304] 0 2024-04-26T04:42:30.247 INFO:tasks.workunit.client.0.smithi090.stdout:2/859: chown db/d11/d14/fc0 1904363895 1 2024-04-26T04:42:30.248 INFO:tasks.workunit.client.0.smithi090.stdout:3/774: mknod d1/d17/d36/d33/d68/dae/dca/d1b/d58/da4/c10a 0 2024-04-26T04:42:30.249 INFO:tasks.workunit.client.0.smithi090.stdout:5/888: dread d4/d1a/d45/f3c [0,4194304] 0 2024-04-26T04:42:30.249 INFO:tasks.workunit.client.0.smithi090.stdout:3/775: creat d1/d17/d36/d33/d68/dae/dca/f10b x:0 0 0 2024-04-26T04:42:30.251 INFO:tasks.workunit.client.0.smithi090.stdout:5/889: dread d4/d1a/df/d36/f5c [0,4194304] 0 2024-04-26T04:42:30.251 INFO:tasks.workunit.client.0.smithi090.stdout:3/776: unlink d1/d17/d36/d33/d68/dae/dca/dcf/d106/fdd 0 2024-04-26T04:42:30.251 INFO:tasks.workunit.client.0.smithi090.stdout:5/890: chown d4/d1a/d2a/d79/d90/db3/ld7 30 1 2024-04-26T04:42:30.252 INFO:tasks.workunit.client.0.smithi090.stdout:3/777: chown d1/d17/d36/d33/d68/dae/dca/d1b/l4d 229937 1 2024-04-26T04:42:30.252 INFO:tasks.workunit.client.0.smithi090.stdout:5/891: write d4/d1a/d45/f97 [368742,44951] 0 2024-04-26T04:42:30.252 INFO:tasks.workunit.client.0.smithi090.stdout:5/892: readlink d4/d1a/d2a/d79/d90/d49/l5f 0 2024-04-26T04:42:30.252 INFO:tasks.workunit.client.0.smithi090.stdout:5/893: stat d4/d1a/d45/c35 0 2024-04-26T04:42:30.252 INFO:tasks.workunit.client.0.smithi090.stdout:5/894: mknod d4/d1a/d2a/d79/d90/d49/c119 0 2024-04-26T04:42:30.252 INFO:tasks.workunit.client.0.smithi090.stdout:5/895: dread - d4/d1a/d45/fa6 zero size 2024-04-26T04:42:30.256 INFO:tasks.workunit.client.0.smithi090.stdout:7/916: dwrite d0/d109/d65/db2/fcd [0,4194304] 0 2024-04-26T04:42:30.256 INFO:tasks.workunit.client.0.smithi090.stdout:7/917: dread - d0/db/d15/fe9 zero size 2024-04-26T04:42:30.260 INFO:tasks.workunit.client.0.smithi090.stdout:3/778: dread d1/d17/d36/d33/d68/dae/dca/f21 [0,4194304] 0 2024-04-26T04:42:30.260 INFO:tasks.workunit.client.0.smithi090.stdout:3/779: dread - d1/d17/d61/d9d/da7/ff4 zero size 2024-04-26T04:42:30.265 INFO:tasks.workunit.client.0.smithi090.stdout:7/918: read d0/f45 [4114611,2958] 0 2024-04-26T04:42:30.265 INFO:tasks.workunit.client.0.smithi090.stdout:7/919: truncate d0/db/d15/f7e 1336200 0 2024-04-26T04:42:30.268 INFO:tasks.workunit.client.0.smithi090.stdout:7/920: creat d0/db/df/d4d/d64/df2/f140 x:0 0 0 2024-04-26T04:42:30.268 INFO:tasks.workunit.client.0.smithi090.stdout:7/921: read - d0/db/d53/f115 zero size 2024-04-26T04:42:30.270 INFO:tasks.workunit.client.0.smithi090.stdout:7/922: rmdir d0/db/df/d4d/d64/db0/dd6 39 2024-04-26T04:42:30.276 INFO:tasks.workunit.client.0.smithi090.stdout:7/923: rmdir d0/d109/d4a 39 2024-04-26T04:42:30.276 INFO:tasks.workunit.client.0.smithi090.stdout:7/924: write d0/d109/d65/dbb/dc0/fdf [1321206,25378] 0 2024-04-26T04:42:30.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/896: dwrite d4/d1a/d2a/f41 [0,4194304] 0 2024-04-26T04:42:30.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/897: dread - d4/d1a/d2a/d79/d90/d112/f116 zero size 2024-04-26T04:42:30.278 INFO:tasks.workunit.client.0.smithi090.stdout:5/898: dread - d4/d1a/d2a/d79/fcf zero size 2024-04-26T04:42:30.282 INFO:tasks.workunit.client.0.smithi090.stdout:5/899: dread d4/f88 [0,4194304] 0 2024-04-26T04:42:30.312 INFO:tasks.workunit.client.0.smithi090.stdout:7/925: dwrite d0/db/d53/f17 [0,4194304] 0 2024-04-26T04:42:30.312 INFO:tasks.workunit.client.0.smithi090.stdout:7/926: dread - d0/db/de6/f136 zero size 2024-04-26T04:42:30.313 INFO:tasks.workunit.client.0.smithi090.stdout:5/900: dwrite d4/d1a/d45/f4f [0,4194304] 0 2024-04-26T04:42:30.317 INFO:tasks.workunit.client.0.smithi090.stdout:5/901: symlink d4/d1a/d2a/d79/d90/d49/l11a 0 2024-04-26T04:42:30.319 INFO:tasks.workunit.client.0.smithi090.stdout:5/902: truncate d4/d1a/d1f/d53/d59/d5e/f6a 2516650 0 2024-04-26T04:42:30.320 INFO:tasks.workunit.client.0.smithi090.stdout:5/903: stat d4/dab 0 2024-04-26T04:42:30.320 INFO:tasks.workunit.client.0.smithi090.stdout:5/904: write d4/d1a/d2a/d79/d90/d56/d71/d91/fe9 [1028130,102945] 0 2024-04-26T04:42:30.341 INFO:tasks.workunit.client.0.smithi090.stdout:7/927: dwrite d0/db/d15/f7e [0,4194304] 0 2024-04-26T04:42:30.341 INFO:tasks.workunit.client.0.smithi090.stdout:5/905: dwrite d4/d1a/d2a/d79/d90/f103 [0,4194304] 0 2024-04-26T04:42:30.346 INFO:tasks.workunit.client.0.smithi090.stdout:5/906: dread d4/d1a/df/fa7 [0,4194304] 0 2024-04-26T04:42:30.349 INFO:tasks.workunit.client.0.smithi090.stdout:5/907: creat d4/d1a/d2a/d79/d90/d56/d80/db2/f11b x:0 0 0 2024-04-26T04:42:30.350 INFO:tasks.workunit.client.0.smithi090.stdout:5/908: link d4/d1a/d2a/d79/c102 d4/d1a/d2a/d79/d90/d56/d80/c11c 0 2024-04-26T04:42:30.352 INFO:tasks.workunit.client.0.smithi090.stdout:7/928: dread d0/db/d15/d29/fa2 [0,4194304] 0 2024-04-26T04:42:30.352 INFO:tasks.workunit.client.0.smithi090.stdout:7/929: getdents d0/d109/d65/d11c 0 2024-04-26T04:42:30.352 INFO:tasks.workunit.client.0.smithi090.stdout:7/930: readlink d0/db/de6/le8 0 2024-04-26T04:42:30.353 INFO:tasks.workunit.client.0.smithi090.stdout:5/909: rename d4/d1a/d2a/d31/f34 to d4/d1a/d1f/d53/d59/d5e/da0/f11d 0 2024-04-26T04:42:30.354 INFO:tasks.workunit.client.0.smithi090.stdout:7/931: mknod d0/db/df/d4d/daf/dd1/d13c/c141 0 2024-04-26T04:42:30.356 INFO:tasks.workunit.client.0.smithi090.stdout:7/932: rename d0/db/d15/d29/f18 to d0/d105/f142 0 2024-04-26T04:42:30.356 INFO:tasks.workunit.client.0.smithi090.stdout:7/933: readlink d0/db/df/d4d/l79 0 2024-04-26T04:42:30.365 INFO:tasks.workunit.client.0.smithi090.stdout:8/997: sync 2024-04-26T04:42:30.365 INFO:tasks.workunit.client.0.smithi090.stdout:9/742: sync 2024-04-26T04:42:30.365 INFO:tasks.workunit.client.0.smithi090.stdout:4/811: sync 2024-04-26T04:42:30.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/873: sync 2024-04-26T04:42:30.365 INFO:tasks.workunit.client.0.smithi090.stdout:1/818: sync 2024-04-26T04:42:30.365 INFO:tasks.workunit.client.0.smithi090.stdout:1/819: stat d0/d55/dda/d7/d49/cc4 0 2024-04-26T04:42:30.366 INFO:tasks.workunit.client.0.smithi090.stdout:5/910: write d4/d1a/d2a/fd0 [408943,124500] 0 2024-04-26T04:42:30.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/812: rename d3/db/d12/d20/d24/d43 to d3/db/d12/d20/d2d/d93/d102 0 2024-04-26T04:42:30.367 INFO:tasks.workunit.client.0.smithi090.stdout:4/813: fdatasync d3/db/d12/d20/d2d/d58/d6a/d74/da2/f5a 0 2024-04-26T04:42:30.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/743: mknod db/dd/d44/d82/d85/df6/c10e 0 2024-04-26T04:42:30.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/744: creat db/dd/d18/d38/d39/dba/f10f x:0 0 0 2024-04-26T04:42:30.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/745: write db/dd/d18/d38/fd1 [757575,79054] 0 2024-04-26T04:42:30.368 INFO:tasks.workunit.client.0.smithi090.stdout:9/746: fdatasync db/dd/d44/d82/d85/db4/f4e 0 2024-04-26T04:42:30.370 INFO:tasks.workunit.client.0.smithi090.stdout:1/820: creat d0/d55/dda/d7/d4a/d75/d80/d58/d97/f106 x:0 0 0 2024-04-26T04:42:30.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/874: getdents d9/d1c/d31/d69/db7/d117 0 2024-04-26T04:42:30.370 INFO:tasks.workunit.client.0.smithi090.stdout:6/875: truncate d9/d21/d3b/d86/fe1 428042 0 2024-04-26T04:42:30.373 INFO:tasks.workunit.client.0.smithi090.stdout:5/911: unlink d4/d1a/d3f/f111 0 2024-04-26T04:42:30.374 INFO:tasks.workunit.client.0.smithi090.stdout:5/912: chown d4/d1a/df/d36/d8e 230479 1 2024-04-26T04:42:30.374 INFO:tasks.workunit.client.0.smithi090.stdout:5/913: read d4/d1a/d45/f32 [2787792,44644] 0 2024-04-26T04:42:30.374 INFO:tasks.workunit.client.0.smithi090.stdout:5/914: truncate d4/d1a/df/d4a/fa4 1045527 0 2024-04-26T04:42:30.375 INFO:tasks.workunit.client.0.smithi090.stdout:7/934: write d0/db/d53/f56 [3763280,108739] 0 2024-04-26T04:42:30.375 INFO:tasks.workunit.client.0.smithi090.stdout:9/747: dread db/f47 [0,4194304] 0 2024-04-26T04:42:30.375 INFO:tasks.workunit.client.0.smithi090.stdout:9/748: chown db/dd/d44/d82/d85/db4/d6a/db6/cd0 13 1 2024-04-26T04:42:30.377 INFO:tasks.workunit.client.0.smithi090.stdout:4/814: link d3/db/d12/d20/d2d/d93/d102/dd4/f92 d3/db/d12/d20/d24/da9/f103 0 2024-04-26T04:42:30.380 INFO:tasks.workunit.client.0.smithi090.stdout:7/935: mknod d0/db/df/d1c/d23/d8d/d133/c143 0 2024-04-26T04:42:30.380 INFO:tasks.workunit.client.0.smithi090.stdout:5/915: dread d4/d1a/d2a/f83 [0,4194304] 0 2024-04-26T04:42:30.381 INFO:tasks.workunit.client.0.smithi090.stdout:6/876: link d9/l33 d9/l133 0 2024-04-26T04:42:30.382 INFO:tasks.workunit.client.0.smithi090.stdout:5/916: creat d4/d1a/d2a/d31/f11e x:0 0 0 2024-04-26T04:42:30.383 INFO:tasks.workunit.client.0.smithi090.stdout:7/936: dread d0/db/d53/d5e/f8b [0,4194304] 0 2024-04-26T04:42:30.383 INFO:tasks.workunit.client.0.smithi090.stdout:7/937: write d0/d1/f71 [1052898,80945] 0 2024-04-26T04:42:30.386 INFO:tasks.workunit.client.0.smithi090.stdout:7/938: creat d0/db/df/d4d/daf/dd1/f144 x:0 0 0 2024-04-26T04:42:30.389 INFO:tasks.workunit.client.0.smithi090.stdout:9/749: write db/dd/d18/d38/d39/d6b/dc0/fad [3360163,91292] 0 2024-04-26T04:42:30.389 INFO:tasks.workunit.client.0.smithi090.stdout:4/815: dread d3/db/d12/f5b [0,4194304] 0 2024-04-26T04:42:30.389 INFO:tasks.workunit.client.0.smithi090.stdout:4/816: readlink d3/db/d12/d3b/le6 0 2024-04-26T04:42:30.389 INFO:tasks.workunit.client.0.smithi090.stdout:7/939: mkdir d0/db/df/d4d/d64/db0/dd6/d145 0 2024-04-26T04:42:30.389 INFO:tasks.workunit.client.0.smithi090.stdout:7/940: chown d0/db/d53/db7/l72 46 1 2024-04-26T04:42:30.389 INFO:tasks.workunit.client.0.smithi090.stdout:7/941: creat d0/d1/f146 x:0 0 0 2024-04-26T04:42:30.392 INFO:tasks.workunit.client.0.smithi090.stdout:6/877: dread d9/d77/db0/db5/dda/d6d/d66/ffa [0,4194304] 0 2024-04-26T04:42:30.392 INFO:tasks.workunit.client.0.smithi090.stdout:6/878: fdatasync d9/d77/db0/db5/dda/d6d/d66/fb8 0 2024-04-26T04:42:30.392 INFO:tasks.workunit.client.0.smithi090.stdout:6/879: dread - d9/d77/db0/db5/dda/d58/f108 zero size 2024-04-26T04:42:30.394 INFO:tasks.workunit.client.0.smithi090.stdout:9/750: mknod db/dd/d44/d82/d85/db4/d99/c110 0 2024-04-26T04:42:30.395 INFO:tasks.workunit.client.0.smithi090.stdout:7/942: mknod d0/db/df/d4d/d64/c147 0 2024-04-26T04:42:30.402 INFO:tasks.workunit.client.0.smithi090.stdout:1/821: dwrite d0/d55/dda/d7/d4a/f9d [0,4194304] 0 2024-04-26T04:42:30.402 INFO:tasks.workunit.client.0.smithi090.stdout:1/822: truncate d0/d55/dda/d7/d4b/d53/f92 264310 0 2024-04-26T04:42:30.402 INFO:tasks.workunit.client.0.smithi090.stdout:9/751: dread db/dd/d18/d38/d39/f7f [0,4194304] 0 2024-04-26T04:42:30.406 INFO:tasks.workunit.client.0.smithi090.stdout:1/823: symlink d0/d55/dda/d7/d4b/d53/d7b/l107 0 2024-04-26T04:42:30.407 INFO:tasks.workunit.client.0.smithi090.stdout:4/817: dread d3/f7d [0,4194304] 0 2024-04-26T04:42:30.410 INFO:tasks.workunit.client.0.smithi090.stdout:1/824: mkdir d0/d55/dda/d7/d17/d22/d31/d108 0 2024-04-26T04:42:30.413 INFO:tasks.workunit.client.0.smithi090.stdout:4/818: creat d3/d13/d6c/d7e/f104 x:0 0 0 2024-04-26T04:42:30.413 INFO:tasks.workunit.client.0.smithi090.stdout:4/819: fdatasync d3/d13/d1d/d50/fe3 0 2024-04-26T04:42:30.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/820: mknod d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/dd8/c105 0 2024-04-26T04:42:30.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/821: creat d3/db/f106 x:0 0 0 2024-04-26T04:42:30.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/822: stat d3/db/d12/d20/d2d/d2e/fe1 0 2024-04-26T04:42:30.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/823: creat d3/db/d12/d3b/f107 x:0 0 0 2024-04-26T04:42:30.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/825: write d0/d55/dda/d7/d49/f6f [5031449,73927] 0 2024-04-26T04:42:30.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/826: write d0/d55/dda/d7/d4a/ff5 [16607,47426] 0 2024-04-26T04:42:30.419 INFO:tasks.workunit.client.0.smithi090.stdout:1/827: stat d0/d55/dda/c1d 0 2024-04-26T04:42:30.419 INFO:tasks.workunit.client.0.smithi090.stdout:4/824: mkdir d3/d13/d1d/d108 0 2024-04-26T04:42:30.423 INFO:tasks.workunit.client.0.smithi090.stdout:4/825: mkdir d3/d13/d6c/d8c/d109 0 2024-04-26T04:42:30.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/943: dwrite d0/db/d53/db7/f9b [0,4194304] 0 2024-04-26T04:42:30.429 INFO:tasks.workunit.client.0.smithi090.stdout:6/880: dwrite d9/d1c/f2b [0,4194304] 0 2024-04-26T04:42:30.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/944: write d0/db/df/d4d/d64/db0/dd6/f112 [237345,65927] 0 2024-04-26T04:42:30.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/945: creat d0/db/df/d4d/d7b/db3/f148 x:0 0 0 2024-04-26T04:42:30.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/946: read - d0/d1/f146 zero size 2024-04-26T04:42:30.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/947: creat d0/d105/f149 x:0 0 0 2024-04-26T04:42:30.429 INFO:tasks.workunit.client.0.smithi090.stdout:7/948: fdatasync d0/db/de6/f136 0 2024-04-26T04:42:30.436 INFO:tasks.workunit.client.0.smithi090.stdout:9/752: dwrite db/f47 [0,4194304] 0 2024-04-26T04:42:30.436 INFO:tasks.workunit.client.0.smithi090.stdout:6/881: rename d9/d21/d40/d4c/d6f/d9f/dae/fc1 to d9/d1c/da2/f134 0 2024-04-26T04:42:30.438 INFO:tasks.workunit.client.0.smithi090.stdout:7/949: mkdir d0/d1/d4/d3e/dd3/d14a 0 2024-04-26T04:42:30.441 INFO:tasks.workunit.client.0.smithi090.stdout:9/753: mknod db/d5f/c111 0 2024-04-26T04:42:30.443 INFO:tasks.workunit.client.0.smithi090.stdout:7/950: link d0/d1/f38 d0/d109/d65/f14b 0 2024-04-26T04:42:30.443 INFO:tasks.workunit.client.0.smithi090.stdout:6/882: getdents d9/d77/db0 0 2024-04-26T04:42:30.444 INFO:tasks.workunit.client.0.smithi090.stdout:1/828: dwrite d0/d55/dda/d7/d4a/f56 [0,4194304] 0 2024-04-26T04:42:30.446 INFO:tasks.workunit.client.0.smithi090.stdout:7/951: mkdir d0/d109/d4a/d14c 0 2024-04-26T04:42:30.446 INFO:tasks.workunit.client.0.smithi090.stdout:7/952: dread - d0/d109/d65/d11c/f12e zero size 2024-04-26T04:42:30.446 INFO:tasks.workunit.client.0.smithi090.stdout:7/953: write d0/db/df/d4d/daf/dd1/f144 [495723,919] 0 2024-04-26T04:42:30.448 INFO:tasks.workunit.client.0.smithi090.stdout:7/954: link d0/d1/da3/la5 d0/db/d53/l14d 0 2024-04-26T04:42:30.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/955: mkdir d0/db/df/d4d/daf/d14e 0 2024-04-26T04:42:30.449 INFO:tasks.workunit.client.0.smithi090.stdout:7/956: dread - d0/db/de6/f136 zero size 2024-04-26T04:42:30.452 INFO:tasks.workunit.client.0.smithi090.stdout:7/957: unlink d0/db/d53/db7/ca7 0 2024-04-26T04:42:30.454 INFO:tasks.workunit.client.0.smithi090.stdout:1/829: dread d0/d55/dda/d7/d49/f5e [4194304,4194304] 0 2024-04-26T04:42:30.454 INFO:tasks.workunit.client.0.smithi090.stdout:1/830: chown d0/d55/dda/d7/d17/d22/d31 3869565 1 2024-04-26T04:42:30.455 INFO:tasks.workunit.client.0.smithi090.stdout:1/831: chown d0/d55/dda/d7/c1a 27908 1 2024-04-26T04:42:30.456 INFO:tasks.workunit.client.0.smithi090.stdout:1/832: truncate d0/d55/dda/d7/d17/f18 2891944 0 2024-04-26T04:42:30.465 INFO:tasks.workunit.client.0.smithi090.stdout:6/883: write d9/d1c/dc5/fce [2803114,52835] 0 2024-04-26T04:42:30.465 INFO:tasks.workunit.client.0.smithi090.stdout:4/826: dwrite d3/db/d12/d20/d2d/d93/d102/dd4/f92 [4194304,4194304] 0 2024-04-26T04:42:30.465 INFO:tasks.workunit.client.0.smithi090.stdout:9/754: dwrite db/da3/da8/fea [0,4194304] 0 2024-04-26T04:42:30.465 INFO:tasks.workunit.client.0.smithi090.stdout:9/755: chown db/dd/d18/d57/dcc/ff2 3695 1 2024-04-26T04:42:30.467 INFO:tasks.workunit.client.0.smithi090.stdout:7/958: read d0/db/df/d1c/d23/f31 [3488142,3929] 0 2024-04-26T04:42:30.472 INFO:tasks.workunit.client.0.smithi090.stdout:9/756: truncate db/dd/d44/d82/d85/db4/d6a/d70/f96 167985 0 2024-04-26T04:42:30.476 INFO:tasks.workunit.client.0.smithi090.stdout:7/959: creat d0/d109/f14f x:0 0 0 2024-04-26T04:42:30.483 INFO:tasks.workunit.client.0.smithi090.stdout:7/960: creat d0/db/df/d4d/f150 x:0 0 0 2024-04-26T04:42:30.490 INFO:tasks.workunit.client.0.smithi090.stdout:1/833: dwrite d0/d55/dda/d7/d17/d22/f73 [0,4194304] 0 2024-04-26T04:42:30.491 INFO:tasks.workunit.client.0.smithi090.stdout:6/884: dwrite d9/d21/d40/d4c/f11f [0,4194304] 0 2024-04-26T04:42:30.492 INFO:tasks.workunit.client.0.smithi090.stdout:1/834: mknod d0/d55/dda/d7/d17/d22/d93/c109 0 2024-04-26T04:42:30.495 INFO:tasks.workunit.client.0.smithi090.stdout:1/835: creat d0/d55/dda/d7/f10a x:0 0 0 2024-04-26T04:42:30.495 INFO:tasks.workunit.client.0.smithi090.stdout:1/836: readlink d0/d55/dda/d7/d4b/d53/l60 0 2024-04-26T04:42:30.496 INFO:tasks.workunit.client.0.smithi090.stdout:1/837: rmdir d0/d55/dda/d7/d17/d22/d93/db0 39 2024-04-26T04:42:30.497 INFO:tasks.workunit.client.0.smithi090.stdout:7/961: dread d0/d1/d4/d3e/fcf [4194304,4194304] 0 2024-04-26T04:42:30.497 INFO:tasks.workunit.client.0.smithi090.stdout:7/962: write d0/d1/f146 [933801,117398] 0 2024-04-26T04:42:30.497 INFO:tasks.workunit.client.0.smithi090.stdout:7/963: write d0/d105/f121 [1052357,62246] 0 2024-04-26T04:42:30.497 INFO:tasks.workunit.client.0.smithi090.stdout:7/964: write d0/d1/f10d [444294,48449] 0 2024-04-26T04:42:30.500 INFO:tasks.workunit.client.0.smithi090.stdout:1/838: rename d0/d55/dda/d7/d17/d22/d31/f39 to d0/d55/dda/d7/d4b/dca/f10b 0 2024-04-26T04:42:30.501 INFO:tasks.workunit.client.0.smithi090.stdout:6/885: write d9/d77/f9d [1329087,85070] 0 2024-04-26T04:42:30.501 INFO:tasks.workunit.client.0.smithi090.stdout:4/827: dwrite d3/db/d12/d20/d2d/d2e/dda/f100 [0,4194304] 0 2024-04-26T04:42:30.502 INFO:tasks.workunit.client.0.smithi090.stdout:7/965: mkdir d0/db/df/d4d/d64/db0/dd6/d10e/d151 0 2024-04-26T04:42:30.505 INFO:tasks.workunit.client.0.smithi090.stdout:1/839: unlink d0/d55/dda/d7/d4a/d75/d80/fc6 0 2024-04-26T04:42:30.505 INFO:tasks.workunit.client.0.smithi090.stdout:1/840: creat d0/d55/dda/d7/d17/d22/d93/db8/de9/f10c x:0 0 0 2024-04-26T04:42:30.507 INFO:tasks.workunit.client.0.smithi090.stdout:6/886: truncate d9/d3c/fb3 190138 0 2024-04-26T04:42:30.507 INFO:tasks.workunit.client.0.smithi090.stdout:6/887: fsync d9/d77/db0/db5/dda/d6d/f45 0 2024-04-26T04:42:30.508 INFO:tasks.workunit.client.0.smithi090.stdout:4/828: creat d3/db/d12/d20/d2d/d58/d6a/da3/f10a x:0 0 0 2024-04-26T04:42:30.515 INFO:tasks.workunit.client.0.smithi090.stdout:7/966: rmdir d0/d109/d4a 39 2024-04-26T04:42:30.515 INFO:tasks.workunit.client.0.smithi090.stdout:7/967: chown d0/db/d53/d5e/d13a 5 1 2024-04-26T04:42:30.515 INFO:tasks.workunit.client.0.smithi090.stdout:6/888: dread d9/d77/db0/db5/dda/d6d/d92/f25 [0,4194304] 0 2024-04-26T04:42:30.517 INFO:tasks.workunit.client.0.smithi090.stdout:1/841: symlink d0/d55/dda/d7/d17/d22/d31/d108/l10d 0 2024-04-26T04:42:30.520 INFO:tasks.workunit.client.0.smithi090.stdout:4/829: creat d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/f10b x:0 0 0 2024-04-26T04:42:30.521 INFO:tasks.workunit.client.0.smithi090.stdout:7/968: creat d0/f152 x:0 0 0 2024-04-26T04:42:30.523 INFO:tasks.workunit.client.0.smithi090.stdout:0/930: sync 2024-04-26T04:42:30.524 INFO:tasks.workunit.client.0.smithi090.stdout:4/830: creat d3/d13/d6c/dba/f10c x:0 0 0 2024-04-26T04:42:30.525 INFO:tasks.workunit.client.0.smithi090.stdout:6/889: truncate d9/d77/f9d 2353858 0 2024-04-26T04:42:30.527 INFO:tasks.workunit.client.0.smithi090.stdout:1/842: mkdir d0/d55/dda/d7/d17/d22/d10e 0 2024-04-26T04:42:30.527 INFO:tasks.workunit.client.0.smithi090.stdout:0/931: dread d0/d4b/d5d/d5c/d8a/f44 [0,4194304] 0 2024-04-26T04:42:30.527 INFO:tasks.workunit.client.0.smithi090.stdout:0/932: fsync d0/d4b/d5d/d5c/fa8 0 2024-04-26T04:42:30.527 INFO:tasks.workunit.client.0.smithi090.stdout:0/933: write d0/d106/f12e [1014410,10858] 0 2024-04-26T04:42:30.529 INFO:tasks.workunit.client.0.smithi090.stdout:4/831: rename d3/db/d12/d3b/dfb to d3/db/d12/d20/d2d/d93/d102/d72/d10d 0 2024-04-26T04:42:30.530 INFO:tasks.workunit.client.0.smithi090.stdout:6/890: mkdir d9/d1c/d31/d69/db7/d117/d135 0 2024-04-26T04:42:30.530 INFO:tasks.workunit.client.0.smithi090.stdout:6/891: write d9/d1c/ff9 [41237,54078] 0 2024-04-26T04:42:30.532 INFO:tasks.workunit.client.0.smithi090.stdout:0/934: mknod d0/d4b/d6d/d13d/c141 0 2024-04-26T04:42:30.533 INFO:tasks.workunit.client.0.smithi090.stdout:0/935: creat d0/d4b/d5d/d5c/d8a/d45/d53/dab/f142 x:0 0 0 2024-04-26T04:42:30.535 INFO:tasks.workunit.client.0.smithi090.stdout:0/936: unlink d0/d4b/db3/ce3 0 2024-04-26T04:42:30.537 INFO:tasks.workunit.client.0.smithi090.stdout:0/937: symlink d0/d4b/d5d/d5c/d8a/l143 0 2024-04-26T04:42:30.537 INFO:tasks.workunit.client.0.smithi090.stdout:0/938: dread - d0/d4b/feb zero size 2024-04-26T04:42:30.537 INFO:tasks.workunit.client.0.smithi090.stdout:0/939: chown d0/d4b/d5d/d5c/lff 692 1 2024-04-26T04:42:30.537 INFO:tasks.workunit.client.0.smithi090.stdout:1/843: dread d0/d55/dda/f91 [4194304,4194304] 0 2024-04-26T04:42:30.537 INFO:tasks.workunit.client.0.smithi090.stdout:6/892: read d9/d1c/d31/d69/ff2 [166962,41164] 0 2024-04-26T04:42:30.538 INFO:tasks.workunit.client.0.smithi090.stdout:0/940: creat d0/d4b/d5d/d48/f144 x:0 0 0 2024-04-26T04:42:30.539 INFO:tasks.workunit.client.0.smithi090.stdout:7/969: dwrite d0/db/d53/db7/d4f/f76 [0,4194304] 0 2024-04-26T04:42:30.540 INFO:tasks.workunit.client.0.smithi090.stdout:1/844: rename d0/d55/dda/d7/d17/d22/f73 to d0/d55/dda/d62/f10f 0 2024-04-26T04:42:30.540 INFO:tasks.workunit.client.0.smithi090.stdout:1/845: stat d0/d55/dda/d62/d78/l95 0 2024-04-26T04:42:30.540 INFO:tasks.workunit.client.0.smithi090.stdout:4/832: dread d3/f2c [0,4194304] 0 2024-04-26T04:42:30.541 INFO:tasks.workunit.client.0.smithi090.stdout:6/893: rmdir d9/d21/d40/d4c/d6f/d9f/dae 39 2024-04-26T04:42:30.541 INFO:tasks.workunit.client.0.smithi090.stdout:6/894: creat d9/d77/db0/db5/dda/d58/f136 x:0 0 0 2024-04-26T04:42:30.542 INFO:tasks.workunit.client.0.smithi090.stdout:0/941: creat d0/d4b/d5d/d5c/d8a/f145 x:0 0 0 2024-04-26T04:42:30.542 INFO:tasks.workunit.client.0.smithi090.stdout:0/942: truncate d0/d4b/f11e 845610 0 2024-04-26T04:42:30.547 INFO:tasks.workunit.client.0.smithi090.stdout:7/970: mkdir d0/d109/d65/d153 0 2024-04-26T04:42:30.547 INFO:tasks.workunit.client.0.smithi090.stdout:7/971: chown d0/db/df/d1c/d23/d8d/d133 20538387 1 2024-04-26T04:42:30.549 INFO:tasks.workunit.client.0.smithi090.stdout:1/846: link d0/d55/dda/d7/d4b/lf3 d0/d55/dda/d7/d17/d22/d93/db0/l110 0 2024-04-26T04:42:30.553 INFO:tasks.workunit.client.0.smithi090.stdout:4/833: symlink d3/db/d12/l10e 0 2024-04-26T04:42:30.553 INFO:tasks.workunit.client.0.smithi090.stdout:4/834: write d3/f6d [737208,40342] 0 2024-04-26T04:42:30.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/835: truncate d3/db/d12/d20/f41 1038816 0 2024-04-26T04:42:30.556 INFO:tasks.workunit.client.0.smithi090.stdout:4/836: creat d3/d13/d1d/d50/f10f x:0 0 0 2024-04-26T04:42:30.556 INFO:tasks.workunit.client.0.smithi090.stdout:4/837: chown d3/db/d60/dee/dc1 6 1 2024-04-26T04:42:30.556 INFO:tasks.workunit.client.0.smithi090.stdout:7/972: dread d0/f3c [0,4194304] 0 2024-04-26T04:42:30.557 INFO:tasks.workunit.client.0.smithi090.stdout:4/838: link d3/db/d12/d20/d2d/d58/d6a/da3/df5/lf6 d3/db/d12/d20/d2d/d93/d102/d72/d7a/l110 0 2024-04-26T04:42:30.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/839: read - d3/d13/d6c/fbb zero size 2024-04-26T04:42:30.558 INFO:tasks.workunit.client.0.smithi090.stdout:4/840: write d3/db/d60/dee/fd1 [379732,30304] 0 2024-04-26T04:42:30.560 INFO:tasks.workunit.client.0.smithi090.stdout:4/841: mkdir d3/db/d12/d20/d24/d111 0 2024-04-26T04:42:30.563 INFO:tasks.workunit.client.0.smithi090.stdout:7/973: rename d0/d109/d65/dbb to d0/db/df/d4d/d64/db0/dd6/d10e/d151/d154 0 2024-04-26T04:42:30.569 INFO:tasks.workunit.client.0.smithi090.stdout:1/847: dwrite d0/d55/dbd/fd2 [0,4194304] 0 2024-04-26T04:42:30.573 INFO:tasks.workunit.client.0.smithi090.stdout:4/842: dread d3/db/f19 [0,4194304] 0 2024-04-26T04:42:30.575 INFO:tasks.workunit.client.0.smithi090.stdout:7/974: dread d0/db/d53/fa [4194304,4194304] 0 2024-04-26T04:42:30.575 INFO:tasks.workunit.client.0.smithi090.stdout:4/843: truncate d3/d13/d6c/f90 8373230 0 2024-04-26T04:42:30.577 INFO:tasks.workunit.client.0.smithi090.stdout:7/975: rename d0/d109/d4a/fde to d0/d109/d65/d11c/f155 0 2024-04-26T04:42:30.577 INFO:tasks.workunit.client.0.smithi090.stdout:7/976: chown d0/db/df/d4d/d64/df2/f140 6292792 1 2024-04-26T04:42:30.578 INFO:tasks.workunit.client.0.smithi090.stdout:4/844: creat d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/dd8/f112 x:0 0 0 2024-04-26T04:42:30.581 INFO:tasks.workunit.client.0.smithi090.stdout:7/977: symlink d0/db/de6/d135/l156 0 2024-04-26T04:42:30.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/845: truncate f1 249094 0 2024-04-26T04:42:30.584 INFO:tasks.workunit.client.0.smithi090.stdout:4/846: symlink d3/db/d12/d20/d2d/d93/d102/d72/l113 0 2024-04-26T04:42:30.584 INFO:tasks.workunit.client.0.smithi090.stdout:4/847: rename d3/d13/d1d to d3/d13/d1d/d50/d51/de8/d114 22 2024-04-26T04:42:30.585 INFO:tasks.workunit.client.0.smithi090.stdout:4/848: write d3/f48 [5138098,33761] 0 2024-04-26T04:42:30.585 INFO:tasks.workunit.client.0.smithi090.stdout:4/849: chown d3/d13/d1d/d50 64063 1 2024-04-26T04:42:30.587 INFO:tasks.workunit.client.0.smithi090.stdout:1/848: dwrite d0/d55/dda/f27 [0,4194304] 0 2024-04-26T04:42:30.587 INFO:tasks.workunit.client.0.smithi090.stdout:1/849: readlink d0/d55/dda/d7/d4a/d75/d80/ld9 0 2024-04-26T04:42:30.592 INFO:tasks.workunit.client.0.smithi090.stdout:7/978: dread d0/db/d53/f17 [0,4194304] 0 2024-04-26T04:42:30.593 INFO:tasks.workunit.client.0.smithi090.stdout:7/979: mknod d0/db/d53/c157 0 2024-04-26T04:42:30.613 INFO:tasks.workunit.client.0.smithi090.stdout:1/850: dwrite d0/d55/dda/d7/d4b/fc7 [0,4194304] 0 2024-04-26T04:42:30.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/980: dwrite d0/db/d15/d29/f44 [0,4194304] 0 2024-04-26T04:42:30.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/981: stat d0/db/df/d4d/d64/c147 0 2024-04-26T04:42:30.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/982: dread - d0/d109/d65/db2/fcb zero size 2024-04-26T04:42:30.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/983: chown d0/db/d15/d29/c3f 21 1 2024-04-26T04:42:30.613 INFO:tasks.workunit.client.0.smithi090.stdout:7/984: chown d0/d105/c128 0 1 2024-04-26T04:42:30.617 INFO:tasks.workunit.client.0.smithi090.stdout:7/985: rename d0/d1/d4 to d0/d109/d65/db2/d120/d158 0 2024-04-26T04:42:30.627 INFO:tasks.workunit.client.0.smithi090.stdout:7/986: read d0/d109/d65/db2/d120/d158/d3e/dd3/de2/fea [3074594,97393] 0 2024-04-26T04:42:30.628 INFO:tasks.workunit.client.0.smithi090.stdout:1/851: dwrite d0/d55/dda/d7/d4a/d75/d80/fa2 [0,4194304] 0 2024-04-26T04:42:30.629 INFO:tasks.workunit.client.0.smithi090.stdout:7/987: link d0/db/d15/d29/f6e d0/db/df/d4d/daf/f159 0 2024-04-26T04:42:30.629 INFO:tasks.workunit.client.0.smithi090.stdout:7/988: write d0/db/df/d4d/d64/db0/dd6/f122 [1521853,113005] 0 2024-04-26T04:42:30.629 INFO:tasks.workunit.client.0.smithi090.stdout:7/989: stat d0/d1/c134 0 2024-04-26T04:42:30.629 INFO:tasks.workunit.client.0.smithi090.stdout:1/852: truncate d0/d55/dda/d7/d4a/f96 454579 0 2024-04-26T04:42:30.631 INFO:tasks.workunit.client.0.smithi090.stdout:1/853: rename d0/d55/dda/d7/d17/d22/d31/f34 to d0/d55/dda/d7/d17/d22/d93/db8/de9/f111 0 2024-04-26T04:42:30.632 INFO:tasks.workunit.client.0.smithi090.stdout:1/854: rename d0/d55/dda/d43/f47 to d0/d55/dda/d62/d78/dcc/f112 0 2024-04-26T04:42:30.633 INFO:tasks.workunit.client.0.smithi090.stdout:1/855: mknod d0/d55/dda/d7/d17/d22/d93/db8/de9/c113 0 2024-04-26T04:42:30.633 INFO:tasks.workunit.client.0.smithi090.stdout:1/856: creat d0/f114 x:0 0 0 2024-04-26T04:42:30.634 INFO:tasks.workunit.client.0.smithi090.stdout:1/857: truncate d0/d55/dda/d7/d4a/da8/fad 606487 0 2024-04-26T04:42:30.635 INFO:tasks.workunit.client.0.smithi090.stdout:1/858: mkdir d0/d55/dda/d7/d4a/d75/d115 0 2024-04-26T04:42:30.636 INFO:tasks.workunit.client.0.smithi090.stdout:7/990: dread d0/d109/d65/db2/fba [0,4194304] 0 2024-04-26T04:42:30.638 INFO:tasks.workunit.client.0.smithi090.stdout:1/859: rename d0/d55/dda/d7/d17/d22/d93/db0/l110 to d0/d55/dda/d7/d4b/d53/d7b/l116 0 2024-04-26T04:42:30.638 INFO:tasks.workunit.client.0.smithi090.stdout:1/860: chown d0/d55/dda/d62/lf4 0 1 2024-04-26T04:42:30.640 INFO:tasks.workunit.client.0.smithi090.stdout:7/991: rename d0/db/df/d1c/cc3 to d0/d109/d4a/d14c/c15a 0 2024-04-26T04:42:30.643 INFO:tasks.workunit.client.0.smithi090.stdout:7/992: creat d0/db/df/d4d/d64/db0/dd6/d10e/d151/f15b x:0 0 0 2024-04-26T04:42:30.648 INFO:tasks.workunit.client.0.smithi090.stdout:3/780: sync 2024-04-26T04:42:30.648 INFO:tasks.workunit.client.0.smithi090.stdout:8/998: sync 2024-04-26T04:42:30.648 INFO:tasks.workunit.client.0.smithi090.stdout:2/860: sync 2024-04-26T04:42:30.648 INFO:tasks.workunit.client.0.smithi090.stdout:2/861: chown db/d11/d14/d27/d28/l68 1 1 2024-04-26T04:42:30.648 INFO:tasks.workunit.client.0.smithi090.stdout:2/862: fdatasync db/d11/d14/d25/d2d/fe2 0 2024-04-26T04:42:30.648 INFO:tasks.workunit.client.0.smithi090.stdout:2/863: creat db/d11/d14/d25/d2d/d48/f11f x:0 0 0 2024-04-26T04:42:30.649 INFO:tasks.workunit.client.0.smithi090.stdout:8/999: symlink dc/d1f/l160 0 2024-04-26T04:42:30.651 INFO:tasks.workunit.client.0.smithi090.stdout:3/781: symlink d1/d17/d36/d33/l10c 0 2024-04-26T04:42:30.655 INFO:tasks.workunit.client.0.smithi090.stdout:1/861: dwrite d0/f114 [0,4194304] 0 2024-04-26T04:42:30.658 INFO:tasks.workunit.client.0.smithi090.stdout:1/862: mknod d0/d55/dda/d7/d4a/d102/c117 0 2024-04-26T04:42:30.658 INFO:tasks.workunit.client.0.smithi090.stdout:1/863: truncate d0/d55/dda/d7/d4b/d53/f6a 822245 0 2024-04-26T04:42:30.663 INFO:tasks.workunit.client.0.smithi090.stdout:7/993: dwrite d0/db/d53/f115 [0,4194304] 0 2024-04-26T04:42:30.663 INFO:tasks.workunit.client.0.smithi090.stdout:7/994: chown d0/db/d53/f17 0 1 2024-04-26T04:42:30.663 INFO:tasks.workunit.client.0.smithi090.stdout:7/995: readlink d0/db/d53/db7/l80 0 2024-04-26T04:42:30.663 INFO:tasks.workunit.client.0.smithi090.stdout:7/996: fsync d0/d109/d65/db2/d120/d158/d3e/dd3/de2/fea 0 2024-04-26T04:42:30.665 INFO:tasks.workunit.client.0.smithi090.stdout:7/997: creat d0/db/df/d4d/d64/db0/dd6/d10e/d151/d154/f15c x:0 0 0 2024-04-26T04:42:30.672 INFO:tasks.workunit.client.0.smithi090.stdout:1/864: read d0/d55/dda/d7/d17/f21 [3754541,60981] 0 2024-04-26T04:42:30.672 INFO:tasks.workunit.client.0.smithi090.stdout:3/782: dwrite d1/d17/d61/d9d/fec [0,4194304] 0 2024-04-26T04:42:30.672 INFO:tasks.workunit.client.0.smithi090.stdout:3/783: fdatasync d1/d17/f2f 0 2024-04-26T04:42:30.672 INFO:tasks.workunit.client.0.smithi090.stdout:3/784: chown d1/d17/d36/d33/d68/dae/dca/dcf/d106/d7e/le6 758850560 1 2024-04-26T04:42:30.673 INFO:tasks.workunit.client.0.smithi090.stdout:7/998: dread d0/d1/f38 [0,4194304] 0 2024-04-26T04:42:30.673 INFO:tasks.workunit.client.0.smithi090.stdout:7/999: chown d0/d1/da3/lce 15082291 1 2024-04-26T04:42:30.675 INFO:tasks.workunit.client.0.smithi090.stdout:1/865: dread d0/d55/dda/f20 [0,4194304] 0 2024-04-26T04:42:30.676 INFO:tasks.workunit.client.0.smithi090.stdout:1/866: creat d0/d55/dda/d7/d4b/dca/f118 x:0 0 0 2024-04-26T04:42:30.678 INFO:tasks.workunit.client.0.smithi090.stdout:1/867: mknod d0/d55/dda/d7/d17/d22/d31/d108/c119 0 2024-04-26T04:42:30.679 INFO:tasks.workunit.client.0.smithi090.stdout:1/868: mknod d0/d55/dda/d7/d17/d22/d10e/c11a 0 2024-04-26T04:42:30.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/869: getdents d0/d55/dda/d7/d17/d22/d93/db8/de9 0 2024-04-26T04:42:30.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/870: chown d0/d55/dda/fa4 23534 1 2024-04-26T04:42:30.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/871: truncate d0/d55/dda/d7/d4a/d75/d80/d58/fe2 137749 0 2024-04-26T04:42:30.681 INFO:tasks.workunit.client.0.smithi090.stdout:1/872: write d0/d55/dda/d7/d4a/d75/d80/d58/fed [342325,110070] 0 2024-04-26T04:42:30.681 INFO:tasks.workunit.client.0.smithi090.stdout:3/785: write d1/d17/f4f [3747214,89790] 0 2024-04-26T04:42:30.683 INFO:tasks.workunit.client.0.smithi090.stdout:3/786: dread d1/d17/f2f [0,4194304] 0 2024-04-26T04:42:30.685 INFO:tasks.workunit.client.0.smithi090.stdout:3/787: creat d1/d17/d36/d2a/d3a/f10d x:0 0 0 2024-04-26T04:42:30.690 INFO:tasks.workunit.client.0.smithi090.stdout:1/873: read d0/d55/dda/d7/d4a/d75/d80/d58/d97/f103 [4125733,80321] 0 2024-04-26T04:42:30.719 INFO:tasks.workunit.client.0.smithi090.stdout:3/788: dwrite d1/d9/fb [4194304,4194304] 0 2024-04-26T04:42:30.719 INFO:tasks.workunit.client.0.smithi090.stdout:1/874: dwrite d0/d55/dbd/ff8 [0,4194304] 0 2024-04-26T04:42:30.719 INFO:tasks.workunit.client.0.smithi090.stdout:1/875: fsync d0/d55/dda/d7/d4b/f4c 0 2024-04-26T04:42:30.748 INFO:tasks.workunit.client.0.smithi090.stdout:3/789: dwrite d1/d17/d36/f32 [4194304,4194304] 0 2024-04-26T04:42:30.748 INFO:tasks.workunit.client.0.smithi090.stdout:3/790: write d1/d17/f2f [1451728,104776] 0 2024-04-26T04:42:30.748 INFO:tasks.workunit.client.0.smithi090.stdout:1/876: dwrite d0/d55/dda/d7/d4a/ff5 [0,4194304] 0 2024-04-26T04:42:30.748 INFO:tasks.workunit.client.0.smithi090.stdout:1/877: fsync d0/d55/dda/d7/d4a/d75/d80/d58/fe2 0 2024-04-26T04:42:30.749 INFO:tasks.workunit.client.0.smithi090.stdout:3/791: getdents d1/d17/d36/d33/d68/dae/dca/d1b 0 2024-04-26T04:42:30.749 INFO:tasks.workunit.client.0.smithi090.stdout:3/792: write d1/d59/dc9/d7d/fbf [1521295,32354] 0 2024-04-26T04:42:30.751 INFO:tasks.workunit.client.0.smithi090.stdout:1/878: mkdir d0/d55/dda/d43/d11b 0 2024-04-26T04:42:30.752 INFO:tasks.workunit.client.0.smithi090.stdout:1/879: mkdir d0/d55/dda/d7/d17/d22/d93/db8/de9/d11c 0 2024-04-26T04:42:30.753 INFO:tasks.workunit.client.0.smithi090.stdout:1/880: mkdir d0/d55/dda/d7/d4b/d11d 0 2024-04-26T04:42:30.755 INFO:tasks.workunit.client.0.smithi090.stdout:1/881: creat d0/d55/dda/d7/d17/d22/d10e/f11e x:0 0 0 2024-04-26T04:42:30.755 INFO:tasks.workunit.client.0.smithi090.stdout:1/882: chown d0/d55/dda/d7/cfd 0 1 2024-04-26T04:42:30.758 INFO:tasks.workunit.client.0.smithi090.stdout:3/793: dread d1/d59/fa1 [0,4194304] 0 2024-04-26T04:42:30.758 INFO:tasks.workunit.client.0.smithi090.stdout:3/794: chown d1/d17/d36/d33/d68/dae/dca/d1b/d58/da0/fc8 362344466 1 2024-04-26T04:42:30.758 INFO:tasks.workunit.client.0.smithi090.stdout:3/795: truncate d1/d17/d36/d2a/fe3 90937 0 2024-04-26T04:42:30.760 INFO:tasks.workunit.client.0.smithi090.stdout:3/796: rmdir d1/d17/d61/d9d 39 2024-04-26T04:42:30.760 INFO:tasks.workunit.client.0.smithi090.stdout:3/797: getdents d1/d17/d36/d33/d68/dae/dca/dcf/d106/d82/df8 0 2024-04-26T04:42:30.763 INFO:tasks.workunit.client.0.smithi090.stdout:3/798: link d1/d17/d36/d2a/d3a/f10d d1/d17/d61/dce/f10e 0 2024-04-26T04:42:30.764 INFO:tasks.workunit.client.0.smithi090.stdout:1/883: dread d0/d55/dbd/ff8 [0,4194304] 0 2024-04-26T04:42:30.764 INFO:tasks.workunit.client.0.smithi090.stdout:5/917: sync 2024-04-26T04:42:30.766 INFO:tasks.workunit.client.0.smithi090.stdout:1/884: mknod d0/d55/dda/d7/d4b/dca/c11f 0 2024-04-26T04:42:30.766 INFO:tasks.workunit.client.0.smithi090.stdout:1/885: read - d0/d55/dda/d43/fc5 zero size 2024-04-26T04:42:30.766 INFO:tasks.workunit.client.0.smithi090.stdout:5/918: dread d4/d1a/d2a/f83 [0,4194304] 0 2024-04-26T04:42:30.768 INFO:tasks.workunit.client.0.smithi090.stdout:1/886: creat d0/d55/dda/d7/d4a/d75/d80/f120 x:0 0 0 2024-04-26T04:42:30.768 INFO:tasks.workunit.client.0.smithi090.stdout:1/887: readlink d0/d55/dda/d7/d4b/d53/l60 0 2024-04-26T04:42:30.769 INFO:tasks.workunit.client.0.smithi090.stdout:5/919: truncate d4/d1a/d1f/d53/f76 2406805 0 2024-04-26T04:42:30.771 INFO:tasks.workunit.client.0.smithi090.stdout:1/888: unlink d0/d55/dda/d7/d17/d22/d93/db8/de9/f111 0 2024-04-26T04:42:30.773 INFO:tasks.workunit.client.0.smithi090.stdout:5/920: getdents d4/d1a/d1f/d53 0 2024-04-26T04:42:30.773 INFO:tasks.workunit.client.0.smithi090.stdout:5/921: chown d4/d1a/d2a/d31/lda 375070 1 2024-04-26T04:42:30.778 INFO:tasks.workunit.client.0.smithi090.stdout:1/889: dread d0/d55/dda/d7/d17/d22/f52 [0,4194304] 0 2024-04-26T04:42:30.778 INFO:tasks.workunit.client.0.smithi090.stdout:1/890: chown d0/d55/dda/d7/d17/d22/d93/db0 3 1 2024-04-26T04:42:30.780 INFO:tasks.workunit.client.0.smithi090.stdout:1/891: creat d0/d55/dda/d7/d17/d22/d31/d108/f121 x:0 0 0 2024-04-26T04:42:30.780 INFO:tasks.workunit.client.0.smithi090.stdout:1/892: fdatasync d0/d55/dda/d7/d17/d22/d93/db8/de9/f10c 0 2024-04-26T04:42:30.781 INFO:tasks.workunit.client.0.smithi090.stdout:1/893: rmdir d0/d55/dda/d7/d4b 39 2024-04-26T04:42:30.783 INFO:tasks.workunit.client.0.smithi090.stdout:1/894: creat d0/d55/dda/d7/d17/d22/d93/db8/f122 x:0 0 0 2024-04-26T04:42:30.788 INFO:tasks.workunit.client.0.smithi090.stdout:3/799: dwrite d1/d17/f2f [0,4194304] 0 2024-04-26T04:42:30.792 INFO:tasks.workunit.client.0.smithi090.stdout:3/800: rename d1/d17/d36/d33/d68/dae/dca/d2c/d78 to d1/d59/dc9/d7d/d10f 0 2024-04-26T04:42:30.793 INFO:tasks.workunit.client.0.smithi090.stdout:3/801: fsync d1/d17/d36/d2a/d3a/f66 0 2024-04-26T04:42:30.794 INFO:tasks.workunit.client.0.smithi090.stdout:3/802: creat d1/d17/d36/d33/d68/dae/f110 x:0 0 0 2024-04-26T04:42:30.796 INFO:tasks.workunit.client.0.smithi090.stdout:3/803: creat d1/d17/f111 x:0 0 0 2024-04-26T04:42:30.798 INFO:tasks.workunit.client.0.smithi090.stdout:3/804: mkdir d1/d17/d36/d33/daa/d112 0 2024-04-26T04:42:30.799 INFO:tasks.workunit.client.0.smithi090.stdout:3/805: symlink d1/d17/d36/d33/d68/dae/dca/dcf/d106/d82/df8/l113 0 2024-04-26T04:42:30.799 INFO:tasks.workunit.client.0.smithi090.stdout:3/806: write d1/d17/f1f [4274911,110078] 0 2024-04-26T04:42:30.799 INFO:tasks.workunit.client.0.smithi090.stdout:3/807: stat d1/d17/d36/d2a/d3a/f66 0 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:4/850: sync 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:6/895: sync 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:9/757: sync 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:0/943: sync 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:2/864: sync 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:9/758: truncate db/dd/d44/d82/ff9 431641 0 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:2/865: dread - db/d11/d14/d25/d2d/d48/fbf zero size 2024-04-26T04:42:30.801 INFO:tasks.workunit.client.0.smithi090.stdout:6/896: write d9/d77/d9a/dc8/f10d [2849,106012] 0 2024-04-26T04:42:30.802 INFO:tasks.workunit.client.0.smithi090.stdout:3/808: rmdir d1/d9 39 2024-04-26T04:42:30.802 INFO:tasks.workunit.client.0.smithi090.stdout:3/809: write d1/f3c [1554356,108621] 0 2024-04-26T04:42:30.803 INFO:tasks.workunit.client.0.smithi090.stdout:4/851: link d3/db/f23 d3/db/d12/d3b/f115 0 2024-04-26T04:42:30.804 INFO:tasks.workunit.client.0.smithi090.stdout:6/897: symlink d9/d77/l137 0 2024-04-26T04:42:30.805 INFO:tasks.workunit.client.0.smithi090.stdout:3/810: rename d1/d17/d61/d9d/fa5 to d1/d17/d36/d2a/d3a/f114 0 2024-04-26T04:42:30.805 INFO:tasks.workunit.client.0.smithi090.stdout:3/811: chown d1/d17/d36/c44 26 1 2024-04-26T04:42:30.806 INFO:tasks.workunit.client.0.smithi090.stdout:4/852: creat d3/db/d60/dee/dc1/dcf/f116 x:0 0 0 2024-04-26T04:42:30.806 INFO:tasks.workunit.client.0.smithi090.stdout:6/898: creat d9/d3c/db4/ddb/f138 x:0 0 0 2024-04-26T04:42:30.807 INFO:tasks.workunit.client.0.smithi090.stdout:3/812: getdents d1/d17/d36/d2a/dc7 0 2024-04-26T04:42:30.808 INFO:tasks.workunit.client.0.smithi090.stdout:4/853: mkdir d3/db/d60/d5c/d117 0 2024-04-26T04:42:30.809 INFO:tasks.workunit.client.0.smithi090.stdout:6/899: mkdir d9/d1c/d31/d69/db7/dea/d139 0 2024-04-26T04:42:30.809 INFO:tasks.workunit.client.0.smithi090.stdout:6/900: write d9/d1c/dc5/dd9/fe8 [895480,52874] 0 2024-04-26T04:42:30.809 INFO:tasks.workunit.client.0.smithi090.stdout:6/901: chown d9/d3c/f64 454 1 2024-04-26T04:42:30.810 INFO:tasks.workunit.client.0.smithi090.stdout:3/813: mkdir d1/d17/d36/d33/d115 0 2024-04-26T04:42:30.810 INFO:tasks.workunit.client.0.smithi090.stdout:1/895: dwrite d0/d55/dda/f27 [4194304,4194304] 0 2024-04-26T04:42:30.813 INFO:tasks.workunit.client.0.smithi090.stdout:1/896: dread d0/d55/dda/f20 [0,4194304] 0 2024-04-26T04:42:30.816 INFO:tasks.workunit.client.0.smithi090.stdout:0/944: dread d0/d4b/d5d/d48/fa5 [0,4194304] 0 2024-04-26T04:42:30.816 INFO:tasks.workunit.client.0.smithi090.stdout:0/945: fsync d0/df/f13c 0 2024-04-26T04:42:30.821 INFO:tasks.workunit.client.0.smithi090.stdout:3/814: rename d1/d17/d36/d33/d68/dae/dca/dcf/d106/d82/df8 to d1/d17/d36/d33/d68/d76/d116 0 2024-04-26T04:42:30.822 INFO:tasks.workunit.client.0.smithi090.stdout:9/759: write db/dd/d18/d38/d39/d6b/de2/f5a [5574544,42438] 0 2024-04-26T04:42:30.824 INFO:tasks.workunit.client.0.smithi090.stdout:1/897: mkdir d0/d55/dda/d7/d4b/dca/d123 0 2024-04-26T04:42:30.825 INFO:tasks.workunit.client.0.smithi090.stdout:0/946: truncate d0/d4b/d5d/d48/f56 616516 0 2024-04-26T04:42:30.825 INFO:tasks.workunit.client.0.smithi090.stdout:0/947: getdents d0/d4b/d5d/d5c 0 2024-04-26T04:42:30.827 INFO:tasks.workunit.client.0.smithi090.stdout:3/815: creat d1/d17/d61/dce/f117 x:0 0 0 2024-04-26T04:42:30.830 INFO:tasks.workunit.client.0.smithi090.stdout:9/760: unlink db/dd/d18/d38/d39/d6b/de2/f9a 0 2024-04-26T04:42:30.832 INFO:tasks.workunit.client.0.smithi090.stdout:0/948: creat d0/d4b/d6d/d13d/f146 x:0 0 0 2024-04-26T04:42:30.832 INFO:tasks.workunit.client.0.smithi090.stdout:0/949: truncate d0/d4b/d5d/d48/f144 843658 0 2024-04-26T04:42:30.833 INFO:tasks.workunit.client.0.smithi090.stdout:3/816: symlink d1/d9/l118 0 2024-04-26T04:42:30.833 INFO:tasks.workunit.client.0.smithi090.stdout:3/817: chown d1/d17/d36/d33/d43 0 1 2024-04-26T04:42:30.833 INFO:tasks.workunit.client.0.smithi090.stdout:3/818: truncate d1/d17/d61/d9d/da7/ff4 693308 0 2024-04-26T04:42:30.833 INFO:tasks.workunit.client.0.smithi090.stdout:9/761: rmdir db/dd/d44/d82/d85/db4/d99/db2/dcd 39 2024-04-26T04:42:30.834 INFO:tasks.workunit.client.0.smithi090.stdout:9/762: getdents db/dd/d18/d38/d108 0 2024-04-26T04:42:30.834 INFO:tasks.workunit.client.0.smithi090.stdout:1/898: getdents d0/d55/dda 0 2024-04-26T04:42:30.835 INFO:tasks.workunit.client.0.smithi090.stdout:0/950: mkdir d0/de/d103/d147 0 2024-04-26T04:42:30.836 INFO:tasks.workunit.client.0.smithi090.stdout:3/819: symlink d1/d17/d36/d33/d68/dae/dca/d1b/d58/db5/l119 0 2024-04-26T04:42:30.838 INFO:tasks.workunit.client.0.smithi090.stdout:1/899: mknod d0/d55/dda/d7/d17/d22/d93/db8/de9/d11c/c124 0 2024-04-26T04:42:30.838 INFO:tasks.workunit.client.0.smithi090.stdout:3/820: write d1/d17/d36/d33/d42/f63 [499881,57902] 0 2024-04-26T04:42:30.843 INFO:tasks.workunit.client.0.smithi090.stdout:9/763: dread db/f13 [0,4194304] 0 2024-04-26T04:42:30.846 INFO:tasks.workunit.client.0.smithi090.stdout:9/764: rename db/dd/d44/d82/d85/df6 to db/d5f/d112 0 2024-04-26T04:42:30.846 INFO:tasks.workunit.client.0.smithi090.stdout:9/765: write db/dd/d44/d82/d85/db4/f4a [3195880,4204] 0 2024-04-26T04:42:30.851 INFO:tasks.workunit.client.0.smithi090.stdout:3/821: write d1/d5c/f99 [1532106,120231] 0 2024-04-26T04:42:30.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/822: dread d1/d17/d36/d33/d68/dae/dca/dcf/ff9 [0,4194304] 0 2024-04-26T04:42:30.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/823: readlink d1/d17/d36/d33/d68/dae/dca/dcf/d106/l86 0 2024-04-26T04:42:30.853 INFO:tasks.workunit.client.0.smithi090.stdout:3/824: readlink d1/d17/d36/d33/d68/dae/dca/dcf/d106/le0 0 2024-04-26T04:42:30.854 INFO:tasks.workunit.client.0.smithi090.stdout:1/900: dread d0/d55/dda/fa4 [0,4194304] 0 2024-04-26T04:42:30.854 INFO:tasks.workunit.client.0.smithi090.stdout:3/825: link d1/d17/d36/d33/d68/dae/dca/l87 d1/d5c/l11a 0 2024-04-26T04:42:30.854 INFO:tasks.workunit.client.0.smithi090.stdout:1/901: mknod d0/d55/dda/d7/d17/d22/d93/db8/c125 0 2024-04-26T04:42:30.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/766: write db/dd/d18/d38/d39/d3f/f91 [257872,91177] 0 2024-04-26T04:42:30.858 INFO:tasks.workunit.client.0.smithi090.stdout:6/902: dwrite d9/d77/db0/db5/fa3 [0,4194304] 0 2024-04-26T04:42:30.858 INFO:tasks.workunit.client.0.smithi090.stdout:6/903: dread - d9/d1c/d31/d79/ffe zero size 2024-04-26T04:42:30.858 INFO:tasks.workunit.client.0.smithi090.stdout:4/854: dwrite d3/db/f23 [0,4194304] 0 2024-04-26T04:42:30.858 INFO:tasks.workunit.client.0.smithi090.stdout:0/951: dwrite d0/d4b/d5d/d5c/d8a/f13 [0,4194304] 0 2024-04-26T04:42:30.865 INFO:tasks.workunit.client.0.smithi090.stdout:9/767: dread db/dd/d18/d57/f4d [0,4194304] 0 2024-04-26T04:42:30.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/768: truncate db/dd/d18/d57/f4d 2638983 0 2024-04-26T04:42:30.867 INFO:tasks.workunit.client.0.smithi090.stdout:9/769: creat db/dd/d18/d38/d39/dba/f113 x:0 0 0 2024-04-26T04:42:30.868 INFO:tasks.workunit.client.0.smithi090.stdout:3/826: dread d1/d17/d36/d33/f80 [0,4194304] 0 2024-04-26T04:42:30.868 INFO:tasks.workunit.client.0.smithi090.stdout:3/827: chown d1/d17/d36/d33/d68/dae/dca/d1b/lea 4985790 1 2024-04-26T04:42:30.868 INFO:tasks.workunit.client.0.smithi090.stdout:3/828: truncate d1/d17/d36/d33/d68/d71/fd0 116740 0 2024-04-26T04:42:30.870 INFO:tasks.workunit.client.0.smithi090.stdout:9/770: rename db/da3/da8/cda to db/dd/d18/d38/d39/c114 0 2024-04-26T04:42:30.872 INFO:tasks.workunit.client.0.smithi090.stdout:9/771: creat db/dd/d18/d38/d39/d3f/dff/f115 x:0 0 0 2024-04-26T04:42:30.872 INFO:tasks.workunit.client.0.smithi090.stdout:6/904: dread d9/d1c/d31/f95 [0,4194304] 0 2024-04-26T04:42:30.872 INFO:tasks.workunit.client.0.smithi090.stdout:6/905: chown d9/d21/d40/d4c/d6f/d9f/dae/l10a 27477 1 2024-04-26T04:42:30.880 INFO:tasks.workunit.client.0.smithi090.stdout:0/952: dwrite d0/d4b/db3/dcc/fe2 [0,4194304] 0 2024-04-26T04:42:30.880 INFO:tasks.workunit.client.0.smithi090.stdout:0/953: write d0/d4b/fad [4693797,98233] 0 2024-04-26T04:42:30.884 INFO:tasks.workunit.client.0.smithi090.stdout:3/829: dwrite d1/d17/d36/d2a/fe3 [0,4194304] 0 2024-04-26T04:42:30.884 INFO:tasks.workunit.client.0.smithi090.stdout:3/830: read d1/d17/f4f [2561803,87831] 0 2024-04-26T04:42:30.884 INFO:tasks.workunit.client.0.smithi090.stdout:3/831: truncate d1/d59/dc9/f7c 4914766 0 2024-04-26T04:42:30.887 INFO:tasks.workunit.client.0.smithi090.stdout:9/772: rename db/dd/d18/d57/f4d to db/f116 0 2024-04-26T04:42:30.889 INFO:tasks.workunit.client.0.smithi090.stdout:6/906: write d9/d1c/d31/f41 [995688,37986] 0 2024-04-26T04:42:30.911 INFO:tasks.workunit.client.0.smithi090.stdout:0/954: dwrite d0/d4b/d6d/fc6 [0,4194304] 0 2024-04-26T04:42:30.911 INFO:tasks.workunit.client.0.smithi090.stdout:9/773: dwrite db/dd/d18/d57/dcc/f102 [0,4194304] 0 2024-04-26T04:42:30.911 INFO:tasks.workunit.client.0.smithi090.stdout:6/907: dwrite d9/d1c/f2b [0,4194304] 0 2024-04-26T04:42:30.911 INFO:tasks.workunit.client.0.smithi090.stdout:9/774: chown db/dd/d44/d82/d85/c9e 1538384881 1 2024-04-26T04:42:30.911 INFO:tasks.workunit.client.0.smithi090.stdout:6/908: write d9/d77/db0/f121 [75019,55960] 0 2024-04-26T04:42:30.911 INFO:tasks.workunit.client.0.smithi090.stdout:6/909: write d9/d1c/d31/f3a [4450420,12858] 0 2024-04-26T04:42:30.912 INFO:tasks.workunit.client.0.smithi090.stdout:0/955: rename d0/df/l1a to d0/d4b/d5d/d48/d7d/l148 0 2024-04-26T04:42:30.919 INFO:tasks.workunit.client.0.smithi090.stdout:9/775: rmdir db/dd/d44/d82/d85/db4/d58 39 2024-04-26T04:42:30.922 INFO:tasks.workunit.client.0.smithi090.stdout:6/910: symlink d9/d77/db0/db5/dda/d6d/d92/l13a 0 2024-04-26T04:42:30.923 INFO:tasks.workunit.client.0.smithi090.stdout:9/776: readlink db/dd/l22 0 2024-04-26T04:42:30.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/777: dwrite db/dd/d18/d38/d39/d6b/dc0/fbf [0,4194304] 0 2024-04-26T04:42:30.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/778: truncate db/d5f/f10b 5069239 0 2024-04-26T04:42:30.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/779: write f2 [903429,87228] 0 2024-04-26T04:42:30.945 INFO:tasks.workunit.client.0.smithi090.stdout:6/911: dwrite d9/d77/db0/db5/dda/d6d/f84 [0,4194304] 0 2024-04-26T04:42:30.945 INFO:tasks.workunit.client.0.smithi090.stdout:9/780: write db/dd/d44/d82/d85/db4/d6a/f92 [1490432,111069] 0 2024-04-26T04:42:30.946 INFO:tasks.workunit.client.0.smithi090.stdout:6/912: creat d9/d3c/db4/f13b x:0 0 0 2024-04-26T04:42:30.948 INFO:tasks.workunit.client.0.smithi090.stdout:6/913: getdents d9/d1c/d31/d69/dbc 0 2024-04-26T04:42:30.948 INFO:tasks.workunit.client.0.smithi090.stdout:6/914: chown d9/d77/cbb 75143 1 2024-04-26T04:42:30.949 INFO:tasks.workunit.client.0.smithi090.stdout:6/915: fdatasync d9/d21/d40/d4c/f7b 0 2024-04-26T04:42:30.952 INFO:tasks.workunit.client.0.smithi090.stdout:9/781: rename db/dd/d44 to db/d5f/d117 0 2024-04-26T04:42:30.952 INFO:tasks.workunit.client.0.smithi090.stdout:9/782: write db/dd/d18/d57/dcc/ff2 [268853,117661] 0 2024-04-26T04:42:30.952 INFO:tasks.workunit.client.0.smithi090.stdout:6/916: mkdir d9/d21/d40/d4c/d6f/d9f/dae/d13c 0 2024-04-26T04:42:30.953 INFO:tasks.workunit.client.0.smithi090.stdout:9/783: link db/d5f/d117/d82/d85/db4/d6a/db6/ld8 db/d5f/d117/d82/l118 0 2024-04-26T04:42:30.953 INFO:tasks.workunit.client.0.smithi090.stdout:6/917: truncate d9/d77/db0/db5/dda/d6d/d92/fa8 632844 0 2024-04-26T04:42:30.954 INFO:tasks.workunit.client.0.smithi090.stdout:6/918: dread - d9/d77/db0/db5/dca/dd8/f114 zero size 2024-04-26T04:42:30.954 INFO:tasks.workunit.client.0.smithi090.stdout:9/784: symlink db/d5f/d117/d82/dbd/d109/l119 0 2024-04-26T04:42:30.955 INFO:tasks.workunit.client.0.smithi090.stdout:6/919: creat d9/d1c/d31/d69/db7/dea/d139/f13d x:0 0 0 2024-04-26T04:42:30.956 INFO:tasks.workunit.client.0.smithi090.stdout:9/785: unlink db/dd/d18/d38/d39/d6b/de2/l1f 0 2024-04-26T04:42:30.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/786: mknod db/d5f/d117/d82/dbd/d109/c11a 0 2024-04-26T04:42:30.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/787: stat db/dd/d18/d38/d39/dba/ce0 0 2024-04-26T04:42:30.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/788: dread - db/d5f/d117/d82/f10d zero size 2024-04-26T04:42:30.964 INFO:tasks.workunit.client.0.smithi090.stdout:6/920: dread d9/f5c [0,4194304] 0 2024-04-26T04:42:30.967 INFO:tasks.workunit.client.0.smithi090.stdout:6/921: rename d9/d77/db0/db5/f129 to d9/d1c/d31/d122/f13e 0 2024-04-26T04:42:30.967 INFO:tasks.workunit.client.0.smithi090.stdout:6/922: write d9/d1c/d31/d69/f11c [588380,64356] 0 2024-04-26T04:42:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:30 smithi090 ceph-mon[26648]: pgmap v257: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 334 MiB/s rd, 418 MiB/s wr, 620 op/s 2024-04-26T04:42:30.978 INFO:tasks.workunit.client.0.smithi090.stdout:9/789: dwrite db/d5f/d117/d82/ff9 [0,4194304] 0 2024-04-26T04:42:30.985 INFO:tasks.workunit.client.0.smithi090.stdout:9/790: write db/d5f/d117/f8e [599065,55556] 0 2024-04-26T04:42:30.986 INFO:tasks.workunit.client.0.smithi090.stdout:9/791: creat db/dd/d18/d95/f11b x:0 0 0 2024-04-26T04:42:30.986 INFO:tasks.workunit.client.0.smithi090.stdout:9/792: readlink db/d5f/d117/l30 0 2024-04-26T04:42:30.989 INFO:tasks.workunit.client.0.smithi090.stdout:6/923: dwrite d9/d21/d3b/f51 [0,4194304] 0 2024-04-26T04:42:30.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/793: mknod db/d5f/d117/d82/d85/db4/d6a/c11c 0 2024-04-26T04:42:30.991 INFO:tasks.workunit.client.0.smithi090.stdout:9/794: truncate db/dd/d18/d95/fb9 382040 0 2024-04-26T04:42:30.999 INFO:tasks.workunit.client.0.smithi090.stdout:9/795: mkdir db/da3/d11d 0 2024-04-26T04:42:31.001 INFO:tasks.workunit.client.0.smithi090.stdout:9/796: dread db/dd/d18/d57/fbb [0,4194304] 0 2024-04-26T04:42:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:30 smithi104 ceph-mon[32369]: pgmap v257: 65 pgs: 65 active+clean; 3.6 GiB data, 10 GiB used, 526 GiB / 536 GiB avail; 334 MiB/s rd, 418 MiB/s wr, 620 op/s 2024-04-26T04:42:31.011 INFO:tasks.workunit.client.0.smithi090.stdout:9/797: read db/dd/d18/d38/d39/d6b/dc0/fbf [4146012,93770] 0 2024-04-26T04:42:31.013 INFO:tasks.workunit.client.0.smithi090.stdout:6/924: dwrite d9/d1c/d31/d69/f11c [0,4194304] 0 2024-04-26T04:42:31.025 INFO:tasks.workunit.client.0.smithi090.stdout:6/925: dwrite d9/d21/f48 [0,4194304] 0 2024-04-26T04:42:31.033 INFO:tasks.workunit.client.0.smithi090.stdout:6/926: dread d9/d77/db0/db5/dda/d6d/f12a [0,4194304] 0 2024-04-26T04:42:31.047 INFO:tasks.workunit.client.0.smithi090.stdout:6/927: dwrite d9/d21/d40/d4c/d6f/f100 [0,4194304] 0 2024-04-26T04:42:31.049 INFO:tasks.workunit.client.0.smithi090.stdout:6/928: symlink d9/d77/dc0/l13f 0 2024-04-26T04:42:31.055 INFO:tasks.workunit.client.0.smithi090.stdout:2/866: sync 2024-04-26T04:42:31.055 INFO:tasks.workunit.client.0.smithi090.stdout:5/922: sync 2024-04-26T04:42:31.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/867: symlink db/d11/d14/d25/d41/l120 0 2024-04-26T04:42:31.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/868: dread - db/d11/d14/d27/d28/dd1/dfe/d8b/f11c zero size 2024-04-26T04:42:31.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/869: chown db/d11/d14/d1b/d20/d4c/d98/lcb 37913 1 2024-04-26T04:42:31.057 INFO:tasks.workunit.client.0.smithi090.stdout:6/929: read d9/f15 [3836372,51411] 0 2024-04-26T04:42:31.057 INFO:tasks.workunit.client.0.smithi090.stdout:2/870: truncate db/d11/d14/d25/d2d/d48/d5b/f7d 3514066 0 2024-04-26T04:42:31.059 INFO:tasks.workunit.client.0.smithi090.stdout:6/930: unlink d9/d21/d3b/d86/fe1 0 2024-04-26T04:42:31.061 INFO:tasks.workunit.client.0.smithi090.stdout:6/931: mknod d9/d1c/d31/d69/dbc/c140 0 2024-04-26T04:42:31.063 INFO:tasks.workunit.client.0.smithi090.stdout:6/932: truncate d9/d1c/d31/f54 334897 0 2024-04-26T04:42:31.067 INFO:tasks.workunit.client.0.smithi090.stdout:6/933: link d9/d1c/d31/d69/dbc/cbd d9/d77/db0/db5/dda/d58/c141 0 2024-04-26T04:42:31.067 INFO:tasks.workunit.client.0.smithi090.stdout:6/934: chown d9/d77/db0/db5/dda/d6d/d92/c11b 135 1 2024-04-26T04:42:31.067 INFO:tasks.workunit.client.0.smithi090.stdout:5/923: dread d4/fe [0,4194304] 0 2024-04-26T04:42:31.072 INFO:tasks.workunit.client.0.smithi090.stdout:6/935: rename d9/d1c/dc5 to d9/d21/d3b/d142 0 2024-04-26T04:42:31.073 INFO:tasks.workunit.client.0.smithi090.stdout:6/936: symlink d9/d1c/d31/d122/l143 0 2024-04-26T04:42:31.075 INFO:tasks.workunit.client.0.smithi090.stdout:6/937: link d9/d1c/c60 d9/d77/db0/db5/dda/d6d/d66/c144 0 2024-04-26T04:42:31.078 INFO:tasks.workunit.client.0.smithi090.stdout:6/938: unlink d9/d21/d40/d4c/c10e 0 2024-04-26T04:42:31.078 INFO:tasks.workunit.client.0.smithi090.stdout:2/871: dwrite db/d11/d14/d27/d2f/d42/d5c/f80 [0,4194304] 0 2024-04-26T04:42:31.082 INFO:tasks.workunit.client.0.smithi090.stdout:6/939: truncate d9/d21/d3b/d142/dd9/fed 37867 0 2024-04-26T04:42:31.084 INFO:tasks.workunit.client.0.smithi090.stdout:2/872: dread db/d11/d14/d25/d2d/f36 [0,4194304] 0 2024-04-26T04:42:31.085 INFO:tasks.workunit.client.0.smithi090.stdout:2/873: rmdir db/d11/d14/d1b/d20/d4c 39 2024-04-26T04:42:31.087 INFO:tasks.workunit.client.0.smithi090.stdout:2/874: symlink db/d11/d14/d1b/d119/l121 0 2024-04-26T04:42:31.088 INFO:tasks.workunit.client.0.smithi090.stdout:2/875: rmdir db/d11/d14/d27/d28 39 2024-04-26T04:42:31.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/940: dread d9/d21/d3b/d86/fdc [0,4194304] 0 2024-04-26T04:42:31.091 INFO:tasks.workunit.client.0.smithi090.stdout:6/941: readlink d9/d1c/lf0 0 2024-04-26T04:42:31.091 INFO:tasks.workunit.client.0.smithi090.stdout:2/876: creat db/d11/d14/d1b/d20/d4c/d98/db4/f122 x:0 0 0 2024-04-26T04:42:31.093 INFO:tasks.workunit.client.0.smithi090.stdout:6/942: mknod d9/d1c/da2/c145 0 2024-04-26T04:42:31.093 INFO:tasks.workunit.client.0.smithi090.stdout:6/943: stat d9/d77/db0/db5/dda/d6d/d66/f9e 0 2024-04-26T04:42:31.093 INFO:tasks.workunit.client.0.smithi090.stdout:6/944: fsync d9/d21/d3b/d142/dd9/fe4 0 2024-04-26T04:42:31.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/877: symlink db/d11/d14/d25/d2d/d48/d5b/d107/l123 0 2024-04-26T04:42:31.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/878: fdatasync db/d11/d14/d25/d2d/d48/d5b/f7d 0 2024-04-26T04:42:31.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/879: readlink db/d11/d14/l18 0 2024-04-26T04:42:31.094 INFO:tasks.workunit.client.0.smithi090.stdout:2/880: dread - db/d11/d14/d1b/d20/f115 zero size 2024-04-26T04:42:31.097 INFO:tasks.workunit.client.0.smithi090.stdout:6/945: rename d9/d77/f9d to d9/d1c/d31/d69/f146 0 2024-04-26T04:42:31.099 INFO:tasks.workunit.client.0.smithi090.stdout:2/881: creat db/d59/f124 x:0 0 0 2024-04-26T04:42:31.108 INFO:tasks.workunit.client.0.smithi090.stdout:2/882: dread db/dc/f3c [4194304,4194304] 0 2024-04-26T04:42:31.116 INFO:tasks.workunit.client.0.smithi090.stdout:6/946: dwrite d9/f11 [0,4194304] 0 2024-04-26T04:42:31.116 INFO:tasks.workunit.client.0.smithi090.stdout:6/947: write d9/f68 [4468868,24937] 0 2024-04-26T04:42:31.118 INFO:tasks.workunit.client.0.smithi090.stdout:2/883: write db/f84 [3793549,39744] 0 2024-04-26T04:42:31.121 INFO:tasks.workunit.client.0.smithi090.stdout:2/884: getdents db/d59/d66/d77 0 2024-04-26T04:42:31.140 INFO:tasks.workunit.client.0.smithi090.stdout:2/885: dwrite db/d59/f124 [0,4194304] 0 2024-04-26T04:42:31.143 INFO:tasks.workunit.client.0.smithi090.stdout:2/886: creat db/d11/d14/d25/d41/f125 x:0 0 0 2024-04-26T04:42:31.144 INFO:tasks.workunit.client.0.smithi090.stdout:2/887: mkdir db/d11/d14/d1b/d126 0 2024-04-26T04:42:31.146 INFO:tasks.workunit.client.0.smithi090.stdout:2/888: rmdir db/d11/d14/d63/d11a 0 2024-04-26T04:42:31.146 INFO:tasks.workunit.client.0.smithi090.stdout:2/889: chown db/d11/d14/d27/d28/l10e 1 1 2024-04-26T04:42:31.147 INFO:tasks.workunit.client.0.smithi090.stdout:2/890: symlink db/d11/d14/d25/d2d/l127 0 2024-04-26T04:42:31.147 INFO:tasks.workunit.client.0.smithi090.stdout:2/891: truncate db/d11/d14/d27/d2f/f32 693024 0 2024-04-26T04:42:31.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/892: unlink db/d11/d14/d27/ld4 0 2024-04-26T04:42:31.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/893: fdatasync db/d11/d14/fc0 0 2024-04-26T04:42:31.149 INFO:tasks.workunit.client.0.smithi090.stdout:2/894: creat db/d11/d14/f128 x:0 0 0 2024-04-26T04:42:31.171 INFO:tasks.workunit.client.0.smithi090.stdout:2/895: dwrite db/d11/d14/d25/fe5 [0,4194304] 0 2024-04-26T04:42:31.196 INFO:tasks.workunit.client.0.smithi090.stdout:2/896: dwrite db/d11/d14/d25/d2d/f6a [0,4194304] 0 2024-04-26T04:42:31.196 INFO:tasks.workunit.client.0.smithi090.stdout:2/897: readlink db/d11/l2e 0 2024-04-26T04:42:31.199 INFO:tasks.workunit.client.0.smithi090.stdout:2/898: rename db/d11/d14/d27/d2f/d42/fdd to db/d11/d14/d25/d2d/d48/f129 0 2024-04-26T04:42:31.204 INFO:tasks.workunit.client.0.smithi090.stdout:2/899: dread db/d11/d14/d27/d2f/d42/d5c/f4f [0,4194304] 0 2024-04-26T04:42:31.208 INFO:tasks.workunit.client.0.smithi090.stdout:1/902: sync 2024-04-26T04:42:31.209 INFO:tasks.workunit.client.0.smithi090.stdout:1/903: read d0/d55/dda/d7/d4a/f77 [931799,89599] 0 2024-04-26T04:42:31.213 INFO:tasks.workunit.client.0.smithi090.stdout:1/904: write d0/d55/dbd/ff8 [3630518,125744] 0 2024-04-26T04:42:31.215 INFO:tasks.workunit.client.0.smithi090.stdout:1/905: symlink d0/d55/dda/l126 0 2024-04-26T04:42:31.215 INFO:tasks.workunit.client.0.smithi090.stdout:1/906: readlink d0/d55/dda/d7/d17/d22/d31/d108/l10d 0 2024-04-26T04:42:31.218 INFO:tasks.workunit.client.0.smithi090.stdout:1/907: write d0/d55/dda/f91 [4961070,15302] 0 2024-04-26T04:42:31.219 INFO:tasks.workunit.client.0.smithi090.stdout:1/908: creat d0/d55/dda/d7/d4b/f127 x:0 0 0 2024-04-26T04:42:31.219 INFO:tasks.workunit.client.0.smithi090.stdout:1/909: fdatasync d0/d55/dda/d7/d17/d22/d31/f6c 0 2024-04-26T04:42:31.220 INFO:tasks.workunit.client.0.smithi090.stdout:1/910: mkdir d0/d55/dda/d7/d17/d22/d93/db8/de9/d11c/d128 0 2024-04-26T04:42:31.221 INFO:tasks.workunit.client.0.smithi090.stdout:1/911: creat d0/d55/dda/d7/d17/f129 x:0 0 0 2024-04-26T04:42:31.221 INFO:tasks.workunit.client.0.smithi090.stdout:1/912: mknod d0/d55/dda/d7/d4b/dca/d123/c12a 0 2024-04-26T04:42:31.223 INFO:tasks.workunit.client.0.smithi090.stdout:1/913: link d0/d55/dda/d7/d4a/da8/fad d0/d55/dbd/f12b 0 2024-04-26T04:42:31.231 INFO:tasks.workunit.client.0.smithi090.stdout:1/914: dread d0/d55/dda/d62/d78/f8f [0,4194304] 0 2024-04-26T04:42:31.232 INFO:tasks.workunit.client.0.smithi090.stdout:1/915: mkdir d0/d55/dda/d7/d4b/d12c 0 2024-04-26T04:42:31.238 INFO:tasks.workunit.client.0.smithi090.stdout:1/916: rename d0/d55/dda/d7/d17/d22/d93 to d0/d55/dda/d43/d11b/d12d 0 2024-04-26T04:42:31.238 INFO:tasks.workunit.client.0.smithi090.stdout:1/917: chown d0/c9a 1 1 2024-04-26T04:42:31.239 INFO:tasks.workunit.client.0.smithi090.stdout:1/918: dread d0/d55/f63 [0,4194304] 0 2024-04-26T04:42:31.239 INFO:tasks.workunit.client.0.smithi090.stdout:1/919: stat d0/d55/dda/d7/d4a/d75/d80/d58/fbf 0 2024-04-26T04:42:31.240 INFO:tasks.workunit.client.0.smithi090.stdout:1/920: creat d0/d55/dda/d7/d4b/d53/f12e x:0 0 0 2024-04-26T04:42:31.241 INFO:tasks.workunit.client.0.smithi090.stdout:1/921: mknod d0/d55/c12f 0 2024-04-26T04:42:31.261 INFO:tasks.workunit.client.0.smithi090.stdout:1/922: dwrite d0/d55/dda/d7/d4a/f51 [0,4194304] 0 2024-04-26T04:42:31.261 INFO:tasks.workunit.client.0.smithi090.stdout:1/923: truncate d0/d55/dda/d7/d4b/dca/f118 927822 0 2024-04-26T04:42:31.262 INFO:tasks.workunit.client.0.smithi090.stdout:1/924: link d0/d55/dda/d7/d4a/c4f d0/d55/dda/d7/d4b/c130 0 2024-04-26T04:42:31.263 INFO:tasks.workunit.client.0.smithi090.stdout:1/925: rmdir d0/d55/dda/d62/d78/dcc 39 2024-04-26T04:42:31.270 INFO:tasks.workunit.client.0.smithi090.stdout:1/926: read d0/d55/dda/d7/d17/d22/d31/f6d [2091650,91113] 0 2024-04-26T04:42:31.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/927: getdents d0/d55/dda/d7/d49 0 2024-04-26T04:42:31.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/928: fdatasync d0/d55/dda/d7/d4a/d75/d80/d58/fbf 0 2024-04-26T04:42:31.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/929: dread - d0/d55/dda/d7/d4b/d53/f12e zero size 2024-04-26T04:42:31.272 INFO:tasks.workunit.client.0.smithi090.stdout:1/930: chown d0/d55/f83 48 1 2024-04-26T04:42:31.276 INFO:tasks.workunit.client.0.smithi090.stdout:1/931: write d0/d55/dda/d7/d4a/f96 [32042,77401] 0 2024-04-26T04:42:31.277 INFO:tasks.workunit.client.0.smithi090.stdout:1/932: mkdir d0/d55/dda/d43/d11b/d12d/d131 0 2024-04-26T04:42:31.278 INFO:tasks.workunit.client.0.smithi090.stdout:1/933: creat d0/f132 x:0 0 0 2024-04-26T04:42:31.279 INFO:tasks.workunit.client.0.smithi090.stdout:1/934: creat d0/d55/dda/d43/d11b/d12d/f133 x:0 0 0 2024-04-26T04:42:31.280 INFO:tasks.workunit.client.0.smithi090.stdout:1/935: link d0/d55/dda/d7/d17/d22/f89 d0/d55/dda/d43/d11b/d12d/d131/f134 0 2024-04-26T04:42:31.281 INFO:tasks.workunit.client.0.smithi090.stdout:1/936: creat d0/d55/dda/d43/d11b/d12d/db0/dfc/d104/f135 x:0 0 0 2024-04-26T04:42:31.281 INFO:tasks.workunit.client.0.smithi090.stdout:1/937: creat d0/d55/f136 x:0 0 0 2024-04-26T04:42:31.283 INFO:tasks.workunit.client.0.smithi090.stdout:1/938: dread d0/d55/dda/d43/d11b/d12d/ff6 [0,4194304] 0 2024-04-26T04:42:31.283 INFO:tasks.workunit.client.0.smithi090.stdout:1/939: creat d0/d55/dda/d7/d4a/d75/d80/d58/d97/f137 x:0 0 0 2024-04-26T04:42:31.283 INFO:tasks.workunit.client.0.smithi090.stdout:1/940: fsync d0/d55/dda/d7/d4a/f105 0 2024-04-26T04:42:31.285 INFO:tasks.workunit.client.0.smithi090.stdout:1/941: symlink d0/d55/dda/d7/d4a/d75/d80/d58/l138 0 2024-04-26T04:42:31.285 INFO:tasks.workunit.client.0.smithi090.stdout:1/942: stat d0/c11 0 2024-04-26T04:42:31.286 INFO:tasks.workunit.client.0.smithi090.stdout:1/943: mknod d0/d55/dda/d7/d4a/da8/c139 0 2024-04-26T04:42:31.287 INFO:tasks.workunit.client.0.smithi090.stdout:1/944: rmdir d0/d55/dda/d7/d4a/da8 39 2024-04-26T04:42:31.304 INFO:tasks.workunit.client.0.smithi090.stdout:1/945: dwrite d0/d55/dda/d7/d4a/d75/d80/d58/d97/f137 [0,4194304] 0 2024-04-26T04:42:31.312 INFO:tasks.workunit.client.0.smithi090.stdout:1/946: write d0/f86 [2547460,125699] 0 2024-04-26T04:42:31.313 INFO:tasks.workunit.client.0.smithi090.stdout:1/947: symlink d0/d55/dda/d7/d69/l13a 0 2024-04-26T04:42:31.314 INFO:tasks.workunit.client.0.smithi090.stdout:1/948: chown d0/d55/dda/d7/d4a/d75/d115 28755454 1 2024-04-26T04:42:31.314 INFO:tasks.workunit.client.0.smithi090.stdout:1/949: readlink d0/d55/dda/d7/d4a/d75/d80/d58/la3 0 2024-04-26T04:42:31.314 INFO:tasks.workunit.client.0.smithi090.stdout:1/950: creat d0/d55/dda/d7/d17/d22/f13b x:0 0 0 2024-04-26T04:42:31.315 INFO:tasks.workunit.client.0.smithi090.stdout:1/951: getdents d0/d55 0 2024-04-26T04:42:31.339 INFO:tasks.workunit.client.0.smithi090.stdout:1/952: dwrite d0/d55/dda/d7/d4a/d75/d80/d58/fe2 [0,4194304] 0 2024-04-26T04:42:31.341 INFO:tasks.workunit.client.0.smithi090.stdout:1/953: creat d0/d55/dda/d43/d11b/d12d/db8/de9/d11c/d128/f13c x:0 0 0 2024-04-26T04:42:31.341 INFO:tasks.workunit.client.0.smithi090.stdout:1/954: symlink d0/d55/dda/d43/d11b/d12d/d131/l13d 0 2024-04-26T04:42:31.344 INFO:tasks.workunit.client.0.smithi090.stdout:1/955: getdents d0/d55/dda/d43/d11b/d12d/db8/de9/d11c/d128 0 2024-04-26T04:42:31.347 INFO:tasks.workunit.client.0.smithi090.stdout:1/956: write d0/d55/dda/d62/d78/f8f [2679917,121597] 0 2024-04-26T04:42:31.348 INFO:tasks.workunit.client.0.smithi090.stdout:1/957: mknod d0/d55/dda/d62/c13e 0 2024-04-26T04:42:31.349 INFO:tasks.workunit.client.0.smithi090.stdout:1/958: creat d0/f13f x:0 0 0 2024-04-26T04:42:31.349 INFO:tasks.workunit.client.0.smithi090.stdout:1/959: readlink d0/d55/dda/d7/d4b/d53/l65 0 2024-04-26T04:42:31.349 INFO:tasks.workunit.client.0.smithi090.stdout:1/960: truncate d0/d55/dda/d43/d11b/d12d/db8/de9/f10c 64011 0 2024-04-26T04:42:31.350 INFO:tasks.workunit.client.0.smithi090.stdout:1/961: rename d0/d55/dda/d7/d17/d22/d31/f6d to d0/d55/dda/d7/d17/f140 0 2024-04-26T04:42:31.354 INFO:tasks.workunit.client.0.smithi090.stdout:1/962: dread d0/d55/dda/d7/d17/f3c [0,4194304] 0 2024-04-26T04:42:31.354 INFO:tasks.workunit.client.0.smithi090.stdout:1/963: write d0/f12 [1793808,45489] 0 2024-04-26T04:42:31.355 INFO:tasks.workunit.client.0.smithi090.stdout:1/964: mknod d0/d55/dda/d62/d78/dcc/c141 0 2024-04-26T04:42:31.360 INFO:tasks.workunit.client.0.smithi090.stdout:1/965: rename d0/d55/dda/d62/d78/dcc to d0/d55/dda/d7/d142 0 2024-04-26T04:42:31.361 INFO:tasks.workunit.client.0.smithi090.stdout:1/966: mknod d0/d55/dda/d7/d49/c143 0 2024-04-26T04:42:31.362 INFO:tasks.workunit.client.0.smithi090.stdout:4/855: sync 2024-04-26T04:42:31.362 INFO:tasks.workunit.client.0.smithi090.stdout:3/832: sync 2024-04-26T04:42:31.363 INFO:tasks.workunit.client.0.smithi090.stdout:1/967: fsync d0/d55/dda/d7/d17/f29 0 2024-04-26T04:42:31.364 INFO:tasks.workunit.client.0.smithi090.stdout:4/856: truncate d3/db/d12/d20/d2d/d93/d102/dd4/fb7 340698 0 2024-04-26T04:42:31.366 INFO:tasks.workunit.client.0.smithi090.stdout:1/968: rename d0/d55/dda/d7/d17/d22/f68 to d0/d55/dda/d7/d4a/d75/f144 0 2024-04-26T04:42:31.366 INFO:tasks.workunit.client.0.smithi090.stdout:1/969: chown d0/d55/dda/d7/d4b/f127 60 1 2024-04-26T04:42:31.373 INFO:tasks.workunit.client.0.smithi090.stdout:1/970: rename d0/d55/dda/d7/d4b/dca/ff0 to d0/d55/dda/d7/d142/f145 0 2024-04-26T04:42:31.374 INFO:tasks.workunit.client.0.smithi090.stdout:4/857: mknod d3/db/d12/c118 0 2024-04-26T04:42:31.374 INFO:tasks.workunit.client.0.smithi090.stdout:4/858: creat d3/db/d12/d20/d24/da9/f119 x:0 0 0 2024-04-26T04:42:31.381 INFO:tasks.workunit.client.0.smithi090.stdout:3/833: dwrite d1/d59/ff0 [0,4194304] 0 2024-04-26T04:42:31.386 INFO:tasks.workunit.client.0.smithi090.stdout:4/859: write d3/f2c [23239,51330] 0 2024-04-26T04:42:31.386 INFO:tasks.workunit.client.0.smithi090.stdout:4/860: truncate f1 321216 0 2024-04-26T04:42:31.391 INFO:tasks.workunit.client.0.smithi090.stdout:4/861: rename d3/db/d12/d99 to d3/db/d60/dee/d11a 0 2024-04-26T04:42:31.393 INFO:tasks.workunit.client.0.smithi090.stdout:4/862: link d3/db/d12/d20/d24/f52 d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/f11b 0 2024-04-26T04:42:31.393 INFO:tasks.workunit.client.0.smithi090.stdout:4/863: readlink d3/l33 0 2024-04-26T04:42:31.395 INFO:tasks.workunit.client.0.smithi090.stdout:4/864: mkdir d3/db/d12/d20/d2d/dbe/dde/d11c 0 2024-04-26T04:42:31.395 INFO:tasks.workunit.client.0.smithi090.stdout:4/865: creat d3/db/d12/d20/d2d/d58/d6a/d74/d85/f11d x:0 0 0 2024-04-26T04:42:31.399 INFO:tasks.workunit.client.0.smithi090.stdout:1/971: dwrite d0/d55/dda/d7/d4a/f77 [0,4194304] 0 2024-04-26T04:42:31.402 INFO:tasks.workunit.client.0.smithi090.stdout:1/972: symlink d0/d55/dda/d43/d11b/d12d/d131/l146 0 2024-04-26T04:42:31.402 INFO:tasks.workunit.client.0.smithi090.stdout:1/973: write d0/d55/dda/d7/d17/d22/ff9 [944897,88116] 0 2024-04-26T04:42:31.403 INFO:tasks.workunit.client.0.smithi090.stdout:1/974: stat d0/d55/dda/d7/d4a/ff5 0 2024-04-26T04:42:31.403 INFO:tasks.workunit.client.0.smithi090.stdout:1/975: dread - d0/d55/dda/d7/d4a/d75/f101 zero size 2024-04-26T04:42:31.403 INFO:tasks.workunit.client.0.smithi090.stdout:3/834: dwrite d1/d5c/f6e [0,4194304] 0 2024-04-26T04:42:31.403 INFO:tasks.workunit.client.0.smithi090.stdout:3/835: fdatasync d1/d17/d36/d2a/d3a/f114 0 2024-04-26T04:42:31.406 INFO:tasks.workunit.client.0.smithi090.stdout:3/836: mkdir d1/d17/d36/d33/d42/d70/d11b 0 2024-04-26T04:42:31.406 INFO:tasks.workunit.client.0.smithi090.stdout:3/837: rmdir d1/d17/d36/d33/d68 39 2024-04-26T04:42:31.408 INFO:tasks.workunit.client.0.smithi090.stdout:3/838: link d1/d9/fd6 d1/d17/d36/d33/d42/f11c 0 2024-04-26T04:42:31.408 INFO:tasks.workunit.client.0.smithi090.stdout:3/839: rmdir d1/d17/d36/d2a 39 2024-04-26T04:42:31.408 INFO:tasks.workunit.client.0.smithi090.stdout:3/840: fsync d1/d17/d61/d9d/fec 0 2024-04-26T04:42:31.409 INFO:tasks.workunit.client.0.smithi090.stdout:3/841: dread - d1/d17/d61/dce/f10e zero size 2024-04-26T04:42:31.409 INFO:tasks.workunit.client.0.smithi090.stdout:3/842: dread - d1/d5c/ff2 zero size 2024-04-26T04:42:31.409 INFO:tasks.workunit.client.0.smithi090.stdout:3/843: write d1/d17/d36/d33/d42/faf [126908,107147] 0 2024-04-26T04:42:31.409 INFO:tasks.workunit.client.0.smithi090.stdout:3/844: symlink d1/d59/d5e/l11d 0 2024-04-26T04:42:31.412 INFO:tasks.workunit.client.0.smithi090.stdout:4/866: dwrite d3/db/d12/d3b/f78 [0,4194304] 0 2024-04-26T04:42:31.415 INFO:tasks.workunit.client.0.smithi090.stdout:4/867: dread d3/db/d12/d20/d2d/d93/d102/dd4/fb7 [0,4194304] 0 2024-04-26T04:42:31.416 INFO:tasks.workunit.client.0.smithi090.stdout:4/868: dread - d3/db/d60/dee/dc1/dcf/f116 zero size 2024-04-26T04:42:31.416 INFO:tasks.workunit.client.0.smithi090.stdout:4/869: write d3/db/d12/d20/d2d/d93/d102/d72/f80 [1290704,120215] 0 2024-04-26T04:42:31.417 INFO:tasks.workunit.client.0.smithi090.stdout:3/845: read d1/d17/d36/f67 [2299782,89754] 0 2024-04-26T04:42:31.420 INFO:tasks.workunit.client.0.smithi090.stdout:4/870: creat d3/db/d12/d20/d2d/d58/d6a/d74/da2/f11e x:0 0 0 2024-04-26T04:42:31.421 INFO:tasks.workunit.client.0.smithi090.stdout:3/846: truncate d1/d17/d36/d33/d68/dae/dca/dcf/d106/d7e/fe1 3530264 0 2024-04-26T04:42:31.423 INFO:tasks.workunit.client.0.smithi090.stdout:4/871: mkdir d3/db/d12/d20/d2d/d93/d102/dd4/d11f 0 2024-04-26T04:42:31.426 INFO:tasks.workunit.client.0.smithi090.stdout:0/956: sync 2024-04-26T04:42:31.428 INFO:tasks.workunit.client.0.smithi090.stdout:0/957: mkdir d0/de/d12d/d134/d149 0 2024-04-26T04:42:31.428 INFO:tasks.workunit.client.0.smithi090.stdout:0/958: write d0/d4b/d5d/d5c/d8a/d45/d53/dab/ffd [1002352,127037] 0 2024-04-26T04:42:31.432 INFO:tasks.workunit.client.0.smithi090.stdout:0/959: creat d0/d4b/db3/dcc/f14a x:0 0 0 2024-04-26T04:42:31.434 INFO:tasks.workunit.client.0.smithi090.stdout:4/872: dread d3/d13/f1f [0,4194304] 0 2024-04-26T04:42:31.434 INFO:tasks.workunit.client.0.smithi090.stdout:0/960: mknod d0/d4b/d5d/d5c/d8a/c14b 0 2024-04-26T04:42:31.434 INFO:tasks.workunit.client.0.smithi090.stdout:0/961: readlink d0/d4b/d5d/d5c/d8a/l83 0 2024-04-26T04:42:31.435 INFO:tasks.workunit.client.0.smithi090.stdout:4/873: creat d3/db/d60/dee/dfc/f120 x:0 0 0 2024-04-26T04:42:31.436 INFO:tasks.workunit.client.0.smithi090.stdout:4/874: write d3/f2c [1883519,119466] 0 2024-04-26T04:42:31.436 INFO:tasks.workunit.client.0.smithi090.stdout:4/875: write d3/d13/d6c/fbb [621773,71162] 0 2024-04-26T04:42:31.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/962: mknod d0/d4b/d5d/d48/c14c 0 2024-04-26T04:42:31.436 INFO:tasks.workunit.client.0.smithi090.stdout:0/963: truncate d0/d4b/d5d/f6a 71796 0 2024-04-26T04:42:31.437 INFO:tasks.workunit.client.0.smithi090.stdout:0/964: write d0/d4b/feb [718407,39469] 0 2024-04-26T04:42:31.438 INFO:tasks.workunit.client.0.smithi090.stdout:3/847: dwrite d1/d17/d61/dce/f10e [0,4194304] 0 2024-04-26T04:42:31.438 INFO:tasks.workunit.client.0.smithi090.stdout:0/965: mknod d0/d4b/d5d/d5c/d8a/d45/d53/dc2/c14d 0 2024-04-26T04:42:31.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/876: rmdir d3/db/d12/d20/d2d/d2e 39 2024-04-26T04:42:31.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/877: creat d3/d13/d6c/dba/f121 x:0 0 0 2024-04-26T04:42:31.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/878: write d3/db/d12/d20/d2d/d2e/ff8 [693377,84803] 0 2024-04-26T04:42:31.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/879: chown d3/db/d12/d20/d2d/d58/d6a/da3/f10a 23712753 1 2024-04-26T04:42:31.444 INFO:tasks.workunit.client.0.smithi090.stdout:0/966: mkdir d0/d4b/db3/d14e 0 2024-04-26T04:42:31.444 INFO:tasks.workunit.client.0.smithi090.stdout:0/967: write d0/d4b/d5d/d5c/d8a/d45/d53/faf [3313369,23136] 0 2024-04-26T04:42:31.445 INFO:tasks.workunit.client.0.smithi090.stdout:0/968: read d0/d4b/d5d/d5c/d8a/d7c/fdb [425354,89856] 0 2024-04-26T04:42:31.450 INFO:tasks.workunit.client.0.smithi090.stdout:0/969: unlink d0/d4b/f114 0 2024-04-26T04:42:31.451 INFO:tasks.workunit.client.0.smithi090.stdout:0/970: write d0/d4b/d5d/fb1 [391347,111269] 0 2024-04-26T04:42:31.452 INFO:tasks.workunit.client.0.smithi090.stdout:0/971: creat d0/d4b/d5d/d5c/d8a/d45/d53/dc2/dfa/f14f x:0 0 0 2024-04-26T04:42:31.453 INFO:tasks.workunit.client.0.smithi090.stdout:0/972: truncate d0/d4b/d5d/d48/d7d/fb2 573631 0 2024-04-26T04:42:31.453 INFO:tasks.workunit.client.0.smithi090.stdout:0/973: mknod d0/d4b/d5d/d5c/d8a/d64/c150 0 2024-04-26T04:42:31.455 INFO:tasks.workunit.client.0.smithi090.stdout:0/974: link d0/d4b/db3/dcc/c135 d0/d4b/d5d/d5c/d117/c151 0 2024-04-26T04:42:31.455 INFO:tasks.workunit.client.0.smithi090.stdout:0/975: symlink d0/d4b/d5d/d5c/d117/l152 0 2024-04-26T04:42:31.457 INFO:tasks.workunit.client.0.smithi090.stdout:0/976: mkdir d0/d4b/d6d/df2/d153 0 2024-04-26T04:42:31.457 INFO:tasks.workunit.client.0.smithi090.stdout:0/977: dread - d0/de/d103/d105/f13f zero size 2024-04-26T04:42:31.458 INFO:tasks.workunit.client.0.smithi090.stdout:0/978: rename d0/d4b/d5d/d48/d100/cfb to d0/d4b/d5d/d5c/d8a/d45/dcf/c154 0 2024-04-26T04:42:31.458 INFO:tasks.workunit.client.0.smithi090.stdout:0/979: creat d0/d106/f155 x:0 0 0 2024-04-26T04:42:31.459 INFO:tasks.workunit.client.0.smithi090.stdout:0/980: mknod d0/d4b/d5d/d5c/d117/c156 0 2024-04-26T04:42:31.459 INFO:tasks.workunit.client.0.smithi090.stdout:0/981: chown d0/d4b/d5d/d5c/d8a/d7c/fdb 0 1 2024-04-26T04:42:31.465 INFO:tasks.workunit.client.0.smithi090.stdout:4/880: dwrite d3/db/f23 [0,4194304] 0 2024-04-26T04:42:31.481 INFO:tasks.workunit.client.0.smithi090.stdout:0/982: dwrite d0/de/d103/d105/f13f [0,4194304] 0 2024-04-26T04:42:31.481 INFO:tasks.workunit.client.0.smithi090.stdout:4/881: dwrite d3/d13/d6c/dba/f121 [0,4194304] 0 2024-04-26T04:42:31.484 INFO:tasks.workunit.client.0.smithi090.stdout:4/882: symlink d3/db/d60/l122 0 2024-04-26T04:42:31.484 INFO:tasks.workunit.client.0.smithi090.stdout:4/883: stat d3/db/d60/l122 0 2024-04-26T04:42:31.485 INFO:tasks.workunit.client.0.smithi090.stdout:4/884: symlink d3/db/d12/d20/d2d/dbe/dde/l123 0 2024-04-26T04:42:31.487 INFO:tasks.workunit.client.0.smithi090.stdout:4/885: link d3/db/d60/d5c/lb5 d3/db/d12/d20/d2d/d58/d6a/d74/l124 0 2024-04-26T04:42:31.488 INFO:tasks.workunit.client.0.smithi090.stdout:0/983: write d0/d4b/d5d/d5c/d8a/d45/d93/fd5 [205547,83466] 0 2024-04-26T04:42:31.491 INFO:tasks.workunit.client.0.smithi090.stdout:0/984: rename d0/d4b/d5d/d48/d7d/lbe to d0/de/d12d/d134/l157 0 2024-04-26T04:42:31.491 INFO:tasks.workunit.client.0.smithi090.stdout:0/985: fsync d0/d4b/d5d/d48/f11b 0 2024-04-26T04:42:31.494 INFO:tasks.workunit.client.0.smithi090.stdout:0/986: creat d0/d4b/d5d/d5c/d8a/d45/d53/dab/d115/f158 x:0 0 0 2024-04-26T04:42:31.495 INFO:tasks.workunit.client.0.smithi090.stdout:0/987: dread d0/de/f89 [0,4194304] 0 2024-04-26T04:42:31.496 INFO:tasks.workunit.client.0.smithi090.stdout:0/988: mknod d0/d4b/d5d/d5c/d8a/d45/d93/c159 0 2024-04-26T04:42:31.506 INFO:tasks.workunit.client.0.smithi090.stdout:0/989: write d0/d4b/d6d/fc6 [1287765,64717] 0 2024-04-26T04:42:31.506 INFO:tasks.workunit.client.0.smithi090.stdout:0/990: dread - d0/d4b/d5d/d5c/f9c zero size 2024-04-26T04:42:31.507 INFO:tasks.workunit.client.0.smithi090.stdout:0/991: stat d0/fdc 0 2024-04-26T04:42:31.507 INFO:tasks.workunit.client.0.smithi090.stdout:0/992: getdents d0/d4b/d6d/d13d 0 2024-04-26T04:42:31.508 INFO:tasks.workunit.client.0.smithi090.stdout:0/993: creat d0/de/d103/d147/f15a x:0 0 0 2024-04-26T04:42:31.511 INFO:tasks.workunit.client.0.smithi090.stdout:0/994: rename d0/d4b/d6d/c98 to d0/d4b/d6d/d12f/c15b 0 2024-04-26T04:42:31.511 INFO:tasks.workunit.client.0.smithi090.stdout:0/995: write d0/de/f90 [485365,315] 0 2024-04-26T04:42:31.512 INFO:tasks.workunit.client.0.smithi090.stdout:0/996: truncate d0/d4b/db3/fcb 451267 0 2024-04-26T04:42:31.514 INFO:tasks.workunit.client.0.smithi090.stdout:0/997: mkdir d0/d4b/d6d/d12f/d15c 0 2024-04-26T04:42:31.514 INFO:tasks.workunit.client.0.smithi090.stdout:0/998: read d0/d4b/d5d/d5c/d117/d120/d123/f12b [158337,100457] 0 2024-04-26T04:42:31.522 INFO:tasks.workunit.client.0.smithi090.stdout:0/999: write d0/d4b/d5d/d5c/d8a/d45/f96 [1841905,60796] 0 2024-04-26T04:42:31.645 INFO:tasks.workunit.client.0.smithi090.stdout:5/924: sync 2024-04-26T04:42:31.645 INFO:tasks.workunit.client.0.smithi090.stdout:2/900: sync 2024-04-26T04:42:31.645 INFO:tasks.workunit.client.0.smithi090.stdout:6/948: sync 2024-04-26T04:42:31.645 INFO:tasks.workunit.client.0.smithi090.stdout:9/798: sync 2024-04-26T04:42:31.647 INFO:tasks.workunit.client.0.smithi090.stdout:9/799: mknod db/d5f/d117/d82/d85/db4/c11e 0 2024-04-26T04:42:31.648 INFO:tasks.workunit.client.0.smithi090.stdout:6/949: getdents d9/d1c/d31 0 2024-04-26T04:42:31.649 INFO:tasks.workunit.client.0.smithi090.stdout:9/800: mknod db/dd/d18/d38/d39/c11f 0 2024-04-26T04:42:31.650 INFO:tasks.workunit.client.0.smithi090.stdout:6/950: mknod d9/d21/d40/dc7/c147 0 2024-04-26T04:42:31.650 INFO:tasks.workunit.client.0.smithi090.stdout:6/951: read d9/d1c/d31/d79/fba [502548,125542] 0 2024-04-26T04:42:31.652 INFO:tasks.workunit.client.0.smithi090.stdout:9/801: truncate db/f13 1476964 0 2024-04-26T04:42:31.653 INFO:tasks.workunit.client.0.smithi090.stdout:5/925: read d4/d1a/d45/f4f [2113053,18036] 0 2024-04-26T04:42:31.654 INFO:tasks.workunit.client.0.smithi090.stdout:2/901: write db/d11/d14/d27/d2f/f8d [2691264,36651] 0 2024-04-26T04:42:31.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/926: symlink d4/d1a/d1f/l11f 0 2024-04-26T04:42:31.658 INFO:tasks.workunit.client.0.smithi090.stdout:5/927: chown d4/d1a/d1f 2 1 2024-04-26T04:42:31.660 INFO:tasks.workunit.client.0.smithi090.stdout:9/802: link db/dd/d18/d57/lbc db/dd/l120 0 2024-04-26T04:42:31.660 INFO:tasks.workunit.client.0.smithi090.stdout:9/803: stat db/d5f/d117/d82/d85/faf 0 2024-04-26T04:42:31.660 INFO:tasks.workunit.client.0.smithi090.stdout:9/804: truncate db/d5f/d117/f8e 5017878 0 2024-04-26T04:42:31.660 INFO:tasks.workunit.client.0.smithi090.stdout:5/928: creat d4/d1a/f120 x:0 0 0 2024-04-26T04:42:31.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/929: unlink d4/d1a/c39 0 2024-04-26T04:42:31.664 INFO:tasks.workunit.client.0.smithi090.stdout:5/930: dread - d4/d1a/df/d36/d8e/f109 zero size 2024-04-26T04:42:31.666 INFO:tasks.workunit.client.0.smithi090.stdout:5/931: creat d4/d1a/d45/f121 x:0 0 0 2024-04-26T04:42:31.669 INFO:tasks.workunit.client.0.smithi090.stdout:9/805: rmdir db/dd/d18/d38/d108 0 2024-04-26T04:42:31.669 INFO:tasks.workunit.client.0.smithi090.stdout:9/806: fsync db/dd/d18/d38/d39/d3f/f76 0 2024-04-26T04:42:31.669 INFO:tasks.workunit.client.0.smithi090.stdout:9/807: truncate db/d5f/d117/d82/d85/db4/d6a/d70/fc9 274838 0 2024-04-26T04:42:31.671 INFO:tasks.workunit.client.0.smithi090.stdout:6/952: dwrite d9/d1c/d31/d79/fba [0,4194304] 0 2024-04-26T04:42:31.672 INFO:tasks.workunit.client.0.smithi090.stdout:5/932: mkdir d4/d1a/d2a/d79/d90/d56/d71/dea/d122 0 2024-04-26T04:42:31.686 INFO:tasks.workunit.client.0.smithi090.stdout:6/953: dread d9/d1c/f47 [4194304,4194304] 0 2024-04-26T04:42:31.686 INFO:tasks.workunit.client.0.smithi090.stdout:6/954: creat d9/d21/f148 x:0 0 0 2024-04-26T04:42:31.686 INFO:tasks.workunit.client.0.smithi090.stdout:6/955: read - d9/d1c/d31/d69/db7/dea/d139/f13d zero size 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:6/956: rmdir d9/d77/db0/db5/dca 39 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:9/808: dread db/dd/d18/d38/f79 [0,4194304] 0 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/902: dwrite db/d11/d14/d27/d28/dd1/dfe/d71/f83 [0,4194304] 0 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/903: write db/dc/f1f [4833101,46903] 0 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/904: fdatasync db/d11/d14/d25/d2d/d48/f11f 0 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/905: creat db/d11/d14/d25/d65/d95/f12a x:0 0 0 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/906: read db/d11/d14/d27/f38 [243816,115086] 0 2024-04-26T04:42:31.690 INFO:tasks.workunit.client.0.smithi090.stdout:2/907: write db/d11/d14/d25/f4e [2362043,17163] 0 2024-04-26T04:42:31.697 INFO:tasks.workunit.client.0.smithi090.stdout:6/957: write d9/d77/db0/db5/dda/d6d/d66/fb8 [461712,97946] 0 2024-04-26T04:42:31.697 INFO:tasks.workunit.client.0.smithi090.stdout:5/933: dwrite d4/d1a/f78 [8388608,4194304] 0 2024-04-26T04:42:31.697 INFO:tasks.workunit.client.0.smithi090.stdout:5/934: chown d4/d1a/d1f/d53/d59/d5e 0 1 2024-04-26T04:42:31.707 INFO:tasks.workunit.client.0.smithi090.stdout:5/935: write d4/d1a/d1f/f3d [2764251,122229] 0 2024-04-26T04:42:31.707 INFO:tasks.workunit.client.0.smithi090.stdout:5/936: dread - d4/d1a/f120 zero size 2024-04-26T04:42:31.707 INFO:tasks.workunit.client.0.smithi090.stdout:5/937: fdatasync d4/d1a/d2a/d79/d90/d56/d80/db2/f11b 0 2024-04-26T04:42:31.710 INFO:tasks.workunit.client.0.smithi090.stdout:9/809: write db/f41 [1256748,3586] 0 2024-04-26T04:42:31.712 INFO:tasks.workunit.client.0.smithi090.stdout:9/810: mknod db/d5f/d117/d82/d85/db4/d99/db2/dcd/d107/c121 0 2024-04-26T04:42:31.713 INFO:tasks.workunit.client.0.smithi090.stdout:9/811: dread - db/dd/d18/d38/d39/dba/f113 zero size 2024-04-26T04:42:31.713 INFO:tasks.workunit.client.0.smithi090.stdout:9/812: fsync db/dd/d18/d95/f11b 0 2024-04-26T04:42:31.714 INFO:tasks.workunit.client.0.smithi090.stdout:9/813: link db/d5f/d117/d82/f10d db/d5f/d117/d82/d85/db4/d58/f122 0 2024-04-26T04:42:31.725 INFO:tasks.workunit.client.0.smithi090.stdout:1/976: sync 2024-04-26T04:42:31.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/848: sync 2024-04-26T04:42:31.725 INFO:tasks.workunit.client.0.smithi090.stdout:3/849: write d1/d17/d36/d33/d43/f92 [238894,89790] 0 2024-04-26T04:42:31.725 INFO:tasks.workunit.client.0.smithi090.stdout:2/908: dwrite db/d11/d14/d27/d28/dd1/dfe/d8b/ffb [0,4194304] 0 2024-04-26T04:42:31.726 INFO:tasks.workunit.client.0.smithi090.stdout:1/977: link d0/d55/dda/c98 d0/d55/dda/d43/d11b/d12d/db0/dfc/d104/c147 0 2024-04-26T04:42:31.726 INFO:tasks.workunit.client.0.smithi090.stdout:1/978: stat d0/d55/dda/d7/d17/f2a 0 2024-04-26T04:42:31.728 INFO:tasks.workunit.client.0.smithi090.stdout:3/850: unlink d1/d17/d36/d33/d68/dae/dca/d2c/fb8 0 2024-04-26T04:42:31.733 INFO:tasks.workunit.client.0.smithi090.stdout:2/909: creat db/dc/d6f/dd8/f12b x:0 0 0 2024-04-26T04:42:31.733 INFO:tasks.workunit.client.0.smithi090.stdout:2/910: fdatasync db/d11/d14/d27/d28/dd1/dfe/d8b/f10b 0 2024-04-26T04:42:31.735 INFO:tasks.workunit.client.0.smithi090.stdout:2/911: getdents db/d11/d14/d27/d28/dd1/dfe 0 2024-04-26T04:42:31.736 INFO:tasks.workunit.client.0.smithi090.stdout:6/958: dwrite d9/d21/d3b/d142/f104 [0,4194304] 0 2024-04-26T04:42:31.738 INFO:tasks.workunit.client.0.smithi090.stdout:2/912: creat db/d11/d14/d25/d86/f12c x:0 0 0 2024-04-26T04:42:31.738 INFO:tasks.workunit.client.0.smithi090.stdout:2/913: write db/d11/d14/d27/d28/dd1/dfe/d8b/f10b [1048184,91647] 0 2024-04-26T04:42:31.739 INFO:tasks.workunit.client.0.smithi090.stdout:2/914: truncate db/d11/d14/d25/d41/f54 23215 0 2024-04-26T04:42:31.739 INFO:tasks.workunit.client.0.smithi090.stdout:2/915: write db/d11/d14/d1b/f51 [3121475,19438] 0 2024-04-26T04:42:31.742 INFO:tasks.workunit.client.0.smithi090.stdout:5/938: dwrite d4/d1a/d1f/d53/d59/d5e/da0/f11d [0,4194304] 0 2024-04-26T04:42:31.743 INFO:tasks.workunit.client.0.smithi090.stdout:9/814: dwrite db/d5f/d117/f5e [0,4194304] 0 2024-04-26T04:42:31.749 INFO:tasks.workunit.client.0.smithi090.stdout:1/979: dwrite d0/d55/dda/d7/d4a/f105 [0,4194304] 0 2024-04-26T04:42:31.749 INFO:tasks.workunit.client.0.smithi090.stdout:1/980: chown d0/d55/dda/d7/f30 185763475 1 2024-04-26T04:42:31.750 INFO:tasks.workunit.client.0.smithi090.stdout:9/815: getdents db/d5f/d117/d82/d85/db4/d58 0 2024-04-26T04:42:31.754 INFO:tasks.workunit.client.0.smithi090.stdout:1/981: creat d0/d55/dda/d43/d11b/f148 x:0 0 0 2024-04-26T04:42:31.754 INFO:tasks.workunit.client.0.smithi090.stdout:1/982: fsync d0/d55/dda/d7/d4b/d53/f12e 0 2024-04-26T04:42:31.755 INFO:tasks.workunit.client.0.smithi090.stdout:9/816: creat db/d5f/d117/d82/d85/db4/dde/f123 x:0 0 0 2024-04-26T04:42:31.756 INFO:tasks.workunit.client.0.smithi090.stdout:9/817: truncate db/dd/d18/f36 273581 0 2024-04-26T04:42:31.759 INFO:tasks.workunit.client.0.smithi090.stdout:9/818: unlink db/dd/d18/d57/dcc/ff2 0 2024-04-26T04:42:31.763 INFO:tasks.workunit.client.0.smithi090.stdout:4/886: sync 2024-04-26T04:42:31.763 INFO:tasks.workunit.client.0.smithi090.stdout:4/887: stat d3/db/d60/dee/fc0 0 2024-04-26T04:42:31.763 INFO:tasks.workunit.client.0.smithi090.stdout:9/819: rename db/dd/f60 to db/da3/d11d/f124 0 2024-04-26T04:42:31.763 INFO:tasks.workunit.client.0.smithi090.stdout:6/959: dwrite d9/d77/db0/db5/dda/d58/f108 [0,4194304] 0 2024-04-26T04:42:31.764 INFO:tasks.workunit.client.0.smithi090.stdout:4/888: getdents d3/db/d12/d20/d2d/d93/d102/d72 0 2024-04-26T04:42:31.764 INFO:tasks.workunit.client.0.smithi090.stdout:2/916: dwrite db/d11/d14/d1b/d20/fb8 [4194304,4194304] 0 2024-04-26T04:42:31.764 INFO:tasks.workunit.client.0.smithi090.stdout:2/917: chown db/dc/f8e 72955 1 2024-04-26T04:42:31.765 INFO:tasks.workunit.client.0.smithi090.stdout:5/939: dwrite d4/d1a/d2a/d79/f85 [0,4194304] 0 2024-04-26T04:42:31.771 INFO:tasks.workunit.client.0.smithi090.stdout:9/820: creat db/d5f/d117/d82/d85/db4/d6a/d70/f125 x:0 0 0 2024-04-26T04:42:31.771 INFO:tasks.workunit.client.0.smithi090.stdout:9/821: dread - db/d5f/d117/d82/d85/db4/d58/ffe zero size 2024-04-26T04:42:31.771 INFO:tasks.workunit.client.0.smithi090.stdout:9/822: chown db/d5f/d117/d82/dbd/d109 61638948 1 2024-04-26T04:42:31.771 INFO:tasks.workunit.client.0.smithi090.stdout:1/983: dread d0/d55/dda/d7/d49/f5f [0,4194304] 0 2024-04-26T04:42:31.773 INFO:tasks.workunit.client.0.smithi090.stdout:9/823: mknod db/da3/c126 0 2024-04-26T04:42:31.776 INFO:tasks.workunit.client.0.smithi090.stdout:9/824: rename db/l83 to db/d5f/d117/dfb/l127 0 2024-04-26T04:42:31.776 INFO:tasks.workunit.client.0.smithi090.stdout:6/960: write d9/d1c/f2b [469352,37752] 0 2024-04-26T04:42:31.776 INFO:tasks.workunit.client.0.smithi090.stdout:9/825: write db/dd/d18/d38/d39/fc4 [916676,93575] 0 2024-04-26T04:42:31.776 INFO:tasks.workunit.client.0.smithi090.stdout:6/961: dread - d9/d21/d40/d4c/ff5 zero size 2024-04-26T04:42:31.776 INFO:tasks.workunit.client.0.smithi090.stdout:9/826: write db/dd/d18/d38/d39/d6b/de2/f5a [2051796,87871] 0 2024-04-26T04:42:31.777 INFO:tasks.workunit.client.0.smithi090.stdout:5/940: dread d4/d1a/df/f28 [4194304,4194304] 0 2024-04-26T04:42:31.777 INFO:tasks.workunit.client.0.smithi090.stdout:1/984: getdents d0/d55/dda/d43/d11b/d12d 0 2024-04-26T04:42:31.783 INFO:tasks.workunit.client.0.smithi090.stdout:2/918: dwrite db/d11/d14/d27/d28/f61 [0,4194304] 0 2024-04-26T04:42:31.783 INFO:tasks.workunit.client.0.smithi090.stdout:2/919: chown db/d11/d14/f4a 217838083 1 2024-04-26T04:42:31.786 INFO:tasks.workunit.client.0.smithi090.stdout:5/941: symlink d4/d1a/d2a/d79/d90/d118/l123 0 2024-04-26T04:42:31.787 INFO:tasks.workunit.client.0.smithi090.stdout:1/985: truncate d0/d55/dbd/ff8 4025552 0 2024-04-26T04:42:31.792 INFO:tasks.workunit.client.0.smithi090.stdout:5/942: symlink d4/d1a/d2a/d79/d90/db3/d9b/l124 0 2024-04-26T04:42:31.793 INFO:tasks.workunit.client.0.smithi090.stdout:1/986: symlink d0/d55/dda/d7/d4a/l149 0 2024-04-26T04:42:31.793 INFO:tasks.workunit.client.0.smithi090.stdout:1/987: creat d0/f14a x:0 0 0 2024-04-26T04:42:31.796 INFO:tasks.workunit.client.0.smithi090.stdout:1/988: creat d0/d55/dda/d7/d17/d22/d31/f14b x:0 0 0 2024-04-26T04:42:31.797 INFO:tasks.workunit.client.0.smithi090.stdout:2/920: read db/d11/d14/d27/d28/dd1/dfe/f10c [237952,119770] 0 2024-04-26T04:42:31.797 INFO:tasks.workunit.client.0.smithi090.stdout:2/921: fdatasync db/d11/d14/d25/f3e 0 2024-04-26T04:42:31.798 INFO:tasks.workunit.client.0.smithi090.stdout:2/922: dread db/d11/d14/d27/d2f/d42/fd9 [0,4194304] 0 2024-04-26T04:42:31.798 INFO:tasks.workunit.client.0.smithi090.stdout:2/923: fsync db/d11/d14/d27/d28/dd1/dfe/d71/f83 0 2024-04-26T04:42:31.799 INFO:tasks.workunit.client.0.smithi090.stdout:2/924: dread db/d11/d14/d27/d28/f3d [4194304,4194304] 0 2024-04-26T04:42:31.799 INFO:tasks.workunit.client.0.smithi090.stdout:6/962: dwrite d9/d21/d40/d4c/f7b [0,4194304] 0 2024-04-26T04:42:31.800 INFO:tasks.workunit.client.0.smithi090.stdout:9/827: dwrite db/dd/d18/d57/fbb [0,4194304] 0 2024-04-26T04:42:31.800 INFO:tasks.workunit.client.0.smithi090.stdout:9/828: fdatasync db/f35 0 2024-04-26T04:42:31.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/989: dread d0/f94 [0,4194304] 0 2024-04-26T04:42:31.802 INFO:tasks.workunit.client.0.smithi090.stdout:1/990: fsync d0/d55/dda/f15 0 2024-04-26T04:42:31.802 INFO:tasks.workunit.client.0.smithi090.stdout:6/963: mkdir d9/d21/d3b/d142/de7/d149 0 2024-04-26T04:42:31.806 INFO:tasks.workunit.client.0.smithi090.stdout:9/829: truncate db/dd/d18/d38/f90 585763 0 2024-04-26T04:42:31.806 INFO:tasks.workunit.client.0.smithi090.stdout:9/830: chown db/dd/d18/d38/d39/d3f/f93 66 1 2024-04-26T04:42:31.807 INFO:tasks.workunit.client.0.smithi090.stdout:1/991: rmdir d0/d55/dda/d7/d4b/d12c 0 2024-04-26T04:42:31.808 INFO:tasks.workunit.client.0.smithi090.stdout:1/992: chown d0/d55/dda/d7/d4a/da8/lac 404219 1 2024-04-26T04:42:31.808 INFO:tasks.workunit.client.0.smithi090.stdout:1/993: fsync d0/d55/dda/d62/d78/f7d 0 2024-04-26T04:42:31.809 INFO:tasks.workunit.client.0.smithi090.stdout:6/964: mknod d9/d1c/d31/c14a 0 2024-04-26T04:42:31.809 INFO:tasks.workunit.client.0.smithi090.stdout:1/994: write d0/d55/dda/d43/d11b/d12d/d131/f134 [331087,111933] 0 2024-04-26T04:42:31.809 INFO:tasks.workunit.client.0.smithi090.stdout:1/995: readlink d0/d55/dda/d7/d4b/d53/l65 0 2024-04-26T04:42:31.810 INFO:tasks.workunit.client.0.smithi090.stdout:9/831: mknod db/dd/d18/d57/c128 0 2024-04-26T04:42:31.811 INFO:tasks.workunit.client.0.smithi090.stdout:1/996: mknod d0/d55/dda/d7/d4b/dca/c14c 0 2024-04-26T04:42:31.812 INFO:tasks.workunit.client.0.smithi090.stdout:9/832: symlink db/d5f/d117/d82/d85/db4/d6a/db6/l129 0 2024-04-26T04:42:31.814 INFO:tasks.workunit.client.0.smithi090.stdout:5/943: write d4/f46 [1560618,33649] 0 2024-04-26T04:42:31.814 INFO:tasks.workunit.client.0.smithi090.stdout:5/944: fsync d4/d1a/d45/fa6 0 2024-04-26T04:42:31.814 INFO:tasks.workunit.client.0.smithi090.stdout:5/945: fdatasync d4/d1a/d1f/d53/d59/f8f 0 2024-04-26T04:42:31.814 INFO:tasks.workunit.client.0.smithi090.stdout:5/946: fsync d4/d1a/d1f/dc5/dca/fee 0 2024-04-26T04:42:31.815 INFO:tasks.workunit.client.0.smithi090.stdout:2/925: dread db/d11/d14/d27/d2f/d42/d5c/f4f [0,4194304] 0 2024-04-26T04:42:31.815 INFO:tasks.workunit.client.0.smithi090.stdout:1/997: mkdir d0/d55/dda/d7/d4b/d53/d14d 0 2024-04-26T04:42:31.815 INFO:tasks.workunit.client.0.smithi090.stdout:1/998: dread d0/d55/f83 [0,4194304] 0 2024-04-26T04:42:31.818 INFO:tasks.workunit.client.0.smithi090.stdout:9/833: creat db/d5f/d117/d82/d85/db4/d6a/f12a x:0 0 0 2024-04-26T04:42:31.819 INFO:tasks.workunit.client.0.smithi090.stdout:2/926: mkdir db/d11/d14/d63/d12d 0 2024-04-26T04:42:31.819 INFO:tasks.workunit.client.0.smithi090.stdout:2/927: write db/d11/d14/d27/d2f/ff6 [308331,44755] 0 2024-04-26T04:42:31.819 INFO:tasks.workunit.client.0.smithi090.stdout:1/999: truncate d0/d55/dda/d7/d4a/faf 3827309 0 2024-04-26T04:42:31.821 INFO:tasks.workunit.client.0.smithi090.stdout:9/834: unlink db/dd/d18/l1a 0 2024-04-26T04:42:31.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/928: creat db/d11/f12e x:0 0 0 2024-04-26T04:42:31.823 INFO:tasks.workunit.client.0.smithi090.stdout:2/929: chown db/dc/f8e 12044 1 2024-04-26T04:42:31.823 INFO:tasks.workunit.client.0.smithi090.stdout:5/947: read d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 [1294092,18925] 0 2024-04-26T04:42:31.824 INFO:tasks.workunit.client.0.smithi090.stdout:9/835: creat db/d5f/d117/d82/d85/db4/d6a/db6/f12b x:0 0 0 2024-04-26T04:42:31.825 INFO:tasks.workunit.client.0.smithi090.stdout:9/836: dread db/dd/d18/d38/f9b [0,4194304] 0 2024-04-26T04:42:31.826 INFO:tasks.workunit.client.0.smithi090.stdout:2/930: creat db/d11/d14/d1b/f12f x:0 0 0 2024-04-26T04:42:31.826 INFO:tasks.workunit.client.0.smithi090.stdout:2/931: fsync db/d11/d14/d25/f11e 0 2024-04-26T04:42:31.826 INFO:tasks.workunit.client.0.smithi090.stdout:2/932: chown db/d11/d14/d1b/d20/d4c/l9e 156192529 1 2024-04-26T04:42:31.827 INFO:tasks.workunit.client.0.smithi090.stdout:9/837: mkdir db/dd/d18/d38/d39/d3f/d12c 0 2024-04-26T04:42:31.830 INFO:tasks.workunit.client.0.smithi090.stdout:9/838: symlink db/d5f/d117/d82/d85/db4/d58/l12d 0 2024-04-26T04:42:31.832 INFO:tasks.workunit.client.0.smithi090.stdout:9/839: unlink db/d5f/d117/d82/d85/db4/d99/lee 0 2024-04-26T04:42:31.834 INFO:tasks.workunit.client.0.smithi090.stdout:9/840: link db/d5f/d117/d82/d85/db4/d6a/db6/cf8 db/d5f/d117/d82/d85/db4/d6a/db6/c12e 0 2024-04-26T04:42:31.844 INFO:tasks.workunit.client.0.smithi090.stdout:2/933: dwrite db/d11/d14/d27/d28/dd1/dfe/d71/dea/ff5 [0,4194304] 0 2024-04-26T04:42:31.850 INFO:tasks.workunit.client.0.smithi090.stdout:9/841: dread db/d5f/d117/d82/ff9 [0,4194304] 0 2024-04-26T04:42:31.850 INFO:tasks.workunit.client.0.smithi090.stdout:9/842: chown db/d5f/d117/d82/d85/la1 0 1 2024-04-26T04:42:31.851 INFO:tasks.workunit.client.0.smithi090.stdout:9/843: symlink db/dd/d18/d38/d39/d3f/l12f 0 2024-04-26T04:42:31.853 INFO:tasks.workunit.client.0.smithi090.stdout:2/934: dread db/d11/d14/d27/fab [0,4194304] 0 2024-04-26T04:42:31.853 INFO:tasks.workunit.client.0.smithi090.stdout:2/935: write db/d11/d14/d1b/d20/d4c/fae [320052,95045] 0 2024-04-26T04:42:31.854 INFO:tasks.workunit.client.0.smithi090.stdout:2/936: dread - db/d11/d14/d25/d86/f12c zero size 2024-04-26T04:42:31.856 INFO:tasks.workunit.client.0.smithi090.stdout:2/937: getdents db/dc/d6f/dd8 0 2024-04-26T04:42:31.866 INFO:tasks.workunit.client.0.smithi090.stdout:2/938: dread db/d11/d14/d1b/d20/f24 [0,4194304] 0 2024-04-26T04:42:31.867 INFO:tasks.workunit.client.0.smithi090.stdout:2/939: mkdir db/d11/d14/d1b/d20/d130 0 2024-04-26T04:42:31.868 INFO:tasks.workunit.client.0.smithi090.stdout:9/844: dwrite db/d5f/d117/d82/d85/db4/d58/f10a [0,4194304] 0 2024-04-26T04:42:31.877 INFO:tasks.workunit.client.0.smithi090.stdout:2/940: read db/d11/d14/d1b/d20/f115 [3255129,20732] 0 2024-04-26T04:42:31.877 INFO:tasks.workunit.client.0.smithi090.stdout:9/845: write db/dd/d18/d57/dcc/f102 [1674282,85813] 0 2024-04-26T04:42:31.879 INFO:tasks.workunit.client.0.smithi090.stdout:9/846: mkdir db/da3/da8/d130 0 2024-04-26T04:42:31.879 INFO:tasks.workunit.client.0.smithi090.stdout:9/847: fdatasync db/dd/f1d 0 2024-04-26T04:42:31.893 INFO:tasks.workunit.client.0.smithi090.stdout:2/941: dwrite db/d59/d66/d69/f111 [0,4194304] 0 2024-04-26T04:42:31.896 INFO:tasks.workunit.client.0.smithi090.stdout:2/942: mknod db/d59/d66/d77/c131 0 2024-04-26T04:42:31.898 INFO:tasks.workunit.client.0.smithi090.stdout:2/943: rename db/d11/d14/d25/d2d/fe2 to db/d59/f132 0 2024-04-26T04:42:31.902 INFO:tasks.workunit.client.0.smithi090.stdout:9/848: dwrite db/dd/d18/d57/dcc/f102 [0,4194304] 0 2024-04-26T04:42:31.903 INFO:tasks.workunit.client.0.smithi090.stdout:9/849: dread db/dd/d18/d95/fb9 [0,4194304] 0 2024-04-26T04:42:31.922 INFO:tasks.workunit.client.0.smithi090.stdout:2/944: dwrite db/d11/d14/d25/f11e [0,4194304] 0 2024-04-26T04:42:31.922 INFO:tasks.workunit.client.0.smithi090.stdout:9/850: dwrite db/f84 [0,4194304] 0 2024-04-26T04:42:31.924 INFO:tasks.workunit.client.0.smithi090.stdout:2/945: creat db/dc/d6f/de3/de4/f133 x:0 0 0 2024-04-26T04:42:31.926 INFO:tasks.workunit.client.0.smithi090.stdout:9/851: symlink db/d5f/d117/d82/l131 0 2024-04-26T04:42:31.926 INFO:tasks.workunit.client.0.smithi090.stdout:9/852: stat db/d5f/d117/d82/d85/db4/d99/fcf 0 2024-04-26T04:42:31.928 INFO:tasks.workunit.client.0.smithi090.stdout:9/853: truncate db/dd/d18/f36 1087271 0 2024-04-26T04:42:31.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/854: mknod db/d5f/d117/d82/d85/db4/d99/c132 0 2024-04-26T04:42:31.939 INFO:tasks.workunit.client.0.smithi090.stdout:9/855: dread db/f84 [0,4194304] 0 2024-04-26T04:42:31.941 INFO:tasks.workunit.client.0.smithi090.stdout:2/946: dwrite db/d11/d14/d27/d28/dd1/dfe/d8b/f10b [0,4194304] 0 2024-04-26T04:42:31.944 INFO:tasks.workunit.client.0.smithi090.stdout:2/947: rmdir db/d11/d14/d25/d41 39 2024-04-26T04:42:31.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/948: rename db/dc/l116 to db/d11/d14/d63/d12d/l134 0 2024-04-26T04:42:31.946 INFO:tasks.workunit.client.0.smithi090.stdout:2/949: write db/d11/d14/f4a [1107178,94683] 0 2024-04-26T04:42:31.949 INFO:tasks.workunit.client.0.smithi090.stdout:2/950: creat db/d11/d14/d1b/d126/f135 x:0 0 0 2024-04-26T04:42:31.957 INFO:tasks.workunit.client.0.smithi090.stdout:9/856: dwrite db/dd/d18/d38/d39/d6b/de2/f5a [4194304,4194304] 0 2024-04-26T04:42:31.961 INFO:tasks.workunit.client.0.smithi090.stdout:9/857: write db/f84 [1356487,34970] 0 2024-04-26T04:42:31.964 INFO:tasks.workunit.client.0.smithi090.stdout:2/951: dwrite db/d11/d14/d25/d2d/f36 [0,4194304] 0 2024-04-26T04:42:31.965 INFO:tasks.workunit.client.0.smithi090.stdout:9/858: write db/d5f/d117/d82/d85/db4/d58/f97 [35180,22707] 0 2024-04-26T04:42:31.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/859: dread db/dd/d18/d95/fb9 [0,4194304] 0 2024-04-26T04:42:31.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/860: read - db/dd/d18/d38/d39/dba/ff1 zero size 2024-04-26T04:42:31.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/861: write db/f13 [1644710,10115] 0 2024-04-26T04:42:31.969 INFO:tasks.workunit.client.0.smithi090.stdout:9/862: write db/dd/d18/d38/d39/dba/f113 [613009,52838] 0 2024-04-26T04:42:31.980 INFO:tasks.workunit.client.0.smithi090.stdout:2/952: dwrite db/d11/d14/d25/f3e [0,4194304] 0 2024-04-26T04:42:31.982 INFO:tasks.workunit.client.0.smithi090.stdout:2/953: symlink db/d59/d66/d77/l136 0 2024-04-26T04:42:31.989 INFO:tasks.workunit.client.0.smithi090.stdout:9/863: write db/d5f/d117/d82/fe4 [3843211,50823] 0 2024-04-26T04:42:32.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/864: dwrite db/dd/d18/d38/d39/d3f/f93 [0,4194304] 0 2024-04-26T04:42:32.005 INFO:tasks.workunit.client.0.smithi090.stdout:9/865: write db/dd/f8d [4672320,60692] 0 2024-04-26T04:42:32.012 INFO:tasks.workunit.client.0.smithi090.stdout:9/866: dread db/da3/d11d/f124 [0,4194304] 0 2024-04-26T04:42:32.014 INFO:tasks.workunit.client.0.smithi090.stdout:9/867: unlink db/d5f/d117/c80 0 2024-04-26T04:42:32.014 INFO:tasks.workunit.client.0.smithi090.stdout:9/868: read db/dd/d18/d38/d39/f7e [1489735,102502] 0 2024-04-26T04:42:32.026 INFO:tasks.workunit.client.0.smithi090.stdout:9/869: dwrite db/dd/d18/d38/d39/dba/f113 [0,4194304] 0 2024-04-26T04:42:32.027 INFO:tasks.workunit.client.0.smithi090.stdout:9/870: creat db/d5f/d117/dfb/f133 x:0 0 0 2024-04-26T04:42:32.027 INFO:tasks.workunit.client.0.smithi090.stdout:9/871: creat db/dd/d18/d57/dcc/f134 x:0 0 0 2024-04-26T04:42:32.035 INFO:tasks.workunit.client.0.smithi090.stdout:9/872: read db/da3/da8/fea [1045164,100284] 0 2024-04-26T04:42:32.037 INFO:tasks.workunit.client.0.smithi090.stdout:9/873: unlink db/d5f/d117/d82/d85/db4/d6a/d70/cd9 0 2024-04-26T04:42:32.038 INFO:tasks.workunit.client.0.smithi090.stdout:9/874: mknod db/d5f/d117/d82/d85/db4/d99/db2/dcd/d107/c135 0 2024-04-26T04:42:32.039 INFO:tasks.workunit.client.0.smithi090.stdout:9/875: creat db/d5f/d117/f136 x:0 0 0 2024-04-26T04:42:32.040 INFO:tasks.workunit.client.0.smithi090.stdout:9/876: link db/d5f/d117/ld5 db/dd/d18/d95/l137 0 2024-04-26T04:42:32.041 INFO:tasks.workunit.client.0.smithi090.stdout:9/877: link db/d5f/d117/d82/d85/db4/dde/cfc db/da3/da8/c138 0 2024-04-26T04:42:32.043 INFO:tasks.workunit.client.0.smithi090.stdout:9/878: symlink db/dd/d18/d38/d39/dba/l139 0 2024-04-26T04:42:32.052 INFO:tasks.workunit.client.0.smithi090.stdout:9/879: write db/d5f/d117/d82/d85/faf [1839457,119140] 0 2024-04-26T04:42:32.052 INFO:tasks.workunit.client.0.smithi090.stdout:9/880: fdatasync db/d5f/d117/d82/d85/db4/dde/f123 0 2024-04-26T04:42:32.053 INFO:tasks.workunit.client.0.smithi090.stdout:9/881: getdents db/dd/d18/d57 0 2024-04-26T04:42:32.055 INFO:tasks.workunit.client.0.smithi090.stdout:9/882: link db/d5f/d117/d82/fe4 db/dd/d18/d95/f13a 0 2024-04-26T04:42:32.056 INFO:tasks.workunit.client.0.smithi090.stdout:9/883: link db/d5f/d117/d82/d85/db4/d58/la5 db/dd/d18/d38/d39/d6b/dc0/l13b 0 2024-04-26T04:42:32.071 INFO:tasks.workunit.client.0.smithi090.stdout:9/884: dwrite db/dd/d18/f3a [0,4194304] 0 2024-04-26T04:42:32.075 INFO:tasks.workunit.client.0.smithi090.stdout:9/885: dread db/dd/d18/d38/d39/fc4 [0,4194304] 0 2024-04-26T04:42:32.076 INFO:tasks.workunit.client.0.smithi090.stdout:9/886: getdents db/da3/da8 0 2024-04-26T04:42:32.077 INFO:tasks.workunit.client.0.smithi090.stdout:9/887: creat db/d5f/d117/d82/d85/db4/d99/db2/dcd/f13c x:0 0 0 2024-04-26T04:42:32.094 INFO:tasks.workunit.client.0.smithi090.stdout:9/888: dwrite db/d5f/d117/d82/d85/db4/d99/db2/fe9 [0,4194304] 0 2024-04-26T04:42:32.148 INFO:tasks.workunit.client.0.smithi090.stdout:9/889: rmdir db/d5f/d117/d82/d85/db4/d99/db2/dcd/d107 39 2024-04-26T04:42:32.230 INFO:tasks.workunit.client.0.smithi090.stdout:3/851: sync 2024-04-26T04:42:32.230 INFO:tasks.workunit.client.0.smithi090.stdout:3/852: fsync d1/d9/f38 0 2024-04-26T04:42:32.230 INFO:tasks.workunit.client.0.smithi090.stdout:3/853: write d1/d17/d36/d33/d68/dae/f110 [121495,128024] 0 2024-04-26T04:42:32.231 INFO:tasks.workunit.client.0.smithi090.stdout:6/965: sync 2024-04-26T04:42:32.231 INFO:tasks.workunit.client.0.smithi090.stdout:4/889: sync 2024-04-26T04:42:32.232 INFO:tasks.workunit.client.0.smithi090.stdout:4/890: fsync d3/ffa 0 2024-04-26T04:42:32.248 INFO:tasks.workunit.client.0.smithi090.stdout:9/890: link db/d5f/d117/dfb/f133 db/d5f/d117/dfb/f13d 0 2024-04-26T04:42:32.248 INFO:tasks.workunit.client.0.smithi090.stdout:6/966: dwrite d9/d3c/f61 [0,4194304] 0 2024-04-26T04:42:32.250 INFO:tasks.workunit.client.0.smithi090.stdout:3/854: mkdir d1/d17/d36/d33/d68/dae/dca/d2c/d97/d11e 0 2024-04-26T04:42:32.254 INFO:tasks.workunit.client.0.smithi090.stdout:6/967: rmdir d9/d77/db0/db5/dda/d6d/d66 39 2024-04-26T04:42:32.257 INFO:tasks.workunit.client.0.smithi090.stdout:9/891: creat db/d5f/d112/f13e x:0 0 0 2024-04-26T04:42:32.258 INFO:tasks.workunit.client.0.smithi090.stdout:6/968: symlink d9/d3c/l14b 0 2024-04-26T04:42:32.260 INFO:tasks.workunit.client.0.smithi090.stdout:3/855: rmdir d1/d17/d61/dce 39 2024-04-26T04:42:32.264 INFO:tasks.workunit.client.0.smithi090.stdout:3/856: link d1/d17/d36/d2a/d3a/fd3 d1/d17/d36/d33/daa/dff/f11f 0 2024-04-26T04:42:32.271 INFO:tasks.workunit.client.0.smithi090.stdout:3/857: mknod d1/d17/d36/d33/d68/dae/dca/d2c/d97/d11e/c120 0 2024-04-26T04:42:32.285 INFO:tasks.workunit.client.0.smithi090.stdout:3/858: dread d1/f3c [0,4194304] 0 2024-04-26T04:42:32.293 INFO:tasks.workunit.client.0.smithi090.stdout:9/892: dwrite db/dd/d18/f3a [4194304,4194304] 0 2024-04-26T04:42:32.295 INFO:tasks.workunit.client.0.smithi090.stdout:6/969: dwrite d9/d1c/d31/d79/fba [0,4194304] 0 2024-04-26T04:42:32.295 INFO:tasks.workunit.client.0.smithi090.stdout:6/970: write d9/d21/d3b/d142/dd9/fed [815341,9282] 0 2024-04-26T04:42:32.297 INFO:tasks.workunit.client.0.smithi090.stdout:3/859: dread d1/d17/d36/d33/d68/d76/f8e [0,4194304] 0 2024-04-26T04:42:32.301 INFO:tasks.workunit.client.0.smithi090.stdout:3/860: link d1/d17/d61/d9d/da7/ccd d1/d17/c121 0 2024-04-26T04:42:32.301 INFO:tasks.workunit.client.0.smithi090.stdout:3/861: write d1/d17/d36/d33/d68/dae/fb3 [5183455,33270] 0 2024-04-26T04:42:32.303 INFO:tasks.workunit.client.0.smithi090.stdout:3/862: symlink d1/d17/d36/d33/d68/dae/dca/dcf/l122 0 2024-04-26T04:42:32.304 INFO:tasks.workunit.client.0.smithi090.stdout:3/863: chown d1/d17/d36/d33/d68/dae/dca/l87 0 1 2024-04-26T04:42:32.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/893: symlink db/d5f/d117/d82/d85/db4/d6a/d70/l13f 0 2024-04-26T04:42:32.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/894: chown db/dd/d18/d95/f13a 131591079 1 2024-04-26T04:42:32.305 INFO:tasks.workunit.client.0.smithi090.stdout:9/895: write db/d5f/d117/dfb/f13d [125084,72107] 0 2024-04-26T04:42:32.311 INFO:tasks.workunit.client.0.smithi090.stdout:2/954: sync 2024-04-26T04:42:32.311 INFO:tasks.workunit.client.0.smithi090.stdout:2/955: dread - db/d11/d7f/fc6 zero size 2024-04-26T04:42:32.311 INFO:tasks.workunit.client.0.smithi090.stdout:5/948: sync 2024-04-26T04:42:32.314 INFO:tasks.workunit.client.0.smithi090.stdout:2/956: mkdir db/dc/d6f/de3/de4/d137 0 2024-04-26T04:42:32.315 INFO:tasks.workunit.client.0.smithi090.stdout:4/891: mkdir d3/db/d12/d125 0 2024-04-26T04:42:32.315 INFO:tasks.workunit.client.0.smithi090.stdout:4/892: truncate d3/db/d12/d20/d2d/d2e/dda/f100 4248195 0 2024-04-26T04:42:32.321 INFO:tasks.workunit.client.0.smithi090.stdout:3/864: dread d1/d17/f2f [0,4194304] 0 2024-04-26T04:42:32.321 INFO:tasks.workunit.client.0.smithi090.stdout:3/865: chown d1/d17/f4f 2622120 1 2024-04-26T04:42:32.323 INFO:tasks.workunit.client.0.smithi090.stdout:3/866: mknod d1/d17/d36/d33/d68/dae/dca/d2c/d97/de4/c123 0 2024-04-26T04:42:32.328 INFO:tasks.workunit.client.0.smithi090.stdout:3/867: unlink d1/d17/d36/d33/d68/dae/dca/f21 0 2024-04-26T04:42:32.328 INFO:tasks.workunit.client.0.smithi090.stdout:6/971: dwrite d9/d77/db0/db5/dda/d58/f6c [0,4194304] 0 2024-04-26T04:42:32.328 INFO:tasks.workunit.client.0.smithi090.stdout:6/972: write d9/d1c/f2b [1442129,113308] 0 2024-04-26T04:42:32.329 INFO:tasks.workunit.client.0.smithi090.stdout:4/893: write d3/db/d12/f5b [3787110,32527] 0 2024-04-26T04:42:32.329 INFO:tasks.workunit.client.0.smithi090.stdout:3/868: creat d1/d17/d36/d33/d68/dae/dca/d1b/f124 x:0 0 0 2024-04-26T04:42:32.329 INFO:tasks.workunit.client.0.smithi090.stdout:3/869: write d1/d17/f4f [2501107,102617] 0 2024-04-26T04:42:32.330 INFO:tasks.workunit.client.0.smithi090.stdout:9/896: dwrite db/dd/d18/d38/fd1 [0,4194304] 0 2024-04-26T04:42:32.330 INFO:tasks.workunit.client.0.smithi090.stdout:4/894: symlink d3/l126 0 2024-04-26T04:42:32.330 INFO:tasks.workunit.client.0.smithi090.stdout:4/895: getdents d3/d13/d6c/d8c/d109 0 2024-04-26T04:42:32.332 INFO:tasks.workunit.client.0.smithi090.stdout:4/896: rename d3/db/d60/dee/dae to d3/db/d12/d20/d2d/d93/d102/d127 0 2024-04-26T04:42:32.334 INFO:tasks.workunit.client.0.smithi090.stdout:4/897: mknod d3/db/d12/d125/c128 0 2024-04-26T04:42:32.334 INFO:tasks.workunit.client.0.smithi090.stdout:4/898: chown d3/db/d12/c15 3413 1 2024-04-26T04:42:32.335 INFO:tasks.workunit.client.0.smithi090.stdout:4/899: mknod d3/db/d12/d20/c129 0 2024-04-26T04:42:32.346 INFO:tasks.workunit.client.0.smithi090.stdout:2/957: dwrite db/d11/d14/d25/f4e [0,4194304] 0 2024-04-26T04:42:32.349 INFO:tasks.workunit.client.0.smithi090.stdout:5/949: dwrite d4/d1a/d1f/f3a [4194304,4194304] 0 2024-04-26T04:42:32.349 INFO:tasks.workunit.client.0.smithi090.stdout:9/897: dread db/dd/d18/d57/dcc/f102 [0,4194304] 0 2024-04-26T04:42:32.349 INFO:tasks.workunit.client.0.smithi090.stdout:2/958: symlink db/d11/d14/d27/d28/l138 0 2024-04-26T04:42:32.352 INFO:tasks.workunit.client.0.smithi090.stdout:5/950: unlink d4/d1a/df/d4a/laf 0 2024-04-26T04:42:32.354 INFO:tasks.workunit.client.0.smithi090.stdout:2/959: truncate db/d11/d14/d27/d28/dd1/dfe/d8b/ffb 792922 0 2024-04-26T04:42:32.354 INFO:tasks.workunit.client.0.smithi090.stdout:2/960: readlink db/d11/d14/d25/d86/db9/lef 0 2024-04-26T04:42:32.354 INFO:tasks.workunit.client.0.smithi090.stdout:2/961: getdents db/dc/d6f/dd8/d108 0 2024-04-26T04:42:32.355 INFO:tasks.workunit.client.0.smithi090.stdout:5/951: mknod d4/d1a/d2a/d79/d90/d49/d105/c125 0 2024-04-26T04:42:32.355 INFO:tasks.workunit.client.0.smithi090.stdout:5/952: fsync d4/d1a/d2a/d79/fcf 0 2024-04-26T04:42:32.356 INFO:tasks.workunit.client.0.smithi090.stdout:2/962: rmdir db/d11/d14/d25/d65/d95/d112 0 2024-04-26T04:42:32.358 INFO:tasks.workunit.client.0.smithi090.stdout:5/953: mknod d4/d1a/df/d36/d8e/de3/c126 0 2024-04-26T04:42:32.365 INFO:tasks.workunit.client.0.smithi090.stdout:6/973: dwrite d9/d21/f148 [0,4194304] 0 2024-04-26T04:42:32.365 INFO:tasks.workunit.client.0.smithi090.stdout:3/870: dwrite d1/d17/d36/d33/d42/ffb [0,4194304] 0 2024-04-26T04:42:32.368 INFO:tasks.workunit.client.0.smithi090.stdout:5/954: write d4/d1a/d1f/d53/d59/d5e/f6a [1928250,35727] 0 2024-04-26T04:42:32.369 INFO:tasks.workunit.client.0.smithi090.stdout:5/955: creat d4/d1a/df/d72/dc0/f127 x:0 0 0 2024-04-26T04:42:32.369 INFO:tasks.workunit.client.0.smithi090.stdout:4/900: dwrite d3/f2c [0,4194304] 0 2024-04-26T04:42:32.369 INFO:tasks.workunit.client.0.smithi090.stdout:4/901: chown d3/db/d12/d20/d24/fc6 415 1 2024-04-26T04:42:32.369 INFO:tasks.workunit.client.0.smithi090.stdout:9/898: write db/dd/f1d [4938727,18162] 0 2024-04-26T04:42:32.371 INFO:tasks.workunit.client.0.smithi090.stdout:6/974: rename d9/d3c/db4/ddb/l10c to d9/d21/d40/d4c/d6f/l14c 0 2024-04-26T04:42:32.377 INFO:tasks.workunit.client.0.smithi090.stdout:3/871: link d1/d17/d36/d33/d68/dae/dca/d1b/d58/db5/c104 d1/d17/d36/d33/d68/d76/c125 0 2024-04-26T04:42:32.377 INFO:tasks.workunit.client.0.smithi090.stdout:3/872: truncate d1/d17/d36/d33/d68/dae/dca/d2c/f5f 536365 0 2024-04-26T04:42:32.381 INFO:tasks.workunit.client.0.smithi090.stdout:2/963: dwrite db/d11/d14/d25/f4e [4194304,4194304] 0 2024-04-26T04:42:32.382 INFO:tasks.workunit.client.0.smithi090.stdout:5/956: mknod d4/d1a/d2a/d31/c128 0 2024-04-26T04:42:32.383 INFO:tasks.workunit.client.0.smithi090.stdout:5/957: write d4/d1a/df/d4a/fde [1299436,116483] 0 2024-04-26T04:42:32.383 INFO:tasks.workunit.client.0.smithi090.stdout:4/902: truncate d3/db/d12/d3b/f115 839273 0 2024-04-26T04:42:32.383 INFO:tasks.workunit.client.0.smithi090.stdout:4/903: chown d3/db/d12/l10e 33951977 1 2024-04-26T04:42:32.385 INFO:tasks.workunit.client.0.smithi090.stdout:9/899: rename db/d5f/d117/d82/d85/db4/d6a/fb5 to db/f140 0 2024-04-26T04:42:32.387 INFO:tasks.workunit.client.0.smithi090.stdout:6/975: getdents d9/d1c/d31/d69/db7/dea 0 2024-04-26T04:42:32.394 INFO:tasks.workunit.client.0.smithi090.stdout:2/964: mkdir db/d11/d14/d1b/d119/d139 0 2024-04-26T04:42:32.395 INFO:tasks.workunit.client.0.smithi090.stdout:5/958: symlink d4/d1a/d1f/dc5/dca/l129 0 2024-04-26T04:42:32.399 INFO:tasks.workunit.client.0.smithi090.stdout:9/900: mkdir db/d5f/d117/d82/d85/db4/d58/d141 0 2024-04-26T04:42:32.399 INFO:tasks.workunit.client.0.smithi090.stdout:9/901: read - db/d5f/d117/d82/d85/db4/d99/db2/dcd/f13c zero size 2024-04-26T04:42:32.401 INFO:tasks.workunit.client.0.smithi090.stdout:3/873: dwrite d1/d17/d36/d2a/d3a/f103 [0,4194304] 0 2024-04-26T04:42:32.401 INFO:tasks.workunit.client.0.smithi090.stdout:2/965: dread db/d59/f62 [0,4194304] 0 2024-04-26T04:42:32.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/976: creat d9/d21/d3b/d86/f14d x:0 0 0 2024-04-26T04:42:32.403 INFO:tasks.workunit.client.0.smithi090.stdout:6/977: chown d9/d77/db0/db5/dda/d6d/fc6 281351 1 2024-04-26T04:42:32.403 INFO:tasks.workunit.client.0.smithi090.stdout:4/904: truncate d3/f36 4071394 0 2024-04-26T04:42:32.407 INFO:tasks.workunit.client.0.smithi090.stdout:9/902: dread db/d5f/d117/dfb/f133 [0,4194304] 0 2024-04-26T04:42:32.410 INFO:tasks.workunit.client.0.smithi090.stdout:2/966: link db/d11/d14/d27/d2f/d42/d5c/l97 db/d11/d14/d1b/d20/d4c/d98/l13a 0 2024-04-26T04:42:32.412 INFO:tasks.workunit.client.0.smithi090.stdout:6/978: creat d9/d1c/d31/d69/db7/f14e x:0 0 0 2024-04-26T04:42:32.414 INFO:tasks.workunit.client.0.smithi090.stdout:4/905: mkdir d3/d13/d6c/d7e/d12a 0 2024-04-26T04:42:32.415 INFO:tasks.workunit.client.0.smithi090.stdout:2/967: mknod db/d11/d14/d1b/d119/c13b 0 2024-04-26T04:42:32.417 INFO:tasks.workunit.client.0.smithi090.stdout:4/906: symlink d3/db/d12/d125/l12b 0 2024-04-26T04:42:32.419 INFO:tasks.workunit.client.0.smithi090.stdout:2/968: mkdir db/d11/d14/d25/d2d/d48/d5b/d13c 0 2024-04-26T04:42:32.420 INFO:tasks.workunit.client.0.smithi090.stdout:4/907: unlink d3/db/d12/d20/l44 0 2024-04-26T04:42:32.421 INFO:tasks.workunit.client.0.smithi090.stdout:2/969: creat db/d11/d14/d63/d12d/f13d x:0 0 0 2024-04-26T04:42:32.422 INFO:tasks.workunit.client.0.smithi090.stdout:9/903: dread db/d5f/d117/d82/d85/db4/d99/db2/fe9 [0,4194304] 0 2024-04-26T04:42:32.422 INFO:tasks.workunit.client.0.smithi090.stdout:9/904: write db/dd/d18/d38/d39/dba/ff1 [283140,79298] 0 2024-04-26T04:42:32.423 INFO:tasks.workunit.client.0.smithi090.stdout:3/874: dwrite d1/d17/d36/d33/d68/dae/dca/dcf/ff9 [0,4194304] 0 2024-04-26T04:42:32.423 INFO:tasks.workunit.client.0.smithi090.stdout:4/908: rename d3/db/d12/d3b/f107 to d3/db/d60/d5c/d117/f12c 0 2024-04-26T04:42:32.424 INFO:tasks.workunit.client.0.smithi090.stdout:5/959: dwrite d4/d1a/f17 [0,4194304] 0 2024-04-26T04:42:32.424 INFO:tasks.workunit.client.0.smithi090.stdout:5/960: write d4/d1a/d2a/d79/d90/d56/f8b [4937604,74383] 0 2024-04-26T04:42:32.425 INFO:tasks.workunit.client.0.smithi090.stdout:2/970: unlink db/d11/d14/d1b/c110 0 2024-04-26T04:42:32.430 INFO:tasks.workunit.client.0.smithi090.stdout:3/875: getdents d1/d17/d36 0 2024-04-26T04:42:32.434 INFO:tasks.workunit.client.0.smithi090.stdout:3/876: creat d1/d17/d36/d33/d115/f126 x:0 0 0 2024-04-26T04:42:32.436 INFO:tasks.workunit.client.0.smithi090.stdout:9/905: read db/d5f/d117/d82/d85/db4/d58/f10a [3694989,1861] 0 2024-04-26T04:42:32.441 INFO:tasks.workunit.client.0.smithi090.stdout:4/909: write d3/db/d12/d20/d2d/d93/d102/d72/f9b [137702,82150] 0 2024-04-26T04:42:32.443 INFO:tasks.workunit.client.0.smithi090.stdout:9/906: mkdir db/dd/d18/d38/d39/dba/d142 0 2024-04-26T04:42:32.444 INFO:tasks.workunit.client.0.smithi090.stdout:2/971: dread db/d11/d14/d25/f4e [4194304,4194304] 0 2024-04-26T04:42:32.446 INFO:tasks.workunit.client.0.smithi090.stdout:6/979: write d9/d77/db0/db5/dda/d6d/d92/fa8 [304054,21858] 0 2024-04-26T04:42:32.447 INFO:tasks.workunit.client.0.smithi090.stdout:4/910: mknod d3/db/d12/d3b/c12d 0 2024-04-26T04:42:32.447 INFO:tasks.workunit.client.0.smithi090.stdout:6/980: read d9/d77/f7d [131362,51563] 0 2024-04-26T04:42:32.447 INFO:tasks.workunit.client.0.smithi090.stdout:6/981: chown d9/d77/db0/db5/dda/d6d/d92/l2e 0 1 2024-04-26T04:42:32.455 INFO:tasks.workunit.client.0.smithi090.stdout:9/907: dread db/dd/d18/f36 [0,4194304] 0 2024-04-26T04:42:32.458 INFO:tasks.workunit.client.0.smithi090.stdout:2/972: mknod db/d11/d14/d25/d86/db9/c13e 0 2024-04-26T04:42:32.458 INFO:tasks.workunit.client.0.smithi090.stdout:2/973: chown db/d11/d14/d25/l31 404787487 1 2024-04-26T04:42:32.460 INFO:tasks.workunit.client.0.smithi090.stdout:9/908: mkdir db/dd/d18/d38/d39/d3f/d12c/d143 0 2024-04-26T04:42:32.461 INFO:tasks.workunit.client.0.smithi090.stdout:4/911: read d3/f2c [2137529,3469] 0 2024-04-26T04:42:32.462 INFO:tasks.workunit.client.0.smithi090.stdout:2/974: rename db/d11/d14/d25/d86/db9/ce7 to db/d11/d14/d27/c13f 0 2024-04-26T04:42:32.463 INFO:tasks.workunit.client.0.smithi090.stdout:5/961: truncate d4/d1a/d1f/f3a 4002975 0 2024-04-26T04:42:32.464 INFO:tasks.workunit.client.0.smithi090.stdout:6/982: write d9/d1c/f47 [3387155,101664] 0 2024-04-26T04:42:32.464 INFO:tasks.workunit.client.0.smithi090.stdout:9/909: mknod db/d5f/d117/d82/d85/db4/d99/db2/c144 0 2024-04-26T04:42:32.465 INFO:tasks.workunit.client.0.smithi090.stdout:5/962: dread d4/d1a/df/d72/dc0/ff4 [0,4194304] 0 2024-04-26T04:42:32.468 INFO:tasks.workunit.client.0.smithi090.stdout:4/912: rename d3/db/d12/d125 to d3/d13/d6c/d8c/d109/d12e 0 2024-04-26T04:42:32.469 INFO:tasks.workunit.client.0.smithi090.stdout:2/975: mkdir db/d140 0 2024-04-26T04:42:32.472 INFO:tasks.workunit.client.0.smithi090.stdout:5/963: link d4/d1a/df/d36/d8e/fe8 d4/d1a/d2a/d79/d90/d56/d80/f12a 0 2024-04-26T04:42:32.472 INFO:tasks.workunit.client.0.smithi090.stdout:5/964: creat d4/d1a/d2a/d79/d90/d56/d71/deb/f12b x:0 0 0 2024-04-26T04:42:32.472 INFO:tasks.workunit.client.0.smithi090.stdout:5/965: write d4/d1a/d2a/d79/d10a/f113 [1291065,87185] 0 2024-04-26T04:42:32.477 INFO:tasks.workunit.client.0.smithi090.stdout:4/913: link d3/d13/d6c/ffd d3/db/d12/d20/d2d/d58/d6a/da3/f12f 0 2024-04-26T04:42:32.477 INFO:tasks.workunit.client.0.smithi090.stdout:4/914: stat d3/d13/d6c/dba/f10c 0 2024-04-26T04:42:32.477 INFO:tasks.workunit.client.0.smithi090.stdout:9/910: dread db/dd/d18/f3a [0,4194304] 0 2024-04-26T04:42:32.477 INFO:tasks.workunit.client.0.smithi090.stdout:9/911: chown db/cac 190329 1 2024-04-26T04:42:32.477 INFO:tasks.workunit.client.0.smithi090.stdout:6/983: getdents d9/d3c 0 2024-04-26T04:42:32.477 INFO:tasks.workunit.client.0.smithi090.stdout:9/912: write db/d5f/d117/f8e [901221,115576] 0 2024-04-26T04:42:32.479 INFO:tasks.workunit.client.0.smithi090.stdout:5/966: getdents d4/d1a/d2a/d79/d90/d118 0 2024-04-26T04:42:32.481 INFO:tasks.workunit.client.0.smithi090.stdout:4/915: rmdir d3/db/d60/dee/dc1 39 2024-04-26T04:42:32.483 INFO:tasks.workunit.client.0.smithi090.stdout:9/913: symlink db/dd/d18/d38/d39/d3f/dff/l145 0 2024-04-26T04:42:32.484 INFO:tasks.workunit.client.0.smithi090.stdout:5/967: truncate d4/d1a/d2a/d79/d90/d49/f7d 1493275 0 2024-04-26T04:42:32.488 INFO:tasks.workunit.client.0.smithi090.stdout:4/916: symlink d3/d13/l130 0 2024-04-26T04:42:32.488 INFO:tasks.workunit.client.0.smithi090.stdout:4/917: write d3/db/d12/d20/d2d/d93/d102/d127/da5/fca [959243,66967] 0 2024-04-26T04:42:32.488 INFO:tasks.workunit.client.0.smithi090.stdout:4/918: write d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/f10b [37222,1855] 0 2024-04-26T04:42:32.490 INFO:tasks.workunit.client.0.smithi090.stdout:5/968: creat d4/d1a/d2a/d79/d90/d56/d71/f12c x:0 0 0 2024-04-26T04:42:32.490 INFO:tasks.workunit.client.0.smithi090.stdout:5/969: truncate d4/d1a/df/d36/f84 1537845 0 2024-04-26T04:42:32.495 INFO:tasks.workunit.client.0.smithi090.stdout:2/976: dwrite db/dc/f8e [4194304,4194304] 0 2024-04-26T04:42:32.496 INFO:tasks.workunit.client.0.smithi090.stdout:5/970: rename d4/d1a/d45/f97 to d4/d1a/df/d4a/f12d 0 2024-04-26T04:42:32.496 INFO:tasks.workunit.client.0.smithi090.stdout:5/971: creat d4/d1a/df/d72/dc0/f12e x:0 0 0 2024-04-26T04:42:32.497 INFO:tasks.workunit.client.0.smithi090.stdout:2/977: rmdir db/d11/d14/d1b/d20/d4c/d98 39 2024-04-26T04:42:32.497 INFO:tasks.workunit.client.0.smithi090.stdout:2/978: readlink db/d11/d14/d27/d28/dd1/dfe/d71/lb1 0 2024-04-26T04:42:32.499 INFO:tasks.workunit.client.0.smithi090.stdout:5/972: mknod d4/d1a/d2a/d79/d90/d56/d71/dea/d122/c12f 0 2024-04-26T04:42:32.499 INFO:tasks.workunit.client.0.smithi090.stdout:5/973: chown d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 1 1 2024-04-26T04:42:32.499 INFO:tasks.workunit.client.0.smithi090.stdout:5/974: chown d4/d1a/df 222179573 1 2024-04-26T04:42:32.499 INFO:tasks.workunit.client.0.smithi090.stdout:4/919: write d3/d13/f87 [1781969,39186] 0 2024-04-26T04:42:32.500 INFO:tasks.workunit.client.0.smithi090.stdout:4/920: fsync d3/d13/f21 0 2024-04-26T04:42:32.500 INFO:tasks.workunit.client.0.smithi090.stdout:2/979: rename db/d11/d14/d25/d2d/l64 to db/d11/d14/d27/l141 0 2024-04-26T04:42:32.500 INFO:tasks.workunit.client.0.smithi090.stdout:2/980: stat db/d11/d14/d27/d28/dd1/dfe/d71/dea/ff5 0 2024-04-26T04:42:32.504 INFO:tasks.workunit.client.0.smithi090.stdout:4/921: rmdir d3/db/d12/d20/d2d/dbe 39 2024-04-26T04:42:32.505 INFO:tasks.workunit.client.0.smithi090.stdout:2/981: mknod db/d11/c142 0 2024-04-26T04:42:32.506 INFO:tasks.workunit.client.0.smithi090.stdout:5/975: getdents d4/d1a/d2a/d79/d90/d112 0 2024-04-26T04:42:32.510 INFO:tasks.workunit.client.0.smithi090.stdout:6/984: dwrite d9/d3c/ffc [0,4194304] 0 2024-04-26T04:42:32.510 INFO:tasks.workunit.client.0.smithi090.stdout:6/985: write d9/d1c/d31/f54 [267136,58] 0 2024-04-26T04:42:32.512 INFO:tasks.workunit.client.0.smithi090.stdout:2/982: mknod db/d11/d7f/c143 0 2024-04-26T04:42:32.515 INFO:tasks.workunit.client.0.smithi090.stdout:9/914: dwrite db/dd/d18/d38/fd1 [0,4194304] 0 2024-04-26T04:42:32.515 INFO:tasks.workunit.client.0.smithi090.stdout:4/922: rename d3/db/d12/d20/d24 to d3/d13/d1d/d108/d131 0 2024-04-26T04:42:32.519 INFO:tasks.workunit.client.0.smithi090.stdout:2/983: truncate db/d11/d14/d25/d65/d95/faf 352752 0 2024-04-26T04:42:32.521 INFO:tasks.workunit.client.0.smithi090.stdout:9/915: link db/d5f/d117/d82/d85/db4/d58/ffe db/da3/da8/d130/f146 0 2024-04-26T04:42:32.522 INFO:tasks.workunit.client.0.smithi090.stdout:4/923: creat d3/d13/d6c/d7e/d12a/f132 x:0 0 0 2024-04-26T04:42:32.522 INFO:tasks.workunit.client.0.smithi090.stdout:4/924: write d3/db/d60/dee/dc1/dcf/f116 [344295,125811] 0 2024-04-26T04:42:32.522 INFO:tasks.workunit.client.0.smithi090.stdout:4/925: readlink d3/d13/d6c/d8c/d109/d12e/l12b 0 2024-04-26T04:42:32.522 INFO:tasks.workunit.client.0.smithi090.stdout:4/926: dread - d3/db/d12/d20/d2d/d2e/fe1 zero size 2024-04-26T04:42:32.527 INFO:tasks.workunit.client.0.smithi090.stdout:4/927: mknod d3/d13/d6c/d8c/d109/c133 0 2024-04-26T04:42:32.543 INFO:tasks.workunit.client.0.smithi090.stdout:6/986: dwrite d9/d3c/f12e [0,4194304] 0 2024-04-26T04:42:32.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/987: creat d9/d77/db0/db5/dda/d6d/d92/f14f x:0 0 0 2024-04-26T04:42:32.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/988: chown d9/d21/d3b/d142/dd9 61680411 1 2024-04-26T04:42:32.545 INFO:tasks.workunit.client.0.smithi090.stdout:6/989: mknod d9/d21/d40/d130/c150 0 2024-04-26T04:42:32.556 INFO:tasks.workunit.client.0.smithi090.stdout:5/976: dwrite d4/d1a/d1f/d53/d59/f8a [0,4194304] 0 2024-04-26T04:42:32.556 INFO:tasks.workunit.client.0.smithi090.stdout:4/928: dwrite d3/db/fac [0,4194304] 0 2024-04-26T04:42:32.559 INFO:tasks.workunit.client.0.smithi090.stdout:2/984: dwrite db/d11/d14/d27/d28/f61 [0,4194304] 0 2024-04-26T04:42:32.559 INFO:tasks.workunit.client.0.smithi090.stdout:5/977: creat d4/d1a/d2a/d79/d90/d56/d80/f130 x:0 0 0 2024-04-26T04:42:32.559 INFO:tasks.workunit.client.0.smithi090.stdout:5/978: chown d4/d1a/df/d4a/f51 2185553 1 2024-04-26T04:42:32.560 INFO:tasks.workunit.client.0.smithi090.stdout:9/916: dwrite f2 [0,4194304] 0 2024-04-26T04:42:32.560 INFO:tasks.workunit.client.0.smithi090.stdout:9/917: write db/dd/d18/d38/d39/d6b/dc0/fbf [1070630,101820] 0 2024-04-26T04:42:32.560 INFO:tasks.workunit.client.0.smithi090.stdout:9/918: fdatasync db/dd/d18/d57/fbb 0 2024-04-26T04:42:32.564 INFO:tasks.workunit.client.0.smithi090.stdout:4/929: rename d3/db/d12/d20/d2d/d58/d6a/d74/da2/f5a to d3/db/d60/dee/d11a/f134 0 2024-04-26T04:42:32.573 INFO:tasks.workunit.client.0.smithi090.stdout:2/985: creat db/d11/d14/d25/d2d/d48/d5b/d107/f144 x:0 0 0 2024-04-26T04:42:32.575 INFO:tasks.workunit.client.0.smithi090.stdout:5/979: symlink d4/d1a/df/d36/d8e/de3/l131 0 2024-04-26T04:42:32.575 INFO:tasks.workunit.client.0.smithi090.stdout:5/980: creat d4/d1a/d2a/d79/d90/d56/d80/db2/f132 x:0 0 0 2024-04-26T04:42:32.575 INFO:tasks.workunit.client.0.smithi090.stdout:5/981: write d4/d1a/d2a/d79/d90/d49/d105/f106 [3366125,112770] 0 2024-04-26T04:42:32.582 INFO:tasks.workunit.client.0.smithi090.stdout:4/930: link d3/db/d12/d20/d2d/d58/d6a/d74/d85/f11d d3/db/d12/d20/d2d/d58/f135 0 2024-04-26T04:42:32.583 INFO:tasks.workunit.client.0.smithi090.stdout:2/986: rename db/d11/d14/d1b/d20/d130 to db/d11/d14/d1b/d119/d139/d145 0 2024-04-26T04:42:32.589 INFO:tasks.workunit.client.0.smithi090.stdout:9/919: getdents db/dd/d18/d57/dcc 0 2024-04-26T04:42:32.589 INFO:tasks.workunit.client.0.smithi090.stdout:9/920: chown db/d5f/d117/d82/d85/db4/c6e 7 1 2024-04-26T04:42:32.601 INFO:tasks.workunit.client.0.smithi090.stdout:4/931: read d3/d13/d1d/d108/d131/da9/f103 [906959,73288] 0 2024-04-26T04:42:32.602 INFO:tasks.workunit.client.0.smithi090.stdout:4/932: write d3/db/d12/d20/d2d/d2e/f35 [1626099,130416] 0 2024-04-26T04:42:32.602 INFO:tasks.workunit.client.0.smithi090.stdout:4/933: dread - d3/d13/d1d/d108/d131/fc6 zero size 2024-04-26T04:42:32.602 INFO:tasks.workunit.client.0.smithi090.stdout:5/982: dwrite d4/d1a/d45/f121 [0,4194304] 0 2024-04-26T04:42:32.602 INFO:tasks.workunit.client.0.smithi090.stdout:4/934: write d3/db/d60/dee/fd1 [602985,51723] 0 2024-04-26T04:42:32.617 INFO:tasks.workunit.client.0.smithi090.stdout:2/987: dwrite db/d11/d14/d1b/d20/d4c/d98/db4/f122 [0,4194304] 0 2024-04-26T04:42:32.617 INFO:tasks.workunit.client.0.smithi090.stdout:4/935: read d3/db/d12/ff [6164191,96966] 0 2024-04-26T04:42:32.617 INFO:tasks.workunit.client.0.smithi090.stdout:2/988: write db/d11/d14/d25/f9a [505025,55709] 0 2024-04-26T04:42:32.617 INFO:tasks.workunit.client.0.smithi090.stdout:2/989: fsync db/d11/d14/d27/d2f/f8d 0 2024-04-26T04:42:32.617 INFO:tasks.workunit.client.0.smithi090.stdout:9/921: dwrite db/dd/f1d [4194304,4194304] 0 2024-04-26T04:42:32.617 INFO:tasks.workunit.client.0.smithi090.stdout:9/922: write db/d5f/d117/d82/d85/db4/d99/db2/ff5 [477341,85132] 0 2024-04-26T04:42:32.624 INFO:tasks.workunit.client.0.smithi090.stdout:4/936: creat d3/db/d60/f136 x:0 0 0 2024-04-26T04:42:32.624 INFO:tasks.workunit.client.0.smithi090.stdout:4/937: dread - d3/d13/d1d/d108/d131/da9/f119 zero size 2024-04-26T04:42:32.630 INFO:tasks.workunit.client.0.smithi090.stdout:4/938: getdents d3/db/d12/d20/d2d/d93/d102/d72/d7a 0 2024-04-26T04:42:32.630 INFO:tasks.workunit.client.0.smithi090.stdout:4/939: write d3/db/d12/d20/d2d/d93/d102/d72/f9b [194329,99844] 0 2024-04-26T04:42:32.630 INFO:tasks.workunit.client.0.smithi090.stdout:4/940: write d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/f10b [302262,27088] 0 2024-04-26T04:42:32.633 INFO:tasks.workunit.client.0.smithi090.stdout:4/941: link d3/l126 d3/d13/d6c/d7e/l137 0 2024-04-26T04:42:32.633 INFO:tasks.workunit.client.0.smithi090.stdout:4/942: rmdir d3/db/d60/dee/d11a 39 2024-04-26T04:42:32.637 INFO:tasks.workunit.client.0.smithi090.stdout:4/943: dread d3/d13/f81 [0,4194304] 0 2024-04-26T04:42:32.637 INFO:tasks.workunit.client.0.smithi090.stdout:2/990: dwrite db/d11/d14/d27/d28/dd1/dfe/d8b/f10b [0,4194304] 0 2024-04-26T04:42:32.637 INFO:tasks.workunit.client.0.smithi090.stdout:2/991: dread - db/dc/d6f/de3/de4/f133 zero size 2024-04-26T04:42:32.639 INFO:tasks.workunit.client.0.smithi090.stdout:4/944: mkdir d3/db/d12/d20/d2d/d93/d102/d72/d10d/d138 0 2024-04-26T04:42:32.641 INFO:tasks.workunit.client.0.smithi090.stdout:2/992: creat db/d11/d14/d1b/d119/d139/f146 x:0 0 0 2024-04-26T04:42:32.642 INFO:tasks.workunit.client.0.smithi090.stdout:9/923: truncate f2 708119 0 2024-04-26T04:42:32.643 INFO:tasks.workunit.client.0.smithi090.stdout:9/924: mknod db/dd/d18/d57/dcc/c147 0 2024-04-26T04:42:32.663 INFO:tasks.workunit.client.0.smithi090.stdout:2/993: dread db/d11/d14/d25/f11e [0,4194304] 0 2024-04-26T04:42:32.663 INFO:tasks.workunit.client.0.smithi090.stdout:2/994: truncate db/d11/d14/d27/d2f/d42/d5c/f4f 3558853 0 2024-04-26T04:42:32.667 INFO:tasks.workunit.client.0.smithi090.stdout:4/945: dwrite d3/db/f106 [0,4194304] 0 2024-04-26T04:42:32.669 INFO:tasks.workunit.client.0.smithi090.stdout:9/925: dwrite db/dd/d18/d38/d39/d6b/de2/f5a [4194304,4194304] 0 2024-04-26T04:42:32.669 INFO:tasks.workunit.client.0.smithi090.stdout:9/926: getdents db/dd/d18/d38/d39/dba/d142 0 2024-04-26T04:42:32.671 INFO:tasks.workunit.client.0.smithi090.stdout:4/946: mkdir d3/db/d12/d3b/d139 0 2024-04-26T04:42:32.671 INFO:tasks.workunit.client.0.smithi090.stdout:4/947: dread - d3/d13/d6c/dba/f10c zero size 2024-04-26T04:42:32.673 INFO:tasks.workunit.client.0.smithi090.stdout:9/927: unlink db/d5f/d117/d82/d85/db4/d58/c5b 0 2024-04-26T04:42:32.673 INFO:tasks.workunit.client.0.smithi090.stdout:9/928: dread - db/dd/d18/d57/dcc/f134 zero size 2024-04-26T04:42:32.674 INFO:tasks.workunit.client.0.smithi090.stdout:9/929: chown db/dd/d18/le6 420 1 2024-04-26T04:42:32.675 INFO:tasks.workunit.client.0.smithi090.stdout:4/948: unlink d3/db/cb4 0 2024-04-26T04:42:32.675 INFO:tasks.workunit.client.0.smithi090.stdout:4/949: chown d3/d13/d6c/dba/f10c 26984364 1 2024-04-26T04:42:32.676 INFO:tasks.workunit.client.0.smithi090.stdout:9/930: rename db/da3/da8/d130 to db/dd/d18/d38/d39/d6b/de2/d148 0 2024-04-26T04:42:32.678 INFO:tasks.workunit.client.0.smithi090.stdout:9/931: mkdir db/dd/d18/d38/d39/dba/d142/d149 0 2024-04-26T04:42:32.678 INFO:tasks.workunit.client.0.smithi090.stdout:9/932: creat db/dd/d18/d95/f14a x:0 0 0 2024-04-26T04:42:32.678 INFO:tasks.workunit.client.0.smithi090.stdout:9/933: write db/dd/d18/f43 [1355913,73898] 0 2024-04-26T04:42:32.680 INFO:tasks.workunit.client.0.smithi090.stdout:2/995: dwrite db/d11/f44 [0,4194304] 0 2024-04-26T04:42:32.684 INFO:tasks.workunit.client.0.smithi090.stdout:9/934: mknod db/d5f/d117/d82/d85/db4/d58/c14b 0 2024-04-26T04:42:32.687 INFO:tasks.workunit.client.0.smithi090.stdout:2/996: rename db/d11/d14/d27/d28/fa3 to db/d11/d14/d25/d2d/d48/d5b/d107/f147 0 2024-04-26T04:42:32.689 INFO:tasks.workunit.client.0.smithi090.stdout:9/935: creat db/dd/d18/d38/d39/d3f/d12c/f14c x:0 0 0 2024-04-26T04:42:32.698 INFO:tasks.workunit.client.0.smithi090.stdout:4/950: dwrite d3/db/d60/f136 [0,4194304] 0 2024-04-26T04:42:32.701 INFO:tasks.workunit.client.0.smithi090.stdout:4/951: rmdir d3/db/d12/d20/d2d/d93/d102/d127/da5 39 2024-04-26T04:42:32.701 INFO:tasks.workunit.client.0.smithi090.stdout:2/997: dread db/d11/d14/d25/d2d/f91 [0,4194304] 0 2024-04-26T04:42:32.702 INFO:tasks.workunit.client.0.smithi090.stdout:4/952: mkdir d3/db/d12/d20/d13a 0 2024-04-26T04:42:32.704 INFO:tasks.workunit.client.0.smithi090.stdout:4/953: symlink d3/db/d12/d20/d2d/d58/d6a/l13b 0 2024-04-26T04:42:32.709 INFO:tasks.workunit.client.0.smithi090.stdout:9/936: dwrite db/dd/d18/d57/dcc/f102 [0,4194304] 0 2024-04-26T04:42:32.709 INFO:tasks.workunit.client.0.smithi090.stdout:9/937: fsync db/f116 0 2024-04-26T04:42:32.710 INFO:tasks.workunit.client.0.smithi090.stdout:9/938: mknod db/d5f/d117/d82/d85/db4/d6a/db6/c14d 0 2024-04-26T04:42:32.712 INFO:tasks.workunit.client.0.smithi090.stdout:9/939: creat db/d5f/d117/d82/dbd/f14e x:0 0 0 2024-04-26T04:42:32.712 INFO:tasks.workunit.client.0.smithi090.stdout:9/940: fdatasync db/dd/d18/d38/d39/dba/ff1 0 2024-04-26T04:42:32.712 INFO:tasks.workunit.client.0.smithi090.stdout:9/941: write db/dd/d18/d57/f94 [689535,53584] 0 2024-04-26T04:42:32.735 INFO:tasks.workunit.client.0.smithi090.stdout:4/954: dwrite d3/d13/f9f [0,4194304] 0 2024-04-26T04:42:32.736 INFO:tasks.workunit.client.0.smithi090.stdout:9/942: dwrite db/dd/d18/d38/d39/d6b/de2/f5a [0,4194304] 0 2024-04-26T04:42:32.737 INFO:tasks.workunit.client.0.smithi090.stdout:2/998: dwrite db/d11/d14/d1b/d20/d4c/d98/db4/f122 [0,4194304] 0 2024-04-26T04:42:32.743 INFO:tasks.workunit.client.0.smithi090.stdout:9/943: mkdir db/d5f/d117/d82/d85/db4/d99/d14f 0 2024-04-26T04:42:32.743 INFO:tasks.workunit.client.0.smithi090.stdout:9/944: dread - db/dd/d18/d38/d39/d6b/de2/fc2 zero size 2024-04-26T04:42:32.745 INFO:tasks.workunit.client.0.smithi090.stdout:2/999: creat db/d11/d14/d27/d28/dd1/f148 x:0 0 0 2024-04-26T04:42:32.758 INFO:tasks.workunit.client.0.smithi090.stdout:4/955: dwrite d3/f2c [4194304,4194304] 0 2024-04-26T04:42:32.758 INFO:tasks.workunit.client.0.smithi090.stdout:4/956: fsync d3/db/d12/f4c 0 2024-04-26T04:42:32.760 INFO:tasks.workunit.client.0.smithi090.stdout:4/957: mknod d3/db/d12/d20/d2d/dbe/c13c 0 2024-04-26T04:42:32.762 INFO:tasks.workunit.client.0.smithi090.stdout:4/958: mknod d3/db/d12/d20/d2d/d58/c13d 0 2024-04-26T04:42:32.762 INFO:tasks.workunit.client.0.smithi090.stdout:4/959: fsync d3/d13/d1d/d108/d131/ff4 0 2024-04-26T04:42:32.762 INFO:tasks.workunit.client.0.smithi090.stdout:4/960: write d3/db/d12/d20/d2d/d58/fbf [955327,92998] 0 2024-04-26T04:42:32.772 INFO:tasks.workunit.client.0.smithi090.stdout:9/945: dwrite db/f3d [0,4194304] 0 2024-04-26T04:42:32.777 INFO:tasks.workunit.client.0.smithi090.stdout:4/961: read d3/db/d12/d20/d2d/d58/d6a/d74/da2/f37 [1563673,19629] 0 2024-04-26T04:42:32.777 INFO:tasks.workunit.client.0.smithi090.stdout:4/962: chown d3/d13/f81 53256 1 2024-04-26T04:42:32.784 INFO:tasks.workunit.client.0.smithi090.stdout:4/963: read d3/d13/d6c/d7e/f94 [5804636,85282] 0 2024-04-26T04:42:32.788 INFO:tasks.workunit.client.0.smithi090.stdout:9/946: dwrite db/d5f/d117/d82/d85/db4/d99/fc3 [0,4194304] 0 2024-04-26T04:42:32.790 INFO:tasks.workunit.client.0.smithi090.stdout:9/947: link db/laa db/dd/d18/d38/d39/d3f/d12c/l150 0 2024-04-26T04:42:32.791 INFO:tasks.workunit.client.0.smithi090.stdout:9/948: mknod db/dd/d18/d38/d39/d6b/dc0/c151 0 2024-04-26T04:42:32.792 INFO:tasks.workunit.client.0.smithi090.stdout:4/964: dread d3/db/d12/d20/d2d/d58/fbf [0,4194304] 0 2024-04-26T04:42:32.792 INFO:tasks.workunit.client.0.smithi090.stdout:9/949: link db/dd/d18/d57/c61 db/d5f/d117/dfb/c152 0 2024-04-26T04:42:32.794 INFO:tasks.workunit.client.0.smithi090.stdout:4/965: symlink d3/db/d12/d20/d2d/dbe/dde/l13e 0 2024-04-26T04:42:32.796 INFO:tasks.workunit.client.0.smithi090.stdout:9/950: creat db/dd/d18/d38/d39/d3f/f153 x:0 0 0 2024-04-26T04:42:32.796 INFO:tasks.workunit.client.0.smithi090.stdout:9/951: chown db/d5f/d117/d82/d85/db4/d99/db2/c144 2 1 2024-04-26T04:42:32.796 INFO:tasks.workunit.client.0.smithi090.stdout:9/952: mkdir db/dd/d18/d154 0 2024-04-26T04:42:32.820 INFO:tasks.workunit.client.0.smithi090.stdout:4/966: dwrite d3/f2c [8388608,4194304] 0 2024-04-26T04:42:32.820 INFO:tasks.workunit.client.0.smithi090.stdout:9/953: dwrite db/dd/d18/d57/dcc/f102 [0,4194304] 0 2024-04-26T04:42:32.821 INFO:tasks.workunit.client.0.smithi090.stdout:4/967: creat d3/db/d12/d20/d2d/dbe/f13f x:0 0 0 2024-04-26T04:42:32.822 INFO:tasks.workunit.client.0.smithi090.stdout:9/954: rename db/dd/d18/d38/d39/d6b/l104 to db/d5f/d112/l155 0 2024-04-26T04:42:32.822 INFO:tasks.workunit.client.0.smithi090.stdout:4/968: mknod d3/db/d12/d20/d2d/d58/d6a/da3/c140 0 2024-04-26T04:42:32.824 INFO:tasks.workunit.client.0.smithi090.stdout:4/969: mkdir d3/db/d12/d20/d2d/d58/d6a/d74/d141 0 2024-04-26T04:42:32.824 INFO:tasks.workunit.client.0.smithi090.stdout:4/970: write d3/f6d [1390752,110321] 0 2024-04-26T04:42:32.824 INFO:tasks.workunit.client.0.smithi090.stdout:4/971: creat d3/db/d12/d20/d2d/d58/d6a/da3/f142 x:0 0 0 2024-04-26T04:42:32.824 INFO:tasks.workunit.client.0.smithi090.stdout:4/972: write d3/f6d [1800549,25424] 0 2024-04-26T04:42:32.836 INFO:tasks.workunit.client.0.smithi090.stdout:9/955: dwrite db/d5f/d117/d82/dbd/f14e [0,4194304] 0 2024-04-26T04:42:32.839 INFO:tasks.workunit.client.0.smithi090.stdout:9/956: dread f2 [0,4194304] 0 2024-04-26T04:42:32.839 INFO:tasks.workunit.client.0.smithi090.stdout:9/957: fdatasync db/d5f/d117/f56 0 2024-04-26T04:42:32.841 INFO:tasks.workunit.client.0.smithi090.stdout:9/958: truncate db/dd/d18/d38/d39/d6b/dc0/fad 1968683 0 2024-04-26T04:42:32.841 INFO:tasks.workunit.client.0.smithi090.stdout:9/959: dread - db/dd/d18/d95/f11b zero size 2024-04-26T04:42:32.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/960: dwrite db/dd/d18/d38/d39/d3f/f93 [4194304,4194304] 0 2024-04-26T04:42:32.854 INFO:tasks.workunit.client.0.smithi090.stdout:9/961: fdatasync db/dd/d18/d38/d39/dba/ff1 0 2024-04-26T04:42:32.855 INFO:tasks.workunit.client.0.smithi090.stdout:9/962: readlink db/d5f/d112/l101 0 2024-04-26T04:42:32.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/963: symlink db/d5f/d117/d82/d85/db4/d99/db2/dcd/l156 0 2024-04-26T04:42:32.856 INFO:tasks.workunit.client.0.smithi090.stdout:9/964: truncate db/f140 422208 0 2024-04-26T04:42:32.857 INFO:tasks.workunit.client.0.smithi090.stdout:9/965: creat db/dd/d18/d38/f157 x:0 0 0 2024-04-26T04:42:32.857 INFO:tasks.workunit.client.0.smithi090.stdout:9/966: creat db/d5f/d117/d82/d85/db4/d58/d141/f158 x:0 0 0 2024-04-26T04:42:32.866 INFO:tasks.workunit.client.0.smithi090.stdout:9/967: dread db/dd/d18/d38/d39/d6b/de2/f5a [0,4194304] 0 2024-04-26T04:42:32.867 INFO:tasks.workunit.client.0.smithi090.stdout:9/968: write db/d5f/d117/d82/d85/db4/d6a/d70/f96 [1133106,28812] 0 2024-04-26T04:42:32.868 INFO:tasks.workunit.client.0.smithi090.stdout:9/969: mknod db/d5f/d117/d82/d85/db4/d99/d14f/c159 0 2024-04-26T04:42:32.868 INFO:tasks.workunit.client.0.smithi090.stdout:9/970: creat db/d5f/d117/d82/d85/db4/d99/db2/dcd/f15a x:0 0 0 2024-04-26T04:42:32.868 INFO:tasks.workunit.client.0.smithi090.stdout:9/971: truncate db/d5f/d117/d82/d85/db4/d58/f97 1521957 0 2024-04-26T04:42:32.870 INFO:tasks.workunit.client.0.smithi090.stdout:9/972: symlink db/dd/d18/d38/d39/d6b/dc0/l15b 0 2024-04-26T04:42:32.871 INFO:tasks.workunit.client.0.smithi090.stdout:9/973: symlink db/dd/d18/d38/d39/dba/l15c 0 2024-04-26T04:42:32.872 INFO:tasks.workunit.client.0.smithi090.stdout:9/974: creat db/d5f/d117/d82/d85/db4/d58/f15d x:0 0 0 2024-04-26T04:42:32.886 INFO:tasks.workunit.client.0.smithi090.stdout:9/975: dwrite db/d5f/d117/d82/d85/db4/d6a/d70/f125 [0,4194304] 0 2024-04-26T04:42:32.895 INFO:tasks.workunit.client.0.smithi090.stdout:9/976: dread db/f10c [0,4194304] 0 2024-04-26T04:42:32.897 INFO:tasks.workunit.client.0.smithi090.stdout:9/977: truncate db/d5f/d117/d82/d85/faf 787499 0 2024-04-26T04:42:32.900 INFO:tasks.workunit.client.0.smithi090.stdout:9/978: unlink db/dd/d18/d38/d39/d3f/l12f 0 2024-04-26T04:42:32.900 INFO:tasks.workunit.client.0.smithi090.stdout:9/979: truncate db/dd/d18/d95/f14a 616571 0 2024-04-26T04:42:32.903 INFO:tasks.workunit.client.0.smithi090.stdout:9/980: creat db/d5f/d117/d82/f15e x:0 0 0 2024-04-26T04:42:32.905 INFO:tasks.workunit.client.0.smithi090.stdout:9/981: unlink db/dd/d18/d38/d39/cd4 0 2024-04-26T04:42:32.905 INFO:tasks.workunit.client.0.smithi090.stdout:9/982: write db/d5f/d117/d82/d85/db4/d58/f122 [77406,49468] 0 2024-04-26T04:42:32.907 INFO:tasks.workunit.client.0.smithi090.stdout:9/983: link db/dd/d18/d38/d39/dba/f10f db/d5f/d117/d82/d85/db4/f15f 0 2024-04-26T04:42:32.910 INFO:tasks.workunit.client.0.smithi090.stdout:9/984: creat db/d5f/d117/d82/d85/db4/d99/f160 x:0 0 0 2024-04-26T04:42:32.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/985: dwrite db/d5f/d117/d82/d85/db4/d6a/f12a [0,4194304] 0 2024-04-26T04:42:32.929 INFO:tasks.workunit.client.0.smithi090.stdout:9/986: chown db/d5f/d117/d82/d85/db4/dde/cfd 1 1 2024-04-26T04:42:32.939 INFO:tasks.workunit.client.0.smithi090.stdout:9/987: dread db/dd/d18/d38/d39/d3f/f76 [0,4194304] 0 2024-04-26T04:42:32.941 INFO:tasks.workunit.client.0.smithi090.stdout:9/988: mknod db/da3/da8/c161 0 2024-04-26T04:42:32.942 INFO:tasks.workunit.client.0.smithi090.stdout:9/989: truncate db/d5f/d112/ff7 212501 0 2024-04-26T04:42:32.942 INFO:tasks.workunit.client.0.smithi090.stdout:9/990: write db/d5f/d117/d82/d85/db4/d58/f15d [480450,100131] 0 2024-04-26T04:42:32.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/991: rmdir db 39 2024-04-26T04:42:32.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/992: truncate db/dd/d18/d38/d39/d3f/f153 694985 0 2024-04-26T04:42:32.944 INFO:tasks.workunit.client.0.smithi090.stdout:9/993: creat db/dd/d18/d38/f162 x:0 0 0 2024-04-26T04:42:32.945 INFO:tasks.workunit.client.0.smithi090.stdout:9/994: symlink db/d5f/d112/l163 0 2024-04-26T04:42:32.945 INFO:tasks.workunit.client.0.smithi090.stdout:9/995: mknod db/dd/d18/d38/d39/d6b/de2/c164 0 2024-04-26T04:42:32.946 INFO:tasks.workunit.client.0.smithi090.stdout:9/996: mkdir db/d5f/d117/dfb/d165 0 2024-04-26T04:42:32.964 INFO:tasks.workunit.client.0.smithi090.stdout:9/997: dwrite db/d5f/d117/d82/f10d [0,4194304] 0 2024-04-26T04:42:32.965 INFO:tasks.workunit.client.0.smithi090.stdout:9/998: creat db/d5f/d117/d82/dbd/d109/f166 x:0 0 0 2024-04-26T04:42:32.967 INFO:tasks.workunit.client.0.smithi090.stdout:9/999: link db/d5f/d117/ld5 db/dd/d18/d154/l167 0 2024-04-26T04:42:32.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:32 smithi090 ceph-mon[26648]: pgmap v258: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 301 MiB/s rd, 370 MiB/s wr, 500 op/s 2024-04-26T04:42:32.975 INFO:tasks.workunit.client.0.smithi090.stdout:3/877: sync 2024-04-26T04:42:32.976 INFO:tasks.workunit.client.0.smithi090.stdout:3/878: creat d1/d17/d36/f127 x:0 0 0 2024-04-26T04:42:32.977 INFO:tasks.workunit.client.0.smithi090.stdout:3/879: write d1/d17/d36/d2a/d3a/fd3 [670979,23822] 0 2024-04-26T04:42:32.977 INFO:tasks.workunit.client.0.smithi090.stdout:3/880: fsync d1/d17/f2f 0 2024-04-26T04:42:32.978 INFO:tasks.workunit.client.0.smithi090.stdout:3/881: symlink d1/d17/d36/d33/d68/dae/dca/d1b/d58/l128 0 2024-04-26T04:42:32.978 INFO:tasks.workunit.client.0.smithi090.stdout:3/882: creat d1/d59/f129 x:0 0 0 2024-04-26T04:42:32.978 INFO:tasks.workunit.client.0.smithi090.stdout:3/883: fsync d1/d17/d36/d33/d42/ffb 0 2024-04-26T04:42:32.995 INFO:tasks.workunit.client.0.smithi090.stdout:3/884: dwrite d1/f27 [4194304,4194304] 0 2024-04-26T04:42:33.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:32 smithi104 ceph-mon[32369]: pgmap v258: 65 pgs: 65 active+clean; 3.8 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 301 MiB/s rd, 370 MiB/s wr, 500 op/s 2024-04-26T04:42:33.013 INFO:tasks.workunit.client.0.smithi090.stdout:3/885: dwrite d1/d59/f64 [0,4194304] 0 2024-04-26T04:42:33.013 INFO:tasks.workunit.client.0.smithi090.stdout:3/886: fdatasync d1/d17/d36/f6a 0 2024-04-26T04:42:33.015 INFO:tasks.workunit.client.0.smithi090.stdout:3/887: mknod d1/d17/d36/d33/d68/dae/dca/d2c/d97/d11e/c12a 0 2024-04-26T04:42:33.016 INFO:tasks.workunit.client.0.smithi090.stdout:3/888: symlink d1/d17/d36/d33/d68/dae/dca/l12b 0 2024-04-26T04:42:33.017 INFO:tasks.workunit.client.0.smithi090.stdout:3/889: rename d1/d17/d36/d33/d68/dae/dca/dcf/d106/d7e to d1/d17/d36/d33/d68/dae/d12c 0 2024-04-26T04:42:33.026 INFO:tasks.workunit.client.0.smithi090.stdout:3/890: write d1/d17/d36/d2a/d3a/f10d [1964441,57242] 0 2024-04-26T04:42:33.027 INFO:tasks.workunit.client.0.smithi090.stdout:3/891: mknod d1/d59/dc9/c12d 0 2024-04-26T04:42:33.028 INFO:tasks.workunit.client.0.smithi090.stdout:3/892: symlink d1/d59/dc9/d7d/d10f/l12e 0 2024-04-26T04:42:33.028 INFO:tasks.workunit.client.0.smithi090.stdout:3/893: write d1/d17/d36/d33/daa/dff/f11f [255572,67354] 0 2024-04-26T04:42:33.030 INFO:tasks.workunit.client.0.smithi090.stdout:3/894: mkdir d1/d17/d36/d33/d68/d76/d12f 0 2024-04-26T04:42:33.030 INFO:tasks.workunit.client.0.smithi090.stdout:3/895: chown d1/d17/d36/d33/d68/dae/dca/dcf 2008571 1 2024-04-26T04:42:33.030 INFO:tasks.workunit.client.0.smithi090.stdout:3/896: dread - d1/d17/d36/d33/d68/dae/dca/dcf/feb zero size 2024-04-26T04:42:33.030 INFO:tasks.workunit.client.0.smithi090.stdout:3/897: chown d1/d17/d36/d33/daa/d112 5780 1 2024-04-26T04:42:33.031 INFO:tasks.workunit.client.0.smithi090.stdout:3/898: write d1/d17/d36/d33/d115/f126 [863900,105947] 0 2024-04-26T04:42:33.031 INFO:tasks.workunit.client.0.smithi090.stdout:3/899: fsync d1/d17/d61/d9d/fd5 0 2024-04-26T04:42:33.031 INFO:tasks.workunit.client.0.smithi090.stdout:3/900: mknod d1/d17/d36/d33/d68/dae/dca/d1b/d58/c130 0 2024-04-26T04:42:33.032 INFO:tasks.workunit.client.0.smithi090.stdout:3/901: creat d1/d17/d61/dce/f131 x:0 0 0 2024-04-26T04:42:33.034 INFO:tasks.workunit.client.0.smithi090.stdout:3/902: truncate d1/d17/d36/d33/d68/dae/fb3 554904 0 2024-04-26T04:42:33.035 INFO:tasks.workunit.client.0.smithi090.stdout:3/903: mknod d1/d17/d36/d33/d42/d70/d11b/c132 0 2024-04-26T04:42:33.036 INFO:tasks.workunit.client.0.smithi090.stdout:3/904: write d1/d59/f64 [7345611,2020] 0 2024-04-26T04:42:33.038 INFO:tasks.workunit.client.0.smithi090.stdout:3/905: rename d1/d17/d36/d33/d43/cbd to d1/d17/d36/d33/d68/d76/d116/c133 0 2024-04-26T04:42:33.038 INFO:tasks.workunit.client.0.smithi090.stdout:3/906: truncate d1/d17/d36/f67 4337084 0 2024-04-26T04:42:33.038 INFO:tasks.workunit.client.0.smithi090.stdout:3/907: read d1/d17/d36/d33/d43/f92 [3058511,12890] 0 2024-04-26T04:42:33.039 INFO:tasks.workunit.client.0.smithi090.stdout:3/908: mkdir d1/d17/d61/d134 0 2024-04-26T04:42:33.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/909: readlink d1/d17/d61/de7/le9 0 2024-04-26T04:42:33.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/910: fsync d1/d17/d36/dab/fb7 0 2024-04-26T04:42:33.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/911: chown d1/d17/d36/d33/daa/dff/f11f 2265 1 2024-04-26T04:42:33.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/912: chown d1/d17/d36/d33/d115/f126 16765716 1 2024-04-26T04:42:33.041 INFO:tasks.workunit.client.0.smithi090.stdout:3/913: write d1/d17/d36/d33/d68/dae/f110 [166654,68120] 0 2024-04-26T04:42:33.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/914: mkdir d1/d17/d36/d33/daa/dff/d135 0 2024-04-26T04:42:33.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/915: rmdir d1/d17/d105 0 2024-04-26T04:42:33.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/916: creat d1/d17/d36/d33/d68/d76/d12f/f136 x:0 0 0 2024-04-26T04:42:33.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/917: fsync d1/d17/d61/d9d/da7/fd9 0 2024-04-26T04:42:33.043 INFO:tasks.workunit.client.0.smithi090.stdout:3/918: chown d1/d17/d36/d33/d42/f63 1 1 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:6/990: sync 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:5/983: sync 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:5/984: chown d4/d1a/d2a/d79/d90/d56/d80/fc7 8754471 1 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:5/985: dread - d4/d1a/df/d72/dc0/fdf zero size 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:3/919: dread d1/d17/d36/d33/d68/dae/dca/f14 [0,4194304] 0 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:3/920: chown d1/d17/d61/de7/le9 2 1 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:6/991: creat d9/d77/dc0/f151 x:0 0 0 2024-04-26T04:42:33.052 INFO:tasks.workunit.client.0.smithi090.stdout:6/992: truncate d9/d21/d3b/d86/f14d 179231 0 2024-04-26T04:42:33.053 INFO:tasks.workunit.client.0.smithi090.stdout:5/986: link d4/d1a/d1f/fe1 d4/d1a/d2a/d79/d90/d112/f133 0 2024-04-26T04:42:33.054 INFO:tasks.workunit.client.0.smithi090.stdout:3/921: symlink d1/d17/d36/d33/d115/l137 0 2024-04-26T04:42:33.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/922: rename d1/d59/dc9/d7d/c85 to d1/d17/d36/d33/d68/dae/c138 0 2024-04-26T04:42:33.058 INFO:tasks.workunit.client.0.smithi090.stdout:3/923: rename d1/d17/d36/d33/d68/dae/dca/d2c/d97/d11e to d1/d17/d36/d33/d68/dae/dca/d1b/d58/d139 0 2024-04-26T04:42:33.074 INFO:tasks.workunit.client.0.smithi090.stdout:5/987: write d4/d1a/d2a/d79/f85 [1291454,21713] 0 2024-04-26T04:42:33.074 INFO:tasks.workunit.client.0.smithi090.stdout:6/993: dwrite d9/d77/db0/db5/dda/d6d/d92/f26 [0,4194304] 0 2024-04-26T04:42:33.076 INFO:tasks.workunit.client.0.smithi090.stdout:5/988: symlink d4/d1a/d2a/d79/d90/d56/d71/dea/d122/l134 0 2024-04-26T04:42:33.076 INFO:tasks.workunit.client.0.smithi090.stdout:6/994: dread d9/d21/d3b/d142/dd9/fe8 [0,4194304] 0 2024-04-26T04:42:33.077 INFO:tasks.workunit.client.0.smithi090.stdout:5/989: unlink d4/d1a/d2a/d79/d90/d56/d80/l10d 0 2024-04-26T04:42:33.104 INFO:tasks.workunit.client.0.smithi090.stdout:5/990: dwrite d4/d1a/df/d4a/fe4 [0,4194304] 0 2024-04-26T04:42:33.104 INFO:tasks.workunit.client.0.smithi090.stdout:5/991: write d4/d1a/df/d72/dc0/fdf [620653,57005] 0 2024-04-26T04:42:33.104 INFO:tasks.workunit.client.0.smithi090.stdout:6/995: dwrite d9/d77/db0/db5/dda/d6d/d92/f29 [8388608,4194304] 0 2024-04-26T04:42:33.104 INFO:tasks.workunit.client.0.smithi090.stdout:6/996: fsync d9/d1c/d31/d69/db7/f14e 0 2024-04-26T04:42:33.105 INFO:tasks.workunit.client.0.smithi090.stdout:5/992: creat d4/d1a/d2a/d79/d90/d56/f135 x:0 0 0 2024-04-26T04:42:33.106 INFO:tasks.workunit.client.0.smithi090.stdout:6/997: creat d9/d21/d3b/d142/de7/d149/f152 x:0 0 0 2024-04-26T04:42:33.107 INFO:tasks.workunit.client.0.smithi090.stdout:5/993: truncate d4/d1a/df/d36/f4d 1256003 0 2024-04-26T04:42:33.107 INFO:tasks.workunit.client.0.smithi090.stdout:5/994: write d4/d1a/d2a/d79/d90/d56/d71/d91/fb4 [4897622,50150] 0 2024-04-26T04:42:33.108 INFO:tasks.workunit.client.0.smithi090.stdout:6/998: unlink d9/d77/db0/db5/dda/d6d/f84 0 2024-04-26T04:42:33.109 INFO:tasks.workunit.client.0.smithi090.stdout:5/995: rmdir d4/d1a/d45/db8 39 2024-04-26T04:42:33.110 INFO:tasks.workunit.client.0.smithi090.stdout:6/999: symlink d9/d1c/d31/d122/l153 0 2024-04-26T04:42:33.111 INFO:tasks.workunit.client.0.smithi090.stdout:5/996: mknod d4/d1a/d2a/d79/d90/d56/c136 0 2024-04-26T04:42:33.111 INFO:tasks.workunit.client.0.smithi090.stdout:5/997: fsync d4/d1a/d2a/d79/d10a/f113 0 2024-04-26T04:42:33.126 INFO:tasks.workunit.client.0.smithi090.stdout:5/998: dwrite d4/d1a/d45/f121 [0,4194304] 0 2024-04-26T04:42:33.132 INFO:tasks.workunit.client.0.smithi090.stdout:5/999: rmdir d4/d1a/de5 0 2024-04-26T04:42:33.240 INFO:tasks.workunit.client.0.smithi090.stdout:4/973: sync 2024-04-26T04:42:33.248 INFO:tasks.workunit.client.0.smithi090.stdout:4/974: write d3/d13/d6c/dba/f121 [1996676,47483] 0 2024-04-26T04:42:33.248 INFO:tasks.workunit.client.0.smithi090.stdout:4/975: readlink d3/db/d12/d20/d2d/dbe/dde/l123 0 2024-04-26T04:42:33.248 INFO:tasks.workunit.client.0.smithi090.stdout:4/976: creat d3/db/d12/d20/d2d/d58/d6a/d74/d85/dcb/f143 x:0 0 0 2024-04-26T04:42:33.554 INFO:tasks.workunit.client.0.smithi090.stdout:3/924: sync 2024-04-26T04:42:33.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/977: sync 2024-04-26T04:42:33.555 INFO:tasks.workunit.client.0.smithi090.stdout:3/925: write d1/d17/d36/d33/d42/f11c [433713,101475] 0 2024-04-26T04:42:33.555 INFO:tasks.workunit.client.0.smithi090.stdout:4/978: getdents d3/db/d60/d5c 0 2024-04-26T04:42:33.557 INFO:tasks.workunit.client.0.smithi090.stdout:3/926: rename d1/d17/d36/d33/d68/d76/d116 to d1/d17/d61/d9d/da7/d13a 0 2024-04-26T04:42:33.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/927: creat d1/f13b x:0 0 0 2024-04-26T04:42:33.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/928: write d1/d17/d36/d33/d42/faf [442952,84994] 0 2024-04-26T04:42:33.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/929: stat d1/d17/d36/f127 0 2024-04-26T04:42:33.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/930: dread - d1/d17/d36/d33/d68/dae/dca/d1b/f124 zero size 2024-04-26T04:42:33.558 INFO:tasks.workunit.client.0.smithi090.stdout:3/931: chown d1/d17/d36/d33/d68/dae/dca/dcf/d106/l93 9876257 1 2024-04-26T04:42:33.559 INFO:tasks.workunit.client.0.smithi090.stdout:3/932: creat d1/d17/d36/dee/f13c x:0 0 0 2024-04-26T04:42:33.577 INFO:tasks.workunit.client.0.smithi090.stdout:3/933: dwrite d1/d5c/f99 [0,4194304] 0 2024-04-26T04:42:33.595 INFO:tasks.workunit.client.0.smithi090.stdout:3/934: dwrite d1/d17/d36/d33/d68/dae/dca/d1b/f35 [0,4194304] 0 2024-04-26T04:42:33.596 INFO:tasks.workunit.client.0.smithi090.stdout:3/935: creat d1/d17/d36/d33/f13d x:0 0 0 2024-04-26T04:42:33.604 INFO:tasks.workunit.client.0.smithi090.stdout:3/936: dread d1/d9/fb [4194304,4194304] 0 2024-04-26T04:42:33.605 INFO:tasks.workunit.client.0.smithi090.stdout:3/937: creat d1/d17/d36/d33/d68/dae/dca/dcf/d106/f13e x:0 0 0 2024-04-26T04:42:33.606 INFO:tasks.workunit.client.0.smithi090.stdout:3/938: symlink d1/d17/d36/d33/d68/dae/dca/dcf/d106/d82/l13f 0 2024-04-26T04:42:33.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/939: mkdir d1/d17/d61/d9d/da7/d140 0 2024-04-26T04:42:33.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/940: readlink d1/d17/d36/d33/d68/dae/dca/dcf/d106/le0 0 2024-04-26T04:42:33.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/941: chown d1/d17/d36/d2a/l88 236 1 2024-04-26T04:42:33.607 INFO:tasks.workunit.client.0.smithi090.stdout:3/942: write d1/d9/fa [3751259,9798] 0 2024-04-26T04:42:33.608 INFO:tasks.workunit.client.0.smithi090.stdout:3/943: mknod d1/d17/d36/d33/d68/dae/dca/d1b/d58/d139/c141 0 2024-04-26T04:42:33.608 INFO:tasks.workunit.client.0.smithi090.stdout:3/944: creat d1/d17/d36/d33/d42/f142 x:0 0 0 2024-04-26T04:42:33.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/945: dwrite d1/d17/f4f [0,4194304] 0 2024-04-26T04:42:33.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/946: chown d1/d17/d61/d9d/da7/fd9 1766616 1 2024-04-26T04:42:33.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/947: dread - d1/d17/d61/d9d/da7/fd9 zero size 2024-04-26T04:42:33.628 INFO:tasks.workunit.client.0.smithi090.stdout:3/948: chown d1/d17/d36/d33/d68/dae/dca/d1b/d58/d139/c120 178266 1 2024-04-26T04:42:33.629 INFO:tasks.workunit.client.0.smithi090.stdout:3/949: symlink d1/d17/d36/d33/d68/dae/dca/dcf/l143 0 2024-04-26T04:42:33.630 INFO:tasks.workunit.client.0.smithi090.stdout:3/950: symlink d1/d17/d36/d33/d68/d76/d12f/l144 0 2024-04-26T04:42:33.630 INFO:tasks.workunit.client.0.smithi090.stdout:3/951: unlink d1/d17/d36/f67 0 2024-04-26T04:42:33.631 INFO:tasks.workunit.client.0.smithi090.stdout:3/952: unlink d1/d17/c18 0 2024-04-26T04:42:33.631 INFO:tasks.workunit.client.0.smithi090.stdout:3/953: fsync d1/d17/d36/f6a 0 2024-04-26T04:42:33.631 INFO:tasks.workunit.client.0.smithi090.stdout:3/954: creat d1/d17/d61/dce/f145 x:0 0 0 2024-04-26T04:42:33.645 INFO:tasks.workunit.client.0.smithi090.stdout:3/955: dwrite d1/d17/d36/d2a/fe3 [0,4194304] 0 2024-04-26T04:42:33.645 INFO:tasks.workunit.client.0.smithi090.stdout:3/956: truncate d1/d17/d61/dce/f145 303959 0 2024-04-26T04:42:33.646 INFO:tasks.workunit.client.0.smithi090.stdout:3/957: chown d1/d59/ff0 296 1 2024-04-26T04:42:33.647 INFO:tasks.workunit.client.0.smithi090.stdout:3/958: getdents d1/d17/d36/d33/d43 0 2024-04-26T04:42:33.647 INFO:tasks.workunit.client.0.smithi090.stdout:3/959: fdatasync d1/d17/d61/dce/f117 0 2024-04-26T04:42:33.647 INFO:tasks.workunit.client.0.smithi090.stdout:3/960: stat d1/d17/d61/d9d/da7/d13a 0 2024-04-26T04:42:33.647 INFO:tasks.workunit.client.0.smithi090.stdout:3/961: fdatasync d1/d59/dc9/f7c 0 2024-04-26T04:42:33.648 INFO:tasks.workunit.client.0.smithi090.stdout:3/962: stat d1/d17/d36/d33/d43/c51 0 2024-04-26T04:42:33.648 INFO:tasks.workunit.client.0.smithi090.stdout:3/963: chown d1/d17/d36/d33/d68/dae/dca/d2c/d97 20637718 1 2024-04-26T04:42:33.649 INFO:tasks.workunit.client.0.smithi090.stdout:3/964: mkdir d1/d17/d36/d33/d68/dae/dca/dcf/d146 0 2024-04-26T04:42:33.651 INFO:tasks.workunit.client.0.smithi090.stdout:3/965: rename d1/d9/f38 to d1/d17/f147 0 2024-04-26T04:42:33.652 INFO:tasks.workunit.client.0.smithi090.stdout:3/966: mknod d1/d17/d36/d2a/c148 0 2024-04-26T04:42:33.652 INFO:tasks.workunit.client.0.smithi090.stdout:3/967: read - d1/d17/d36/d33/d68/dae/dca/dcf/d106/f13e zero size 2024-04-26T04:42:33.910 INFO:tasks.workunit.client.0.smithi090.stdout:4/979: sync 2024-04-26T04:42:33.910 INFO:tasks.workunit.client.0.smithi090.stdout:4/980: write d3/db/d12/f4c [2976108,74878] 0 2024-04-26T04:42:33.926 INFO:tasks.workunit.client.0.smithi090.stdout:4/981: dwrite d3/d13/d6c/dba/f121 [4194304,4194304] 0 2024-04-26T04:42:33.926 INFO:tasks.workunit.client.0.smithi090.stdout:4/982: dread - d3/ffa zero size 2024-04-26T04:42:33.927 INFO:tasks.workunit.client.0.smithi090.stdout:4/983: truncate d3/db/d12/d3b/f115 11793 0 2024-04-26T04:42:33.951 INFO:tasks.workunit.client.0.smithi090.stdout:4/984: dwrite d3/db/d12/f4e [0,4194304] 0 2024-04-26T04:42:33.951 INFO:tasks.workunit.client.0.smithi090.stdout:4/985: write d3/db/d12/fa0 [3608,50856] 0 2024-04-26T04:42:33.973 INFO:tasks.workunit.client.0.smithi090.stdout:4/986: dwrite d3/db/d12/d3b/f115 [0,4194304] 0 2024-04-26T04:42:33.982 INFO:tasks.workunit.client.0.smithi090.stdout:4/987: write d3/db/f106 [3862132,15608] 0 2024-04-26T04:42:33.991 INFO:tasks.workunit.client.0.smithi090.stdout:4/988: dread d3/f7d [0,4194304] 0 2024-04-26T04:42:33.991 INFO:tasks.workunit.client.0.smithi090.stdout:4/989: truncate d3/d13/d1d/d108/d131/da9/f119 793333 0 2024-04-26T04:42:33.994 INFO:tasks.workunit.client.0.smithi090.stdout:4/990: dread d3/db/d12/d20/d2d/d58/d6a/d74/da2/f37 [0,4194304] 0 2024-04-26T04:42:33.996 INFO:tasks.workunit.client.0.smithi090.stdout:4/991: symlink d3/db/d12/d20/d2d/d58/d6a/da3/l144 0 2024-04-26T04:42:33.997 INFO:tasks.workunit.client.0.smithi090.stdout:4/992: mknod d3/db/d12/d20/d2d/d93/d102/dd4/c145 0 2024-04-26T04:42:33.997 INFO:tasks.workunit.client.0.smithi090.stdout:4/993: dread - d3/db/d12/d20/d2d/d2e/fe7 zero size 2024-04-26T04:42:33.997 INFO:tasks.workunit.client.0.smithi090.stdout:4/994: dread d3/d13/d1d/d108/d131/da9/f119 [0,4194304] 0 2024-04-26T04:42:33.998 INFO:tasks.workunit.client.0.smithi090.stdout:4/995: read - d3/db/d12/d20/d2d/d2e/fe1 zero size 2024-04-26T04:42:34.006 INFO:tasks.workunit.client.0.smithi090.stdout:4/996: read d3/db/d12/d3b/f78 [285990,51811] 0 2024-04-26T04:42:34.015 INFO:tasks.workunit.client.0.smithi090.stdout:4/997: write d3/f48 [2121375,97772] 0 2024-04-26T04:42:34.015 INFO:tasks.workunit.client.0.smithi090.stdout:4/998: write d3/d13/d1d/d108/d131/fc6 [165537,63377] 0 2024-04-26T04:42:34.015 INFO:tasks.workunit.client.0.smithi090.stdout:4/999: chown d3/la1 5908711 1 2024-04-26T04:42:34.020 INFO:tasks.workunit.client.0.smithi090.stdout:3/968: sync 2024-04-26T04:42:34.020 INFO:tasks.workunit.client.0.smithi090.stdout:3/969: mknod d1/d17/d36/d33/d68/dae/dca/dcf/d106/c149 0 2024-04-26T04:42:34.029 INFO:tasks.workunit.client.0.smithi090.stdout:3/970: read d1/d17/d36/d33/d68/dae/dca/d1b/f35 [2787409,109778] 0 2024-04-26T04:42:34.030 INFO:tasks.workunit.client.0.smithi090.stdout:3/971: unlink d1/d17/d36/d2a/d3a/ldc 0 2024-04-26T04:42:34.030 INFO:tasks.workunit.client.0.smithi090.stdout:3/972: truncate d1/d17/d61/d9d/fec 3545297 0 2024-04-26T04:42:34.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/973: dread d1/d59/dc9/f7c [0,4194304] 0 2024-04-26T04:42:34.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/974: chown d1/d17/d36/d2a/d3a/lb9 10351346 1 2024-04-26T04:42:34.040 INFO:tasks.workunit.client.0.smithi090.stdout:3/975: write d1/d17/d61/dce/f145 [1185970,41532] 0 2024-04-26T04:42:34.042 INFO:tasks.workunit.client.0.smithi090.stdout:3/976: rename d1/d17/d36/d33/d68/dae/dca/f91 to d1/d17/d36/d33/d68/d76/f14a 0 2024-04-26T04:42:34.043 INFO:tasks.workunit.client.0.smithi090.stdout:3/977: link d1/d17/d36/d33/d68/dae/dca/d1b/f35 d1/d17/d36/d33/d68/dae/dca/d1b/d58/d139/f14b 0 2024-04-26T04:42:34.044 INFO:tasks.workunit.client.0.smithi090.stdout:3/978: symlink d1/d59/l14c 0 2024-04-26T04:42:34.045 INFO:tasks.workunit.client.0.smithi090.stdout:3/979: link d1/d17/d36/d33/d68/dae/dca/dcf/d106/l86 d1/d17/d36/d33/d68/dae/dca/l14d 0 2024-04-26T04:42:34.053 INFO:tasks.workunit.client.0.smithi090.stdout:3/980: dread d1/d59/d5e/f62 [0,4194304] 0 2024-04-26T04:42:34.054 INFO:tasks.workunit.client.0.smithi090.stdout:3/981: mkdir d1/d17/d36/d33/d68/d76/d12f/d14e 0 2024-04-26T04:42:34.055 INFO:tasks.workunit.client.0.smithi090.stdout:3/982: creat d1/d17/d36/f14f x:0 0 0 2024-04-26T04:42:34.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/983: mknod d1/d17/d36/d33/d115/c150 0 2024-04-26T04:42:34.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/984: chown d1/d17/d36/d33/d68/dae/fb3 0 1 2024-04-26T04:42:34.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/985: chown d1/d17/d36/dee/f13c 505945 1 2024-04-26T04:42:34.056 INFO:tasks.workunit.client.0.smithi090.stdout:3/986: write d1/d17/d36/dee/f13c [856723,96892] 0 2024-04-26T04:42:34.057 INFO:tasks.workunit.client.0.smithi090.stdout:3/987: rename d1/f13b to d1/d17/d36/d33/d68/d76/d12f/d14e/f151 0 2024-04-26T04:42:34.072 INFO:tasks.workunit.client.0.smithi090.stdout:3/988: dwrite d1/d17/d36/d33/f13d [0,4194304] 0 2024-04-26T04:42:34.074 INFO:tasks.workunit.client.0.smithi090.stdout:3/989: link d1/d59/l14c d1/d17/d36/d33/d42/l152 0 2024-04-26T04:42:34.075 INFO:tasks.workunit.client.0.smithi090.stdout:3/990: mkdir d1/d17/d61/dce/d101/d153 0 2024-04-26T04:42:34.077 INFO:tasks.workunit.client.0.smithi090.stdout:3/991: creat d1/d17/d36/d33/d68/dae/dca/d1b/d58/f154 x:0 0 0 2024-04-26T04:42:34.091 INFO:tasks.workunit.client.0.smithi090.stdout:3/992: rename d1/d17/d36/d33 to d1/d9/d155 0 2024-04-26T04:42:34.091 INFO:tasks.workunit.client.0.smithi090.stdout:3/993: rmdir d1/d9/d155/d68/dae/dca/d1b 39 2024-04-26T04:42:34.094 INFO:tasks.workunit.client.0.smithi090.stdout:3/994: getdents d1/d9 0 2024-04-26T04:42:34.095 INFO:tasks.workunit.client.0.smithi090.stdout:3/995: mkdir d1/d9/d155/d68/dae/dca/d2c/d97/d156 0 2024-04-26T04:42:34.096 INFO:tasks.workunit.client.0.smithi090.stdout:3/996: dread d1/d9/d155/d68/d71/fd0 [0,4194304] 0 2024-04-26T04:42:34.096 INFO:tasks.workunit.client.0.smithi090.stdout:3/997: read d1/d9/d155/d68/dae/dca/f14 [2953107,86224] 0 2024-04-26T04:42:34.097 INFO:tasks.workunit.client.0.smithi090.stdout:3/998: rmdir d1/d17/d61/dce/d101 39 2024-04-26T04:42:34.097 INFO:tasks.workunit.client.0.smithi090.stdout:3/999: read - d1/d17/d36/f127 zero size 2024-04-26T04:42:34.099 INFO:tasks.workunit.client.0.smithi090.stderr:+ rm -rf -- ./tmp.YPJSjuIeJ8 2024-04-26T04:42:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:34 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:34 smithi104 ceph-mon[32369]: pgmap v259: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 303 MiB/s rd, 364 MiB/s wr, 560 op/s 2024-04-26T04:42:34.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:34 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:34.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:34 smithi090 ceph-mon[26648]: pgmap v259: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 303 MiB/s rd, 364 MiB/s wr, 560 op/s 2024-04-26T04:42:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:36 smithi090 ceph-mon[26648]: pgmap v260: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 261 MiB/s rd, 316 MiB/s wr, 509 op/s 2024-04-26T04:42:36.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:36 smithi104 ceph-mon[32369]: pgmap v260: 65 pgs: 65 active+clean; 3.9 GiB data, 12 GiB used, 525 GiB / 536 GiB avail; 261 MiB/s rd, 316 MiB/s wr, 509 op/s 2024-04-26T04:42:38.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:38 smithi090 ceph-mon[26648]: pgmap v261: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 210 MiB/s rd, 262 MiB/s wr, 410 op/s 2024-04-26T04:42:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:38 smithi104 ceph-mon[32369]: pgmap v261: 65 pgs: 65 active+clean; 3.7 GiB data, 11 GiB used, 525 GiB / 536 GiB avail; 210 MiB/s rd, 262 MiB/s wr, 410 op/s 2024-04-26T04:42:40.040 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T04:42:40.041 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1/tmp 2024-04-26T04:42:40.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:40 smithi090 ceph-mon[26648]: pgmap v262: 65 pgs: 65 active+clean; 2.8 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 172 MiB/s rd, 210 MiB/s wr, 393 op/s 2024-04-26T04:42:40.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:40 smithi104 ceph-mon[32369]: pgmap v262: 65 pgs: 65 active+clean; 2.8 GiB data, 9.4 GiB used, 527 GiB / 536 GiB avail; 172 MiB/s rd, 210 MiB/s wr, 393 op/s 2024-04-26T04:42:42.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:42 smithi090 ceph-mon[26648]: pgmap v263: 65 pgs: 65 active+clean; 2.1 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 109 MiB/s rd, 131 MiB/s wr, 359 op/s 2024-04-26T04:42:42.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:42 smithi104 ceph-mon[32369]: pgmap v263: 65 pgs: 65 active+clean; 2.1 GiB data, 7.4 GiB used, 529 GiB / 536 GiB avail; 109 MiB/s rd, 131 MiB/s wr, 359 op/s 2024-04-26T04:42:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:44 smithi104 ceph-mon[32369]: pgmap v264: 65 pgs: 65 active+clean; 1.4 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 76 MiB/s rd, 92 MiB/s wr, 365 op/s 2024-04-26T04:42:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:44 smithi090 ceph-mon[26648]: pgmap v264: 65 pgs: 65 active+clean; 1.4 GiB data, 4.7 GiB used, 532 GiB / 536 GiB avail; 76 MiB/s rd, 92 MiB/s wr, 365 op/s 2024-04-26T04:42:45.319 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-26T04:42:45.319 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-26T04:42:46.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:46 smithi090 ceph-mon[26648]: pgmap v265: 65 pgs: 65 active+clean; 930 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 33 MiB/s rd, 46 MiB/s wr, 302 op/s 2024-04-26T04:42:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:46 smithi104 ceph-mon[32369]: pgmap v265: 65 pgs: 65 active+clean; 930 MiB data, 3.7 GiB used, 533 GiB / 536 GiB avail; 33 MiB/s rd, 46 MiB/s wr, 302 op/s 2024-04-26T04:42:48.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:48 smithi090 ceph-mon[26648]: pgmap v266: 65 pgs: 65 active+clean; 703 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 14 MiB/s wr, 320 op/s 2024-04-26T04:42:48.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:48 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:48 smithi104 ceph-mon[32369]: pgmap v266: 65 pgs: 65 active+clean; 703 MiB data, 2.8 GiB used, 534 GiB / 536 GiB avail; 11 MiB/s rd, 14 MiB/s wr, 320 op/s 2024-04-26T04:42:48.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:48 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:42:50.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:50 smithi090 ceph-mon[26648]: pgmap v267: 65 pgs: 65 active+clean; 375 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 431 op/s 2024-04-26T04:42:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:50 smithi104 ceph-mon[32369]: pgmap v267: 65 pgs: 65 active+clean; 375 MiB data, 1.9 GiB used, 535 GiB / 536 GiB avail; 2.4 MiB/s rd, 6.2 MiB/s wr, 431 op/s 2024-04-26T04:42:52.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:52 smithi104 ceph-mon[32369]: pgmap v268: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.8 MiB/s wr, 446 op/s 2024-04-26T04:42:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:52 smithi090 ceph-mon[26648]: pgmap v268: 65 pgs: 65 active+clean; 319 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.8 MiB/s wr, 446 op/s 2024-04-26T04:42:54.514 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:54 smithi104 ceph-mon[32369]: pgmap v269: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.6 MiB/s wr, 488 op/s 2024-04-26T04:42:54.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:54 smithi090 ceph-mon[26648]: pgmap v269: 65 pgs: 65 active+clean; 312 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.6 MiB/s wr, 488 op/s 2024-04-26T04:42:54.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2970276151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88069270 msgr2=0x7f1b881051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2970276151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88069270 0x7f1b881051d0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009910 tx=0x7f1b78009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2970276151 shutdown_connections 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2970276151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b88105710 0x7f1b88107af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2970276151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88069270 0x7f1b881051d0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2970276151 >> 172.21.15.90:0/2970276151 conn(0x7f1b880fa780 msgr2=0x7f1b880fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2970276151 shutdown_connections 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.799+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2970276151 wait complete. 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b8e3b6700 1 Processor -- start 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b8e3b6700 1 -- start start 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b8e3b6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b88105710 0x7f1b88197ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b8e3b6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88198220 0x7f1b8819d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b8e3b6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88198690 con 0x7f1b88198220 2024-04-26T04:42:54.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b8e3b6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b88198800 con 0x7f1b88105710 2024-04-26T04:42:54.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88198220 0x7f1b8819d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:54.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b87fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b88105710 0x7f1b88197ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:54.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b877fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88198220 0x7f1b8819d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57548/0 (socket says 172.21.15.90:57548) 2024-04-26T04:42:54.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.800+0000 7f1b877fe700 1 -- 172.21.15.90:0/2280964530 learned_addr learned my addr 172.21.15.90:0/2280964530 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b877fe700 1 -- 172.21.15.90:0/2280964530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b88105710 msgr2=0x7f1b88197ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b877fe700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b88105710 0x7f1b88197ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b877fe700 1 -- 172.21.15.90:0/2280964530 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b7c009520 con 0x7f1b88198220 2024-04-26T04:42:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b87fff700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b88105710 0x7f1b88197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:42:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b877fe700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88198220 0x7f1b8819d250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00eab0 tx=0x7f1b7c00edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:54.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b857fa700 1 -- 172.21.15.90:0/2280964530 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b7c00cba0 con 0x7f1b88198220 2024-04-26T04:42:54.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b857fa700 1 -- 172.21.15.90:0/2280964530 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f1b7c0044d0 con 0x7f1b88198220 2024-04-26T04:42:54.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b780095f0 con 0x7f1b88198220 2024-04-26T04:42:54.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b857fa700 1 -- 172.21.15.90:0/2280964530 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b7c005020 con 0x7f1b88198220 2024-04-26T04:42:54.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.801+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b8819db80 con 0x7f1b88198220 2024-04-26T04:42:54.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.803+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b88191dd0 con 0x7f1b88198220 2024-04-26T04:42:54.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.804+0000 7f1b857fa700 1 -- 172.21.15.90:0/2280964530 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f1b7c00cd00 con 0x7f1b88198220 2024-04-26T04:42:54.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.804+0000 7f1b857fa700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1b7006c680 0x7f1b7006eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:54.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.804+0000 7f1b857fa700 1 -- 172.21.15.90:0/2280964530 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f1b7c014070 con 0x7f1b88198220 2024-04-26T04:42:54.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.804+0000 7f1b87fff700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1b7006c680 0x7f1b7006eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:54.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.805+0000 7f1b87fff700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1b7006c680 0x7f1b7006eb30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009fd0 tx=0x7f1b78005fd0 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:54.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:54.811+0000 7f1b857fa700 1 -- 172.21.15.90:0/2280964530 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f1b7c059b00 con 0x7f1b88198220 2024-04-26T04:42:55.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.085+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1b880694e0 con 0x7f1b7006c680 2024-04-26T04:42:55.088 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.087+0000 7f1b857fa700 1 -- 172.21.15.90:0/2280964530 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7f1b880694e0 con 0x7f1b7006c680 2024-04-26T04:42:55.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1b7006c680 msgr2=0x7f1b7006eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1b7006c680 0x7f1b7006eb30 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1b78009fd0 tx=0x7f1b78005fd0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88198220 msgr2=0x7f1b8819d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88198220 0x7f1b8819d250 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c00eab0 tx=0x7f1b7c00edc0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 shutdown_connections 2024-04-26T04:42:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f1b7006c680 0x7f1b7006eb30 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b88105710 0x7f1b88197ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 --2- 172.21.15.90:0/2280964530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b88198220 0x7f1b8819d250 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 >> 172.21.15.90:0/2280964530 conn(0x7f1b880fa780 msgr2=0x7f1b880fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 shutdown_connections 2024-04-26T04:42:55.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.089+0000 7f1b8e3b6700 1 -- 172.21.15.90:0/2280964530 wait complete. 2024-04-26T04:42:55.101 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:42:55.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 -- 172.21.15.90:0/2122988478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 msgr2=0x7fbc68100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/2122988478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc68100790 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fbc50009910 tx=0x7fbc50009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:55.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 -- 172.21.15.90:0/2122988478 shutdown_connections 2024-04-26T04:42:55.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/2122988478 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc68100790 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/2122988478 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc680ff140 0x7fbc680ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 -- 172.21.15.90:0/2122988478 >> 172.21.15.90:0/2122988478 conn(0x7fbc680fa6d0 msgr2=0x7fbc680fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:55.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 -- 172.21.15.90:0/2122988478 shutdown_connections 2024-04-26T04:42:55.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.208+0000 7fbc6e97d700 1 -- 172.21.15.90:0/2122988478 wait complete. 2024-04-26T04:42:55.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc6e97d700 1 Processor -- start 2024-04-26T04:42:55.211 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc6e97d700 1 -- start start 2024-04-26T04:42:55.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc6e97d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc680ff140 0x7fbc68069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:55.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc6e97d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc6819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:55.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc6e97d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6819adc0 con 0x7fbc68100340 2024-04-26T04:42:55.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc6e97d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6819af00 con 0x7fbc680ff140 2024-04-26T04:42:55.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc5ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc6819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc5ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc6819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57568/0 (socket says 172.21.15.90:57568) 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc5ffff700 1 -- 172.21.15.90:0/1339760658 learned_addr learned my addr 172.21.15.90:0/1339760658 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.209+0000 7fbc67fff700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc680ff140 0x7fbc68069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc5ffff700 1 -- 172.21.15.90:0/1339760658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc680ff140 msgr2=0x7fbc68069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc5ffff700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc680ff140 0x7fbc68069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc5ffff700 1 -- 172.21.15.90:0/1339760658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc500095f0 con 0x7fbc68100340 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc5ffff700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc6819a7f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000c00 tx=0x7fbc5000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc65ffb700 1 -- 172.21.15.90:0/1339760658 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc5001d070 con 0x7fbc68100340 2024-04-26T04:42:55.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6819b180 con 0x7fbc68100340 2024-04-26T04:42:55.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc65ffb700 1 -- 172.21.15.90:0/1339760658 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbc50005dc0 con 0x7fbc68100340 2024-04-26T04:42:55.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc65ffb700 1 -- 172.21.15.90:0/1339760658 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbc500226a0 con 0x7fbc68100340 2024-04-26T04:42:55.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.210+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6819b670 con 0x7fbc68100340 2024-04-26T04:42:55.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.211+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc68107c90 con 0x7fbc68100340 2024-04-26T04:42:55.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.212+0000 7fbc65ffb700 1 -- 172.21.15.90:0/1339760658 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fbc5000faf0 con 0x7fbc68100340 2024-04-26T04:42:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.212+0000 7fbc65ffb700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbc5406c680 0x7fbc5406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.213+0000 7fbc65ffb700 1 -- 172.21.15.90:0/1339760658 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fbc5008d080 con 0x7fbc68100340 2024-04-26T04:42:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.213+0000 7fbc67fff700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbc5406c680 0x7fbc5406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:55.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.213+0000 7fbc67fff700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbc5406c680 0x7fbc5406eb30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc681001a0 tx=0x7fbc5800b040 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:55.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.218+0000 7fbc65ffb700 1 -- 172.21.15.90:0/1339760658 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fbc5005b2c0 con 0x7fbc68100340 2024-04-26T04:42:55.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.509+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbc68104c90 con 0x7fbc5406c680 2024-04-26T04:42:55.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.510+0000 7fbc65ffb700 1 -- 172.21.15.90:0/1339760658 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7fbc68104c90 con 0x7fbc5406c680 2024-04-26T04:42:55.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbc5406c680 msgr2=0x7fbc5406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbc5406c680 0x7fbc5406eb30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbc681001a0 tx=0x7fbc5800b040 comp rx=0 tx=0).stop 2024-04-26T04:42:55.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 msgr2=0x7fbc6819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc6819a7f0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000c00 tx=0x7fbc5000f690 comp rx=0 tx=0).stop 2024-04-26T04:42:55.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 shutdown_connections 2024-04-26T04:42:55.515 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fbc5406c680 0x7fbc5406eb30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbc680ff140 0x7fbc68069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 --2- 172.21.15.90:0/1339760658 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbc68100340 0x7fbc6819a7f0 secure :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fbc50000c00 tx=0x7fbc5000f690 comp rx=0 tx=0).stop 2024-04-26T04:42:55.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.514+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 >> 172.21.15.90:0/1339760658 conn(0x7fbc680fa6d0 msgr2=0x7fbc68103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:55.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.515+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 shutdown_connections 2024-04-26T04:42:55.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.516+0000 7fbc6e97d700 1 -- 172.21.15.90:0/1339760658 wait complete. 2024-04-26T04:42:55.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.689+0000 7f9b827fc700 1 -- 172.21.15.90:0/4171550253 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7400e070 con 0x7f9b8410b100 2024-04-26T04:42:55.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4171550253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8410b100 msgr2=0x7f9b8410b510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4171550253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8410b100 0x7f9b8410b510 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b7400b0a0 tx=0x7f9b7400b3b0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4171550253 shutdown_connections 2024-04-26T04:42:55.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4171550253 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410be80 0x7f9b8410c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4171550253 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8410b100 0x7f9b8410b510 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4171550253 >> 172.21.15.90:0/4171550253 conn(0x7f9b8406a6d0 msgr2=0x7f9b8406ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:55.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4171550253 shutdown_connections 2024-04-26T04:42:55.692 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.690+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4171550253 wait complete. 2024-04-26T04:42:55.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.691+0000 7f9b89dc2700 1 Processor -- start 2024-04-26T04:42:55.694 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.691+0000 7f9b89dc2700 1 -- start start 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b89dc2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410b100 0x7f9b841a0570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b89dc2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8410be80 0x7f9b841a0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b89dc2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841a10d0 con 0x7f9b8410b100 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b89dc2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b841a1210 con 0x7f9b8410be80 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b837fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410b100 0x7f9b841a0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b837fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410b100 0x7f9b841a0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57582/0 (socket says 172.21.15.90:57582) 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b837fe700 1 -- 172.21.15.90:0/4184191696 learned_addr learned my addr 172.21.15.90:0/4184191696 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:55.695 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b837fe700 1 -- 172.21.15.90:0/4184191696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8410be80 msgr2=0x7f9b841a0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b837fe700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8410be80 0x7f9b841a0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b837fe700 1 -- 172.21.15.90:0/4184191696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b74009d00 con 0x7f9b8410b100 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.692+0000 7f9b837fe700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410b100 0x7f9b841a0570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9b74003c30 tx=0x7f9b74003d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.693+0000 7f9b80ff9700 1 -- 172.21.15.90:0/4184191696 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7400e070 con 0x7f9b8410b100 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.693+0000 7f9b80ff9700 1 -- 172.21.15.90:0/4184191696 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9b7400be50 con 0x7f9b8410b100 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.693+0000 7f9b80ff9700 1 -- 172.21.15.90:0/4184191696 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7401da40 con 0x7f9b8410b100 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.693+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b841a5c60 con 0x7f9b8410b100 2024-04-26T04:42:55.696 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.693+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b841a61b0 con 0x7f9b8410b100 2024-04-26T04:42:55.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.694+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b84113820 con 0x7f9b8410b100 2024-04-26T04:42:55.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.698+0000 7f9b80ff9700 1 -- 172.21.15.90:0/4184191696 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7f9b74019040 con 0x7f9b8410b100 2024-04-26T04:42:55.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.698+0000 7f9b80ff9700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9b6c06c6d0 0x7f9b6c06eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:55.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.699+0000 7f9b80ff9700 1 -- 172.21.15.90:0/4184191696 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7f9b7408d080 con 0x7f9b8410b100 2024-04-26T04:42:55.700 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.699+0000 7f9b82ffd700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9b6c06c6d0 0x7f9b6c06eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:55.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.701+0000 7f9b82ffd700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9b6c06c6d0 0x7f9b6c06eb80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9b780097b0 tx=0x7f9b78006d20 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:55.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.703+0000 7f9b80ff9700 1 -- 172.21.15.90:0/4184191696 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7f9b7405b2b0 con 0x7f9b8410b100 2024-04-26T04:42:55.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:55.983+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f9b841107d0 con 0x7f9b6c06c6d0 2024-04-26T04:42:56.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.001+0000 7f9b80ff9700 1 -- 172.21.15.90:0/4184191696 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3456 (secure 0 0 0) 0x7f9b841107d0 con 0x7f9b6c06c6d0 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (7m) 2m ago 8m 25.9M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (8m) 2m ago 8m 7407k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (7m) 2m ago 7m 7335k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (8m) 2m ago 8m 6660k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (7m) 2m ago 7m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (6m) 2m ago 8m 83.8M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (3m) 2m ago 2m 15.6M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (3m) 2m ago 3m 11.2M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (2m) 2m ago 2m 12.2M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (3m) 2m ago 3m 18.4M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:9283,8765,8443 running (9m) 2m ago 9m 487M - 18.2.0 dc2bc1663786 a008eb0a2d7e 2024-04-26T04:42:56.003 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (7m) 2m ago 7m 437M - 18.2.0 dc2bc1663786 b809d29d97b5 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (9m) 2m ago 9m 49.1M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (7m) 2m ago 7m 35.3M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (8m) 2m ago 8m 14.3M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (7m) 2m ago 7m 15.5M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (6m) 2m ago 6m 48.1M 3946M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (6m) 2m ago 6m 47.6M 3946M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (5m) 2m ago 5m 47.3M 3946M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (5m) 2m ago 5m 45.4M 4970M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (4m) 2m ago 4m 44.8M 4970M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (4m) 2m ago 4m 44.3M 4970M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:42:56.004 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (6m) 2m ago 8m 60.0M - 2.43.0 a07b618ecd1d 8382d6553d6c 2024-04-26T04:42:56.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9b6c06c6d0 msgr2=0x7f9b6c06eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9b6c06c6d0 0x7f9b6c06eb80 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9b780097b0 tx=0x7f9b78006d20 comp rx=0 tx=0).stop 2024-04-26T04:42:56.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410b100 msgr2=0x7f9b841a0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410b100 0x7f9b841a0570 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9b74003c30 tx=0x7f9b74003d10 comp rx=0 tx=0).stop 2024-04-26T04:42:56.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 shutdown_connections 2024-04-26T04:42:56.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7f9b6c06c6d0 0x7f9b6c06eb80 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b8410b100 0x7f9b841a0570 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 --2- 172.21.15.90:0/4184191696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b8410be80 0x7f9b841a0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.003+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 >> 172.21.15.90:0/4184191696 conn(0x7f9b8406a6d0 msgr2=0x7f9b8410f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:56.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.004+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 shutdown_connections 2024-04-26T04:42:56.006 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.004+0000 7f9b89dc2700 1 -- 172.21.15.90:0/4184191696 wait complete. 2024-04-26T04:42:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/962809491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c100340 msgr2=0x7fb69c100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/962809491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c100340 0x7fb69c100790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fb688009910 tx=0x7fb688009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/962809491 shutdown_connections 2024-04-26T04:42:56.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/962809491 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c100340 0x7fb69c100790 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/962809491 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb69c0ff140 0x7fb69c0ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/962809491 >> 172.21.15.90:0/962809491 conn(0x7fb69c0fa6d0 msgr2=0x7fb69c0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/962809491 shutdown_connections 2024-04-26T04:42:56.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/962809491 wait complete. 2024-04-26T04:42:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 Processor -- start 2024-04-26T04:42:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.121+0000 7fb6a2c1a700 1 -- start start 2024-04-26T04:42:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a2c1a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c0ff140 0x7fb69c069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a2c1a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb69c100340 0x7fb69c19cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a2c1a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c196680 con 0x7fb69c0ff140 2024-04-26T04:42:56.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a2c1a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb69c19d150 con 0x7fb69c100340 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a09b6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c0ff140 0x7fb69c069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a09b6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c0ff140 0x7fb69c069930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57592/0 (socket says 172.21.15.90:57592) 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a09b6700 1 -- 172.21.15.90:0/2894448041 learned_addr learned my addr 172.21.15.90:0/2894448041 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb69bfff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb69c100340 0x7fb69c19cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a09b6700 1 -- 172.21.15.90:0/2894448041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb69c100340 msgr2=0x7fb69c19cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a09b6700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb69c100340 0x7fb69c19cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a09b6700 1 -- 172.21.15.90:0/2894448041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6880095f0 con 0x7fb69c0ff140 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a09b6700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c0ff140 0x7fb69c069930 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb69000e900 tx=0x7fb69000ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb699ffb700 1 -- 172.21.15.90:0/2894448041 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb690009770 con 0x7fb69c0ff140 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb699ffb700 1 -- 172.21.15.90:0/2894448041 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb69000ce80 con 0x7fb69c0ff140 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.122+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb69c19d400 con 0x7fb69c0ff140 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.123+0000 7fb699ffb700 1 -- 172.21.15.90:0/2894448041 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb690003d20 con 0x7fb69c0ff140 2024-04-26T04:42:56.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.123+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb69c19d920 con 0x7fb69c0ff140 2024-04-26T04:42:56.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.124+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb69c107cb0 con 0x7fb69c0ff140 2024-04-26T04:42:56.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.124+0000 7fb699ffb700 1 -- 172.21.15.90:0/2894448041 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7fb69000c9d0 con 0x7fb69c0ff140 2024-04-26T04:42:56.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.125+0000 7fb699ffb700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb68c06c680 0x7fb68c06eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.125+0000 7fb699ffb700 1 -- 172.21.15.90:0/2894448041 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7fb690014070 con 0x7fb69c0ff140 2024-04-26T04:42:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.125+0000 7fb69bfff700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb68c06c680 0x7fb68c06eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:56.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.125+0000 7fb69bfff700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb68c06c680 0x7fb68c06eb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb688009fd0 tx=0x7fb688005940 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:56.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.130+0000 7fb699ffb700 1 -- 172.21.15.90:0/2894448041 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7fb690059dc0 con 0x7fb69c0ff140 2024-04-26T04:42:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: Upgrade: Updating mgr.smithi104.wwoewi 2024-04-26T04:42:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:42:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:42:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:42:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:42:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: Deploying daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:42:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: pgmap v270: 65 pgs: 65 active+clean; 314 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.4 MiB/s wr, 511 op/s 2024-04-26T04:42:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:55 smithi104 ceph-mon[32369]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: Upgrade: Updating mgr.smithi104.wwoewi 2024-04-26T04:42:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:42:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:42:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:42:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:42:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: Deploying daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:42:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: from='client.14620 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: pgmap v270: 65 pgs: 65 active+clean; 314 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 53 KiB/s rd, 2.4 MiB/s wr, 511 op/s 2024-04-26T04:42:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:55 smithi090 ceph-mon[26648]: from='client.14624 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:56.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.478+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb69c19dfc0 con 0x7fb69c0ff140 2024-04-26T04:42:56.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.479+0000 7fb699ffb700 1 -- 172.21.15.90:0/2894448041 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+558 (secure 0 0 0) 0x7fb690059950 con 0x7fb69c0ff140 2024-04-26T04:42:56.480 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:42:56.480 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:42:56.480 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:42:56.480 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:56.480 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:42:56.480 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 14 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:42:56.481 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:42:56.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb68c06c680 msgr2=0x7fb68c06eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb68c06c680 0x7fb68c06eb30 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb688009fd0 tx=0x7fb688005940 comp rx=0 tx=0).stop 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c0ff140 msgr2=0x7fb69c069930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c0ff140 0x7fb69c069930 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fb69000e900 tx=0x7fb69000ec10 comp rx=0 tx=0).stop 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 shutdown_connections 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7fb68c06c680 0x7fb68c06eb30 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb69c0ff140 0x7fb69c069930 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 --2- 172.21.15.90:0/2894448041 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb69c100340 0x7fb69c19cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.481+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 >> 172.21.15.90:0/2894448041 conn(0x7fb69c0fa6d0 msgr2=0x7fb69c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.482+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 shutdown_connections 2024-04-26T04:42:56.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.482+0000 7fb6a2c1a700 1 -- 172.21.15.90:0/2894448041 wait complete. 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 -- 172.21.15.90:0/185870530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1880ff430 msgr2=0x7ff1880ff8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/185870530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1880ff430 0x7ff1880ff8a0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7ff178009910 tx=0x7ff178009c20 comp rx=0 tx=0).stop 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 -- 172.21.15.90:0/185870530 shutdown_connections 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/185870530 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1880ff430 0x7ff1880ff8a0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/185870530 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff188100730 0x7ff188100b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 -- 172.21.15.90:0/185870530 >> 172.21.15.90:0/185870530 conn(0x7ff1880f9f10 msgr2=0x7ff1880fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 -- 172.21.15.90:0/185870530 shutdown_connections 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.620+0000 7ff18fbbb700 1 -- 172.21.15.90:0/185870530 wait complete. 2024-04-26T04:42:56.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18fbbb700 1 Processor -- start 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18fbbb700 1 -- start start 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18fbbb700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1880ff430 0x7ff188195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18fbbb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188100730 0x7ff188196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18fbbb700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff188196620 con 0x7ff188100730 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18fbbb700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff188196760 con 0x7ff1880ff430 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18d156700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188100730 0x7ff188196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18d156700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188100730 0x7ff188196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57608/0 (socket says 172.21.15.90:57608) 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18d156700 1 -- 172.21.15.90:0/4191042560 learned_addr learned my addr 172.21.15.90:0/4191042560 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.621+0000 7ff18d957700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1880ff430 0x7ff188195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff18d156700 1 -- 172.21.15.90:0/4191042560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1880ff430 msgr2=0x7ff188195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff18d156700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1880ff430 0x7ff188195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff18d156700 1 -- 172.21.15.90:0/4191042560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1780095f0 con 0x7ff188100730 2024-04-26T04:42:56.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff18d156700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188100730 0x7ff188196000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff17800b5c0 tx=0x7ff17800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff17effd700 1 -- 172.21.15.90:0/4191042560 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17801d070 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff18819b1b0 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff18819b6a0 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff17effd700 1 -- 172.21.15.90:0/4191042560 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff178022470 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.622+0000 7ff17effd700 1 -- 172.21.15.90:0/4191042560 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff17800f690 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.623+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff188107bf0 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.624+0000 7ff17effd700 1 -- 172.21.15.90:0/4191042560 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 19) v1 ==== 89914+0+0 (secure 0 0 0) 0x7ff178004b70 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.624+0000 7ff17effd700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff17406c680 0x7ff17406eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.624+0000 7ff17effd700 1 -- 172.21.15.90:0/4191042560 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5554+0+0 (secure 0 0 0) 0x7ff17808cec0 con 0x7ff188100730 2024-04-26T04:42:56.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.624+0000 7ff18d957700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff17406c680 0x7ff17406eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:42:56.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.625+0000 7ff18d957700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff17406c680 0x7ff17406eb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff184005fd0 tx=0x7ff18400a560 comp rx=0 tx=0).ready entity=mgr.14213 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:42:56.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.631+0000 7ff17effd700 1 -- 172.21.15.90:0/4191042560 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177933 (secure 0 0 0) 0x7ff17805b100 con 0x7ff188100730 2024-04-26T04:42:56.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.951+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 --> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1880fea30 con 0x7ff17406c680 2024-04-26T04:42:56.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.953+0000 7ff17effd700 1 -- 172.21.15.90:0/4191042560 <== mgr.14213 v2:172.21.15.90:6800/1235211382 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+341 (secure 0 0 0) 0x7ff1880fea30 con 0x7ff17406c680 2024-04-26T04:42:56.954 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:42:56.954 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:42:56.954 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:42:56.954 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:42:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-26T04:42:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "0/2 daemons upgraded", 2024-04-26T04:42:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mgr daemons", 2024-04-26T04:42:56.955 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:42:56.955 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:42:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff17406c680 msgr2=0x7ff17406eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff17406c680 0x7ff17406eb30 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7ff184005fd0 tx=0x7ff18400a560 comp rx=0 tx=0).stop 2024-04-26T04:42:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188100730 msgr2=0x7ff188196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:42:56.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188100730 0x7ff188196000 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7ff17800b5c0 tx=0x7ff17800be20 comp rx=0 tx=0).stop 2024-04-26T04:42:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 shutdown_connections 2024-04-26T04:42:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:6800/1235211382,v1:172.21.15.90:6801/1235211382] conn(0x7ff17406c680 0x7ff17406eb30 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1880ff430 0x7ff188195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 --2- 172.21.15.90:0/4191042560 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff188100730 0x7ff188196000 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:42:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 >> 172.21.15.90:0/4191042560 conn(0x7ff1880f9f10 msgr2=0x7ff1880fe320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:42:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 shutdown_connections 2024-04-26T04:42:56.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:42:56.957+0000 7ff18fbbb700 1 -- 172.21.15.90:0/4191042560 wait complete. 2024-04-26T04:42:57.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:56 smithi090 ceph-mon[26648]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:57.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:56 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2894448041' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:42:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:56 smithi104 ceph-mon[32369]: from='client.14626 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:56 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2894448041' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:42:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:58 smithi104 ceph-mon[32369]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:58 smithi104 ceph-mon[32369]: pgmap v271: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.2 MiB/s wr, 524 op/s 2024-04-26T04:42:58.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:57 smithi090 ceph-mon[26648]: from='client.14634 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:42:58.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:57 smithi090 ceph-mon[26648]: pgmap v271: 65 pgs: 65 active+clean; 309 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.2 MiB/s wr, 524 op/s 2024-04-26T04:42:59.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:42:58 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:42:59.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:42:58 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:43:00.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:00 smithi104 ceph-mon[32369]: pgmap v272: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 3.2 MiB/s wr, 616 op/s 2024-04-26T04:43:00.372 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:00 smithi090 ceph-mon[26648]: pgmap v272: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 3.2 MiB/s wr, 616 op/s 2024-04-26T04:43:02.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:02 smithi090 ceph-mon[26648]: pgmap v273: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.2 MiB/s wr, 547 op/s 2024-04-26T04:43:02.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:02 smithi104 ceph-mon[32369]: pgmap v273: 65 pgs: 65 active+clean; 290 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 56 KiB/s rd, 2.2 MiB/s wr, 547 op/s 2024-04-26T04:43:04.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:04 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:04.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:04 smithi104 ceph-mon[32369]: pgmap v274: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.8 MiB/s wr, 567 op/s 2024-04-26T04:43:04.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:04 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:04.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:04 smithi090 ceph-mon[26648]: pgmap v274: 65 pgs: 65 active+clean; 298 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.8 MiB/s wr, 567 op/s 2024-04-26T04:43:06.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:06 smithi090 ceph-mon[26648]: pgmap v275: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-04-26T04:43:06.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:06 smithi104 ceph-mon[32369]: pgmap v275: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54 KiB/s rd, 2.9 MiB/s wr, 497 op/s 2024-04-26T04:43:08.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:08 smithi090 ceph-mon[26648]: pgmap v276: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 3.1 MiB/s wr, 465 op/s 2024-04-26T04:43:08.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:08 smithi104 ceph-mon[32369]: pgmap v276: 65 pgs: 65 active+clean; 305 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 46 KiB/s rd, 3.1 MiB/s wr, 465 op/s 2024-04-26T04:43:10.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:10 smithi090 ceph-mon[26648]: pgmap v277: 65 pgs: 65 active+clean; 297 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 2.9 MiB/s wr, 479 op/s 2024-04-26T04:43:10.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:10 smithi104 ceph-mon[32369]: pgmap v277: 65 pgs: 65 active+clean; 297 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 2.9 MiB/s wr, 479 op/s 2024-04-26T04:43:12.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:12 smithi090 ceph-mon[26648]: pgmap v278: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.5 MiB/s wr, 391 op/s 2024-04-26T04:43:12.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:12 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:12.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:12 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:12.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:12 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:12.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:12 smithi104 ceph-mon[32369]: pgmap v278: 65 pgs: 65 active+clean; 300 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.5 MiB/s wr, 391 op/s 2024-04-26T04:43:12.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:12 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:12.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:12 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:12.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:12 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:14.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:14 smithi104 ceph-mon[32369]: pgmap v279: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 408 op/s 2024-04-26T04:43:14.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:14 smithi090 ceph-mon[26648]: pgmap v279: 65 pgs: 65 active+clean; 302 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 37 KiB/s rd, 3.1 MiB/s wr, 408 op/s 2024-04-26T04:43:15.889 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:15.889 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:15.889 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:15.889 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:15 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:15.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:15.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:15.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:15.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:15 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:16.917 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:16 smithi104 ceph-mon[32369]: pgmap v280: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.2 MiB/s wr, 382 op/s 2024-04-26T04:43:16.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:16 smithi090 ceph-mon[26648]: pgmap v280: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 44 KiB/s rd, 2.2 MiB/s wr, 382 op/s 2024-04-26T04:43:18.869 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:18 smithi104 ceph-mon[32369]: pgmap v281: 65 pgs: 65 active+clean; 299 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.9 MiB/s wr, 402 op/s 2024-04-26T04:43:18.870 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:18 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:18.870 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:18 smithi090 ceph-mon[26648]: pgmap v281: 65 pgs: 65 active+clean; 299 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 34 KiB/s rd, 1.9 MiB/s wr, 402 op/s 2024-04-26T04:43:18.870 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:18 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:19.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr fail", "who": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: osdmap e47: 6 total, 6 up, 6 in 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "mgr fail", "who": "smithi090.iwrvwa"}]': finished 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: mgrmap e20: smithi104.wwoewi(active, starting, since 0.00748465s) 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: Active manager daemon smithi104.wwoewi restarted 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: Activating manager daemon smithi104.wwoewi 2024-04-26T04:43:19.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: osdmap e48: 6 total, 6 up, 6 in 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: mgrmap e21: smithi104.wwoewi(active, starting, since 0.00615999s) 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:43:19.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: Manager daemon smithi104.wwoewi is now available 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:43:19.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:43:19.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/trash_purge_schedule"}]: dispatch 2024-04-26T04:43:19.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:19 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/trash_purge_schedule"}]: dispatch 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr fail", "who": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: osdmap e47: 6 total, 6 up, 6 in 2024-04-26T04:43:20.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.14213 172.21.15.90:0/1411236129' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "mgr fail", "who": "smithi090.iwrvwa"}]': finished 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: mgrmap e20: smithi104.wwoewi(active, starting, since 0.00748465s) 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: Active manager daemon smithi104.wwoewi restarted 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: Activating manager daemon smithi104.wwoewi 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: osdmap e48: 6 total, 6 up, 6 in 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: mgrmap e21: smithi104.wwoewi(active, starting, since 0.00615999s) 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:43:20.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:43:20.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:43:20.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: Manager daemon smithi104.wwoewi is now available 2024-04-26T04:43:20.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:20.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:20.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:43:20.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:43:20.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/trash_purge_schedule"}]: dispatch 2024-04-26T04:43:20.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:19 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi104.wwoewi/trash_purge_schedule"}]: dispatch 2024-04-26T04:43:21.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: [26/Apr/2024:04:43:19] ENGINE Bus STARTING 2024-04-26T04:43:21.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: Deploying cephadm binary to smithi104 2024-04-26T04:43:21.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: Deploying cephadm binary to smithi090 2024-04-26T04:43:21.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: [26/Apr/2024:04:43:19] ENGINE Serving on http://172.21.15.104:8765 2024-04-26T04:43:21.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: [26/Apr/2024:04:43:19] ENGINE Serving on https://172.21.15.104:7150 2024-04-26T04:43:21.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: [26/Apr/2024:04:43:19] ENGINE Bus STARTED 2024-04-26T04:43:21.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: mgrmap e22: smithi104.wwoewi(active, since 1.00831s) 2024-04-26T04:43:21.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:21 smithi090 ceph-mon[26648]: pgmap v3: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: [26/Apr/2024:04:43:19] ENGINE Bus STARTING 2024-04-26T04:43:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: Deploying cephadm binary to smithi104 2024-04-26T04:43:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: Deploying cephadm binary to smithi090 2024-04-26T04:43:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: [26/Apr/2024:04:43:19] ENGINE Serving on http://172.21.15.104:8765 2024-04-26T04:43:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: [26/Apr/2024:04:43:19] ENGINE Serving on https://172.21.15.104:7150 2024-04-26T04:43:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: [26/Apr/2024:04:43:19] ENGINE Bus STARTED 2024-04-26T04:43:21.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: mgrmap e22: smithi104.wwoewi(active, since 1.00831s) 2024-04-26T04:43:21.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:21 smithi104 ceph-mon[32369]: pgmap v3: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:22.458 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:22 smithi090 ceph-mon[26648]: pgmap v4: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:22.458 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:22 smithi090 ceph-mon[26648]: mgrmap e23: smithi104.wwoewi(active, since 2s) 2024-04-26T04:43:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:22 smithi104 ceph-mon[32369]: pgmap v4: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:22 smithi104 ceph-mon[32369]: mgrmap e23: smithi104.wwoewi(active, since 2s) 2024-04-26T04:43:24.450 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:24 smithi090 ceph-mon[26648]: pgmap v5: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:24.450 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:24 smithi090 ceph-mon[26648]: mgrmap e24: smithi104.wwoewi(active, since 4s) 2024-04-26T04:43:24.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:24 smithi104 ceph-mon[32369]: pgmap v5: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:24.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:24 smithi104 ceph-mon[32369]: mgrmap e24: smithi104.wwoewi(active, since 4s) 2024-04-26T04:43:26.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:26 smithi090 ceph-mon[26648]: pgmap v6: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:26.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:26 smithi104 ceph-mon[32369]: pgmap v6: 65 pgs: 65 active+clean; 301 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 -- 172.21.15.90:0/2628121845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac102490 msgr2=0x7f69ac1028a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/2628121845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac102490 0x7f69ac1028a0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009870 tx=0x7f69a0009b80 comp rx=0 tx=0).stop 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 -- 172.21.15.90:0/2628121845 shutdown_connections 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/2628121845 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69ac103640 0x7f69ac103a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/2628121845 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac102490 0x7f69ac1028a0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 -- 172.21.15.90:0/2628121845 >> 172.21.15.90:0/2628121845 conn(0x7f69ac0fda60 msgr2=0x7f69ac0ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 -- 172.21.15.90:0/2628121845 shutdown_connections 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.115+0000 7f69b2f5a700 1 -- 172.21.15.90:0/2628121845 wait complete. 2024-04-26T04:43:27.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.116+0000 7f69b2f5a700 1 Processor -- start 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.116+0000 7f69b2f5a700 1 -- start start 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.116+0000 7f69b2f5a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69ac102490 0x7f69ac197d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.116+0000 7f69b2f5a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac103640 0x7f69ac198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69b2f5a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac198870 con 0x7f69ac102490 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69b2f5a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69ac19d280 con 0x7f69ac103640 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69b0cf6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69ac102490 0x7f69ac197d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69b0cf6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69ac102490 0x7f69ac197d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54980/0 (socket says 172.21.15.90:54980) 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69b0cf6700 1 -- 172.21.15.90:0/412776878 learned_addr learned my addr 172.21.15.90:0/412776878 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:43:27.119 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69abfff700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac103640 0x7f69ac198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:27.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69abfff700 1 -- 172.21.15.90:0/412776878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69ac102490 msgr2=0x7f69ac197d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69abfff700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69ac102490 0x7f69ac197d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69abfff700 1 -- 172.21.15.90:0/412776878 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69a0009520 con 0x7f69ac103640 2024-04-26T04:43:27.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.117+0000 7f69abfff700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac103640 0x7f69ac198250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f699800e940 tx=0x7f699800ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.118+0000 7f69a9ffb700 1 -- 172.21.15.90:0/412776878 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f699800c990 con 0x7f69ac103640 2024-04-26T04:43:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.118+0000 7f69a9ffb700 1 -- 172.21.15.90:0/412776878 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6998010430 con 0x7f69ac103640 2024-04-26T04:43:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.118+0000 7f69a9ffb700 1 -- 172.21.15.90:0/412776878 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6998018630 con 0x7f69ac103640 2024-04-26T04:43:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.118+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69ac19d480 con 0x7f69ac103640 2024-04-26T04:43:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.118+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69ac19d950 con 0x7f69ac103640 2024-04-26T04:43:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.119+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69ac10af90 con 0x7f69ac103640 2024-04-26T04:43:27.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.119+0000 7f69a9ffb700 1 -- 172.21.15.90:0/412776878 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) v1 ==== 46428+0+0 (secure 0 0 0) 0x7f699800caf0 con 0x7f69ac103640 2024-04-26T04:43:27.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.120+0000 7f69a9ffb700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f699c039360 0x7f699c03b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:27.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.120+0000 7f69a9ffb700 1 -- 172.21.15.90:0/412776878 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6998014070 con 0x7f69ac103640 2024-04-26T04:43:27.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.120+0000 7f69b0cf6700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f699c039360 0x7f699c03b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:27.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.121+0000 7f69b0cf6700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f699c039360 0x7f699c03b810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009f50 tx=0x7f69a0019040 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:27.129 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.128+0000 7f69a9ffb700 1 -- 172.21.15.90:0/412776878 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f6998016330 con 0x7f69ac103640 2024-04-26T04:43:27.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.454+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 --> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f69ac107f90 con 0x7f699c039360 2024-04-26T04:43:27.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.456+0000 7f69a9ffb700 1 -- 172.21.15.90:0/412776878 <== mgr.24395 v2:172.21.15.104:6828/680795988 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f69ac107f90 con 0x7f699c039360 2024-04-26T04:43:27.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f699c039360 msgr2=0x7f699c03b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f699c039360 0x7f699c03b810 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009f50 tx=0x7f69a0019040 comp rx=0 tx=0).stop 2024-04-26T04:43:27.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac103640 msgr2=0x7f69ac198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac103640 0x7f69ac198250 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f699800e940 tx=0x7f699800ec50 comp rx=0 tx=0).stop 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 shutdown_connections 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f699c039360 0x7f699c03b810 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69ac102490 0x7f69ac197d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 --2- 172.21.15.90:0/412776878 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69ac103640 0x7f69ac198250 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 >> 172.21.15.90:0/412776878 conn(0x7f69ac0fda60 msgr2=0x7f69ac106870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 shutdown_connections 2024-04-26T04:43:27.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.458+0000 7f69b2f5a700 1 -- 172.21.15.90:0/412776878 wait complete. 2024-04-26T04:43:27.472 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:43:27.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 -- 172.21.15.90:0/781583846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad81036a0 msgr2=0x7fdad8105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 --2- 172.21.15.90:0/781583846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad81036a0 0x7fdad8105a80 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009910 tx=0x7fdacc009c20 comp rx=0 tx=0).stop 2024-04-26T04:43:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 -- 172.21.15.90:0/781583846 shutdown_connections 2024-04-26T04:43:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 --2- 172.21.15.90:0/781583846 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad81036a0 0x7fdad8105a80 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 --2- 172.21.15.90:0/781583846 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 0x7fdad8103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 -- 172.21.15.90:0/781583846 >> 172.21.15.90:0/781583846 conn(0x7fdad80fa7a0 msgr2=0x7fdad80fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 -- 172.21.15.90:0/781583846 shutdown_connections 2024-04-26T04:43:27.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.605+0000 7fdadda0d700 1 -- 172.21.15.90:0/781583846 wait complete. 2024-04-26T04:43:27.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.606+0000 7fdadda0d700 1 Processor -- start 2024-04-26T04:43:27.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdadda0d700 1 -- start start 2024-04-26T04:43:27.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdadda0d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 0x7fdad81006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:27.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdadda0d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad80fed30 0x7fdad80ff1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:27.608 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdadda0d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad80ff6e0 con 0x7fdad80fed30 2024-04-26T04:43:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdadda0d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdad80ff850 con 0x7fdad8100d80 2024-04-26T04:43:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdad6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 0x7fdad81006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdad67fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad80fed30 0x7fdad80ff1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdad6ffd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 0x7fdad81006e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.90:50580/0 (socket says 172.21.15.90:50580) 2024-04-26T04:43:27.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdad6ffd700 1 -- 172.21.15.90:0/3247955779 learned_addr learned my addr 172.21.15.90:0/3247955779 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:43:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdad6ffd700 1 -- 172.21.15.90:0/3247955779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad80fed30 msgr2=0x7fdad80ff1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdad6ffd700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad80fed30 0x7fdad80ff1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.607+0000 7fdad6ffd700 1 -- 172.21.15.90:0/3247955779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdacc0095f0 con 0x7fdad8100d80 2024-04-26T04:43:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.608+0000 7fdad6ffd700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 0x7fdad81006e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e8f0 tx=0x7fdac800ec00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.608+0000 7fdadca0b700 1 -- 172.21.15.90:0/3247955779 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac8009790 con 0x7fdad8100d80 2024-04-26T04:43:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.608+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdad80ffb30 con 0x7fdad8100d80 2024-04-26T04:43:27.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.608+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdad819e0e0 con 0x7fdad8100d80 2024-04-26T04:43:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.608+0000 7fdadca0b700 1 -- 172.21.15.90:0/3247955779 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fdac800c810 con 0x7fdad8100d80 2024-04-26T04:43:27.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.608+0000 7fdadca0b700 1 -- 172.21.15.90:0/3247955779 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdac80055b0 con 0x7fdad8100d80 2024-04-26T04:43:27.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.609+0000 7fdadca0b700 1 -- 172.21.15.90:0/3247955779 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 24) v1 ==== 46428+0+0 (secure 0 0 0) 0x7fdac8021440 con 0x7fdad8100d80 2024-04-26T04:43:27.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.609+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac4005320 con 0x7fdad8100d80 2024-04-26T04:43:27.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.610+0000 7fdadca0b700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7fdac0039360 0x7fdac003b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:27.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.610+0000 7fdadca0b700 1 -- 172.21.15.90:0/3247955779 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7fdac8014070 con 0x7fdad8100d80 2024-04-26T04:43:27.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.610+0000 7fdad67fc700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7fdac0039360 0x7fdac003b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:27.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.611+0000 7fdad67fc700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7fdac0039360 0x7fdac003b810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009fd0 tx=0x7fdacc000bc0 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:27.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.615+0000 7fdadca0b700 1 -- 172.21.15.90:0/3247955779 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fdac801fdb0 con 0x7fdad8100d80 2024-04-26T04:43:27.975 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.972+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 --> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdac4000bf0 con 0x7fdac0039360 2024-04-26T04:43:27.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.975+0000 7fdadca0b700 1 -- 172.21.15.90:0/3247955779 <== mgr.24395 v2:172.21.15.104:6828/680795988 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7fdac4000bf0 con 0x7fdac0039360 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7fdac0039360 msgr2=0x7fdac003b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7fdac0039360 0x7fdac003b810 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fdacc009fd0 tx=0x7fdacc000bc0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 msgr2=0x7fdad81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 0x7fdad81006e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fdac800e8f0 tx=0x7fdac800ec00 comp rx=0 tx=0).stop 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 shutdown_connections 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7fdac0039360 0x7fdac003b810 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdad8100d80 0x7fdad81006e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 --2- 172.21.15.90:0/3247955779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdad80fed30 0x7fdad80ff1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:27.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 >> 172.21.15.90:0/3247955779 conn(0x7fdad80fa7a0 msgr2=0x7fdad80fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:27.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 shutdown_connections 2024-04-26T04:43:27.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:27.979+0000 7fdadda0d700 1 -- 172.21.15.90:0/3247955779 wait complete. 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 -- 172.21.15.90:0/3056030268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234069230 msgr2=0x7f7234102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 --2- 172.21.15.90:0/3056030268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234069230 0x7f7234102e10 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f7224009910 tx=0x7f7224009c20 comp rx=0 tx=0).stop 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 -- 172.21.15.90:0/3056030268 shutdown_connections 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 --2- 172.21.15.90:0/3056030268 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7234103350 0x7f7234105790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 --2- 172.21.15.90:0/3056030268 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234069230 0x7f7234102e10 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 -- 172.21.15.90:0/3056030268 >> 172.21.15.90:0/3056030268 conn(0x7f72340fa780 msgr2=0x7f72340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 -- 172.21.15.90:0/3056030268 shutdown_connections 2024-04-26T04:43:28.108 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.106+0000 7f723a018700 1 -- 172.21.15.90:0/3056030268 wait complete. 2024-04-26T04:43:28.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.107+0000 7f723a018700 1 Processor -- start 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f723a018700 1 -- start start 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f723a018700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7234069230 0x7f7234195a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f723a018700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234103350 0x7f7234195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f723a018700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72341965e0 con 0x7f7234103350 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f723a018700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7234196720 con 0x7f7234069230 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f7232ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234103350 0x7f7234195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f72337fe700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7234069230 0x7f7234195a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f7232ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234103350 0x7f7234195fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32866/0 (socket says 172.21.15.90:32866) 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f7232ffd700 1 -- 172.21.15.90:0/478136968 learned_addr learned my addr 172.21.15.90:0/478136968 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f7232ffd700 1 -- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7234069230 msgr2=0x7f7234195a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f7232ffd700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7234069230 0x7f7234195a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f7232ffd700 1 -- 172.21.15.90:0/478136968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72240095f0 con 0x7f7234103350 2024-04-26T04:43:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.108+0000 7f72337fe700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7234069230 0x7f7234195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:43:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.109+0000 7f7232ffd700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234103350 0x7f7234195fc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f722800e920 tx=0x7f722800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:28.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.109+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f722800c9d0 con 0x7f7234103350 2024-04-26T04:43:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.109+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f7228010430 con 0x7f7234103350 2024-04-26T04:43:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.109+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7228018650 con 0x7f7234103350 2024-04-26T04:43:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.109+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72340fec50 con 0x7f7234103350 2024-04-26T04:43:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.110+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72340ff1a0 con 0x7f7234103350 2024-04-26T04:43:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.110+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 24) v1 ==== 46428+0+0 (secure 0 0 0) 0x7f72280188c0 con 0x7f7234103350 2024-04-26T04:43:28.112 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.111+0000 7f7230ff9700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f721c039360 0x7f721c03b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.111+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f7228014070 con 0x7f7234103350 2024-04-26T04:43:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.111+0000 7f72337fe700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f721c039360 0x7f721c03b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.111+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7234108cf0 con 0x7f7234103350 2024-04-26T04:43:28.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.111+0000 7f72337fe700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f721c039360 0x7f721c03b810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7224005b40 tx=0x7f7224005a90 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:28.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.116+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f7228018d70 con 0x7f7234103350 2024-04-26T04:43:28.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.153+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f722804bc00 con 0x7f7234103350 2024-04-26T04:43:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:28 smithi090 ceph-mon[26648]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64 KiB/s rd, 1.9 MiB/s wr, 346 op/s 2024-04-26T04:43:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:28 smithi090 ceph-mon[26648]: Standby manager daemon smithi090.iwrvwa started 2024-04-26T04:43:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:28 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/crt"}]: dispatch 2024-04-26T04:43:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:28 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:43:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:28 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/key"}]: dispatch 2024-04-26T04:43:28.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:28 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:43:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:28 smithi104 ceph-mon[32369]: pgmap v7: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 64 KiB/s rd, 1.9 MiB/s wr, 346 op/s 2024-04-26T04:43:28.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:28 smithi104 ceph-mon[32369]: Standby manager daemon smithi090.iwrvwa started 2024-04-26T04:43:28.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:28 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/crt"}]: dispatch 2024-04-26T04:43:28.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:28 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:43:28.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:28 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/key"}]: dispatch 2024-04-26T04:43:28.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:28 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/4230980327' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:43:28.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.447+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 --> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7234104010 con 0x7f721c039360 2024-04-26T04:43:28.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.459+0000 7f7230ff9700 1 -- 172.21.15.90:0/478136968 <== mgr.24395 v2:172.21.15.104:6828/680795988 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7234104010 con 0x7f721c039360 2024-04-26T04:43:28.460 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:43:28.460 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (7m) 3m ago 8m 25.9M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:43:28.460 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (9m) 3m ago 9m 7407k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:43:28.460 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (8m) 13s ago 8m 7620k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:43:28.460 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (9m) 3m ago 9m 6660k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (8m) 13s ago 8m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 3m ago 8m 83.8M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (3m) 3m ago 3m 15.6M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (3m) 3m ago 3m 11.2M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (3m) 13s ago 3m 13.8M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (3m) 13s ago 3m 143M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:9283,8765,8443 running (10m) 3m ago 10m 487M - 18.2.0 dc2bc1663786 a008eb0a2d7e 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (17s) 13s ago 8m 48.5M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 c89355d8b25c 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (10m) 3m ago 10m 49.1M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (7m) 13s ago 7m 35.7M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (9m) 3m ago 9m 14.3M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (8m) 13s ago 8m 15.9M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (7m) 3m ago 7m 48.1M 3946M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (6m) 3m ago 6m 47.6M 3946M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:43:28.461 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (6m) 3m ago 6m 47.3M 3946M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:43:28.462 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (5m) 13s ago 5m 255M 4970M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:43:28.462 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (5m) 13s ago 5m 217M 4970M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:43:28.462 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (5m) 13s ago 5m 227M 4970M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:43:28.462 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (7m) 3m ago 8m 60.0M - 2.43.0 a07b618ecd1d 8382d6553d6c 2024-04-26T04:43:28.462 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f721c039360 msgr2=0x7f721c03b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f721c039360 0x7f721c03b810 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f7224005b40 tx=0x7f7224005a90 comp rx=0 tx=0).stop 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234103350 msgr2=0x7f7234195fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234103350 0x7f7234195fc0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f722800e920 tx=0x7f722800ec30 comp rx=0 tx=0).stop 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 shutdown_connections 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f721c039360 0x7f721c03b810 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7234069230 0x7f7234195a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 --2- 172.21.15.90:0/478136968 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7234103350 0x7f7234195fc0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.461+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 >> 172.21.15.90:0/478136968 conn(0x7f72340fa780 msgr2=0x7f72340fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.462+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 shutdown_connections 2024-04-26T04:43:28.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.462+0000 7f723a018700 1 -- 172.21.15.90:0/478136968 wait complete. 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 -- 172.21.15.90:0/3324458400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07140ff130 msgr2=0x7f07140ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 --2- 172.21.15.90:0/3324458400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07140ff130 0x7f07140ff540 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0704009910 tx=0x7f0704009c20 comp rx=0 tx=0).stop 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 -- 172.21.15.90:0/3324458400 shutdown_connections 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 --2- 172.21.15.90:0/3324458400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07141003d0 0x7f0714100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 --2- 172.21.15.90:0/3324458400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07140ff130 0x7f07140ff540 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 -- 172.21.15.90:0/3324458400 >> 172.21.15.90:0/3324458400 conn(0x7f07140fa780 msgr2=0x7f07140fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 -- 172.21.15.90:0/3324458400 shutdown_connections 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.583+0000 7f071a95e700 1 -- 172.21.15.90:0/3324458400 wait complete. 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f071a95e700 1 Processor -- start 2024-04-26T04:43:28.585 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f071a95e700 1 -- start start 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f071a95e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07140ff130 0x7f07141949b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f071a95e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07141003d0 0x7f0714194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f071a95e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0714195510 con 0x7f07141003d0 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f071a95e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0714195650 con 0x7f07140ff130 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f07137fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07141003d0 0x7f0714194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f07137fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07141003d0 0x7f0714194ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32874/0 (socket says 172.21.15.90:32874) 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f07137fe700 1 -- 172.21.15.90:0/2578996660 learned_addr learned my addr 172.21.15.90:0/2578996660 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.584+0000 7f0713fff700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07140ff130 0x7f07141949b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f07137fe700 1 -- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07140ff130 msgr2=0x7f07141949b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f07137fe700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07140ff130 0x7f07141949b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:28.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f07137fe700 1 -- 172.21.15.90:0/2578996660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07040095f0 con 0x7f07141003d0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f07137fe700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07141003d0 0x7f0714194ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f070800e920 tx=0x7f070800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f07117fa700 1 -- 172.21.15.90:0/2578996660 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f070800c9d0 con 0x7f07141003d0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071419a100 con 0x7f07141003d0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f07117fa700 1 -- 172.21.15.90:0/2578996660 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0708010430 con 0x7f07141003d0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f07117fa700 1 -- 172.21.15.90:0/2578996660 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0708018650 con 0x7f07141003d0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.585+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0714068710 con 0x7f07141003d0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.586+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0714107c10 con 0x7f07141003d0 2024-04-26T04:43:28.588 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.587+0000 7f07117fa700 1 -- 172.21.15.90:0/2578996660 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f07080105a0 con 0x7f07141003d0 2024-04-26T04:43:28.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.588+0000 7f07117fa700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f06fc0717c0 0x7f06fc073c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:28.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.588+0000 7f07117fa700 1 -- 172.21.15.90:0/2578996660 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0708014070 con 0x7f07141003d0 2024-04-26T04:43:28.589 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.588+0000 7f0713fff700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f06fc0717c0 0x7f06fc073c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:28.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.589+0000 7f0713fff700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f06fc0717c0 0x7f06fc073c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0704009fd0 tx=0x7f0704005fd0 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:28.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:28.593+0000 7f07117fa700 1 -- 172.21.15.90:0/2578996660 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f0708053470 con 0x7f07141003d0 2024-04-26T04:43:29.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.045+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f0714107dd0 con 0x7f07141003d0 2024-04-26T04:43:29.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.045+0000 7f07117fa700 1 -- 172.21.15.90:0/2578996660 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+766 (secure 0 0 0) 0x7f070805a300 con 0x7f07141003d0 2024-04-26T04:43:29.047 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:43:29.047 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:43:29.047 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:43:29.047 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:43:29.048 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:43:29.049 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:43:29.049 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:43:29.049 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:43:29.049 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 13, 2024-04-26T04:43:29.049 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:43:29.049 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:43:29.049 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:43:29.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f06fc0717c0 msgr2=0x7f06fc073c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:29.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f06fc0717c0 0x7f06fc073c70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0704009fd0 tx=0x7f0704005fd0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07141003d0 msgr2=0x7f0714194ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:29.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07141003d0 0x7f0714194ef0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f070800e920 tx=0x7f070800ec30 comp rx=0 tx=0).stop 2024-04-26T04:43:29.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 shutdown_connections 2024-04-26T04:43:29.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f06fc0717c0 0x7f06fc073c70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f07140ff130 0x7f07141949b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 --2- 172.21.15.90:0/2578996660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f07141003d0 0x7f0714194ef0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 >> 172.21.15.90:0/2578996660 conn(0x7f07140fa780 msgr2=0x7f0714103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:29.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 shutdown_connections 2024-04-26T04:43:29.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.048+0000 7f071a95e700 1 -- 172.21.15.90:0/2578996660 wait complete. 2024-04-26T04:43:29.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.187+0000 7f04a0e73700 1 -- 172.21.15.90:0/4088853553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 msgr2=0x7f049c0731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:29.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.187+0000 7f04a0e73700 1 --2- 172.21.15.90:0/4088853553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c0731e0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f048c009940 tx=0x7f048c009c50 comp rx=0 tx=0).stop 2024-04-26T04:43:29.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.187+0000 7f04a0e73700 1 -- 172.21.15.90:0/4088853553 shutdown_connections 2024-04-26T04:43:29.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.187+0000 7f04a0e73700 1 --2- 172.21.15.90:0/4088853553 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 0x7f049c073c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.187+0000 7f04a0e73700 1 --2- 172.21.15.90:0/4088853553 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c0731e0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.188+0000 7f04a0e73700 1 -- 172.21.15.90:0/4088853553 >> 172.21.15.90:0/4088853553 conn(0x7f049c0fb7f0 msgr2=0x7f049c0fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:29.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.188+0000 7f04a0e73700 1 -- 172.21.15.90:0/4088853553 shutdown_connections 2024-04-26T04:43:29.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.188+0000 7f04a0e73700 1 -- 172.21.15.90:0/4088853553 wait complete. 2024-04-26T04:43:29.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f04a0e73700 1 Processor -- start 2024-04-26T04:43:29.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f04a0e73700 1 -- start start 2024-04-26T04:43:29.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f04a0e73700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 0x7f049c197c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f04a0e73700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f04a0e73700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c1987c0 con 0x7f049c074d80 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f04a0e73700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f049c198930 con 0x7f049c0737b0 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f049a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 0x7f049c197c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f0499d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f0499d9b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c1981c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:32892/0 (socket says 172.21.15.90:32892) 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f049a59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 0x7f049c197c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.90:50644/0 (socket says 172.21.15.90:50644) 2024-04-26T04:43:29.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.189+0000 7f0499d9b700 1 -- 172.21.15.90:0/548912940 learned_addr learned my addr 172.21.15.90:0/548912940 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:43:29.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f0499d9b700 1 -- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 msgr2=0x7f049c197c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:29.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f0499d9b700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 0x7f049c197c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f0499d9b700 1 -- 172.21.15.90:0/548912940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f048c0095f0 con 0x7f049c074d80 2024-04-26T04:43:29.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f049a59c700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 0x7f049c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:43:29.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f0499d9b700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c1981c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f049000e920 tx=0x7f049000ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:29.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f048b7fe700 1 -- 172.21.15.90:0/548912940 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f049000c9d0 con 0x7f049c074d80 2024-04-26T04:43:29.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f048b7fe700 1 -- 172.21.15.90:0/548912940 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f0490010430 con 0x7f049c074d80 2024-04-26T04:43:29.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f048b7fe700 1 -- 172.21.15.90:0/548912940 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0490018650 con 0x7f049c074d80 2024-04-26T04:43:29.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.190+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f049c19d3c0 con 0x7f049c074d80 2024-04-26T04:43:29.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.191+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f049c19d850 con 0x7f049c074d80 2024-04-26T04:43:29.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.192+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f049c10af80 con 0x7f049c074d80 2024-04-26T04:43:29.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.193+0000 7f048b7fe700 1 -- 172.21.15.90:0/548912940 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 25) v1 ==== 91089+0+0 (secure 0 0 0) 0x7f04900105a0 con 0x7f049c074d80 2024-04-26T04:43:29.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.193+0000 7f048b7fe700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f048406d3c0 0x7f048406f870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:29.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.193+0000 7f048b7fe700 1 -- 172.21.15.90:0/548912940 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f0490014070 con 0x7f049c074d80 2024-04-26T04:43:29.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.193+0000 7f049a59c700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f048406d3c0 0x7f048406f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:29.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.194+0000 7f049a59c700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f048406d3c0 0x7f048406f870 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f048c009fa0 tx=0x7f048c005ab0 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:29.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.201+0000 7f048b7fe700 1 -- 172.21.15.90:0/548912940 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f0490053470 con 0x7f049c074d80 2024-04-26T04:43:29.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:29 smithi090 ceph-mon[26648]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:29.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:29 smithi090 ceph-mon[26648]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:29.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:29 smithi090 ceph-mon[26648]: mgrmap e25: smithi104.wwoewi(active, since 9s), standbys: smithi090.iwrvwa 2024-04-26T04:43:29.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:29 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:43:29.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:29 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2578996660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:43:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:29 smithi104 ceph-mon[32369]: from='client.24427 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:29 smithi104 ceph-mon[32369]: from='client.24431 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:29 smithi104 ceph-mon[32369]: mgrmap e25: smithi104.wwoewi(active, since 9s), standbys: smithi090.iwrvwa 2024-04-26T04:43:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:29 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:43:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:29 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2578996660' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:43:29.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.566+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 --> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f049c107740 con 0x7f048406d3c0 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.568+0000 7f048b7fe700 1 -- 172.21.15.90:0/548912940 <== mgr.24395 v2:172.21.15.104:6828/680795988 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+310 (secure 0 0 0) 0x7f049c107740 con 0x7f048406d3c0 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "1/2 daemons upgraded", 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:43:29.569 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:43:29.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f048406d3c0 msgr2=0x7f048406f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f048406d3c0 0x7f048406f870 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f048c009fa0 tx=0x7f048c005ab0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 msgr2=0x7f049c1981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c1981c0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f049000e920 tx=0x7f049000ec30 comp rx=0 tx=0).stop 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 shutdown_connections 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f048406d3c0 0x7f048406f870 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f049c0737b0 0x7f049c197c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 --2- 172.21.15.90:0/548912940 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f049c074d80 0x7f049c1981c0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 >> 172.21.15.90:0/548912940 conn(0x7f049c0fb7f0 msgr2=0x7f049c106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 shutdown_connections 2024-04-26T04:43:29.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:29.570+0000 7f04a0e73700 1 -- 172.21.15.90:0/548912940 wait complete. 2024-04-26T04:43:30.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:30 smithi104 ceph-mon[32369]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:30.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:30 smithi104 ceph-mon[32369]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.5 MiB/s wr, 269 op/s 2024-04-26T04:43:30.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:30 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:30.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:30 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:30.551 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:30 smithi090 ceph-mon[26648]: from='client.14676 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:30.551 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:30 smithi090 ceph-mon[26648]: pgmap v8: 65 pgs: 65 active+clean; 307 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 1.5 MiB/s wr, 269 op/s 2024-04-26T04:43:30.551 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:30 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:30.551 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:30 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:31.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:31 smithi104 ceph-mon[32369]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:31.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:31 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:31.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:31 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:31.563 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:31 smithi090 ceph-mon[26648]: from='client.14684 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:43:31.564 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:31 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:31.564 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:31 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:32.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:32 smithi104 ceph-mon[32369]: pgmap v9: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63 KiB/s rd, 2.3 MiB/s wr, 467 op/s 2024-04-26T04:43:32.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:32 smithi090 ceph-mon[26648]: pgmap v9: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 63 KiB/s rd, 2.3 MiB/s wr, 467 op/s 2024-04-26T04:43:34.393 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.1... 2024-04-26T04:43:34.393 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.1 /home/ubuntu/cephtest/clone.client.1 2024-04-26T04:43:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:34 smithi104 ceph-mon[32369]: pgmap v10: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 2.1 MiB/s wr, 428 op/s 2024-04-26T04:43:34.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:34 smithi090 ceph-mon[26648]: pgmap v10: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 2.1 MiB/s wr, 428 op/s 2024-04-26T04:43:34.751 DEBUG:teuthology.parallel:result is None 2024-04-26T04:43:35.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:35 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:35.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:35 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:35.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:35 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:35.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:35 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:35.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:35 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:35.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:35 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: pgmap v11: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 2.1 MiB/s wr, 428 op/s 2024-04-26T04:43:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:36 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: pgmap v11: 65 pgs: 65 active+clean; 303 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 57 KiB/s rd, 2.1 MiB/s wr, 428 op/s 2024-04-26T04:43:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:36.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:36.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:36.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:36 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi090 to 1216M 2024-04-26T04:43:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:37.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:43:37.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:37 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:43:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi090 to 1216M 2024-04-26T04:43:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-26T04:43:37.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:37.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:43:37.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:37 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:43:38.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:38 smithi090 ceph-mon[26648]: Adjusting osd_memory_target on smithi104 to 2240M 2024-04-26T04:43:38.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:38 smithi090 ceph-mon[26648]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:43:38.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:38 smithi090 ceph-mon[26648]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:43:38.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:38 smithi090 ceph-mon[26648]: pgmap v12: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 615 op/s 2024-04-26T04:43:38.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:38 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:38.353 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:38 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:38.354 INFO:tasks.workunit:Stopping ['suites/fsstress.sh'] on client.0... 2024-04-26T04:43:38.355 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-26T04:43:38.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:38 smithi104 ceph-mon[32369]: Adjusting osd_memory_target on smithi104 to 2240M 2024-04-26T04:43:38.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:38 smithi104 ceph-mon[32369]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:43:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:38 smithi104 ceph-mon[32369]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:43:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:38 smithi104 ceph-mon[32369]: pgmap v12: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 77 KiB/s rd, 2.8 MiB/s wr, 615 op/s 2024-04-26T04:43:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:38 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:38 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:38.684 DEBUG:teuthology.parallel:result is None 2024-04-26T04:43:38.685 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T04:43:38.722 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-26T04:43:38.723 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.1/client.1 2024-04-26T04:43:38.752 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.1/client.1 2024-04-26T04:43:38.752 DEBUG:teuthology.parallel:result is None 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:39 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.636 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.637 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:39.637 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:39 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:40.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:40 smithi104 ceph-mon[32369]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-26T04:43:40.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:40 smithi104 ceph-mon[32369]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-26T04:43:40.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:40 smithi104 ceph-mon[32369]: pgmap v13: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 413 op/s 2024-04-26T04:43:40.601 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:40 smithi090 ceph-mon[26648]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-26T04:43:40.601 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:40 smithi090 ceph-mon[26648]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-26T04:43:40.601 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:40 smithi090 ceph-mon[26648]: pgmap v13: 65 pgs: 65 active+clean; 298 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 40 KiB/s rd, 1.7 MiB/s wr, 413 op/s 2024-04-26T04:43:42.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:42 smithi090 ceph-mon[26648]: pgmap v14: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.0 MiB/s wr, 514 op/s 2024-04-26T04:43:42.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:42 smithi104 ceph-mon[32369]: pgmap v14: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 50 KiB/s rd, 2.0 MiB/s wr, 514 op/s 2024-04-26T04:43:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:44 smithi104 ceph-mon[32369]: pgmap v15: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-04-26T04:43:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:44 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:44 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:44.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:44 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:43:44.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:44 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:43:44.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:44 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:44.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:44 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:44 smithi090 ceph-mon[26648]: pgmap v15: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-04-26T04:43:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:44 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:44 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:44 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:43:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:44 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:43:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:44 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:44.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:44 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:45.399 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:45 smithi090 ceph-mon[26648]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:43:45.400 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:45 smithi090 ceph-mon[26648]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:43:45.400 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:45 smithi090 ceph-mon[26648]: Upgrade: Need to upgrade myself (mgr.smithi104.wwoewi) 2024-04-26T04:43:45.400 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:45 smithi090 ceph-mon[26648]: Upgrade: Need to upgrade myself (mgr.smithi104.wwoewi) 2024-04-26T04:43:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:45 smithi104 ceph-mon[32369]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:43:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:45 smithi104 ceph-mon[32369]: from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090.front.sepia.ceph.com:9095"}]: dispatch 2024-04-26T04:43:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:45 smithi104 ceph-mon[32369]: Upgrade: Need to upgrade myself (mgr.smithi104.wwoewi) 2024-04-26T04:43:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:45 smithi104 ceph-mon[32369]: Upgrade: Need to upgrade myself (mgr.smithi104.wwoewi) 2024-04-26T04:43:46.587 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:46 smithi090 ceph-mon[26648]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-04-26T04:43:46.587 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:46 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:46.587 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:46 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:43:46.588 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:46 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:43:46.588 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:46 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:43:46.588 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:46 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:43:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:46 smithi104 ceph-mon[32369]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 287 op/s 2024-04-26T04:43:46.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:46 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:46.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:46 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:43:46.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:46 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:43:46.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:46 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:43:46.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:46 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:43:47.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:47 smithi104 ceph-mon[32369]: Upgrade: Updating mgr.smithi090.iwrvwa 2024-04-26T04:43:47.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:47 smithi104 ceph-mon[32369]: Deploying daemon mgr.smithi090.iwrvwa on smithi090 2024-04-26T04:43:47.523 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:47 smithi090 ceph-mon[26648]: Upgrade: Updating mgr.smithi090.iwrvwa 2024-04-26T04:43:47.523 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:47 smithi090 ceph-mon[26648]: Deploying daemon mgr.smithi090.iwrvwa on smithi090 2024-04-26T04:43:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:48 smithi090 ceph-mon[26648]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 288 op/s 2024-04-26T04:43:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:48 smithi104 ceph-mon[32369]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 1.1 MiB/s wr, 288 op/s 2024-04-26T04:43:50.450 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:50 smithi090 ceph-mon[26648]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 338 KiB/s wr, 101 op/s 2024-04-26T04:43:50.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:50 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:50.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:50 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:50.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:50 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:50.451 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:50 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:50 smithi104 ceph-mon[32369]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 338 KiB/s wr, 101 op/s 2024-04-26T04:43:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:50 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:43:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:50 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:50 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:50 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:43:51.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:51 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:51.816 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:51 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:51.817 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:51 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:51.817 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:51 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:52.466 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:52 smithi090 ceph-mon[26648]: pgmap v19: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 338 KiB/s wr, 102 op/s 2024-04-26T04:43:52.699 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:52 smithi104 ceph-mon[32369]: pgmap v19: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 11 KiB/s rd, 338 KiB/s wr, 102 op/s 2024-04-26T04:43:54.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:54 smithi090 ceph-mon[26648]: pgmap v20: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 8.4 KiB/s wr, 1 op/s 2024-04-26T04:43:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:54 smithi104 ceph-mon[32369]: pgmap v20: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 8.4 KiB/s wr, 1 op/s 2024-04-26T04:43:55.556 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:55 smithi090 ceph-mon[26648]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 8.4 KiB/s wr, 1 op/s 2024-04-26T04:43:55.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:55 smithi104 ceph-mon[32369]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 8.4 KiB/s wr, 1 op/s 2024-04-26T04:43:56.869 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:56 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:56.869 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:56 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:56 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:57.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:56 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:57.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: Standby manager daemon smithi090.iwrvwa restarted 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: Standby manager daemon smithi090.iwrvwa started 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/crt"}]: dispatch 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/key"}]: dispatch 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:43:57.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:57 smithi090 ceph-mon[26648]: pgmap v22: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 8.4 KiB/s wr, 1 op/s 2024-04-26T04:43:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:43:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: Standby manager daemon smithi090.iwrvwa restarted 2024-04-26T04:43:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: Standby manager daemon smithi090.iwrvwa started 2024-04-26T04:43:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/crt"}]: dispatch 2024-04-26T04:43:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:43:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi090.iwrvwa/key"}]: dispatch 2024-04-26T04:43:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:43:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:57 smithi104 ceph-mon[32369]: pgmap v22: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s rd, 8.4 KiB/s wr, 1 op/s 2024-04-26T04:43:58.838 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:58 smithi090 ceph-mon[26648]: mgrmap e26: smithi104.wwoewi(active, since 38s), standbys: smithi090.iwrvwa 2024-04-26T04:43:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:58 smithi104 ceph-mon[32369]: mgrmap e26: smithi104.wwoewi(active, since 38s), standbys: smithi090.iwrvwa 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.681+0000 7f6bf8574700 1 -- 172.21.15.90:0/299985743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 msgr2=0x7f6bf0100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.681+0000 7f6bf8574700 1 --2- 172.21.15.90:0/299985743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0100790 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f6bec009910 tx=0x7f6bec009c20 comp rx=0 tx=0).stop 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.681+0000 7f6bf8574700 1 -- 172.21.15.90:0/299985743 shutdown_connections 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.681+0000 7f6bf8574700 1 --2- 172.21.15.90:0/299985743 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0100790 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.681+0000 7f6bf8574700 1 --2- 172.21.15.90:0/299985743 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bf00ff140 0x7f6bf00ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.681+0000 7f6bf8574700 1 -- 172.21.15.90:0/299985743 >> 172.21.15.90:0/299985743 conn(0x7f6bf00fa6d0 msgr2=0x7f6bf00fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.681+0000 7f6bf8574700 1 -- 172.21.15.90:0/299985743 shutdown_connections 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.682+0000 7f6bf8574700 1 -- 172.21.15.90:0/299985743 wait complete. 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.682+0000 7f6bf8574700 1 Processor -- start 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.682+0000 7f6bf8574700 1 -- start start 2024-04-26T04:43:59.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.682+0000 7f6bf8574700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bf00ff140 0x7f6bf0194a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.682+0000 7f6bf8574700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0194f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.682+0000 7f6bf8574700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf0195590 con 0x7f6bf0100340 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.682+0000 7f6bf8574700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bf01956d0 con 0x7f6bf00ff140 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf5b0f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0194f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf5b0f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0194f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58958/0 (socket says 172.21.15.90:58958) 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf5b0f700 1 -- 172.21.15.90:0/2610061995 learned_addr learned my addr 172.21.15.90:0/2610061995 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf6310700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bf00ff140 0x7f6bf0194a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf5b0f700 1 -- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bf00ff140 msgr2=0x7f6bf0194a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf5b0f700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bf00ff140 0x7f6bf0194a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf5b0f700 1 -- 172.21.15.90:0/2610061995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0095f0 con 0x7f6bf0100340 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf5b0f700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0194f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006010 tx=0x7f6bec00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6be77fe700 1 -- 172.21.15.90:0/2610061995 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bec01d070 con 0x7f6bf0100340 2024-04-26T04:43:59.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6be77fe700 1 -- 172.21.15.90:0/2610061995 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6bec00f460 con 0x7f6bf0100340 2024-04-26T04:43:59.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6be77fe700 1 -- 172.21.15.90:0/2610061995 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6bec021640 con 0x7f6bf0100340 2024-04-26T04:43:59.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bf019a120 con 0x7f6bf0100340 2024-04-26T04:43:59.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.683+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bf019a5c0 con 0x7f6bf0100340 2024-04-26T04:43:59.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.684+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bf0107d30 con 0x7f6bf0100340 2024-04-26T04:43:59.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.685+0000 7f6be77fe700 1 -- 172.21.15.90:0/2610061995 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 92160+0+0 (secure 0 0 0) 0x7f6bec00fab0 con 0x7f6bf0100340 2024-04-26T04:43:59.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.685+0000 7f6be77fe700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f6bdc06dfc0 0x7f6bdc070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:43:59.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.685+0000 7f6be77fe700 1 -- 172.21.15.90:0/2610061995 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7f6bec08f150 con 0x7f6bf0100340 2024-04-26T04:43:59.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.685+0000 7f6bf6310700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f6bdc06dfc0 0x7f6bdc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:43:59.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.686+0000 7f6bf6310700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f6bdc06dfc0 0x7f6bdc070470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0008040 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:43:59.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.690+0000 7f6be77fe700 1 -- 172.21.15.90:0/2610061995 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f6bec05c440 con 0x7f6bf0100340 2024-04-26T04:43:59.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.916+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 --> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6bf019ac00 con 0x7f6bdc06dfc0 2024-04-26T04:43:59.918 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:43:59 smithi090 ceph-mon[26648]: pgmap v23: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:43:59.919 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.918+0000 7f6be77fe700 1 -- 172.21.15.90:0/2610061995 <== mgr.24395 v2:172.21.15.104:6828/680795988 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f6bf019ac00 con 0x7f6bdc06dfc0 2024-04-26T04:43:59.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f6bdc06dfc0 msgr2=0x7f6bdc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:59.922 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f6bdc06dfc0 0x7f6bdc070470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6be0006fd0 tx=0x7f6be0008040 comp rx=0 tx=0).stop 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 msgr2=0x7f6bf0194f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0194f70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f6bec006010 tx=0x7f6bec00be20 comp rx=0 tx=0).stop 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 shutdown_connections 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7f6bdc06dfc0 0x7f6bdc070470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6bf00ff140 0x7f6bf0194a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 --2- 172.21.15.90:0/2610061995 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6bf0100340 0x7f6bf0194f70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 >> 172.21.15.90:0/2610061995 conn(0x7f6bf00fa6d0 msgr2=0x7f6bf0103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 shutdown_connections 2024-04-26T04:43:59.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:43:59.921+0000 7f6bf8574700 1 -- 172.21.15.90:0/2610061995 wait complete. 2024-04-26T04:43:59.934 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:44:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:43:59 smithi104 ceph-mon[32369]: pgmap v23: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 426 B/s wr, 0 op/s 2024-04-26T04:44:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 -- 172.21.15.90:0/486578195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 msgr2=0x7ff7d0105790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/486578195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0105790 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8009910 tx=0x7ff7b8009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 -- 172.21.15.90:0/486578195 shutdown_connections 2024-04-26T04:44:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/486578195 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0105790 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/486578195 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7d0069270 0x7ff7d0102dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:00.036 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 -- 172.21.15.90:0/486578195 >> 172.21.15.90:0/486578195 conn(0x7ff7d00fa780 msgr2=0x7ff7d00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 -- 172.21.15.90:0/486578195 shutdown_connections 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.033+0000 7ff7d53d3700 1 -- 172.21.15.90:0/486578195 wait complete. 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7d53d3700 1 Processor -- start 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7d53d3700 1 -- start start 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7d53d3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0195ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7d53d3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7d0196010 0x7ff7d019b080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7d53d3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d0196510 con 0x7ff7d0103310 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7d53d3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d0196680 con 0x7ff7d0196010 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ceffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0195ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ceffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0195ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58984/0 (socket says 172.21.15.90:58984) 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ceffd700 1 -- 172.21.15.90:0/2207669234 learned_addr learned my addr 172.21.15.90:0/2207669234 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:00.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ce7fc700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7d0196010 0x7ff7d019b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ceffd700 1 -- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7d0196010 msgr2=0x7ff7d019b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ceffd700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7d0196010 0x7ff7d019b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ceffd700 1 -- 172.21.15.90:0/2207669234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b80095f0 con 0x7ff7d0103310 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.034+0000 7ff7ce7fc700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7d0196010 0x7ff7d019b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.035+0000 7ff7ceffd700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0195ad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007e90 tx=0x7ff7c800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.035+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c800dda0 con 0x7ff7d0103310 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.035+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d019b620 con 0x7ff7d0103310 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.035+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff7c800f040 con 0x7ff7d0103310 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.035+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff7c8012610 con 0x7ff7d0103310 2024-04-26T04:44:00.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.035+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d019bb40 con 0x7ff7d0103310 2024-04-26T04:44:00.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.036+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7d018fbf0 con 0x7ff7d0103310 2024-04-26T04:44:00.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.036+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 26) v1 ==== 92160+0+0 (secure 0 0 0) 0x7ff7c80128d0 con 0x7ff7d0103310 2024-04-26T04:44:00.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.037+0000 7ff7c7fff700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7ff7bc06dfc0 0x7ff7bc070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.037+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5812+0+0 (secure 0 0 0) 0x7ff7c808d660 con 0x7ff7d0103310 2024-04-26T04:44:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.037+0000 7ff7ce7fc700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7ff7bc06dfc0 0x7ff7bc070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:00.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.037+0000 7ff7ce7fc700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7ff7bc06dfc0 0x7ff7bc070470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005b40 tx=0x7ff7b8005a90 comp rx=0 tx=0).ready entity=mgr.24395 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:00.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.041+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7ff7c805a8d0 con 0x7ff7d0103310 2024-04-26T04:44:00.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.138+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mgrmap(e 27) v1 ==== 45949+0+0 (secure 0 0 0) 0x7ff7c808bb10 con 0x7ff7d0103310 2024-04-26T04:44:00.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.138+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7ff7bc06dfc0 msgr2=0x7ff7bc070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:00.140 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:00.138+0000 7ff7c7fff700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7ff7bc06dfc0 0x7ff7bc070470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005b40 tx=0x7ff7b8005a90 comp rx=0 tx=0).stop 2024-04-26T04:44:01.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.140+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mon.0 v2:172.21.15.90:3300/0 8 ==== mgrmap(e 28) v1 ==== 46370+0+0 (secure 0 0 0) 0x7ff7c8013a10 con 0x7ff7d0103310 2024-04-26T04:44:01.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.140+0000 7ff7c7fff700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff7bc005eb0 0x7ff7bc035df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:01.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.141+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff7bc0328b0 con 0x7ff7bc005eb0 2024-04-26T04:44:01.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.150+0000 7ff7ce7fc700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff7bc005eb0 0x7ff7bc035df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:01.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.152+0000 7ff7ce7fc700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff7bc005eb0 0x7ff7bc035df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005b40 tx=0x7ff7b8022000 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:01.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.153+0000 7ff7c7fff700 1 -- 172.21.15.90:0/2207669234 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7ff7bc0328b0 con 0x7ff7bc005eb0 2024-04-26T04:44:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff7bc005eb0 msgr2=0x7ff7bc035df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff7bc005eb0 0x7ff7bc035df0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005b40 tx=0x7ff7b8022000 comp rx=0 tx=0).stop 2024-04-26T04:44:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 msgr2=0x7ff7d0195ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0195ad0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8007e90 tx=0x7ff7c800d3b0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 shutdown_connections 2024-04-26T04:44:01.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff7bc005eb0 0x7ff7bc035df0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:6828/680795988,v1:172.21.15.104:6829/680795988] conn(0x7ff7bc06dfc0 0x7ff7bc070470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff7d0103310 0x7ff7d0195ad0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 --2- 172.21.15.90:0/2207669234 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff7d0196010 0x7ff7d019b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 >> 172.21.15.90:0/2207669234 conn(0x7ff7d00fa780 msgr2=0x7ff7d00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 shutdown_connections 2024-04-26T04:44:01.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.156+0000 7ff7d53d3700 1 -- 172.21.15.90:0/2207669234 wait complete. 2024-04-26T04:44:01.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr fail", "who": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr fail", "who": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd='[{"prefix": "mgr fail", "who": "smithi104.wwoewi"}]': finished 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: mgrmap e27: smithi090.iwrvwa(active, starting, since 0.00612955s) 2024-04-26T04:44:01.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:44:01.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: [26/Apr/2024:04:44:00] ENGINE Bus STARTING 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: [26/Apr/2024:04:44:00] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: [26/Apr/2024:04:44:00] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T04:44:01.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:00 smithi090 ceph-mon[26648]: [26/Apr/2024:04:44:00] ENGINE Bus STARTED 2024-04-26T04:44:01.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 172.21.15.104:0/3722348764' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr fail", "who": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "mgr fail", "who": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.24395 ' entity='mgr.smithi104.wwoewi' cmd='[{"prefix": "mgr fail", "who": "smithi104.wwoewi"}]': finished 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: mgrmap e27: smithi090.iwrvwa(active, starting, since 0.00612955s) 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:44:01.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:01.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:01.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:44:01.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:44:01.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: [26/Apr/2024:04:44:00] ENGINE Bus STARTING 2024-04-26T04:44:01.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: [26/Apr/2024:04:44:00] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T04:44:01.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: [26/Apr/2024:04:44:00] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T04:44:01.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:00 smithi104 ceph-mon[32369]: [26/Apr/2024:04:44:00] ENGINE Bus STARTED 2024-04-26T04:44:01.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 -- 172.21.15.90:0/1997931714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 msgr2=0x7fad0c102e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 --2- 172.21.15.90:0/1997931714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c102e30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7facfc009910 tx=0x7facfc009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:01.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 -- 172.21.15.90:0/1997931714 shutdown_connections 2024-04-26T04:44:01.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 --2- 172.21.15.90:0/1997931714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad0c103370 0x7fad0c1057b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 --2- 172.21.15.90:0/1997931714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c102e30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 -- 172.21.15.90:0/1997931714 >> 172.21.15.90:0/1997931714 conn(0x7fad0c0fa7a0 msgr2=0x7fad0c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:01.269 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 -- 172.21.15.90:0/1997931714 shutdown_connections 2024-04-26T04:44:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.262+0000 7fad11d26700 1 -- 172.21.15.90:0/1997931714 wait complete. 2024-04-26T04:44:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad11d26700 1 Processor -- start 2024-04-26T04:44:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad11d26700 1 -- start start 2024-04-26T04:44:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad11d26700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:01.270 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad11d26700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad0c103370 0x7fad0c196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad11d26700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c196970 con 0x7fad0c069ac0 2024-04-26T04:44:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad11d26700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad0c196ab0 con 0x7fad0c103370 2024-04-26T04:44:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad0b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad0b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c195e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59112/0 (socket says 172.21.15.90:59112) 2024-04-26T04:44:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad0b7fe700 1 -- 172.21.15.90:0/2606677959 learned_addr learned my addr 172.21.15.90:0/2606677959 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:01.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.263+0000 7fad0affd700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad0c103370 0x7fad0c196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad0b7fe700 1 -- 172.21.15.90:0/2606677959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad0c103370 msgr2=0x7fad0c196350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad0b7fe700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad0c103370 0x7fad0c196350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad0b7fe700 1 -- 172.21.15.90:0/2606677959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facfc0095f0 con 0x7fad0c069ac0 2024-04-26T04:44:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad0b7fe700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c195e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7facfc005850 tx=0x7facfc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad08ff9700 1 -- 172.21.15.90:0/2606677959 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facfc01d070 con 0x7fad0c069ac0 2024-04-26T04:44:01.272 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad08ff9700 1 -- 172.21.15.90:0/2606677959 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7facfc00f460 con 0x7fad0c069ac0 2024-04-26T04:44:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad0c19b500 con 0x7fad0c069ac0 2024-04-26T04:44:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad08ff9700 1 -- 172.21.15.90:0/2606677959 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7facfc021640 con 0x7fad0c069ac0 2024-04-26T04:44:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.264+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad0c19b9f0 con 0x7fad0c069ac0 2024-04-26T04:44:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.265+0000 7fad08ff9700 1 -- 172.21.15.90:0/2606677959 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 46370+0+0 (secure 0 0 0) 0x7facfc02b440 con 0x7fad0c069ac0 2024-04-26T04:44:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.265+0000 7fad08ff9700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7facf40392e0 0x7facf403b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:01.273 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.265+0000 7fad08ff9700 1 -- 172.21.15.90:0/2606677959 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7facfc04e750 con 0x7fad0c069ac0 2024-04-26T04:44:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.265+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad0c18fbf0 con 0x7fad0c069ac0 2024-04-26T04:44:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.265+0000 7fad0affd700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7facf40392e0 0x7facf403b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:01.274 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.266+0000 7fad0affd700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7facf40392e0 0x7facf403b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad00007be0 tx=0x7fad000073d0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:01.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.270+0000 7fad08ff9700 1 -- 172.21.15.90:0/2606677959 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7facfc026020 con 0x7fad0c069ac0 2024-04-26T04:44:01.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.489+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fad0c02cee0 con 0x7facf40392e0 2024-04-26T04:44:01.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.499+0000 7fad08ff9700 1 -- 172.21.15.90:0/2606677959 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fad0c02cee0 con 0x7facf40392e0 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (8m) 5s ago 9m 25.8M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (9m) 5s ago 9m 7763k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (8m) 32s ago 8m 7642k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (9m) 5s ago 9m 6660k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (8m) 32s ago 8m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 5s ago 9m 83.6M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (4m) 5s ago 4m 130M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (4m) 5s ago 4m 13.7M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (4m) 32s ago 4m 14.2M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:44:01.501 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (4m) 32s ago 4m 155M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (12s) 5s ago 10m 98.3M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (50s) 32s ago 8m 488M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 c89355d8b25c 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (10m) 5s ago 11m 59.0M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (8m) 32s ago 8m 37.9M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (9m) 5s ago 9m 14.7M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (8m) 32s ago 8m 15.6M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (7m) 5s ago 7m 228M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (7m) 5s ago 7m 244M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (6m) 5s ago 6m 199M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (6m) 32s ago 6m 262M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (6m) 32s ago 6m 230M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (5m) 32s ago 5m 235M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:44:01.502 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (17s) 5s ago 9m 40.2M - 2.43.0 a07b618ecd1d c3726eced7eb 2024-04-26T04:44:01.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7facf40392e0 msgr2=0x7facf403b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7facf40392e0 0x7facf403b790 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fad00007be0 tx=0x7fad000073d0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 msgr2=0x7fad0c195e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c195e10 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7facfc005850 tx=0x7facfc00bd70 comp rx=0 tx=0).stop 2024-04-26T04:44:01.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 shutdown_connections 2024-04-26T04:44:01.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7facf40392e0 0x7facf403b790 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fad0c069ac0 0x7fad0c195e10 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 --2- 172.21.15.90:0/2606677959 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fad0c103370 0x7fad0c196350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 >> 172.21.15.90:0/2606677959 conn(0x7fad0c0fa7a0 msgr2=0x7fad0c0fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:01.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 shutdown_connections 2024-04-26T04:44:01.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.502+0000 7fad11d26700 1 -- 172.21.15.90:0/2606677959 wait complete. 2024-04-26T04:44:01.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 -- 172.21.15.90:0/3976597725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe981003d0 msgr2=0x7fbe98100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/3976597725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe981003d0 0x7fbe98100840 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fbe88009910 tx=0x7fbe88009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 -- 172.21.15.90:0/3976597725 shutdown_connections 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/3976597725 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe981003d0 0x7fbe98100840 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/3976597725 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe980ff130 0x7fbe980ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 -- 172.21.15.90:0/3976597725 >> 172.21.15.90:0/3976597725 conn(0x7fbe980fa780 msgr2=0x7fbe980fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 -- 172.21.15.90:0/3976597725 shutdown_connections 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.608+0000 7fbe9fe30700 1 -- 172.21.15.90:0/3976597725 wait complete. 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9fe30700 1 Processor -- start 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9fe30700 1 -- start start 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9fe30700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe980ff130 0x7fbe98069100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9fe30700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe981003d0 0x7fbe9819a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9fe30700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9819ad80 con 0x7fbe980ff130 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9fe30700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe9819aec0 con 0x7fbe981003d0 2024-04-26T04:44:01.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9dbcc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe980ff130 0x7fbe98069100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9dbcc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe980ff130 0x7fbe98069100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59128/0 (socket says 172.21.15.90:59128) 2024-04-26T04:44:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9dbcc700 1 -- 172.21.15.90:0/4186355916 learned_addr learned my addr 172.21.15.90:0/4186355916 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.609+0000 7fbe9d3cb700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe981003d0 0x7fbe9819a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe9dbcc700 1 -- 172.21.15.90:0/4186355916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe981003d0 msgr2=0x7fbe9819a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe9dbcc700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe981003d0 0x7fbe9819a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe9dbcc700 1 -- 172.21.15.90:0/4186355916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe880095f0 con 0x7fbe980ff130 2024-04-26T04:44:01.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe9dbcc700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe980ff130 0x7fbe98069100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400d8d0 tx=0x7fbe9400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:01.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe8effd700 1 -- 172.21.15.90:0/4186355916 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe940098a0 con 0x7fbe980ff130 2024-04-26T04:44:01.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe9819b1a0 con 0x7fbe980ff130 2024-04-26T04:44:01.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe8effd700 1 -- 172.21.15.90:0/4186355916 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fbe94010460 con 0x7fbe980ff130 2024-04-26T04:44:01.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe8effd700 1 -- 172.21.15.90:0/4186355916 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbe9400f5f0 con 0x7fbe980ff130 2024-04-26T04:44:01.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.610+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe9819b6f0 con 0x7fbe980ff130 2024-04-26T04:44:01.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.611+0000 7fbe8effd700 1 -- 172.21.15.90:0/4186355916 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 46370+0+0 (secure 0 0 0) 0x7fbe9400f820 con 0x7fbe980ff130 2024-04-26T04:44:01.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.611+0000 7fbe8effd700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbe840392e0 0x7fbe8403b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:01.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.611+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe98107c10 con 0x7fbe980ff130 2024-04-26T04:44:01.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.611+0000 7fbe8effd700 1 -- 172.21.15.90:0/4186355916 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbe9404d440 con 0x7fbe980ff130 2024-04-26T04:44:01.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.611+0000 7fbe9d3cb700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbe840392e0 0x7fbe8403b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:01.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.612+0000 7fbe9d3cb700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbe840392e0 0x7fbe8403b790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b5c0 tx=0x7fbe88005ee0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:01.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.620+0000 7fbe8effd700 1 -- 172.21.15.90:0/4186355916 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fbe940224e0 con 0x7fbe980ff130 2024-04-26T04:44:01.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.942+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbe9819be40 con 0x7fbe980ff130 2024-04-26T04:44:01.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.943+0000 7fbe8effd700 1 -- 172.21.15.90:0/4186355916 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fbe94009a00 con 0x7fbe980ff130 2024-04-26T04:44:01.945 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:44:01.945 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:44:01.945 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:44:01.945 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:01.946 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:44:01.946 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:44:01.946 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:01.946 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:44:01.946 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:44:01.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:01.947 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:44:01.947 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:44:01.947 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:01.947 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:44:01.947 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-26T04:44:01.948 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:44:01.948 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:44:01.948 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:44:01.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbe840392e0 msgr2=0x7fbe8403b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbe840392e0 0x7fbe8403b790 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fbe8800b5c0 tx=0x7fbe88005ee0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe980ff130 msgr2=0x7fbe98069100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:01.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe980ff130 0x7fbe98069100 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fbe9400d8d0 tx=0x7fbe9400dbe0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 shutdown_connections 2024-04-26T04:44:01.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbe840392e0 0x7fbe8403b790 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbe980ff130 0x7fbe98069100 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 --2- 172.21.15.90:0/4186355916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbe981003d0 0x7fbe9819a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:01.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 >> 172.21.15.90:0/4186355916 conn(0x7fbe980fa780 msgr2=0x7fbe98103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:01.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 shutdown_connections 2024-04-26T04:44:01.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:01.945+0000 7fbe9fe30700 1 -- 172.21.15.90:0/4186355916 wait complete. 2024-04-26T04:44:02.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 -- 172.21.15.90:0/800580385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a80ff130 msgr2=0x7fb8a80ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 --2- 172.21.15.90:0/800580385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a80ff130 0x7fb8a80ff540 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb8a000b0a0 tx=0x7fb8a000b3b0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 -- 172.21.15.90:0/800580385 shutdown_connections 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 --2- 172.21.15.90:0/800580385 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8a81003d0 0x7fb8a8100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 --2- 172.21.15.90:0/800580385 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a80ff130 0x7fb8a80ff540 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 -- 172.21.15.90:0/800580385 >> 172.21.15.90:0/800580385 conn(0x7fb8a80fa780 msgr2=0x7fb8a80fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 -- 172.21.15.90:0/800580385 shutdown_connections 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.054+0000 7fb8aec25700 1 -- 172.21.15.90:0/800580385 wait complete. 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8aec25700 1 Processor -- start 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8aec25700 1 -- start start 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8aec25700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8a80ff130 0x7fb8a80690c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8aec25700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a81003d0 0x7fb8a819a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:02.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8aec25700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a819adc0 con 0x7fb8a81003d0 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8aec25700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a819af30 con 0x7fb8a80ff130 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a81003d0 0x7fb8a819a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a81003d0 0x7fb8a819a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:59146/0 (socket says 172.21.15.90:59146) 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8a7fff700 1 -- 172.21.15.90:0/3090852178 learned_addr learned my addr 172.21.15.90:0/3090852178 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8ac9c1700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8a80ff130 0x7fb8a80690c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8a7fff700 1 -- 172.21.15.90:0/3090852178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8a80ff130 msgr2=0x7fb8a80690c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8a7fff700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8a80ff130 0x7fb8a80690c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.055+0000 7fb8a7fff700 1 -- 172.21.15.90:0/3090852178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a0009d00 con 0x7fb8a81003d0 2024-04-26T04:44:02.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.056+0000 7fb8a7fff700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a81003d0 0x7fb8a819a7f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb89800d8d0 tx=0x7fb89800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:02.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.056+0000 7fb8a5ffb700 1 -- 172.21.15.90:0/3090852178 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8980098a0 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.056+0000 7fb8a5ffb700 1 -- 172.21.15.90:0/3090852178 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fb898010460 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.056+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a819b210 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.056+0000 7fb8a5ffb700 1 -- 172.21.15.90:0/3090852178 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb89800f5f0 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.056+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a819b760 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.057+0000 7fb8a5ffb700 1 -- 172.21.15.90:0/3090852178 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 28) v1 ==== 46370+0+0 (secure 0 0 0) 0x7fb89800f820 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.057+0000 7fb8a5ffb700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb8940392e0 0x7fb89403b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.057+0000 7fb8a5ffb700 1 -- 172.21.15.90:0/3090852178 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb89804d710 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.057+0000 7fb8ac9c1700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb8940392e0 0x7fb89403b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.057+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a8107c10 con 0x7fb8a81003d0 2024-04-26T04:44:02.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.058+0000 7fb8ac9c1700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb8940392e0 0x7fb89403b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009fd0 tx=0x7fb8a00060c0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:02.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.062+0000 7fb8a5ffb700 1 -- 172.21.15.90:0/3090852178 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fb8980224e0 con 0x7fb8a81003d0 2024-04-26T04:44:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:02 smithi104 ceph-mon[32369]: mgrmap e28: smithi090.iwrvwa(active, since 1.00938s) 2024-04-26T04:44:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:02 smithi104 ceph-mon[32369]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:02 smithi104 ceph-mon[32369]: pgmap v3: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:02 smithi104 ceph-mon[32369]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:02.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:02 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/4186355916' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:02.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.322+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8a8104d20 con 0x7fb8940392e0 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.323+0000 7fb8a5ffb700 1 -- 172.21.15.90:0/3090852178 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+329 (secure 0 0 0) 0x7fb8a8104d20 con 0x7fb8940392e0 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-26T04:44:02.326 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-26T04:44:02.327 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:44:02.327 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:44:02.327 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb8940392e0 msgr2=0x7fb89403b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb8940392e0 0x7fb89403b790 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb8a0009fd0 tx=0x7fb8a00060c0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a81003d0 msgr2=0x7fb8a819a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a81003d0 0x7fb8a819a7f0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb89800d8d0 tx=0x7fb89800dbe0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 shutdown_connections 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb8940392e0 0x7fb89403b790 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8a80ff130 0x7fb8a80690c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 --2- 172.21.15.90:0/3090852178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8a81003d0 0x7fb8a819a7f0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 >> 172.21.15.90:0/3090852178 conn(0x7fb8a80fa780 msgr2=0x7fb8a8103600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 shutdown_connections 2024-04-26T04:44:02.328 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:02.326+0000 7fb8aec25700 1 -- 172.21.15.90:0/3090852178 wait complete. 2024-04-26T04:44:02.452 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:02 smithi090 ceph-mon[26648]: mgrmap e28: smithi090.iwrvwa(active, since 1.00938s) 2024-04-26T04:44:02.452 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:02 smithi090 ceph-mon[26648]: from='client.14696 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:02.452 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:02 smithi090 ceph-mon[26648]: pgmap v3: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:02.453 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:02 smithi090 ceph-mon[26648]: from='client.14724 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:02.453 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:02 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/4186355916' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:03.249 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:03 smithi104 ceph-mon[32369]: pgmap v4: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:03.249 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:03 smithi104 ceph-mon[32369]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:03.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:03 smithi090 ceph-mon[26648]: pgmap v4: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:03.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:03 smithi090 ceph-mon[26648]: from='client.14732 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:04.461 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:04 smithi090 ceph-mon[26648]: mgrmap e29: smithi090.iwrvwa(active, since 3s) 2024-04-26T04:44:04.461 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:04 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:04.461 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:04 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:04 smithi104 ceph-mon[32369]: mgrmap e29: smithi090.iwrvwa(active, since 3s) 2024-04-26T04:44:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:04 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:04 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:05.705 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:05 smithi104 ceph-mon[32369]: pgmap v5: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:05.705 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:05.705 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:05 smithi090 ceph-mon[26648]: pgmap v5: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:07.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:06 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:07.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:06 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:07.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:06 smithi090 ceph-mon[26648]: pgmap v6: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:07.225 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:06 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:07.225 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:06 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:07.225 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:06 smithi104 ceph-mon[32369]: pgmap v6: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:08.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:07 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:08.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:07 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:08.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:07 smithi090 ceph-mon[26648]: Standby manager daemon smithi104.wwoewi started 2024-04-26T04:44:08.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:07 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:44:08.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:07 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:44:08.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:07 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:44:08.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:07 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:44:08.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:07 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:08.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:07 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:08.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:07 smithi104 ceph-mon[32369]: Standby manager daemon smithi104.wwoewi started 2024-04-26T04:44:08.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:07 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:44:08.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:07 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:44:08.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:07 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:44:08.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:07 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/1596839483' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:44:09.202 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:09 smithi090 ceph-mon[26648]: mgrmap e30: smithi090.iwrvwa(active, since 7s), standbys: smithi104.wwoewi 2024-04-26T04:44:09.202 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:09 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:09.202 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:09 smithi090 ceph-mon[26648]: pgmap v7: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T04:44:09.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:09 smithi104 ceph-mon[32369]: mgrmap e30: smithi090.iwrvwa(active, since 7s), standbys: smithi104.wwoewi 2024-04-26T04:44:09.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:09 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:09.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:09 smithi104 ceph-mon[32369]: pgmap v7: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 29 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T04:44:11.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: pgmap v8: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T04:44:11.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:11 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: pgmap v8: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T04:44:11.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:11.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:11 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:12.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:12 smithi090 ceph-mon[26648]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:44:12.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:12 smithi090 ceph-mon[26648]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:44:12.350 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:12 smithi090 ceph-mon[26648]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:44:12.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:12 smithi090 ceph-mon[26648]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-26T04:44:12.351 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:12 smithi090 ceph-mon[26648]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-26T04:44:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:12 smithi104 ceph-mon[32369]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:44:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:12 smithi104 ceph-mon[32369]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:44:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:12 smithi104 ceph-mon[32369]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:44:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:12 smithi104 ceph-mon[32369]: Reconfiguring prometheus.smithi090 (dependencies changed)... 2024-04-26T04:44:12.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:12 smithi104 ceph-mon[32369]: Reconfiguring daemon prometheus.smithi090 on smithi090 2024-04-26T04:44:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:13 smithi090 ceph-mon[26648]: pgmap v9: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T04:44:13.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:13 smithi104 ceph-mon[32369]: pgmap v9: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T04:44:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:15 smithi104 ceph-mon[32369]: pgmap v10: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:44:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:15 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:15.608 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:15 smithi090 ceph-mon[26648]: pgmap v10: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:44:15.608 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:15 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:17.634 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: pgmap v11: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:44:17.634 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:17.634 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:17.634 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:44:17.634 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:44:17.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:44:17.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:44:17.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:17.635 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:17.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: pgmap v11: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi090:9095"}]: dispatch 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:18.649 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:18 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:44:18.649 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:18 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:44:18.649 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:18 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:18.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:18 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:44:18.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:18 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:44:18.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:18 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:19.496 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:19 smithi104 ceph-mon[32369]: Upgrade: Updating mgr.smithi104.wwoewi 2024-04-26T04:44:19.497 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:19 smithi104 ceph-mon[32369]: Deploying daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:44:19.497 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:19 smithi104 ceph-mon[32369]: pgmap v12: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:44:19.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:19 smithi090 ceph-mon[26648]: Upgrade: Updating mgr.smithi104.wwoewi 2024-04-26T04:44:19.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:19 smithi090 ceph-mon[26648]: Deploying daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:44:19.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:19 smithi090 ceph-mon[26648]: pgmap v12: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:44:21.605 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:21 smithi104 ceph-mon[32369]: pgmap v13: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:21.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:21 smithi090 ceph-mon[26648]: pgmap v13: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:22.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:22.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:22.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:22.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:22 smithi090 ceph-mon[26648]: pgmap v14: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:22.986 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:22.986 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:22.986 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:22.986 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:22 smithi104 ceph-mon[32369]: pgmap v14: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:25.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:25 smithi104 ceph-mon[32369]: pgmap v15: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:25.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:25 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:25.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:25 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:25.605 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:25 smithi090 ceph-mon[26648]: pgmap v15: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:25.605 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:25 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:25.606 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:25 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:26 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:26 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:26 smithi104 ceph-mon[32369]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:26 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:26 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:26 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:26 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:26 smithi090 ceph-mon[26648]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:27.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:26 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:27.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:26 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:29 smithi104 ceph-mon[32369]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:29 smithi104 ceph-mon[32369]: Standby manager daemon smithi104.wwoewi restarted 2024-04-26T04:44:29.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:29 smithi104 ceph-mon[32369]: Standby manager daemon smithi104.wwoewi started 2024-04-26T04:44:29.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:29 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:44:29.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:29 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:44:29.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:29 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:44:29.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:29 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:44:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:29 smithi090 ceph-mon[26648]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:29 smithi090 ceph-mon[26648]: Standby manager daemon smithi104.wwoewi restarted 2024-04-26T04:44:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:29 smithi090 ceph-mon[26648]: Standby manager daemon smithi104.wwoewi started 2024-04-26T04:44:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:29 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:44:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:29 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:44:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:29 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:44:29.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:29 smithi090 ceph-mon[26648]: from='mgr.? 172.21.15.104:0/453882422' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:44:30.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: mgrmap e31: smithi090.iwrvwa(active, since 29s), standbys: smithi104.wwoewi 2024-04-26T04:44:30.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:30.340 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all mgr 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.iwrvwa"}]: dispatch 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.iwrvwa"}]': finished 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi104.wwoewi"}]': finished 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all rgw 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all rbd-mirror 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all iscsi 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all nfs 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all nvmeof 2024-04-26T04:44:30.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.343 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: mgrmap e31: smithi090.iwrvwa(active, since 29s), standbys: smithi104.wwoewi 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all mgr 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.iwrvwa"}]: dispatch 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi090.iwrvwa"}]': finished 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi104.wwoewi"}]: dispatch 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr.smithi104.wwoewi"}]': finished 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all rgw 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all rbd-mirror 2024-04-26T04:44:30.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all cephfs-mirror 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all iscsi 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all nfs 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all nvmeof 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:30.764 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:31.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:31 smithi104 ceph-mon[32369]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:31.875 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:31 smithi090 ceph-mon[26648]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:32.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- 172.21.15.90:0/1627920857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 msgr2=0x7f6d90100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:32.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/1627920857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d90100840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c009910 tx=0x7f6d7c009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:32.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- 172.21.15.90:0/1627920857 shutdown_connections 2024-04-26T04:44:32.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/1627920857 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d90100840 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/1627920857 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d900ff130 0x7f6d900ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- 172.21.15.90:0/1627920857 >> 172.21.15.90:0/1627920857 conn(0x7f6d900fa780 msgr2=0x7f6d900fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:32.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- 172.21.15.90:0/1627920857 shutdown_connections 2024-04-26T04:44:32.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- 172.21.15.90:0/1627920857 wait complete. 2024-04-26T04:44:32.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 Processor -- start 2024-04-26T04:44:32.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- start start 2024-04-26T04:44:32.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d900ff130 0x7f6d90103160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d901017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90101cf0 con 0x7f6d901003d0 2024-04-26T04:44:32.441 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d96ce1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d90101e30 con 0x7f6d900ff130 2024-04-26T04:44:32.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.436+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d901017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d901017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53234/0 (socket says 172.21.15.90:53234) 2024-04-26T04:44:32.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8ffff700 1 -- 172.21.15.90:0/322743111 learned_addr learned my addr 172.21.15.90:0/322743111 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:32.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d94a7d700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d900ff130 0x7f6d90103160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8ffff700 1 -- 172.21.15.90:0/322743111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d900ff130 msgr2=0x7f6d90103160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:32.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8ffff700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d900ff130 0x7f6d90103160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8ffff700 1 -- 172.21.15.90:0/322743111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d7c0095f0 con 0x7f6d901003d0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d94a7d700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d900ff130 0x7f6d90103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8ffff700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d901017b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c006010 tx=0x7f6d7c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8dffb700 1 -- 172.21.15.90:0/322743111 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7c01d070 con 0x7f6d901003d0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d901020b0 con 0x7f6d901003d0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8dffb700 1 -- 172.21.15.90:0/322743111 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f6d7c00f460 con 0x7f6d901003d0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d8dffb700 1 -- 172.21.15.90:0/322743111 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6d7c021640 con 0x7f6d901003d0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.437+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d901025a0 con 0x7f6d901003d0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.438+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d90107c90 con 0x7f6d901003d0 2024-04-26T04:44:32.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.439+0000 7f6d8dffb700 1 -- 172.21.15.90:0/322743111 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f6d7c00fab0 con 0x7f6d901003d0 2024-04-26T04:44:32.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.439+0000 7f6d8dffb700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6d8006dfc0 0x7f6d80070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.439+0000 7f6d8dffb700 1 -- 172.21.15.90:0/322743111 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6d7c08f400 con 0x7f6d901003d0 2024-04-26T04:44:32.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.439+0000 7f6d94a7d700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6d8006dfc0 0x7f6d80070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.446 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.440+0000 7f6d94a7d700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6d8006dfc0 0x7f6d80070470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6d84007900 tx=0x7f6d84008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:32.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.444+0000 7f6d8dffb700 1 -- 172.21.15.90:0/322743111 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f6d7c05c480 con 0x7f6d901003d0 2024-04-26T04:44:32.680 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:32 smithi090 ceph-mon[26648]: Upgrade: Updating node-exporter.smithi090 (1/2) 2024-04-26T04:44:32.680 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:32 smithi090 ceph-mon[26648]: Deploying daemon node-exporter.smithi090 on smithi090 2024-04-26T04:44:32.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.678+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6d90068c50 con 0x7f6d8006dfc0 2024-04-26T04:44:32.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.679+0000 7f6d8dffb700 1 -- 172.21.15.90:0/322743111 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f6d90068c50 con 0x7f6d8006dfc0 2024-04-26T04:44:32.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6d8006dfc0 msgr2=0x7f6d80070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:32.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6d8006dfc0 0x7f6d80070470 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f6d84007900 tx=0x7f6d84008040 comp rx=0 tx=0).stop 2024-04-26T04:44:32.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 msgr2=0x7f6d901017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:32.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d901017b0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6d7c006010 tx=0x7f6d7c00be20 comp rx=0 tx=0).stop 2024-04-26T04:44:32.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 shutdown_connections 2024-04-26T04:44:32.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6d8006dfc0 0x7f6d80070470 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6d900ff130 0x7f6d90103160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 --2- 172.21.15.90:0/322743111 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6d901003d0 0x7f6d901017b0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 >> 172.21.15.90:0/322743111 conn(0x7f6d900fa780 msgr2=0x7f6d90068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:32.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 shutdown_connections 2024-04-26T04:44:32.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.682+0000 7f6d96ce1700 1 -- 172.21.15.90:0/322743111 wait complete. 2024-04-26T04:44:32.693 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:44:32.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:32 smithi104 ceph-mon[32369]: Upgrade: Updating node-exporter.smithi090 (1/2) 2024-04-26T04:44:32.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:32 smithi104 ceph-mon[32369]: Deploying daemon node-exporter.smithi090 on smithi090 2024-04-26T04:44:32.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 -- 172.21.15.90:0/740348424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 msgr2=0x7f9f5c100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:32.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/740348424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c100840 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f9f58009910 tx=0x7f9f58009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:32.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 -- 172.21.15.90:0/740348424 shutdown_connections 2024-04-26T04:44:32.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/740348424 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c100840 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/740348424 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f5c0ff130 0x7f9f5c0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 -- 172.21.15.90:0/740348424 >> 172.21.15.90:0/740348424 conn(0x7f9f5c0fa780 msgr2=0x7f9f5c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:32.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 -- 172.21.15.90:0/740348424 shutdown_connections 2024-04-26T04:44:32.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.788+0000 7f9f63cd3700 1 -- 172.21.15.90:0/740348424 wait complete. 2024-04-26T04:44:32.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f63cd3700 1 Processor -- start 2024-04-26T04:44:32.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f63cd3700 1 -- start start 2024-04-26T04:44:32.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f63cd3700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f5c0ff130 0x7f9f5c195ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f63cd3700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f63cd3700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c196640 con 0x7f9f5c1003d0 2024-04-26T04:44:32.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f63cd3700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f5c196780 con 0x7f9f5c0ff130 2024-04-26T04:44:32.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f6126e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f6126e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53248/0 (socket says 172.21.15.90:53248) 2024-04-26T04:44:32.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f6126e700 1 -- 172.21.15.90:0/4025738479 learned_addr learned my addr 172.21.15.90:0/4025738479 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:32.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.789+0000 7f9f61a6f700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f5c0ff130 0x7f9f5c195ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f6126e700 1 -- 172.21.15.90:0/4025738479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f5c0ff130 msgr2=0x7f9f5c195ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:32.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f6126e700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f5c0ff130 0x7f9f5c195ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:32.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f6126e700 1 -- 172.21.15.90:0/4025738479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f580095f0 con 0x7f9f5c1003d0 2024-04-26T04:44:32.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f6126e700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c196020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800bd70 tx=0x7f9f5800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:32.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f52ffd700 1 -- 172.21.15.90:0/4025738479 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5801d070 con 0x7f9f5c1003d0 2024-04-26T04:44:32.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f5c19b1d0 con 0x7f9f5c1003d0 2024-04-26T04:44:32.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f52ffd700 1 -- 172.21.15.90:0/4025738479 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f9f58022470 con 0x7f9f5c1003d0 2024-04-26T04:44:32.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f52ffd700 1 -- 172.21.15.90:0/4025738479 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9f5800f670 con 0x7f9f5c1003d0 2024-04-26T04:44:32.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.790+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f5c19b6c0 con 0x7f9f5c1003d0 2024-04-26T04:44:32.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.791+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f5c107c90 con 0x7f9f5c1003d0 2024-04-26T04:44:32.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.791+0000 7f9f52ffd700 1 -- 172.21.15.90:0/4025738479 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f9f58022a50 con 0x7f9f5c1003d0 2024-04-26T04:44:32.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.792+0000 7f9f52ffd700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f9f4806dfc0 0x7f9f48070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:32.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.792+0000 7f9f52ffd700 1 -- 172.21.15.90:0/4025738479 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f9f5808f560 con 0x7f9f5c1003d0 2024-04-26T04:44:32.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.792+0000 7f9f61a6f700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f9f4806dfc0 0x7f9f48070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:32.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.792+0000 7f9f61a6f700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f9f4806dfc0 0x7f9f48070470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c006fd0 tx=0x7f9f4c008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:32.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:32.796+0000 7f9f52ffd700 1 -- 172.21.15.90:0/4025738479 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f9f5805c730 con 0x7f9f5c1003d0 2024-04-26T04:44:33.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.044+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9f5c068c50 con 0x7f9f4806dfc0 2024-04-26T04:44:33.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.045+0000 7f9f52ffd700 1 -- 172.21.15.90:0/4025738479 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7f9f5c068c50 con 0x7f9f4806dfc0 2024-04-26T04:44:33.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f9f4806dfc0 msgr2=0x7f9f48070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f9f4806dfc0 0x7f9f48070470 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9f4c006fd0 tx=0x7f9f4c008040 comp rx=0 tx=0).stop 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 msgr2=0x7f9f5c196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c196020 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f9f5800bd70 tx=0x7f9f5800be50 comp rx=0 tx=0).stop 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 shutdown_connections 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f9f4806dfc0 0x7f9f48070470 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9f5c0ff130 0x7f9f5c195ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 --2- 172.21.15.90:0/4025738479 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9f5c1003d0 0x7f9f5c196020 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 >> 172.21.15.90:0/4025738479 conn(0x7f9f5c0fa780 msgr2=0x7f9f5c068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 shutdown_connections 2024-04-26T04:44:33.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.047+0000 7f9f63cd3700 1 -- 172.21.15.90:0/4025738479 wait complete. 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 -- 172.21.15.90:0/1615174863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10981003d0 msgr2=0x7f1098100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 --2- 172.21.15.90:0/1615174863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10981003d0 0x7f1098100840 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f1088009910 tx=0x7f1088009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 -- 172.21.15.90:0/1615174863 shutdown_connections 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 --2- 172.21.15.90:0/1615174863 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10981003d0 0x7f1098100840 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 --2- 172.21.15.90:0/1615174863 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10980ff130 0x7f10980ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 -- 172.21.15.90:0/1615174863 >> 172.21.15.90:0/1615174863 conn(0x7f10980fa780 msgr2=0x7f10980fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 -- 172.21.15.90:0/1615174863 shutdown_connections 2024-04-26T04:44:33.164 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 -- 172.21.15.90:0/1615174863 wait complete. 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 Processor -- start 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.161+0000 7f109e670700 1 -- start start 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f109e670700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10980ff130 0x7f1098195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f109e670700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981003d0 0x7f1098195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f109e670700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098196530 con 0x7f10980ff130 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f109e670700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1098196670 con 0x7f10981003d0 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f1097fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10980ff130 0x7f1098195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f1097fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10980ff130 0x7f1098195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53266/0 (socket says 172.21.15.90:53266) 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f1097fff700 1 -- 172.21.15.90:0/2582092325 learned_addr learned my addr 172.21.15.90:0/2582092325 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f10977fe700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981003d0 0x7f1098195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f1097fff700 1 -- 172.21.15.90:0/2582092325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981003d0 msgr2=0x7f1098195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f1097fff700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981003d0 0x7f1098195fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f1097fff700 1 -- 172.21.15.90:0/2582092325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10880095f0 con 0x7f10980ff130 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f1097fff700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10980ff130 0x7f1098195a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f108c00c930 tx=0x7f108c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:33.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.162+0000 7f10957fa700 1 -- 172.21.15.90:0/2582092325 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c007ad0 con 0x7f10980ff130 2024-04-26T04:44:33.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.163+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f109819b130 con 0x7f10980ff130 2024-04-26T04:44:33.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.163+0000 7f10957fa700 1 -- 172.21.15.90:0/2582092325 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f108c007c30 con 0x7f10980ff130 2024-04-26T04:44:33.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.163+0000 7f10957fa700 1 -- 172.21.15.90:0/2582092325 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f108c018630 con 0x7f10980ff130 2024-04-26T04:44:33.166 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.163+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f109819b650 con 0x7f10980ff130 2024-04-26T04:44:33.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.164+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1098107c10 con 0x7f10980ff130 2024-04-26T04:44:33.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.164+0000 7f10957fa700 1 -- 172.21.15.90:0/2582092325 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f108c0188b0 con 0x7f10980ff130 2024-04-26T04:44:33.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.164+0000 7f10957fa700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f108006dfc0 0x7f1080070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.165+0000 7f10957fa700 1 -- 172.21.15.90:0/2582092325 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f108c08dfa0 con 0x7f10980ff130 2024-04-26T04:44:33.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.165+0000 7f10977fe700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f108006dfc0 0x7f1080070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.167 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.165+0000 7f10977fe700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f108006dfc0 0x7f1080070470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f10880059d0 tx=0x7f1088005920 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:33.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.169+0000 7f10957fa700 1 -- 172.21.15.90:0/2582092325 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f108c05b290 con 0x7f10980ff130 2024-04-26T04:44:33.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.396+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f1098068c50 con 0x7f108006dfc0 2024-04-26T04:44:33.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.407+0000 7f10957fa700 1 -- 172.21.15.90:0/2582092325 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f1098068c50 con 0x7f108006dfc0 2024-04-26T04:44:33.408 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:44:33.408 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (8m) 6s ago 10m 25.8M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:44:33.408 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (10m) 6s ago 10m 7805k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:44:33.408 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (9m) 8s ago 9m 7717k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:44:33.408 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (10m) 6s ago 10m 6665k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:44:33.408 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (9m) 8s ago 9m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (8m) 6s ago 9m 84.1M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (4m) 6s ago 4m 130M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (4m) 6s ago 4m 14.0M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (4m) 8s ago 4m 14.8M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (4m) 8s ago 4m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (44s) 6s ago 11m 494M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (12s) 8s ago 9m 62.7M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (11m) 6s ago 11m 57.8M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (9m) 8s ago 9m 41.4M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (10m) 6s ago 10m 14.2M - 1.5.0 0da6a335fe13 c4d7616e34b0 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (9m) 8s ago 9m 15.6M - 1.5.0 0da6a335fe13 23f92116fe69 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 6s ago 8m 227M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (7m) 6s ago 7m 245M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:44:33.409 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (7m) 6s ago 7m 200M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:44:33.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (6m) 8s ago 6m 269M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:44:33.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (6m) 8s ago 6m 234M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:44:33.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (6m) 8s ago 6m 238M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:44:33.410 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (17s) 6s ago 9m 45.6M - 2.43.0 a07b618ecd1d bc36a70273b7 2024-04-26T04:44:33.410 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f108006dfc0 msgr2=0x7f1080070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f108006dfc0 0x7f1080070470 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f10880059d0 tx=0x7f1088005920 comp rx=0 tx=0).stop 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10980ff130 msgr2=0x7f1098195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10980ff130 0x7f1098195a60 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f108c00c930 tx=0x7f108c00ccf0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 shutdown_connections 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f108006dfc0 0x7f1080070470 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10980ff130 0x7f1098195a60 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 --2- 172.21.15.90:0/2582092325 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10981003d0 0x7f1098195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 >> 172.21.15.90:0/2582092325 conn(0x7f10980fa780 msgr2=0x7f1098068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 shutdown_connections 2024-04-26T04:44:33.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.409+0000 7f109e670700 1 -- 172.21.15.90:0/2582092325 wait complete. 2024-04-26T04:44:33.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 -- 172.21.15.90:0/3959819921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b01003d0 msgr2=0x7f88b0100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/3959819921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b01003d0 0x7f88b0100840 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f88a4009910 tx=0x7f88a4009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:33.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 -- 172.21.15.90:0/3959819921 shutdown_connections 2024-04-26T04:44:33.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/3959819921 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b01003d0 0x7f88b0100840 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/3959819921 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88b00ff130 0x7f88b00ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 -- 172.21.15.90:0/3959819921 >> 172.21.15.90:0/3959819921 conn(0x7f88b00fa780 msgr2=0x7f88b00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:33.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 -- 172.21.15.90:0/3959819921 shutdown_connections 2024-04-26T04:44:33.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.515+0000 7f88b4c5d700 1 -- 172.21.15.90:0/3959819921 wait complete. 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88b4c5d700 1 Processor -- start 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88b4c5d700 1 -- start start 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88b4c5d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b00ff130 0x7f88b0195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88b4c5d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88b01003d0 0x7f88b0195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88b4c5d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b01965c0 con 0x7f88b00ff130 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88b4c5d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88b0196700 con 0x7f88b01003d0 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88ae59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b00ff130 0x7f88b0195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88ae59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b00ff130 0x7f88b0195a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53284/0 (socket says 172.21.15.90:53284) 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88ae59c700 1 -- 172.21.15.90:0/2569836661 learned_addr learned my addr 172.21.15.90:0/2569836661 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88add9b700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88b01003d0 0x7f88b0195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88ae59c700 1 -- 172.21.15.90:0/2569836661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88b01003d0 msgr2=0x7f88b0195fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88ae59c700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88b01003d0 0x7f88b0195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.516+0000 7f88ae59c700 1 -- 172.21.15.90:0/2569836661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a40095f0 con 0x7f88b00ff130 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.517+0000 7f88ae59c700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b00ff130 0x7f88b0195a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f88a800c8a0 tx=0x7f88a800cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.517+0000 7f889f7fe700 1 -- 172.21.15.90:0/2569836661 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a800cea0 con 0x7f88b00ff130 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.517+0000 7f889f7fe700 1 -- 172.21.15.90:0/2569836661 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f88a8004830 con 0x7f88b00ff130 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.517+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88b019b1b0 con 0x7f88b00ff130 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.517+0000 7f889f7fe700 1 -- 172.21.15.90:0/2569836661 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f88a80055b0 con 0x7f88b00ff130 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.517+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88b019b700 con 0x7f88b00ff130 2024-04-26T04:44:33.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.518+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88b0107c10 con 0x7f88b00ff130 2024-04-26T04:44:33.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.518+0000 7f889f7fe700 1 -- 172.21.15.90:0/2569836661 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f88a8007ad0 con 0x7f88b00ff130 2024-04-26T04:44:33.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.519+0000 7f889f7fe700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f889806dfb0 0x7f8898070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.519+0000 7f889f7fe700 1 -- 172.21.15.90:0/2569836661 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f88a808dba0 con 0x7f88b00ff130 2024-04-26T04:44:33.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.519+0000 7f88add9b700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f889806dfb0 0x7f8898070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.519+0000 7f88add9b700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f889806dfb0 0x7f8898070460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88a4006010 tx=0x7f88a400b540 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:33.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.523+0000 7f889f7fe700 1 -- 172.21.15.90:0/2569836661 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f88a805acc0 con 0x7f88b00ff130 2024-04-26T04:44:33.651 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:33 smithi090 ceph-mon[26648]: pgmap v19: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:33.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:33 smithi104 ceph-mon[32369]: pgmap v19: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.832+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f88b019be00 con 0x7f88b00ff130 2024-04-26T04:44:33.834 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.833+0000 7f889f7fe700 1 -- 172.21.15.90:0/2569836661 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f88a805a850 con 0x7f88b00ff130 2024-04-26T04:44:33.834 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:44:33.835 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-26T04:44:33.836 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:44:33.836 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:44:33.836 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:44:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f889806dfb0 msgr2=0x7f8898070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f889806dfb0 0x7f8898070460 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f88a4006010 tx=0x7f88a400b540 comp rx=0 tx=0).stop 2024-04-26T04:44:33.836 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b00ff130 msgr2=0x7f88b0195a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b00ff130 0x7f88b0195a60 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f88a800c8a0 tx=0x7f88a800cc60 comp rx=0 tx=0).stop 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 shutdown_connections 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f889806dfb0 0x7f8898070460 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f88b00ff130 0x7f88b0195a60 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 --2- 172.21.15.90:0/2569836661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f88b01003d0 0x7f88b0195fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 >> 172.21.15.90:0/2569836661 conn(0x7f88b00fa780 msgr2=0x7f88b0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 shutdown_connections 2024-04-26T04:44:33.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.835+0000 7f88b4c5d700 1 -- 172.21.15.90:0/2569836661 wait complete. 2024-04-26T04:44:33.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 -- 172.21.15.90:0/3070137719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 msgr2=0x7fadd4100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 --2- 172.21.15.90:0/3070137719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4100650 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fadc4009910 tx=0x7fadc4009c20 comp rx=0 tx=0).stop 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 -- 172.21.15.90:0/3070137719 shutdown_connections 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 --2- 172.21.15.90:0/3070137719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd41014e0 0x7fadd4101950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 --2- 172.21.15.90:0/3070137719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4100650 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 -- 172.21.15.90:0/3070137719 >> 172.21.15.90:0/3070137719 conn(0x7fadd40fb890 msgr2=0x7fadd40fdcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 -- 172.21.15.90:0/3070137719 shutdown_connections 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.946+0000 7fadd9836700 1 -- 172.21.15.90:0/3070137719 wait complete. 2024-04-26T04:44:33.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd9836700 1 Processor -- start 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd9836700 1 -- start start 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd9836700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4195a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd9836700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd41014e0 0x7fadd4195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd9836700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd41965f0 con 0x7fadd4100240 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd9836700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4196730 con 0x7fadd41014e0 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4195a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd2ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4195a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:53308/0 (socket says 172.21.15.90:53308) 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd2ffd700 1 -- 172.21.15.90:0/3848432791 learned_addr learned my addr 172.21.15.90:0/3848432791 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd27fc700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd41014e0 0x7fadd4195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd2ffd700 1 -- 172.21.15.90:0/3848432791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd41014e0 msgr2=0x7fadd4195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd2ffd700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd41014e0 0x7fadd4195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd2ffd700 1 -- 172.21.15.90:0/3848432791 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc40095f0 con 0x7fadd4100240 2024-04-26T04:44:33.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd27fc700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd41014e0 0x7fadd4195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:44:33.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.947+0000 7fadd2ffd700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4195a90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b5c0 tx=0x7fadc400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:33.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.948+0000 7fadd8834700 1 -- 172.21.15.90:0/3848432791 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc401d070 con 0x7fadd4100240 2024-04-26T04:44:33.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.948+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd419b180 con 0x7fadd4100240 2024-04-26T04:44:33.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.948+0000 7fadd8834700 1 -- 172.21.15.90:0/3848432791 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fadc400f460 con 0x7fadd4100240 2024-04-26T04:44:33.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.948+0000 7fadd8834700 1 -- 172.21.15.90:0/3848432791 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fadc4021640 con 0x7fadd4100240 2024-04-26T04:44:33.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.948+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd419b670 con 0x7fadd4100240 2024-04-26T04:44:33.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.949+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd4108d20 con 0x7fadd4100240 2024-04-26T04:44:33.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.949+0000 7fadd8834700 1 -- 172.21.15.90:0/3848432791 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7fadc400f5d0 con 0x7fadd4100240 2024-04-26T04:44:33.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.950+0000 7fadd8834700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fadbc06dfb0 0x7fadbc070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:44:33.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.950+0000 7fadd8834700 1 -- 172.21.15.90:0/3848432791 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fadc408f200 con 0x7fadd4100240 2024-04-26T04:44:33.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.950+0000 7fadd27fc700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fadbc06dfb0 0x7fadbc070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:44:33.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.950+0000 7fadd27fc700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fadbc06dfb0 0x7fadbc070460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005950 tx=0x7fadc80058e0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:44:33.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:33.954+0000 7fadd8834700 1 -- 172.21.15.90:0/3848432791 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fadc405c320 con 0x7fadd4100240 2024-04-26T04:44:34.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.174+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fadd4105e30 con 0x7fadbc06dfb0 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.176+0000 7fadd8834700 1 -- 172.21.15.90:0/3848432791 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+370 (secure 0 0 0) 0x7fadd4105e30 con 0x7fadbc06dfb0 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-26T04:44:34.177 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading node-exporter daemons", 2024-04-26T04:44:34.178 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:44:34.178 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:44:34.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fadbc06dfb0 msgr2=0x7fadbc070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:34.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fadbc06dfb0 0x7fadbc070460 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fadc8005950 tx=0x7fadc80058e0 comp rx=0 tx=0).stop 2024-04-26T04:44:34.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 msgr2=0x7fadd4195a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:44:34.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4195a90 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fadc400b5c0 tx=0x7fadc400be20 comp rx=0 tx=0).stop 2024-04-26T04:44:34.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 shutdown_connections 2024-04-26T04:44:34.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fadbc06dfb0 0x7fadbc070460 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:34.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fadd4100240 0x7fadd4195a90 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:34.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 --2- 172.21.15.90:0/3848432791 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fadd41014e0 0x7fadd4195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:44:34.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 >> 172.21.15.90:0/3848432791 conn(0x7fadd40fb890 msgr2=0x7fadd4104710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:44:34.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 shutdown_connections 2024-04-26T04:44:34.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:44:34.178+0000 7fadd9836700 1 -- 172.21.15.90:0/3848432791 wait complete. 2024-04-26T04:44:34.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:34 smithi104 ceph-mon[32369]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:34.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:34 smithi104 ceph-mon[32369]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:34.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:34 smithi104 ceph-mon[32369]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:34.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:34 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2569836661' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:34.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:34 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:34.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:34 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:34 smithi090 ceph-mon[26648]: from='client.14742 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:34 smithi090 ceph-mon[26648]: from='client.14746 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:34 smithi090 ceph-mon[26648]: from='client.14750 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:34 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2569836661' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:34 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:34 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:35 smithi104 ceph-mon[32369]: pgmap v20: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:35 smithi104 ceph-mon[32369]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:35.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:35 smithi090 ceph-mon[26648]: pgmap v20: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:35.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:35 smithi090 ceph-mon[26648]: from='client.14758 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:44:36.757 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:36 smithi104 ceph-mon[32369]: Upgrade: Updating node-exporter.smithi104 (2/2) 2024-04-26T04:44:36.758 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:36 smithi104 ceph-mon[32369]: Deploying daemon node-exporter.smithi104 on smithi104 2024-04-26T04:44:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:36 smithi090 ceph-mon[26648]: Upgrade: Updating node-exporter.smithi104 (2/2) 2024-04-26T04:44:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:36 smithi090 ceph-mon[26648]: Deploying daemon node-exporter.smithi104 on smithi104 2024-04-26T04:44:37.744 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:37 smithi104 ceph-mon[32369]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:37.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:37 smithi090 ceph-mon[26648]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:39.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:39 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:39.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:39 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:39.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:39 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:39.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:39 smithi090 ceph-mon[26648]: pgmap v22: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:39.255 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:39 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:39.256 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:39 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:39.256 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:39 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:39.256 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:39 smithi104 ceph-mon[32369]: pgmap v22: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:41.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:41 smithi090 ceph-mon[26648]: pgmap v23: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:41.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:41.345 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:41.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:41 smithi104 ceph-mon[32369]: pgmap v23: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:41.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:41.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:43.219 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:43 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:43.219 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:43 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:43.219 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:43 smithi104 ceph-mon[32369]: pgmap v24: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:43.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:43 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:43.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:43 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:43.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:43 smithi090 ceph-mon[26648]: pgmap v24: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:44 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:44 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:44.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:44 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:44.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:44 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:45 smithi090 ceph-mon[26648]: pgmap v25: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:45.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:45 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:45 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:45 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:45.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:45 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:45 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:45 smithi104 ceph-mon[32369]: pgmap v25: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:45.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:45 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:45 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:45 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:44:45.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:45 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:45.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:45 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:47.395 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:47 smithi090 ceph-mon[26648]: pgmap v26: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:47.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:47 smithi104 ceph-mon[32369]: pgmap v26: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.514 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.515 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.515 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:48 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.733 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:48.733 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:48.734 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:44:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:49 smithi104 ceph-mon[32369]: pgmap v27: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:49.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:49 smithi090 ceph-mon[26648]: pgmap v27: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:50.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:50 smithi090 ceph-mon[26648]: Upgrade: Updating prometheus.smithi090 2024-04-26T04:44:50.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:50 smithi090 ceph-mon[26648]: Deploying daemon prometheus.smithi090 on smithi090 2024-04-26T04:44:50.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:50 smithi104 ceph-mon[32369]: Upgrade: Updating prometheus.smithi090 2024-04-26T04:44:50.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:50 smithi104 ceph-mon[32369]: Deploying daemon prometheus.smithi090 on smithi090 2024-04-26T04:44:51.614 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:51 smithi090 ceph-mon[26648]: pgmap v28: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:51.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:51 smithi104 ceph-mon[32369]: pgmap v28: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:53.715 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:53 smithi090 ceph-mon[26648]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:53.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:53 smithi104 ceph-mon[32369]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 85 B/s wr, 0 op/s 2024-04-26T04:44:55.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:55 smithi104 ceph-mon[32369]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:55.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:55 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:55.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:55 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:55.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:55 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:55.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:55 smithi090 ceph-mon[26648]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:55.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:55 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:55.894 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:55 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:44:55.895 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:55 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:44:57.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:57 smithi090 ceph-mon[26648]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:57 smithi104 ceph-mon[32369]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:44:59.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:44:59 smithi090 ceph-mon[26648]: pgmap v32: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:44:59 smithi104 ceph-mon[32369]: pgmap v32: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:01.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:01 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:01.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:01 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:01.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:01 smithi090 ceph-mon[26648]: pgmap v33: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:01.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:01 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:01.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:01 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:01.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:01 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:01.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:01 smithi104 ceph-mon[32369]: pgmap v33: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:01.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:01 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:02 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:02 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:02.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:02 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:02.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:02 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:03.407 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:03 smithi090 ceph-mon[26648]: pgmap v34: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:03.411 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:03 smithi104 ceph-mon[32369]: pgmap v34: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 -- 172.21.15.90:0/1557045724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4101440 msgr2=0x7f8fa4101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 --2- 172.21.15.90:0/1557045724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4101440 0x7f8fa4101890 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f8f94009910 tx=0x7f8f94009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 -- 172.21.15.90:0/1557045724 shutdown_connections 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 --2- 172.21.15.90:0/1557045724 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4101440 0x7f8fa4101890 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 --2- 172.21.15.90:0/1557045724 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fa4100240 0x7f8fa4100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 -- 172.21.15.90:0/1557045724 >> 172.21.15.90:0/1557045724 conn(0x7f8fa40fb7f0 msgr2=0x7f8fa40fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 -- 172.21.15.90:0/1557045724 shutdown_connections 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 -- 172.21.15.90:0/1557045724 wait complete. 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 Processor -- start 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.286+0000 7f8fab583700 1 -- start start 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fab583700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4100240 0x7f8fa4195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:04.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fab583700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fa4101440 0x7f8fa4195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fab583700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4196610 con 0x7f8fa4100240 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fab583700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fa4196750 con 0x7f8fa4101440 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa931f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4100240 0x7f8fa4195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa931f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4100240 0x7f8fa4195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34434/0 (socket says 172.21.15.90:34434) 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa931f700 1 -- 172.21.15.90:0/2935918696 learned_addr learned my addr 172.21.15.90:0/2935918696 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa8b1e700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fa4101440 0x7f8fa4195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa931f700 1 -- 172.21.15.90:0/2935918696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fa4101440 msgr2=0x7f8fa4195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa931f700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fa4101440 0x7f8fa4195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa931f700 1 -- 172.21.15.90:0/2935918696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f940095f0 con 0x7f8fa4100240 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fa931f700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4100240 0x7f8fa4195ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000d900 tx=0x7f8fa000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8f9a7fc700 1 -- 172.21.15.90:0/2935918696 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fa00041f0 con 0x7f8fa4100240 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8f9a7fc700 1 -- 172.21.15.90:0/2935918696 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f8fa0004350 con 0x7f8fa4100240 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fa419b200 con 0x7f8fa4100240 2024-04-26T04:45:04.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8f9a7fc700 1 -- 172.21.15.90:0/2935918696 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8fa0003d20 con 0x7f8fa4100240 2024-04-26T04:45:04.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.287+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fa419b750 con 0x7f8fa4100240 2024-04-26T04:45:04.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.288+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fa4108d90 con 0x7f8fa4100240 2024-04-26T04:45:04.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.289+0000 7f8f9a7fc700 1 -- 172.21.15.90:0/2935918696 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f8fa0009730 con 0x7f8fa4100240 2024-04-26T04:45:04.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.289+0000 7f8f9a7fc700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8f9006dfc0 0x7f8f90070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:04.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.289+0000 7f8f9a7fc700 1 -- 172.21.15.90:0/2935918696 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8fa008dd20 con 0x7f8fa4100240 2024-04-26T04:45:04.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.289+0000 7f8fa8b1e700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8f9006dfc0 0x7f8f90070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:04.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.290+0000 7f8fa8b1e700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8f9006dfc0 0x7f8f90070470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8f94006010 tx=0x7f8f9400b540 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:04.295 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.294+0000 7f8f9a7fc700 1 -- 172.21.15.90:0/2935918696 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f8fa005ae40 con 0x7f8fa4100240 2024-04-26T04:45:04.529 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.528+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8fa4105d90 con 0x7f8f9006dfc0 2024-04-26T04:45:04.530 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.529+0000 7f8f9a7fc700 1 -- 172.21.15.90:0/2935918696 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+367 (secure 0 0 0) 0x7f8fa4105d90 con 0x7f8f9006dfc0 2024-04-26T04:45:04.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8f9006dfc0 msgr2=0x7f8f90070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8f9006dfc0 0x7f8f90070470 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f8f94006010 tx=0x7f8f9400b540 comp rx=0 tx=0).stop 2024-04-26T04:45:04.533 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4100240 msgr2=0x7f8fa4195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4100240 0x7f8fa4195ab0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f8fa000d900 tx=0x7f8fa000dc10 comp rx=0 tx=0).stop 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 shutdown_connections 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8f9006dfc0 0x7f8f90070470 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8fa4100240 0x7f8fa4195ab0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 --2- 172.21.15.90:0/2935918696 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8fa4101440 0x7f8fa4195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 >> 172.21.15.90:0/2935918696 conn(0x7f8fa40fb7f0 msgr2=0x7f8fa4104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 shutdown_connections 2024-04-26T04:45:04.534 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.532+0000 7f8fab583700 1 -- 172.21.15.90:0/2935918696 wait complete. 2024-04-26T04:45:04.544 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:45:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 -- 172.21.15.90:0/203756644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908103680 msgr2=0x7f9908105a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 --2- 172.21.15.90:0/203756644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908103680 0x7f9908105a60 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f98f8009910 tx=0x7f98f8009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 -- 172.21.15.90:0/203756644 shutdown_connections 2024-04-26T04:45:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 --2- 172.21.15.90:0/203756644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908103680 0x7f9908105a60 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 --2- 172.21.15.90:0/203756644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9908100d60 0x7f9908103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 -- 172.21.15.90:0/203756644 >> 172.21.15.90:0/203756644 conn(0x7f99080fa780 msgr2=0x7f99080fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:04.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 -- 172.21.15.90:0/203756644 shutdown_connections 2024-04-26T04:45:04.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 -- 172.21.15.90:0/203756644 wait complete. 2024-04-26T04:45:04.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 Processor -- start 2024-04-26T04:45:04.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.639+0000 7f990ff35700 1 -- start start 2024-04-26T04:45:04.642 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990ff35700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908100d60 0x7f9908195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990ff35700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9908103680 0x7f9908195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990ff35700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99081965d0 con 0x7f9908100d60 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990ff35700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9908196710 con 0x7f9908103680 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990dcd1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908100d60 0x7f9908195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990dcd1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908100d60 0x7f9908195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34464/0 (socket says 172.21.15.90:34464) 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990dcd1700 1 -- 172.21.15.90:0/1708296639 learned_addr learned my addr 172.21.15.90:0/1708296639 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990d4d0700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9908103680 0x7f9908195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990dcd1700 1 -- 172.21.15.90:0/1708296639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9908103680 msgr2=0x7f9908195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990dcd1700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9908103680 0x7f9908195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990dcd1700 1 -- 172.21.15.90:0/1708296639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f80095f0 con 0x7f9908100d60 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990dcd1700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908100d60 0x7f9908195a70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9904007e60 tx=0x7f990400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f98feffd700 1 -- 172.21.15.90:0/1708296639 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f990400dda0 con 0x7f9908100d60 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f98feffd700 1 -- 172.21.15.90:0/1708296639 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f990400f040 con 0x7f9908100d60 2024-04-26T04:45:04.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f990819b1c0 con 0x7f9908100d60 2024-04-26T04:45:04.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f98feffd700 1 -- 172.21.15.90:0/1708296639 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9904004580 con 0x7f9908100d60 2024-04-26T04:45:04.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.640+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990819b710 con 0x7f9908100d60 2024-04-26T04:45:04.644 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.641+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990818fbe0 con 0x7f9908100d60 2024-04-26T04:45:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.642+0000 7f98feffd700 1 -- 172.21.15.90:0/1708296639 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f9904003a00 con 0x7f9908100d60 2024-04-26T04:45:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.642+0000 7f98feffd700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f98f406dfc0 0x7f98f4070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.642+0000 7f98feffd700 1 -- 172.21.15.90:0/1708296639 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f990408d9c0 con 0x7f9908100d60 2024-04-26T04:45:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.643+0000 7f990d4d0700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f98f406dfc0 0x7f98f4070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:04.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.643+0000 7f990d4d0700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f98f406dfc0 0x7f98f4070470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006010 tx=0x7f98f800b580 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:04.648 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.647+0000 7f98feffd700 1 -- 172.21.15.90:0/1708296639 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f990405aae0 con 0x7f9908100d60 2024-04-26T04:45:04.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.884+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f9908061190 con 0x7f98f406dfc0 2024-04-26T04:45:04.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.885+0000 7f98feffd700 1 -- 172.21.15.90:0/1708296639 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f9908061190 con 0x7f98f406dfc0 2024-04-26T04:45:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f98f406dfc0 msgr2=0x7f98f4070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f98f406dfc0 0x7f98f4070470 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f98f8006010 tx=0x7f98f800b580 comp rx=0 tx=0).stop 2024-04-26T04:45:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908100d60 msgr2=0x7f9908195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908100d60 0x7f9908195a70 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9904007e60 tx=0x7f990400d3b0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 shutdown_connections 2024-04-26T04:45:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f98f406dfc0 0x7f98f4070470 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9908100d60 0x7f9908195a70 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 --2- 172.21.15.90:0/1708296639 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9908103680 0x7f9908195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 >> 172.21.15.90:0/1708296639 conn(0x7f99080fa780 msgr2=0x7f99080fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 shutdown_connections 2024-04-26T04:45:04.891 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.888+0000 7f990ff35700 1 -- 172.21.15.90:0/1708296639 wait complete. 2024-04-26T04:45:04.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 -- 172.21.15.90:0/615086446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac100730 msgr2=0x7fabac100b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:04.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 --2- 172.21.15.90:0/615086446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac100730 0x7fabac100b40 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7faba8009910 tx=0x7faba8009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:04.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 -- 172.21.15.90:0/615086446 shutdown_connections 2024-04-26T04:45:04.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 --2- 172.21.15.90:0/615086446 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabac0ff430 0x7fabac0ff8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 --2- 172.21.15.90:0/615086446 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac100730 0x7fabac100b40 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:04.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 -- 172.21.15.90:0/615086446 >> 172.21.15.90:0/615086446 conn(0x7fabac0f9f10 msgr2=0x7fabac0fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:04.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 -- 172.21.15.90:0/615086446 shutdown_connections 2024-04-26T04:45:04.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.996+0000 7fabb3f92700 1 -- 172.21.15.90:0/615086446 wait complete. 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb3f92700 1 Processor -- start 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb3f92700 1 -- start start 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb3f92700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac0ff430 0x7fabac195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb3f92700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabac100730 0x7fabac196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb3f92700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac196660 con 0x7fabac0ff430 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb3f92700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabac1967a0 con 0x7fabac100730 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb1d2e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac0ff430 0x7fabac195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb1d2e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac0ff430 0x7fabac195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34474/0 (socket says 172.21.15.90:34474) 2024-04-26T04:45:05.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb1d2e700 1 -- 172.21.15.90:0/1399245990 learned_addr learned my addr 172.21.15.90:0/1399245990 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.997+0000 7fabb1d2e700 1 -- 172.21.15.90:0/1399245990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabac100730 msgr2=0x7fabac196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7fabb1d2e700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabac100730 0x7fabac196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7fabb1d2e700 1 -- 172.21.15.90:0/1399245990 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80095f0 con 0x7fabac0ff430 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7fabb1d2e700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac0ff430 0x7fabac195b00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faba8006010 tx=0x7faba800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7faba2ffd700 1 -- 172.21.15.90:0/1399245990 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faba801d070 con 0x7fabac0ff430 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7faba2ffd700 1 -- 172.21.15.90:0/1399245990 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faba800f460 con 0x7fabac0ff430 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabac19b1a0 con 0x7fabac0ff430 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7faba2ffd700 1 -- 172.21.15.90:0/1399245990 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faba8017630 con 0x7fabac0ff430 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.998+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabac19b690 con 0x7fabac0ff430 2024-04-26T04:45:05.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.999+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabac107bf0 con 0x7fabac0ff430 2024-04-26T04:45:05.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:04.999+0000 7faba2ffd700 1 -- 172.21.15.90:0/1399245990 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7faba800f5d0 con 0x7fabac0ff430 2024-04-26T04:45:05.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.000+0000 7faba2ffd700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fab9806dfc0 0x7fab98070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.000+0000 7faba2ffd700 1 -- 172.21.15.90:0/1399245990 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faba808f310 con 0x7fabac0ff430 2024-04-26T04:45:05.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.000+0000 7fabb152d700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fab9806dfc0 0x7fab98070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.000+0000 7fabb152d700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fab9806dfc0 0x7fab98070470 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fab9c005950 tx=0x7fab9c0058e0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:05.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.005+0000 7faba2ffd700 1 -- 172.21.15.90:0/1399245990 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7faba805c430 con 0x7fabac0ff430 2024-04-26T04:45:05.227 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.226+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fabac0fe9f0 con 0x7fab9806dfc0 2024-04-26T04:45:05.237 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.236+0000 7faba2ffd700 1 -- 172.21.15.90:0/1399245990 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fabac0fe9f0 con 0x7fab9806dfc0 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (9m) 5s ago 10m 25.8M - 0.25.0 c8568f914cd2 e521169689de 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (10m) 5s ago 10m 7852k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (9m) 24s ago 9m 7742k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (10m) 5s ago 10m 6665k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (9m) 24s ago 9m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (9m) 5s ago 10m 84.1M - 9.4.7 954c08fa6188 1434edac0da4 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (5m) 5s ago 5m 130M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (5m) 5s ago 5m 14.4M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:45:05.238 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (5m) 24s ago 5m 15.0M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (5m) 24s ago 5m 131M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (75s) 5s ago 12m 499M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (44s) 24s ago 9m 447M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (12m) 5s ago 12m 60.3M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (9m) 24s ago 9m 42.5M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (31s) 5s ago 10m 8300k - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (27s) 24s ago 9m 3481k - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (8m) 5s ago 8m 228M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (8m) 5s ago 8m 245M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (7m) 5s ago 7m 201M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (7m) 24s ago 7m 269M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (7m) 24s ago 7m 234M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (6m) 24s ago 6m 239M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:45:05.239 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (10s) 5s ago 10m 40.9M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:45:05.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.238+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fab9806dfc0 msgr2=0x7fab98070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.238+0000 7fabb3f92700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fab9806dfc0 0x7fab98070470 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fab9c005950 tx=0x7fab9c0058e0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.238+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac0ff430 msgr2=0x7fabac195b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.238+0000 7fabb3f92700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac0ff430 0x7fabac195b00 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7faba8006010 tx=0x7faba800be20 comp rx=0 tx=0).stop 2024-04-26T04:45:05.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.239+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 shutdown_connections 2024-04-26T04:45:05.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.239+0000 7fabb3f92700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fab9806dfc0 0x7fab98070470 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.239+0000 7fabb3f92700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fabac0ff430 0x7fabac195b00 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.239+0000 7fabb3f92700 1 --2- 172.21.15.90:0/1399245990 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fabac100730 0x7fabac196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.239+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 >> 172.21.15.90:0/1399245990 conn(0x7fabac0f9f10 msgr2=0x7fabac0fe2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:05.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.239+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 shutdown_connections 2024-04-26T04:45:05.241 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.239+0000 7fabb3f92700 1 -- 172.21.15.90:0/1399245990 wait complete. 2024-04-26T04:45:05.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 -- 172.21.15.90:0/764384151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 msgr2=0x7faeb00ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 --2- 172.21.15.90:0/764384151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb00ff540 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7faea0009910 tx=0x7faea0009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 -- 172.21.15.90:0/764384151 shutdown_connections 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 --2- 172.21.15.90:0/764384151 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faeb01003d0 0x7faeb0100840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 --2- 172.21.15.90:0/764384151 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb00ff540 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 -- 172.21.15.90:0/764384151 >> 172.21.15.90:0/764384151 conn(0x7faeb00fa780 msgr2=0x7faeb00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 -- 172.21.15.90:0/764384151 shutdown_connections 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 -- 172.21.15.90:0/764384151 wait complete. 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.345+0000 7faeb5561700 1 Processor -- start 2024-04-26T04:45:05.347 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeb5561700 1 -- start start 2024-04-26T04:45:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeb5561700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb0195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeb5561700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faeb01003d0 0x7faeb0195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeb5561700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb0196600 con 0x7faeb00ff130 2024-04-26T04:45:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeb5561700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faeb0196740 con 0x7faeb01003d0 2024-04-26T04:45:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeaeffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb0195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeaeffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb0195aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34494/0 (socket says 172.21.15.90:34494) 2024-04-26T04:45:05.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeaeffd700 1 -- 172.21.15.90:0/4121247304 learned_addr learned my addr 172.21.15.90:0/4121247304 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faea67fc700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faeb01003d0 0x7faeb0195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeaeffd700 1 -- 172.21.15.90:0/4121247304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faeb01003d0 msgr2=0x7faeb0195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeaeffd700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faeb01003d0 0x7faeb0195fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeaeffd700 1 -- 172.21.15.90:0/4121247304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faea00095f0 con 0x7faeb00ff130 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeaeffd700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb0195aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faea0000c00 tx=0x7faea000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeacff9700 1 -- 172.21.15.90:0/4121247304 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faea001d070 con 0x7faeb00ff130 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.346+0000 7faeacff9700 1 -- 172.21.15.90:0/4121247304 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7faea0005dc0 con 0x7faeb00ff130 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.347+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faeb019b190 con 0x7faeb00ff130 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.347+0000 7faeacff9700 1 -- 172.21.15.90:0/4121247304 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faea00226a0 con 0x7faeb00ff130 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.347+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faeb019b680 con 0x7faeb00ff130 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.347+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faeb0107c10 con 0x7faeb00ff130 2024-04-26T04:45:05.350 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.348+0000 7faeacff9700 1 -- 172.21.15.90:0/4121247304 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7faea0003680 con 0x7faeb00ff130 2024-04-26T04:45:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.348+0000 7faeacff9700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fae9c06dfc0 0x7fae9c070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.349+0000 7faeacff9700 1 -- 172.21.15.90:0/4121247304 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7faea008f3e0 con 0x7faeb00ff130 2024-04-26T04:45:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.349+0000 7faea67fc700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fae9c06dfc0 0x7fae9c070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.349+0000 7faea67fc700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fae9c06dfc0 0x7fae9c070470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fae98009b30 tx=0x7fae98009260 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:05.354 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.353+0000 7faeacff9700 1 -- 172.21.15.90:0/4121247304 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7faea005c500 con 0x7faeb00ff130 2024-04-26T04:45:05.656 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.655+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7faeb019c180 con 0x7faeb00ff130 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.656+0000 7faeacff9700 1 -- 172.21.15.90:0/4121247304 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7faea0003930 con 0x7faeb00ff130 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:45:05.657 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:45:05.658 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fae9c06dfc0 msgr2=0x7fae9c070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fae9c06dfc0 0x7fae9c070470 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fae98009b30 tx=0x7fae98009260 comp rx=0 tx=0).stop 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 msgr2=0x7faeb0195aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb0195aa0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7faea0000c00 tx=0x7faea000f740 comp rx=0 tx=0).stop 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 shutdown_connections 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fae9c06dfc0 0x7fae9c070470 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faeb00ff130 0x7faeb0195aa0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 --2- 172.21.15.90:0/4121247304 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faeb01003d0 0x7faeb0195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 >> 172.21.15.90:0/4121247304 conn(0x7faeb00fa780 msgr2=0x7faeb0068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 shutdown_connections 2024-04-26T04:45:05.659 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.658+0000 7faeb5561700 1 -- 172.21.15.90:0/4121247304 wait complete. 2024-04-26T04:45:05.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: pgmap v35: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:05.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.763 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:05 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 -- 172.21.15.90:0/4218959766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 msgr2=0x7f6698100840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 --2- 172.21.15.90:0/4218959766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698100840 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6688009910 tx=0x7f6688009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 -- 172.21.15.90:0/4218959766 shutdown_connections 2024-04-26T04:45:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 --2- 172.21.15.90:0/4218959766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698100840 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 --2- 172.21.15.90:0/4218959766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66980ff130 0x7f66980ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 -- 172.21.15.90:0/4218959766 >> 172.21.15.90:0/4218959766 conn(0x7f66980fa780 msgr2=0x7f66980fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:05.764 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 -- 172.21.15.90:0/4218959766 shutdown_connections 2024-04-26T04:45:05.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.763+0000 7f66a024b700 1 -- 172.21.15.90:0/4218959766 wait complete. 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: pgmap v35: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='client.14762 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-04-26T04:45:05.765 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.766 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:05 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:05.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f66a024b700 1 Processor -- start 2024-04-26T04:45:05.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f66a024b700 1 -- start start 2024-04-26T04:45:05.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f66a024b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66980ff130 0x7f6698195a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.768 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f66a024b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698195fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f66a024b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66981965c0 con 0x7f66981003d0 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f66a024b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6698196700 con 0x7f66980ff130 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669d7e6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698195fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669d7e6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698195fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:34524/0 (socket says 172.21.15.90:34524) 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669d7e6700 1 -- 172.21.15.90:0/631125490 learned_addr learned my addr 172.21.15.90:0/631125490 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669d7e6700 1 -- 172.21.15.90:0/631125490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66980ff130 msgr2=0x7f6698195a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669dfe7700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66980ff130 0x7f6698195a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669d7e6700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66980ff130 0x7f6698195a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669d7e6700 1 -- 172.21.15.90:0/631125490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66880095f0 con 0x7f66981003d0 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.764+0000 7f669dfe7700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66980ff130 0x7f6698195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.765+0000 7f669d7e6700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698195fa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6688006010 tx=0x7f668800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.765+0000 7f668effd700 1 -- 172.21.15.90:0/631125490 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f668801d070 con 0x7f66981003d0 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.765+0000 7f668effd700 1 -- 172.21.15.90:0/631125490 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f668800f460 con 0x7f66981003d0 2024-04-26T04:45:05.769 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.765+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f669819b150 con 0x7f66981003d0 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.765+0000 7f668effd700 1 -- 172.21.15.90:0/631125490 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6688017630 con 0x7f66981003d0 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.765+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f669819b640 con 0x7f66981003d0 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.766+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6698107c10 con 0x7f66981003d0 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.766+0000 7f668effd700 1 -- 172.21.15.90:0/631125490 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 31) v1 ==== 92158+0+0 (secure 0 0 0) 0x7f668800f5d0 con 0x7f66981003d0 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.767+0000 7f668effd700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f668406dfc0 0x7f6684070470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.767+0000 7f669dfe7700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f668406dfc0 0x7f6684070470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.767+0000 7f668effd700 1 -- 172.21.15.90:0/631125490 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f668808f310 con 0x7f66981003d0 2024-04-26T04:45:05.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.767+0000 7f669dfe7700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f668406dfc0 0x7f6684070470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6694005fd0 tx=0x7f6694005ee0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:05.772 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.771+0000 7f668effd700 1 -- 172.21.15.90:0/631125490 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f668805c430 con 0x7f66981003d0 2024-04-26T04:45:05.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.993+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f6698068c50 con 0x7f668406dfc0 2024-04-26T04:45:05.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.994+0000 7f668effd700 1 -- 172.21.15.90:0/631125490 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+369 (secure 0 0 0) 0x7f6698068c50 con 0x7f668406dfc0 2024-04-26T04:45:05.995 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:45:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:45:05.995 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading alertmanager daemons", 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:45:05.996 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:45:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f668406dfc0 msgr2=0x7f6684070470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f668406dfc0 0x7f6684070470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6694005fd0 tx=0x7f6694005ee0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 msgr2=0x7f6698195fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698195fa0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f6688006010 tx=0x7f668800be20 comp rx=0 tx=0).stop 2024-04-26T04:45:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 shutdown_connections 2024-04-26T04:45:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f668406dfc0 0x7f6684070470 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66980ff130 0x7f6698195a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 --2- 172.21.15.90:0/631125490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66981003d0 0x7f6698195fa0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.996+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 >> 172.21.15.90:0/631125490 conn(0x7f66980fa780 msgr2=0x7f6698068540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.997+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 shutdown_connections 2024-04-26T04:45:05.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:05.997+0000 7f66a024b700 1 -- 172.21.15.90:0/631125490 wait complete. 2024-04-26T04:45:06.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:06 smithi090 ceph-mon[26648]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:06.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:06 smithi090 ceph-mon[26648]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:06.792 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:06 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/4121247304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:06.930 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:06 smithi104 ceph-mon[32369]: from='client.14766 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:06.930 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:06 smithi104 ceph-mon[32369]: from='client.14770 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:06.930 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:06 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/4121247304' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:07.852 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:07 smithi104 ceph-mon[32369]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:07.852 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:07 smithi104 ceph-mon[32369]: pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:07.852 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:07 smithi104 ceph-mon[32369]: Upgrade: Updating alertmanager.smithi090 2024-04-26T04:45:07.852 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:07 smithi104 ceph-mon[32369]: Deploying daemon alertmanager.smithi090 on smithi090 2024-04-26T04:45:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:07 smithi090 ceph-mon[26648]: from='client.14778 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:07 smithi090 ceph-mon[26648]: pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:07 smithi090 ceph-mon[26648]: Upgrade: Updating alertmanager.smithi090 2024-04-26T04:45:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:07 smithi090 ceph-mon[26648]: Deploying daemon alertmanager.smithi090 on smithi090 2024-04-26T04:45:09.699 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:09 smithi090 ceph-mon[26648]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:10.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:09 smithi104 ceph-mon[32369]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:11.660 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:11 smithi090 ceph-mon[26648]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:12.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:11 smithi104 ceph-mon[32369]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:12.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:12 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:12.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:12 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:12.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:12 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:12.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:12 smithi090 ceph-mon[26648]: pgmap v39: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:12.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:12 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:12.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:12 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:12 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:12 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:12 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:12 smithi104 ceph-mon[32369]: pgmap v39: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:12 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:12 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:15.658 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:15 smithi090 ceph-mon[26648]: pgmap v40: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:15.658 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:15 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:15.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:15 smithi104 ceph-mon[32369]: pgmap v40: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:15.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:15 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:17.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:17 smithi090 ceph-mon[26648]: pgmap v41: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:17.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:17.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:17 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:17.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:17 smithi104 ceph-mon[32369]: pgmap v41: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:17.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:17 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:18 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:18 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:18 smithi090 ceph-mon[26648]: pgmap v42: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:18 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:19.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:18 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:19.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:18 smithi104 ceph-mon[32369]: pgmap v42: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:21.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:21 smithi090 ceph-mon[26648]: pgmap v43: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:21.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:21 smithi104 ceph-mon[32369]: pgmap v43: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:22.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:22 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:22.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:22 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:23.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:23 smithi104 ceph-mon[32369]: pgmap v44: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:23.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:23 smithi090 ceph-mon[26648]: pgmap v44: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:24.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:24 smithi090 ceph-mon[26648]: Upgrade: Updating grafana.smithi090 2024-04-26T04:45:24.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:24 smithi090 ceph-mon[26648]: Deploying daemon grafana.smithi090 on smithi090 2024-04-26T04:45:24.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:24 smithi104 ceph-mon[32369]: Upgrade: Updating grafana.smithi090 2024-04-26T04:45:24.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:24 smithi104 ceph-mon[32369]: Deploying daemon grafana.smithi090 on smithi090 2024-04-26T04:45:25.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:25 smithi090 ceph-mon[26648]: pgmap v45: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:25.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:25 smithi104 ceph-mon[32369]: pgmap v45: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:27.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:27 smithi090 ceph-mon[26648]: pgmap v46: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:27.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:27 smithi104 ceph-mon[32369]: pgmap v46: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:29.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:29 smithi090 ceph-mon[26648]: pgmap v47: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:29.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:29 smithi104 ceph-mon[32369]: pgmap v47: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:30.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:30.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:31.441 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:31 smithi090 ceph-mon[26648]: pgmap v48: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:31.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:31 smithi104 ceph-mon[32369]: pgmap v48: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:32.624 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:32 smithi090 ceph-mon[26648]: mgrmap e32: smithi090.iwrvwa(active, since 91s), standbys: smithi104.wwoewi 2024-04-26T04:45:32.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:32 smithi104 ceph-mon[32369]: mgrmap e32: smithi090.iwrvwa(active, since 91s), standbys: smithi104.wwoewi 2024-04-26T04:45:33.557 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:33 smithi090 ceph-mon[26648]: pgmap v49: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:33.703 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:33 smithi104 ceph-mon[32369]: pgmap v49: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:35.569 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:35 smithi090 ceph-mon[26648]: pgmap v50: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:35 smithi104 ceph-mon[32369]: pgmap v50: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:36.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.119+0000 7fc0f509e700 1 -- 172.21.15.90:0/3936869763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0101450 msgr2=0x7fc0f01018a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.119+0000 7fc0f509e700 1 --2- 172.21.15.90:0/3936869763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0101450 0x7fc0f01018a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fc0e4009910 tx=0x7fc0e4009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:36.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.119+0000 7fc0f509e700 1 -- 172.21.15.90:0/3936869763 shutdown_connections 2024-04-26T04:45:36.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.119+0000 7fc0f509e700 1 --2- 172.21.15.90:0/3936869763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0101450 0x7fc0f01018a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.119+0000 7fc0f509e700 1 --2- 172.21.15.90:0/3936869763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f0100250 0x7fc0f0100660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.119+0000 7fc0f509e700 1 -- 172.21.15.90:0/3936869763 >> 172.21.15.90:0/3936869763 conn(0x7fc0f00fb7e0 msgr2=0x7fc0f00fdc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:36.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.120+0000 7fc0f509e700 1 -- 172.21.15.90:0/3936869763 shutdown_connections 2024-04-26T04:45:36.122 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.120+0000 7fc0f509e700 1 -- 172.21.15.90:0/3936869763 wait complete. 2024-04-26T04:45:36.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.121+0000 7fc0f509e700 1 Processor -- start 2024-04-26T04:45:36.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.121+0000 7fc0f509e700 1 -- start start 2024-04-26T04:45:36.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0f509e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0100250 0x7fc0f0195a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0f509e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f0101450 0x7fc0f0195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0f509e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01965d0 con 0x7fc0f0100250 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0f509e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f0196710 con 0x7fc0f0101450 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0eed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0100250 0x7fc0f0195a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0eed9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0100250 0x7fc0f0195a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46606/0 (socket says 172.21.15.90:46606) 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0ee59c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f0101450 0x7fc0f0195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0eed9d700 1 -- 172.21.15.90:0/1479783340 learned_addr learned my addr 172.21.15.90:0/1479783340 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0eed9d700 1 -- 172.21.15.90:0/1479783340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f0101450 msgr2=0x7fc0f0195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0eed9d700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f0101450 0x7fc0f0195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0eed9d700 1 -- 172.21.15.90:0/1479783340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40095f0 con 0x7fc0f0100250 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.122+0000 7fc0ee59c700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f0101450 0x7fc0f0195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:45:36.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.123+0000 7fc0eed9d700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0100250 0x7fc0f0195a70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000d900 tx=0x7fc0e000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.123+0000 7fc0d7fff700 1 -- 172.21.15.90:0/1479783340 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e000b920 con 0x7fc0f0100250 2024-04-26T04:45:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.123+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f0072fe0 con 0x7fc0f0100250 2024-04-26T04:45:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.123+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f0073530 con 0x7fc0f0100250 2024-04-26T04:45:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.123+0000 7fc0d7fff700 1 -- 172.21.15.90:0/1479783340 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fc0e0010460 con 0x7fc0f0100250 2024-04-26T04:45:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.123+0000 7fc0d7fff700 1 -- 172.21.15.90:0/1479783340 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc0e000f590 con 0x7fc0f0100250 2024-04-26T04:45:36.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.124+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0dc005320 con 0x7fc0f0100250 2024-04-26T04:45:36.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.125+0000 7fc0d7fff700 1 -- 172.21.15.90:0/1479783340 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fc0e00105d0 con 0x7fc0f0100250 2024-04-26T04:45:36.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.126+0000 7fc0d7fff700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc0d806de80 0x7fc0d8070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.126+0000 7fc0ee59c700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc0d806de80 0x7fc0d8070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.126+0000 7fc0d7fff700 1 -- 172.21.15.90:0/1479783340 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc0e008da30 con 0x7fc0f0100250 2024-04-26T04:45:36.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.126+0000 7fc0ee59c700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc0d806de80 0x7fc0d8070330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400b5c0 tx=0x7fc0e4005fd0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:36.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.131+0000 7fc0d7fff700 1 -- 172.21.15.90:0/1479783340 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fc0e005adf0 con 0x7fc0f0100250 2024-04-26T04:45:36.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.390+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc0dc000bf0 con 0x7fc0d806de80 2024-04-26T04:45:36.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.391+0000 7fc0d7fff700 1 -- 172.21.15.90:0/1479783340 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7fc0dc000bf0 con 0x7fc0d806de80 2024-04-26T04:45:36.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc0d806de80 msgr2=0x7fc0d8070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc0d806de80 0x7fc0d8070330 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400b5c0 tx=0x7fc0e4005fd0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0100250 msgr2=0x7fc0f0195a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0100250 0x7fc0f0195a70 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc0e000d900 tx=0x7fc0e000dc10 comp rx=0 tx=0).stop 2024-04-26T04:45:36.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 shutdown_connections 2024-04-26T04:45:36.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc0d806de80 0x7fc0d8070330 secure :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc0e400b5c0 tx=0x7fc0e4005fd0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc0f0100250 0x7fc0f0195a70 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 --2- 172.21.15.90:0/1479783340 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc0f0101450 0x7fc0f0195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.399+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 >> 172.21.15.90:0/1479783340 conn(0x7fc0f00fb7e0 msgr2=0x7fc0f0104680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:36.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.400+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 shutdown_connections 2024-04-26T04:45:36.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.400+0000 7fc0f509e700 1 -- 172.21.15.90:0/1479783340 wait complete. 2024-04-26T04:45:36.412 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:45:36.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.515+0000 7f4207d68700 1 -- 172.21.15.90:0/382142677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 msgr2=0x7f42001030c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.515+0000 7f4207d68700 1 --2- 172.21.15.90:0/382142677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f42001030c0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f41fc009910 tx=0x7f41fc009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.515+0000 7f4207d68700 1 -- 172.21.15.90:0/382142677 shutdown_connections 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.515+0000 7f4207d68700 1 --2- 172.21.15.90:0/382142677 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4200103600 0x7f42001059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 --2- 172.21.15.90:0/382142677 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f42001030c0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 -- 172.21.15.90:0/382142677 >> 172.21.15.90:0/382142677 conn(0x7f42000fa700 msgr2=0x7f42000fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 -- 172.21.15.90:0/382142677 shutdown_connections 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 -- 172.21.15.90:0/382142677 wait complete. 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 Processor -- start 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 -- start start 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f4200195ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4200103600 0x7f4200195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4200196610 con 0x7f4200100ce0 2024-04-26T04:45:36.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.516+0000 7f4207d68700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4200196750 con 0x7f4200103600 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205b04700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f4200195ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205b04700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f4200195ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46624/0 (socket says 172.21.15.90:46624) 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205b04700 1 -- 172.21.15.90:0/2245034349 learned_addr learned my addr 172.21.15.90:0/2245034349 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205303700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4200103600 0x7f4200195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205b04700 1 -- 172.21.15.90:0/2245034349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4200103600 msgr2=0x7f4200195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205b04700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4200103600 0x7f4200195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205b04700 1 -- 172.21.15.90:0/2245034349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41fc0095f0 con 0x7f4200100ce0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205303700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4200103600 0x7f4200195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4205b04700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f4200195ab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00bd70 tx=0x7f41fc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f41f6ffd700 1 -- 172.21.15.90:0/2245034349 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41fc01d070 con 0x7f4200100ce0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f420019b1a0 con 0x7f4200100ce0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f420019b690 con 0x7f4200100ce0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f41f6ffd700 1 -- 172.21.15.90:0/2245034349 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f41fc022470 con 0x7f4200100ce0 2024-04-26T04:45:36.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.517+0000 7f41f6ffd700 1 -- 172.21.15.90:0/2245034349 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f41fc00f690 con 0x7f4200100ce0 2024-04-26T04:45:36.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.518+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f420018fc70 con 0x7f4200100ce0 2024-04-26T04:45:36.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.519+0000 7f41f6ffd700 1 -- 172.21.15.90:0/2245034349 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f41fc022a30 con 0x7f4200100ce0 2024-04-26T04:45:36.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.519+0000 7f41f6ffd700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f41ec072290 0x7f41ec074740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.519+0000 7f41f6ffd700 1 -- 172.21.15.90:0/2245034349 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f41fc08f0a0 con 0x7f4200100ce0 2024-04-26T04:45:36.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.519+0000 7f4205303700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f41ec072290 0x7f41ec074740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.520+0000 7f4205303700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f41ec072290 0x7f41ec074740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005950 tx=0x7f41f00058e0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:36.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.524+0000 7f41f6ffd700 1 -- 172.21.15.90:0/2245034349 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f41fc05c630 con 0x7f4200100ce0 2024-04-26T04:45:36.762 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.761+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4200061190 con 0x7f41ec072290 2024-04-26T04:45:36.763 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.762+0000 7f41f6ffd700 1 -- 172.21.15.90:0/2245034349 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7f4200061190 con 0x7f41ec072290 2024-04-26T04:45:36.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f41ec072290 msgr2=0x7f41ec074740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f41ec072290 0x7f41ec074740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f41f0005950 tx=0x7f41f00058e0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 msgr2=0x7f4200195ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f4200195ab0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f41fc00bd70 tx=0x7f41fc00be50 comp rx=0 tx=0).stop 2024-04-26T04:45:36.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 shutdown_connections 2024-04-26T04:45:36.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f41ec072290 0x7f41ec074740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4200100ce0 0x7f4200195ab0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 --2- 172.21.15.90:0/2245034349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4200103600 0x7f4200195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 >> 172.21.15.90:0/2245034349 conn(0x7f42000fa700 msgr2=0x7f42000fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:36.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 shutdown_connections 2024-04-26T04:45:36.767 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.764+0000 7f4207d68700 1 -- 172.21.15.90:0/2245034349 wait complete. 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 -- 172.21.15.90:0/2062075196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c0ff140 msgr2=0x7f124c0ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 --2- 172.21.15.90:0/2062075196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c0ff140 0x7f124c0ff550 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1248009910 tx=0x7f1248009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 -- 172.21.15.90:0/2062075196 shutdown_connections 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 --2- 172.21.15.90:0/2062075196 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f124c100340 0x7f124c100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 --2- 172.21.15.90:0/2062075196 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c0ff140 0x7f124c0ff550 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 -- 172.21.15.90:0/2062075196 >> 172.21.15.90:0/2062075196 conn(0x7f124c0fa6d0 msgr2=0x7f124c0fcb20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 -- 172.21.15.90:0/2062075196 shutdown_connections 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 -- 172.21.15.90:0/2062075196 wait complete. 2024-04-26T04:45:36.872 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 Processor -- start 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.869+0000 7f125446c700 1 -- start start 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f125446c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f124c0ff140 0x7f124c1949d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f125446c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c100340 0x7f124c194f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f125446c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c195530 con 0x7f124c100340 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f125446c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124c195670 con 0x7f124c0ff140 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1251a07700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c100340 0x7f124c194f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1251a07700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c100340 0x7f124c194f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46638/0 (socket says 172.21.15.90:46638) 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1251a07700 1 -- 172.21.15.90:0/2808721937 learned_addr learned my addr 172.21.15.90:0/2808721937 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1252208700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f124c0ff140 0x7f124c1949d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1251a07700 1 -- 172.21.15.90:0/2808721937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f124c0ff140 msgr2=0x7f124c1949d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:36.873 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1251a07700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f124c0ff140 0x7f124c1949d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1251a07700 1 -- 172.21.15.90:0/2808721937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12480095f0 con 0x7f124c100340 2024-04-26T04:45:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f1251a07700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c100340 0x7f124c194f10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d8d0 tx=0x7f123c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f12437fe700 1 -- 172.21.15.90:0/2808721937 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123c0098a0 con 0x7f124c100340 2024-04-26T04:45:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f124c19a120 con 0x7f124c100340 2024-04-26T04:45:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f12437fe700 1 -- 172.21.15.90:0/2808721937 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7f123c010460 con 0x7f124c100340 2024-04-26T04:45:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f12437fe700 1 -- 172.21.15.90:0/2808721937 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f123c00f5f0 con 0x7f124c100340 2024-04-26T04:45:36.874 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.870+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f124c19a670 con 0x7f124c100340 2024-04-26T04:45:36.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.871+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f124c107c90 con 0x7f124c100340 2024-04-26T04:45:36.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.872+0000 7f12437fe700 1 -- 172.21.15.90:0/2808721937 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f123c009a00 con 0x7f124c100340 2024-04-26T04:45:36.875 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.872+0000 7f12437fe700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f123806de80 0x7f1238070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:36.875 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:36 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:36.876 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:36 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:36.876 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:36 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.872+0000 7f12437fe700 1 -- 172.21.15.90:0/2808721937 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f123c08dab0 con 0x7f124c100340 2024-04-26T04:45:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.873+0000 7f1252208700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f123806de80 0x7f1238070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:36.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.873+0000 7f1252208700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f123806de80 0x7f1238070330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1248006010 tx=0x7f124800b540 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:36.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:36.877+0000 7f12437fe700 1 -- 172.21.15.90:0/2808721937 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f123c05adf0 con 0x7f124c100340 2024-04-26T04:45:36.927 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:36 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:36.927 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:36 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:36.927 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:36 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:37.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.097+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f124c104c90 con 0x7f123806de80 2024-04-26T04:45:37.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.108+0000 7f12437fe700 1 -- 172.21.15.90:0/2808721937 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f124c104c90 con 0x7f123806de80 2024-04-26T04:45:37.110 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:45:37.110 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (25s) 20s ago 11m 17.0M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:45:37.110 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (11m) 20s ago 11m 7872k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:45:37.110 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (10m) 56s ago 10m 7742k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:45:37.110 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (11m) 20s ago 11m 6665k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:45:37.111 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (10m) 56s ago 10m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:45:37.111 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 starting - - - - 2024-04-26T04:45:37.111 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (5m) 20s ago 5m 130M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:45:37.111 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (5m) 20s ago 5m 14.5M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:45:37.111 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (5m) 56s ago 5m 15.0M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:45:37.111 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (5m) 56s ago 5m 131M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (107s) 20s ago 12m 500M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (75s) 56s ago 10m 447M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (12m) 20s ago 12m 60.6M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (10m) 56s ago 10m 42.5M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (63s) 20s ago 11m 8497k - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (59s) 56s ago 10m 3481k - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (9m) 20s ago 9m 228M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (8m) 20s ago 8m 245M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (8m) 20s ago 8m 201M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (8m) 56s ago 8m 269M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (7m) 56s ago 7m 234M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (7m) 56s ago 7m 239M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:45:37.112 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (42s) 20s ago 10m 50.6M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:45:37.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f123806de80 msgr2=0x7f1238070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f123806de80 0x7f1238070330 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1248006010 tx=0x7f124800b540 comp rx=0 tx=0).stop 2024-04-26T04:45:37.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c100340 msgr2=0x7f124c194f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c100340 0x7f124c194f10 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f123c00d8d0 tx=0x7f123c00dbe0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 shutdown_connections 2024-04-26T04:45:37.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f123806de80 0x7f1238070330 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.113 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f124c0ff140 0x7f124c1949d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 --2- 172.21.15.90:0/2808721937 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f124c100340 0x7f124c194f10 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 >> 172.21.15.90:0/2808721937 conn(0x7f124c0fa6d0 msgr2=0x7f124c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:37.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 shutdown_connections 2024-04-26T04:45:37.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.111+0000 7f125446c700 1 -- 172.21.15.90:0/2808721937 wait complete. 2024-04-26T04:45:37.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 -- 172.21.15.90:0/3580964285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 msgr2=0x7fccec1007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 --2- 172.21.15.90:0/3580964285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec1007a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fcce0009910 tx=0x7fcce0009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:37.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 -- 172.21.15.90:0/3580964285 shutdown_connections 2024-04-26T04:45:37.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 --2- 172.21.15.90:0/3580964285 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec1007a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 --2- 172.21.15.90:0/3580964285 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccec0ff150 0x7fccec0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 -- 172.21.15.90:0/3580964285 >> 172.21.15.90:0/3580964285 conn(0x7fccec0fa700 msgr2=0x7fccec0fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 -- 172.21.15.90:0/3580964285 shutdown_connections 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.215+0000 7fccf105b700 1 -- 172.21.15.90:0/3580964285 wait complete. 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccf105b700 1 Processor -- start 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccf105b700 1 -- start start 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccf105b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccec0ff150 0x7fccec1949a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccf105b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccf105b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec195500 con 0x7fccec100350 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccf105b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec195640 con 0x7fccec0ff150 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccea59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:37.219 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccead9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccec0ff150 0x7fccec1949a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccea59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:46652/0 (socket says 172.21.15.90:46652) 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccea59c700 1 -- 172.21.15.90:0/149284097 learned_addr learned my addr 172.21.15.90:0/149284097 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccea59c700 1 -- 172.21.15.90:0/149284097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccec0ff150 msgr2=0x7fccec1949a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccea59c700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccec0ff150 0x7fccec1949a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccea59c700 1 -- 172.21.15.90:0/149284097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00095f0 con 0x7fccec100350 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.216+0000 7fccead9d700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccec0ff150 0x7fccec1949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.217+0000 7fccea59c700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec194ee0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcce0000c00 tx=0x7fcce000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.217+0000 7fccd3fff700 1 -- 172.21.15.90:0/149284097 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce001d070 con 0x7fccec100350 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.217+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec19a090 con 0x7fccec100350 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.217+0000 7fccd3fff700 1 -- 172.21.15.90:0/149284097 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7fcce0005dc0 con 0x7fccec100350 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.217+0000 7fccd3fff700 1 -- 172.21.15.90:0/149284097 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcce00226a0 con 0x7fccec100350 2024-04-26T04:45:37.220 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.217+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec19a580 con 0x7fccec100350 2024-04-26T04:45:37.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.218+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec107ca0 con 0x7fccec100350 2024-04-26T04:45:37.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.218+0000 7fccd3fff700 1 -- 172.21.15.90:0/149284097 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fcce00038f0 con 0x7fccec100350 2024-04-26T04:45:37.221 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.219+0000 7fccd3fff700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fccd406de80 0x7fccd4070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:37.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.219+0000 7fccd3fff700 1 -- 172.21.15.90:0/149284097 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fcce008f0b0 con 0x7fccec100350 2024-04-26T04:45:37.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.219+0000 7fccead9d700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fccd406de80 0x7fccd4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:37.222 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.219+0000 7fccead9d700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fccd406de80 0x7fccd4070330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fccec1001b0 tx=0x7fcce400b040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:37.224 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.223+0000 7fccd3fff700 1 -- 172.21.15.90:0/149284097 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fcce005c3f0 con 0x7fccec100350 2024-04-26T04:45:37.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.522+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fccec107e90 con 0x7fccec100350 2024-04-26T04:45:37.524 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.523+0000 7fccd3fff700 1 -- 172.21.15.90:0/149284097 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fcce0003680 con 0x7fccec100350 2024-04-26T04:45:37.524 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:45:37.524 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:45:37.524 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:45:37.524 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:45:37.525 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:45:37.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fccd406de80 msgr2=0x7fccd4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fccd406de80 0x7fccd4070330 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fccec1001b0 tx=0x7fcce400b040 comp rx=0 tx=0).stop 2024-04-26T04:45:37.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 msgr2=0x7fccec194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec194ee0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fcce0000c00 tx=0x7fcce000f690 comp rx=0 tx=0).stop 2024-04-26T04:45:37.526 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 shutdown_connections 2024-04-26T04:45:37.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fccd406de80 0x7fccd4070330 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccec0ff150 0x7fccec1949a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 --2- 172.21.15.90:0/149284097 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccec100350 0x7fccec194ee0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 >> 172.21.15.90:0/149284097 conn(0x7fccec0fa700 msgr2=0x7fccec103580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:37.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 shutdown_connections 2024-04-26T04:45:37.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.525+0000 7fccf105b700 1 -- 172.21.15.90:0/149284097 wait complete. 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 -- 172.21.15.90:0/3622800678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 msgr2=0x7ff4080ff560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 --2- 172.21.15.90:0/3622800678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff4080ff560 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff3f8009910 tx=0x7ff3f8009c20 comp rx=0 tx=0).stop 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 -- 172.21.15.90:0/3622800678 shutdown_connections 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 --2- 172.21.15.90:0/3622800678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4081003f0 0x7ff408100860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 --2- 172.21.15.90:0/3622800678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff4080ff560 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 -- 172.21.15.90:0/3622800678 >> 172.21.15.90:0/3622800678 conn(0x7ff4080fa7a0 msgr2=0x7ff4080fcbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 -- 172.21.15.90:0/3622800678 shutdown_connections 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.630+0000 7ff40fd55700 1 -- 172.21.15.90:0/3622800678 wait complete. 2024-04-26T04:45:37.633 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40fd55700 1 Processor -- start 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40fd55700 1 -- start start 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40fd55700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff408069930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40fd55700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4081003f0 0x7ff40819cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40fd55700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4081969c0 con 0x7ff4080ff150 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40fd55700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff40819d490 con 0x7ff4081003f0 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40daf1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff408069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40daf1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff408069930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35232/0 (socket says 172.21.15.90:35232) 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40daf1700 1 -- 172.21.15.90:0/2227722613 learned_addr learned my addr 172.21.15.90:0/2227722613 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40d2f0700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4081003f0 0x7ff40819cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40daf1700 1 -- 172.21.15.90:0/2227722613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4081003f0 msgr2=0x7ff40819cf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.634 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40daf1700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4081003f0 0x7ff40819cf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40daf1700 1 -- 172.21.15.90:0/2227722613 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3f80095f0 con 0x7ff4080ff150 2024-04-26T04:45:37.635 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.631+0000 7ff40daf1700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff408069930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800bd70 tx=0x7ff3f800bda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:37.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.632+0000 7ff3feffd700 1 -- 172.21.15.90:0/2227722613 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f801d070 con 0x7ff4080ff150 2024-04-26T04:45:37.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.632+0000 7ff3feffd700 1 -- 172.21.15.90:0/2227722613 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1101+0+0 (secure 0 0 0) 0x7ff3f8005dc0 con 0x7ff4080ff150 2024-04-26T04:45:37.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.632+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff40819d6e0 con 0x7ff4080ff150 2024-04-26T04:45:37.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.632+0000 7ff3feffd700 1 -- 172.21.15.90:0/2227722613 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff3f8017400 con 0x7ff4080ff150 2024-04-26T04:45:37.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.632+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff40819dbd0 con 0x7ff4080ff150 2024-04-26T04:45:37.636 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.634+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff408068ab0 con 0x7ff4080ff150 2024-04-26T04:45:37.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.634+0000 7ff3feffd700 1 -- 172.21.15.90:0/2227722613 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7ff3f800f7d0 con 0x7ff4080ff150 2024-04-26T04:45:37.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.635+0000 7ff3feffd700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff3f406de80 0x7ff3f4070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:45:37.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.635+0000 7ff3feffd700 1 -- 172.21.15.90:0/2227722613 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7ff3f808f030 con 0x7ff4080ff150 2024-04-26T04:45:37.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.635+0000 7ff40d2f0700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff3f406de80 0x7ff3f4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:45:37.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.635+0000 7ff40d2f0700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff3f406de80 0x7ff3f4070330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff408069eb0 tx=0x7ff40400b410 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:45:37.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.639+0000 7ff3feffd700 1 -- 172.21.15.90:0/2227722613 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7ff3f805c370 con 0x7ff4080ff150 2024-04-26T04:45:37.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.859+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff408104e20 con 0x7ff3f406de80 2024-04-26T04:45:37.861 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:37 smithi090 ceph-mon[26648]: pgmap v51: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:37.861 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:37 smithi090 ceph-mon[26648]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:37.861 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:37 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/149284097' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:37.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.860+0000 7ff3feffd700 1 -- 172.21.15.90:0/2227722613 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+364 (secure 0 0 0) 0x7ff408104e20 con 0x7ff3f406de80 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading daemons of type(s) mgr", 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/2 daemons upgraded", 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading grafana daemons", 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:45:37.862 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff3f406de80 msgr2=0x7ff3f4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff3f406de80 0x7ff3f4070330 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7ff408069eb0 tx=0x7ff40400b410 comp rx=0 tx=0).stop 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 msgr2=0x7ff408069930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff408069930 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff3f800bd70 tx=0x7ff3f800bda0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 shutdown_connections 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7ff3f406de80 0x7ff3f4070330 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff4080ff150 0x7ff408069930 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 --2- 172.21.15.90:0/2227722613 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff4081003f0 0x7ff40819cf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.862+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 >> 172.21.15.90:0/2227722613 conn(0x7ff4080fa7a0 msgr2=0x7ff4081036c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.863+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 shutdown_connections 2024-04-26T04:45:37.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:45:37.863+0000 7ff40fd55700 1 -- 172.21.15.90:0/2227722613 wait complete. 2024-04-26T04:45:38.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:37 smithi104 ceph-mon[32369]: pgmap v51: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:38.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:37 smithi104 ceph-mon[32369]: from='client.14782 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:38.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:37 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/149284097' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:38.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:38 smithi090 ceph-mon[26648]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:38.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:38 smithi090 ceph-mon[26648]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:39.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:38 smithi104 ceph-mon[32369]: from='client.14786 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:39.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:38 smithi104 ceph-mon[32369]: from='client.14790 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:39.896 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:39 smithi090 ceph-mon[26648]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:39.896 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:39 smithi090 ceph-mon[26648]: pgmap v52: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:40.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:39 smithi104 ceph-mon[32369]: from='client.14798 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:45:40.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:39 smithi104 ceph-mon[32369]: pgmap v52: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:41 smithi090 ceph-mon[26648]: pgmap v53: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:41 smithi104 ceph-mon[32369]: pgmap v53: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:43.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:43 smithi090 ceph-mon[26648]: pgmap v54: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:43.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:43 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:43.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:43 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:43 smithi104 ceph-mon[32369]: pgmap v54: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 511 B/s wr, 0 op/s 2024-04-26T04:45:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:43 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:43.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:43 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:45.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:45 smithi090 ceph-mon[26648]: pgmap v55: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:45.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:45 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:45 smithi104 ceph-mon[32369]: pgmap v55: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:45.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:45 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:45:47.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:47 smithi090 ceph-mon[26648]: pgmap v56: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:47 smithi104 ceph-mon[32369]: pgmap v56: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:49.362 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:49 smithi090 ceph-mon[26648]: pgmap v57: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:49 smithi104 ceph-mon[32369]: pgmap v57: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:51.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:51.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:51.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:51.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:51.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-26T04:45:51.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:51.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:51 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:52.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:52.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:52.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:52.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:52.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:52.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:52.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-26T04:45:52.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:52.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:45:52.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:45:52.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:45:52.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:51 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:52 smithi090 ceph-mon[26648]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:45:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:52 smithi090 ceph-mon[26648]: Upgrade: Finalizing container_image settings 2024-04-26T04:45:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:52 smithi090 ceph-mon[26648]: Upgrade: Complete! 2024-04-26T04:45:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:52 smithi104 ceph-mon[32369]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-04-26T04:45:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:52 smithi104 ceph-mon[32369]: Upgrade: Finalizing container_image settings 2024-04-26T04:45:53.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:52 smithi104 ceph-mon[32369]: Upgrade: Complete! 2024-04-26T04:45:53.898 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:53 smithi104 ceph-mon[32369]: pgmap v59: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:53.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:53 smithi090 ceph-mon[26648]: pgmap v59: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:55.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:55 smithi090 ceph-mon[26648]: pgmap v60: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:55.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:55 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:55 smithi104 ceph-mon[32369]: pgmap v60: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:55 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:45:57.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:57 smithi090 ceph-mon[26648]: pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:57 smithi104 ceph-mon[32369]: pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:45:59.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:45:59 smithi090 ceph-mon[26648]: pgmap v62: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:45:59 smithi104 ceph-mon[32369]: pgmap v62: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:00.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:00 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:00 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:01.888 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:01 smithi090 ceph-mon[26648]: pgmap v63: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:01 smithi104 ceph-mon[32369]: pgmap v63: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:03.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:03 smithi090 ceph-mon[26648]: pgmap v64: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:03.996 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:03 smithi104 ceph-mon[32369]: pgmap v64: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:05 smithi090 ceph-mon[26648]: pgmap v65: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:06.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:05 smithi104 ceph-mon[32369]: pgmap v65: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:07 smithi090 ceph-mon[26648]: pgmap v66: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 -- 172.21.15.90:0/1076782980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30100240 msgr2=0x7fdb30100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 --2- 172.21.15.90:0/1076782980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30100240 0x7fdb30100650 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009910 tx=0x7fdb20009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 -- 172.21.15.90:0/1076782980 shutdown_connections 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 --2- 172.21.15.90:0/1076782980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30101440 0x7fdb30101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 --2- 172.21.15.90:0/1076782980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30100240 0x7fdb30100650 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 -- 172.21.15.90:0/1076782980 >> 172.21.15.90:0/1076782980 conn(0x7fdb300fb7f0 msgr2=0x7fdb300fdc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 -- 172.21.15.90:0/1076782980 shutdown_connections 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.989+0000 7fdb358c8700 1 -- 172.21.15.90:0/1076782980 wait complete. 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.990+0000 7fdb358c8700 1 Processor -- start 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.990+0000 7fdb358c8700 1 -- start start 2024-04-26T04:46:07.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.990+0000 7fdb358c8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30100240 0x7fdb30074af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:07.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.990+0000 7fdb358c8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101440 0x7fdb30073140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:07.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.990+0000 7fdb358c8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb30073680 con 0x7fdb30101440 2024-04-26T04:46:07.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.990+0000 7fdb358c8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb300737c0 con 0x7fdb30100240 2024-04-26T04:46:07.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb267fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101440 0x7fdb30073140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:07.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb267fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101440 0x7fdb30073140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58554/0 (socket says 172.21.15.90:58554) 2024-04-26T04:46:07.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb267fc700 1 -- 172.21.15.90:0/2058477988 learned_addr learned my addr 172.21.15.90:0/2058477988 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:07.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb2effd700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30100240 0x7fdb30074af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb267fc700 1 -- 172.21.15.90:0/2058477988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30100240 msgr2=0x7fdb30074af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb267fc700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30100240 0x7fdb30074af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb267fc700 1 -- 172.21.15.90:0/2058477988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb200095f0 con 0x7fdb30101440 2024-04-26T04:46:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb2effd700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30100240 0x7fdb30074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:46:07.994 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb267fc700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101440 0x7fdb30073140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800e920 tx=0x7fdb1800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.991+0000 7fdb348c6700 1 -- 172.21.15.90:0/2058477988 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb1800c9d0 con 0x7fdb30101440 2024-04-26T04:46:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.992+0000 7fdb348c6700 1 -- 172.21.15.90:0/2058477988 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fdb18010430 con 0x7fdb30101440 2024-04-26T04:46:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.992+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb30073aa0 con 0x7fdb30101440 2024-04-26T04:46:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.992+0000 7fdb348c6700 1 -- 172.21.15.90:0/2058477988 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fdb18018630 con 0x7fdb30101440 2024-04-26T04:46:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.992+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3019db10 con 0x7fdb30101440 2024-04-26T04:46:07.995 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.993+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb30108d90 con 0x7fdb30101440 2024-04-26T04:46:07.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.994+0000 7fdb348c6700 1 -- 172.21.15.90:0/2058477988 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fdb180105a0 con 0x7fdb30101440 2024-04-26T04:46:07.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.994+0000 7fdb348c6700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fdb1c06de80 0x7fdb1c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:07.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.994+0000 7fdb348c6700 1 -- 172.21.15.90:0/2058477988 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fdb18014070 con 0x7fdb30101440 2024-04-26T04:46:07.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.994+0000 7fdb2effd700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fdb1c06de80 0x7fdb1c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:07.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.995+0000 7fdb2effd700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fdb1c06de80 0x7fdb1c070330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000b6f0 tx=0x7fdb20005940 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:08.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:07.999+0000 7fdb348c6700 1 -- 172.21.15.90:0/2058477988 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fdb18053470 con 0x7fdb30101440 2024-04-26T04:46:08.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:07 smithi104 ceph-mon[32369]: pgmap v66: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:08.225 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.224+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb30105d90 con 0x7fdb1c06de80 2024-04-26T04:46:08.226 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.225+0000 7fdb348c6700 1 -- 172.21.15.90:0/2058477988 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fdb30105d90 con 0x7fdb1c06de80 2024-04-26T04:46:08.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.227+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fdb1c06de80 msgr2=0x7fdb1c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:08.229 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.227+0000 7fdb358c8700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fdb1c06de80 0x7fdb1c070330 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdb2000b6f0 tx=0x7fdb20005940 comp rx=0 tx=0).stop 2024-04-26T04:46:08.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.227+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101440 msgr2=0x7fdb30073140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:08.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.227+0000 7fdb358c8700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101440 0x7fdb30073140 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fdb1800e920 tx=0x7fdb1800ec30 comp rx=0 tx=0).stop 2024-04-26T04:46:08.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.228+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 shutdown_connections 2024-04-26T04:46:08.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.228+0000 7fdb358c8700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fdb1c06de80 0x7fdb1c070330 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:08.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.228+0000 7fdb358c8700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fdb30100240 0x7fdb30074af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:08.230 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.228+0000 7fdb358c8700 1 --2- 172.21.15.90:0/2058477988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fdb30101440 0x7fdb30073140 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:08.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.228+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 >> 172.21.15.90:0/2058477988 conn(0x7fdb300fb7f0 msgr2=0x7fdb30104670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:08.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.228+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 shutdown_connections 2024-04-26T04:46:08.231 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:08.228+0000 7fdb358c8700 1 -- 172.21.15.90:0/2058477988 wait complete. 2024-04-26T04:46:08.968 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions | jq -e '"'"'.mgr | length == 1'"'"'' 2024-04-26T04:46:09.218 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:09.962 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:09 smithi090 ceph-mon[26648]: pgmap v67: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:09.963 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:09 smithi090 ceph-mon[26648]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:10.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:09 smithi104 ceph-mon[32369]: pgmap v67: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:10.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:09 smithi104 ceph-mon[32369]: from='client.14802 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:10.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 -- 172.21.15.90:0/1691763314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 msgr2=0x7f524c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:10.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 --2- 172.21.15.90:0/1691763314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c100040 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f523c009910 tx=0x7f523c009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:10.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 -- 172.21.15.90:0/1691763314 shutdown_connections 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 --2- 172.21.15.90:0/1691763314 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c100040 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 --2- 172.21.15.90:0/1691763314 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c104130 0x7f524c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 -- 172.21.15.90:0/1691763314 >> 172.21.15.90:0/1691763314 conn(0x7f524c075130 msgr2=0x7f524c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 -- 172.21.15.90:0/1691763314 shutdown_connections 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.784+0000 7f5253436700 1 -- 172.21.15.90:0/1691763314 wait complete. 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f5253436700 1 Processor -- start 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f5253436700 1 -- start start 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f5253436700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f5253436700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c104130 0x7f524c196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f5253436700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c196a20 con 0x7f524c0ffbd0 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f5253436700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f524c19a7b0 con 0x7f524c104130 2024-04-26T04:46:10.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52511d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52511d2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c195e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58576/0 (socket says 172.21.15.90:58576) 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52511d2700 1 -- 172.21.15.90:0/1195578178 learned_addr learned my addr 172.21.15.90:0/1195578178 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52509d1700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c104130 0x7f524c196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52511d2700 1 -- 172.21.15.90:0/1195578178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c104130 msgr2=0x7f524c196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52511d2700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c104130 0x7f524c196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52511d2700 1 -- 172.21.15.90:0/1195578178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f523c0095f0 con 0x7f524c0ffbd0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.785+0000 7f52511d2700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c195e00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f52440060b0 tx=0x7f524400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.786+0000 7f52427fc700 1 -- 172.21.15.90:0/1195578178 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f524400dda0 con 0x7f524c0ffbd0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.786+0000 7f52427fc700 1 -- 172.21.15.90:0/1195578178 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f524400f040 con 0x7f524c0ffbd0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.786+0000 7f52427fc700 1 -- 172.21.15.90:0/1195578178 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5244014790 con 0x7f524c0ffbd0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.786+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f524c19aa90 con 0x7f524c0ffbd0 2024-04-26T04:46:10.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.786+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f524c19afe0 con 0x7f524c0ffbd0 2024-04-26T04:46:10.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.787+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f524c04f2a0 con 0x7f524c0ffbd0 2024-04-26T04:46:10.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.787+0000 7f52427fc700 1 -- 172.21.15.90:0/1195578178 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f5244014980 con 0x7f524c0ffbd0 2024-04-26T04:46:10.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.787+0000 7f52427fc700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f523806de80 0x7f5238070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:10.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.788+0000 7f52427fc700 1 -- 172.21.15.90:0/1195578178 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f524408d670 con 0x7f524c0ffbd0 2024-04-26T04:46:10.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.788+0000 7f52509d1700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f523806de80 0x7f5238070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:10.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.788+0000 7f52509d1700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f523806de80 0x7f5238070330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f523c005b40 tx=0x7f523c01a040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:10.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:10.792+0000 7f52427fc700 1 -- 172.21.15.90:0/1195578178 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f5244056f30 con 0x7f524c0ffbd0 2024-04-26T04:46:11.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.079+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f524c19b300 con 0x7f524c0ffbd0 2024-04-26T04:46:11.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.080+0000 7f52427fc700 1 -- 172.21.15.90:0/1195578178 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f524405a5f0 con 0x7f524c0ffbd0 2024-04-26T04:46:11.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f523806de80 msgr2=0x7f5238070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:11.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f523806de80 0x7f5238070330 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f523c005b40 tx=0x7f523c01a040 comp rx=0 tx=0).stop 2024-04-26T04:46:11.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 msgr2=0x7f524c195e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:11.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c195e00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f52440060b0 tx=0x7f524400d3b0 comp rx=0 tx=0).stop 2024-04-26T04:46:11.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 shutdown_connections 2024-04-26T04:46:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f523806de80 0x7f5238070330 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f524c0ffbd0 0x7f524c195e00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 --2- 172.21.15.90:0/1195578178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f524c104130 0x7f524c196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 >> 172.21.15.90:0/1195578178 conn(0x7f524c075130 msgr2=0x7f524c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 shutdown_connections 2024-04-26T04:46:11.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:11.082+0000 7f5253436700 1 -- 172.21.15.90:0/1195578178 wait complete. 2024-04-26T04:46:11.093 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:46:11.700 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:11 smithi090 ceph-mon[26648]: pgmap v68: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:11.700 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:11 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1195578178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:11.722 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions | jq -e '"'"'.mgr | keys'"'"' | grep $sha1' 2024-04-26T04:46:11.929 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:12.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:11 smithi104 ceph-mon[32369]: pgmap v68: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:12.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:11 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1195578178' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:13.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 -- 172.21.15.90:0/2174105501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ffc60 msgr2=0x7fb65010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:13.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 --2- 172.21.15.90:0/2174105501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ffc60 0x7fb65010a6d0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fb640009910 tx=0x7fb640009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 -- 172.21.15.90:0/2174105501 shutdown_connections 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 --2- 172.21.15.90:0/2174105501 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ffc60 0x7fb65010a6d0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 --2- 172.21.15.90:0/2174105501 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6500ff350 0x7fb6500ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 -- 172.21.15.90:0/2174105501 >> 172.21.15.90:0/2174105501 conn(0x7fb6500748b0 msgr2=0x7fb650074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 -- 172.21.15.90:0/2174105501 shutdown_connections 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.479+0000 7fb656934700 1 -- 172.21.15.90:0/2174105501 wait complete. 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.480+0000 7fb656934700 1 Processor -- start 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.480+0000 7fb656934700 1 -- start start 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.480+0000 7fb656934700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ff350 0x7fb650195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.480+0000 7fb656934700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6500ffc60 0x7fb650196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.480+0000 7fb656934700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb650196b50 con 0x7fb6500ff350 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.480+0000 7fb656934700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb65019a870 con 0x7fb6500ffc60 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ff350 0x7fb650195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:13.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ff350 0x7fb650195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58606/0 (socket says 172.21.15.90:58606) 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64ffff700 1 -- 172.21.15.90:0/2223648895 learned_addr learned my addr 172.21.15.90:0/2223648895 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64f7fe700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6500ffc60 0x7fb650196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64ffff700 1 -- 172.21.15.90:0/2223648895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6500ffc60 msgr2=0x7fb650196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64ffff700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6500ffc60 0x7fb650196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64ffff700 1 -- 172.21.15.90:0/2223648895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6400095f0 con 0x7fb6500ff350 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64ffff700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ff350 0x7fb650195f30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b480 tx=0x7fb63800b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64d7fa700 1 -- 172.21.15.90:0/2223648895 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb638011800 con 0x7fb6500ff350 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64d7fa700 1 -- 172.21.15.90:0/2223648895 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb638011e20 con 0x7fb6500ff350 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb65019ab50 con 0x7fb6500ff350 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb64d7fa700 1 -- 172.21.15.90:0/2223648895 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb638010ea0 con 0x7fb6500ff350 2024-04-26T04:46:13.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.481+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb65019b0a0 con 0x7fb6500ff350 2024-04-26T04:46:13.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.482+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb650107dd0 con 0x7fb6500ff350 2024-04-26T04:46:13.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.483+0000 7fb64d7fa700 1 -- 172.21.15.90:0/2223648895 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fb638010380 con 0x7fb6500ff350 2024-04-26T04:46:13.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.483+0000 7fb64d7fa700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb63c06de80 0x7fb63c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:13.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.483+0000 7fb64d7fa700 1 -- 172.21.15.90:0/2223648895 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb63808d6a0 con 0x7fb6500ff350 2024-04-26T04:46:13.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.483+0000 7fb64f7fe700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb63c06de80 0x7fb63c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:13.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.484+0000 7fb64f7fe700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb63c06de80 0x7fb63c070330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6501974e0 tx=0x7fb64001a040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:13.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.488+0000 7fb64d7fa700 1 -- 172.21.15.90:0/2223648895 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fb638056eb0 con 0x7fb6500ff350 2024-04-26T04:46:13.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.787+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fb6501972c0 con 0x7fb6500ff350 2024-04-26T04:46:13.789 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.788+0000 7fb64d7fa700 1 -- 172.21.15.90:0/2223648895 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7fb63805a570 con 0x7fb6500ff350 2024-04-26T04:46:13.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb63c06de80 msgr2=0x7fb63c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb63c06de80 0x7fb63c070330 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb6501974e0 tx=0x7fb64001a040 comp rx=0 tx=0).stop 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ff350 msgr2=0x7fb650195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ff350 0x7fb650195f30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fb63800b480 tx=0x7fb63800b790 comp rx=0 tx=0).stop 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 shutdown_connections 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb63c06de80 0x7fb63c070330 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb6500ff350 0x7fb650195f30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 --2- 172.21.15.90:0/2223648895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb6500ffc60 0x7fb650196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 >> 172.21.15.90:0/2223648895 conn(0x7fb6500748b0 msgr2=0x7fb650104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 shutdown_connections 2024-04-26T04:46:13.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:13.790+0000 7fb656934700 1 -- 172.21.15.90:0/2223648895 wait complete. 2024-04-26T04:46:13.803 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)" 2024-04-26T04:46:13.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:13 smithi090 ceph-mon[26648]: pgmap v69: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:14.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:13 smithi104 ceph-mon[32369]: pgmap v69: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:14.366 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 2'"'"'' 2024-04-26T04:46:14.573 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:14.941 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:14 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/2223648895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:14 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/2223648895' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:15.962 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:15 smithi090 ceph-mon[26648]: pgmap v70: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:15.962 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:15 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:16.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:15 smithi104 ceph-mon[32369]: pgmap v70: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:16.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:15 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.167+0000 7f10aedce700 1 -- 172.21.15.90:0/3328642661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 msgr2=0x7f10a8104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:16.169 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.167+0000 7f10aedce700 1 --2- 172.21.15.90:0/3328642661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a8104500 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f109c009910 tx=0x7f109c009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.167+0000 7f10aedce700 1 -- 172.21.15.90:0/3328642661 shutdown_connections 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.167+0000 7f10aedce700 1 --2- 172.21.15.90:0/3328642661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10a80ffbd0 0x7f10a8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.167+0000 7f10aedce700 1 --2- 172.21.15.90:0/3328642661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a8104500 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.167+0000 7f10aedce700 1 -- 172.21.15.90:0/3328642661 >> 172.21.15.90:0/3328642661 conn(0x7f10a8075130 msgr2=0x7f10a8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.168+0000 7f10aedce700 1 -- 172.21.15.90:0/3328642661 shutdown_connections 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.168+0000 7f10aedce700 1 -- 172.21.15.90:0/3328642661 wait complete. 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.168+0000 7f10aedce700 1 Processor -- start 2024-04-26T04:46:16.170 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.168+0000 7f10aedce700 1 -- start start 2024-04-26T04:46:16.171 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10aedce700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10a80ffbd0 0x7f10a8194ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10aedce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a81950e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10aedce700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a8195620 con 0x7f10a8104130 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10aedce700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a8195790 con 0x7f10a80ffbd0 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a81950e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a81950e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58610/0 (socket says 172.21.15.90:58610) 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a7fff700 1 -- 172.21.15.90:0/54501466 learned_addr learned my addr 172.21.15.90:0/54501466 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10acb6a700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10a80ffbd0 0x7f10a8194ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a7fff700 1 -- 172.21.15.90:0/54501466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10a80ffbd0 msgr2=0x7f10a8194ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a7fff700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10a80ffbd0 0x7f10a8194ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a7fff700 1 -- 172.21.15.90:0/54501466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c0095f0 con 0x7f10a8104130 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a7fff700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a81950e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f109400b4b0 tx=0x7f109400b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10a5ffb700 1 -- 172.21.15.90:0/54501466 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1094011800 con 0x7f10a8104130 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.169+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a81a2210 con 0x7f10a8104130 2024-04-26T04:46:16.172 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.170+0000 7f10a5ffb700 1 -- 172.21.15.90:0/54501466 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1094011e20 con 0x7f10a8104130 2024-04-26T04:46:16.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.170+0000 7f10a5ffb700 1 -- 172.21.15.90:0/54501466 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f109400f2e0 con 0x7f10a8104130 2024-04-26T04:46:16.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.170+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a81a2570 con 0x7f10a8104130 2024-04-26T04:46:16.173 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.171+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a81076a0 con 0x7f10a8104130 2024-04-26T04:46:16.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.171+0000 7f10a5ffb700 1 -- 172.21.15.90:0/54501466 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f1094010380 con 0x7f10a8104130 2024-04-26T04:46:16.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.172+0000 7f10a5ffb700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f109806de80 0x7f1098070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:16.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.172+0000 7f10a5ffb700 1 -- 172.21.15.90:0/54501466 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f109408d6a0 con 0x7f10a8104130 2024-04-26T04:46:16.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.172+0000 7f10acb6a700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f109806de80 0x7f1098070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:16.174 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.172+0000 7f10acb6a700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f109806de80 0x7f1098070330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f109c006010 tx=0x7f109c01a040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:16.177 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.176+0000 7f10a5ffb700 1 -- 172.21.15.90:0/54501466 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f1094056eb0 con 0x7f10a8104130 2024-04-26T04:46:16.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.463+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f10a804f2a0 con 0x7f10a8104130 2024-04-26T04:46:16.465 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.464+0000 7f10a5ffb700 1 -- 172.21.15.90:0/54501466 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f109405a570 con 0x7f10a8104130 2024-04-26T04:46:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f109806de80 msgr2=0x7f1098070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f109806de80 0x7f1098070330 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f109c006010 tx=0x7f109c01a040 comp rx=0 tx=0).stop 2024-04-26T04:46:16.467 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 msgr2=0x7f10a81950e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a81950e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f109400b4b0 tx=0x7f109400b7c0 comp rx=0 tx=0).stop 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 shutdown_connections 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f109806de80 0x7f1098070330 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f10a80ffbd0 0x7f10a8194ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 --2- 172.21.15.90:0/54501466 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f10a8104130 0x7f10a81950e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 >> 172.21.15.90:0/54501466 conn(0x7f10a8075130 msgr2=0x7f10a80fed20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 shutdown_connections 2024-04-26T04:46:16.468 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:16.466+0000 7f10aedce700 1 -- 172.21.15.90:0/54501466 wait complete. 2024-04-26T04:46:16.477 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:46:16.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:16 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/54501466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:16 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/54501466' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:17.117 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch upgrade check quay.ceph.io/ceph-ci/ceph:$sha1 | jq -e '"'"'.up_to_date | length == 2'"'"'' 2024-04-26T04:46:17.326 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:17.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:17 smithi090 ceph-mon[26648]: pgmap v71: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:18.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:17 smithi104 ceph-mon[32369]: pgmap v71: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:18.982 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 -- 172.21.15.90:0/360447794 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 msgr2=0x7f6a281008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:18.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 --2- 172.21.15.90:0/360447794 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a281008b0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f6a24009910 tx=0x7f6a24009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:18.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 -- 172.21.15.90:0/360447794 shutdown_connections 2024-04-26T04:46:18.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 --2- 172.21.15.90:0/360447794 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a28100e80 0x7f6a28104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:18.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 --2- 172.21.15.90:0/360447794 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a281008b0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:18.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 -- 172.21.15.90:0/360447794 >> 172.21.15.90:0/360447794 conn(0x7f6a280fbda0 msgr2=0x7f6a280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:18.983 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 -- 172.21.15.90:0/360447794 shutdown_connections 2024-04-26T04:46:18.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 -- 172.21.15.90:0/360447794 wait complete. 2024-04-26T04:46:18.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 Processor -- start 2024-04-26T04:46:18.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.978+0000 7f6a30688700 1 -- start start 2024-04-26T04:46:18.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a30688700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a28195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:18.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a30688700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a28100e80 0x7f6a28196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:18.984 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a30688700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28196a80 con 0x7f6a281004e0 2024-04-26T04:46:18.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a30688700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a28196bc0 con 0x7f6a28100e80 2024-04-26T04:46:18.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2e424700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a28195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:18.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2e424700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a28195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56468/0 (socket says 172.21.15.90:56468) 2024-04-26T04:46:18.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2e424700 1 -- 172.21.15.90:0/1641222214 learned_addr learned my addr 172.21.15.90:0/1641222214 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:18.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2dc23700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a28100e80 0x7f6a28196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:18.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2e424700 1 -- 172.21.15.90:0/1641222214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a28100e80 msgr2=0x7f6a28196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2e424700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a28100e80 0x7f6a28196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2e424700 1 -- 172.21.15.90:0/1641222214 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a240095f0 con 0x7f6a281004e0 2024-04-26T04:46:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2dc23700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a28100e80 0x7f6a28196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:46:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a2e424700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a28195ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400b5c0 tx=0x7f6a2400be00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a1f7fe700 1 -- 172.21.15.90:0/1641222214 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a2401d070 con 0x7f6a281004e0 2024-04-26T04:46:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a1f7fe700 1 -- 172.21.15.90:0/1641222214 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6a2400f460 con 0x7f6a281004e0 2024-04-26T04:46:18.986 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a2819a9b0 con 0x7f6a281004e0 2024-04-26T04:46:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a1f7fe700 1 -- 172.21.15.90:0/1641222214 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6a24017610 con 0x7f6a281004e0 2024-04-26T04:46:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.979+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a2819aea0 con 0x7f6a281004e0 2024-04-26T04:46:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.980+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a281087f0 con 0x7f6a281004e0 2024-04-26T04:46:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.981+0000 7f6a1f7fe700 1 -- 172.21.15.90:0/1641222214 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f6a2400f5d0 con 0x7f6a281004e0 2024-04-26T04:46:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.981+0000 7f6a1f7fe700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6a1406de80 0x7f6a14070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:18.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.981+0000 7f6a1f7fe700 1 -- 172.21.15.90:0/1641222214 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f6a2408f220 con 0x7f6a281004e0 2024-04-26T04:46:18.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.981+0000 7f6a2dc23700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6a1406de80 0x7f6a14070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.982+0000 7f6a2dc23700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6a1406de80 0x7f6a14070330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005fd0 tx=0x7f6a18005e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:18.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:18.986+0000 7f6a1f7fe700 1 -- 172.21.15.90:0/1641222214 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f6a24058a30 con 0x7f6a281004e0 2024-04-26T04:46:19.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:19.206+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}) v1 -- 0x7f6a2819b180 con 0x7f6a1406de80 2024-04-26T04:46:19.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:19 smithi090 ceph-mon[26648]: pgmap v72: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:20.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:19 smithi104 ceph-mon[32369]: pgmap v72: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:20.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:20 smithi090 ceph-mon[26648]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:20 smithi104 ceph-mon[32369]: from='client.14818 -' entity='client.admin' cmd=[{"prefix": "orch upgrade check", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:21.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.427+0000 7f6a1f7fe700 1 -- 172.21.15.90:0/1641222214 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+5391 (secure 0 0 0) 0x7f6a2819b180 con 0x7f6a1406de80 2024-04-26T04:46:21.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6a1406de80 msgr2=0x7f6a14070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:21.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6a1406de80 0x7f6a14070330 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6a18005fd0 tx=0x7f6a18005e30 comp rx=0 tx=0).stop 2024-04-26T04:46:21.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 msgr2=0x7f6a28195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:21.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a28195ef0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f6a2400b5c0 tx=0x7f6a2400be00 comp rx=0 tx=0).stop 2024-04-26T04:46:21.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 shutdown_connections 2024-04-26T04:46:21.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f6a1406de80 0x7f6a14070330 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:21.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6a281004e0 0x7f6a28195ef0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:21.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 --2- 172.21.15.90:0/1641222214 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6a28100e80 0x7f6a28196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:21.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 >> 172.21.15.90:0/1641222214 conn(0x7f6a280fbda0 msgr2=0x7f6a280fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:21.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 shutdown_connections 2024-04-26T04:46:21.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:21.430+0000 7f6a30688700 1 -- 172.21.15.90:0/1641222214 wait complete. 2024-04-26T04:46:21.441 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:46:21.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:21 smithi090 ceph-mon[26648]: pgmap v73: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:22.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:21 smithi104 ceph-mon[32369]: pgmap v73: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:22.066 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch ps' 2024-04-26T04:46:22.273 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:23.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.858+0000 7fbe0039f700 1 -- 172.21.15.90:0/4147284754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 msgr2=0x7fbdf8104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:23.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.858+0000 7fbe0039f700 1 --2- 172.21.15.90:0/4147284754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8104e50 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fbde8009910 tx=0x7fbde8009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:23.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 -- 172.21.15.90:0/4147284754 shutdown_connections 2024-04-26T04:46:23.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 --2- 172.21.15.90:0/4147284754 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8104e50 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:23.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 --2- 172.21.15.90:0/4147284754 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf81004e0 0x7fbdf81008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:23.862 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 -- 172.21.15.90:0/4147284754 >> 172.21.15.90:0/4147284754 conn(0x7fbdf80fbda0 msgr2=0x7fbdf80fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 -- 172.21.15.90:0/4147284754 shutdown_connections 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 -- 172.21.15.90:0/4147284754 wait complete. 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 Processor -- start 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.859+0000 7fbe0039f700 1 -- start start 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbe0039f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf81004e0 0x7fbdf8195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbe0039f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbe0039f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8196aa0 con 0x7fbdf8100e80 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbe0039f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdf8196be0 con 0x7fbdf81004e0 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfd93a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfd93a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56492/0 (socket says 172.21.15.90:56492) 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfd93a700 1 -- 172.21.15.90:0/3106634222 learned_addr learned my addr 172.21.15.90:0/3106634222 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfe13b700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf81004e0 0x7fbdf8195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:23.863 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfd93a700 1 -- 172.21.15.90:0/3106634222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf81004e0 msgr2=0x7fbdf8195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfd93a700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf81004e0 0x7fbdf8195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfd93a700 1 -- 172.21.15.90:0/3106634222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbde80095f0 con 0x7fbdf8100e80 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfe13b700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf81004e0 0x7fbdf8195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdfd93a700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8196450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbde800bda0 tx=0x7fbde800be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdef7fe700 1 -- 172.21.15.90:0/3106634222 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde801d070 con 0x7fbdf8100e80 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdf819a9d0 con 0x7fbdf8100e80 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdef7fe700 1 -- 172.21.15.90:0/3106634222 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbde8004fb0 con 0x7fbdf8100e80 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbdef7fe700 1 -- 172.21.15.90:0/3106634222 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbde800f480 con 0x7fbdf8100e80 2024-04-26T04:46:23.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.860+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdf819aec0 con 0x7fbdf8100e80 2024-04-26T04:46:23.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.861+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdf8108850 con 0x7fbdf8100e80 2024-04-26T04:46:23.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.862+0000 7fbdef7fe700 1 -- 172.21.15.90:0/3106634222 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fbde8022870 con 0x7fbdf8100e80 2024-04-26T04:46:23.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.862+0000 7fbdef7fe700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbde406de80 0x7fbde4070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:23.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.862+0000 7fbdef7fe700 1 -- 172.21.15.90:0/3106634222 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbde808f100 con 0x7fbdf8100e80 2024-04-26T04:46:23.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.862+0000 7fbdfe13b700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbde406de80 0x7fbde4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:23.865 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.863+0000 7fbdfe13b700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbde406de80 0x7fbde4070330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005fd0 tx=0x7fbdf4005e30 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:23.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:23.867+0000 7fbdef7fe700 1 -- 172.21.15.90:0/3106634222 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fbde80589c0 con 0x7fbdf8100e80 2024-04-26T04:46:23.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:23 smithi090 ceph-mon[26648]: pgmap v74: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:24.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:23 smithi104 ceph-mon[32369]: pgmap v74: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:24.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.075+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fbdf81972e0 con 0x7fbde406de80 2024-04-26T04:46:24.089 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.086+0000 7fbdef7fe700 1 -- 172.21.15.90:0/3106634222 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fbdf81972e0 con 0x7fbde406de80 2024-04-26T04:46:24.089 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (72s) 43s ago 11m 17.0M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (12m) 43s ago 12m 7906k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (11m) 103s ago 11m 7742k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (12m) 43s ago 12m 6665k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (11m) 103s ago 11m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (48s) 43s ago 11m 52.8M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (6m) 43s ago 6m 130M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (6m) 43s ago 6m 14.7M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (6m) 103s ago 6m 15.0M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (6m) 103s ago 6m 131M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (2m) 43s ago 13m 508M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:46:24.090 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (2m) 103s ago 10m 447M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (13m) 43s ago 13m 61.6M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (10m) 103s ago 10m 42.5M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (110s) 43s ago 12m 8484k - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (106s) 103s ago 11m 3481k - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (10m) 43s ago 10m 229M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (9m) 43s ago 9m 244M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (9m) 43s ago 9m 201M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (8m) 103s ago 8m 269M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (8m) 103s ago 8m 234M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (7m) 103s ago 7m 239M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:46:24.091 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (89s) 43s ago 11m 51.4M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbde406de80 msgr2=0x7fbde4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbde406de80 0x7fbde4070330 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fbdf4005fd0 tx=0x7fbdf4005e30 comp rx=0 tx=0).stop 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 msgr2=0x7fbdf8196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8196450 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fbde800bda0 tx=0x7fbde800be80 comp rx=0 tx=0).stop 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 shutdown_connections 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbde406de80 0x7fbde4070330 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbdf81004e0 0x7fbdf8195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:24.092 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 --2- 172.21.15.90:0/3106634222 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbdf8100e80 0x7fbdf8196450 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 >> 172.21.15.90:0/3106634222 conn(0x7fbdf80fbda0 msgr2=0x7fbdf80fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 shutdown_connections 2024-04-26T04:46:24.093 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:24.088+0000 7fbe0039f700 1 -- 172.21.15.90:0/3106634222 wait complete. 2024-04-26T04:46:24.701 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-26T04:46:24.702 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:46:24.703 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mgr mgr/orchestrator/fail_fs true' 2024-04-26T04:46:24.907 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:25.916 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:25 smithi090 ceph-mon[26648]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:25.916 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:25 smithi090 ceph-mon[26648]: pgmap v75: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:25 smithi104 ceph-mon[32369]: from='client.14822 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:25 smithi104 ceph-mon[32369]: pgmap v75: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.380+0000 7fc3e7d29700 1 -- 172.21.15.90:0/2312807678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ffc60 msgr2=0x7fc3e010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.380+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/2312807678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ffc60 0x7fc3e010a6d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc009910 tx=0x7fc3dc009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.380+0000 7fc3e7d29700 1 -- 172.21.15.90:0/2312807678 shutdown_connections 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.380+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/2312807678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ffc60 0x7fc3e010a6d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.380+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/2312807678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e00ff350 0x7fc3e00ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.380+0000 7fc3e7d29700 1 -- 172.21.15.90:0/2312807678 >> 172.21.15.90:0/2312807678 conn(0x7fc3e00748b0 msgr2=0x7fc3e0074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 -- 172.21.15.90:0/2312807678 shutdown_connections 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 -- 172.21.15.90:0/2312807678 wait complete. 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 Processor -- start 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 -- start start 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ff350 0x7fc3e0195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e00ffc60 0x7fc3e0196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e0196b50 con 0x7fc3e00ff350 2024-04-26T04:46:26.384 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e7d29700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3e019a8e0 con 0x7fc3e00ffc60 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e5ac5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ff350 0x7fc3e0195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e5ac5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ff350 0x7fc3e0195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:56518/0 (socket says 172.21.15.90:56518) 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.381+0000 7fc3e5ac5700 1 -- 172.21.15.90:0/3051787746 learned_addr learned my addr 172.21.15.90:0/3051787746 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3e52c4700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e00ffc60 0x7fc3e0196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3e5ac5700 1 -- 172.21.15.90:0/3051787746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e00ffc60 msgr2=0x7fc3e0196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3e5ac5700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e00ffc60 0x7fc3e0196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3e5ac5700 1 -- 172.21.15.90:0/3051787746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3dc0095f0 con 0x7fc3e00ff350 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3e5ac5700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ff350 0x7fc3e0195f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b480 tx=0x7fc3d000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3d6ffd700 1 -- 172.21.15.90:0/3051787746 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d00118c0 con 0x7fc3e00ff350 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3e019abc0 con 0x7fc3e00ff350 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3d6ffd700 1 -- 172.21.15.90:0/3051787746 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc3d0010380 con 0x7fc3e00ff350 2024-04-26T04:46:26.385 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3d6ffd700 1 -- 172.21.15.90:0/3051787746 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3d000f4f0 con 0x7fc3e00ff350 2024-04-26T04:46:26.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.382+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3e019b110 con 0x7fc3e00ff350 2024-04-26T04:46:26.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.383+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3e0107dd0 con 0x7fc3e00ff350 2024-04-26T04:46:26.386 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.384+0000 7fc3d6ffd700 1 -- 172.21.15.90:0/3051787746 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fc3d00104f0 con 0x7fc3e00ff350 2024-04-26T04:46:26.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.384+0000 7fc3d6ffd700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc3cc06de80 0x7fc3cc070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:26.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.384+0000 7fc3d6ffd700 1 -- 172.21.15.90:0/3051787746 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc3d008d840 con 0x7fc3e00ff350 2024-04-26T04:46:26.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.384+0000 7fc3e52c4700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc3cc06de80 0x7fc3cc070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:26.387 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.384+0000 7fc3e52c4700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc3cc06de80 0x7fc3cc070330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0059d0 tx=0x7fc3dc01a040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:26.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.389+0000 7fc3d6ffd700 1 -- 172.21.15.90:0/3051787746 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fc3d0057050 con 0x7fc3e00ff350 2024-04-26T04:46:26.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.589+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mgr/orchestrator/fail_fs}] v 0) v1 -- 0x7fc3e004f2a0 con 0x7fc3e00ff350 2024-04-26T04:46:26.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.594+0000 7fc3d6ffd700 1 -- 172.21.15.90:0/3051787746 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/orchestrator/fail_fs}]=0 v46)=0 v46) v1 ==== 125+0+0 (secure 0 0 0) 0x7fc3d005a710 con 0x7fc3e00ff350 2024-04-26T04:46:26.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc3cc06de80 msgr2=0x7fc3cc070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:26.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc3cc06de80 0x7fc3cc070330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fc3dc0059d0 tx=0x7fc3dc01a040 comp rx=0 tx=0).stop 2024-04-26T04:46:26.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ff350 msgr2=0x7fc3e0195f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:26.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ff350 0x7fc3e0195f30 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7fc3d000b480 tx=0x7fc3d000b840 comp rx=0 tx=0).stop 2024-04-26T04:46:26.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 shutdown_connections 2024-04-26T04:46:26.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc3cc06de80 0x7fc3cc070330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:26.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc3e00ff350 0x7fc3e0195f30 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:26.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 --2- 172.21.15.90:0/3051787746 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc3e00ffc60 0x7fc3e0196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:26.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 >> 172.21.15.90:0/3051787746 conn(0x7fc3e00748b0 msgr2=0x7fc3e0104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:26.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 shutdown_connections 2024-04-26T04:46:26.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:26.596+0000 7fc3e7d29700 1 -- 172.21.15.90:0/3051787746 wait complete. 2024-04-26T04:46:27.181 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-26T04:46:27.182 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:46:27.182 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim false --force' 2024-04-26T04:46:27.387 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:27.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:27 smithi090 ceph-mon[26648]: pgmap v76: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:27.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:27 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3051787746' entity='client.admin' 2024-04-26T04:46:27.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:27 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:27.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:27 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:27.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:27 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:27.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:27 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:28.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:27 smithi104 ceph-mon[32369]: pgmap v76: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:28.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:27 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3051787746' entity='client.admin' 2024-04-26T04:46:28.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:27 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:28.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:27 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:28.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:27 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:28.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:27 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:28.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:28 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 -- 172.21.15.90:0/3542543853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 msgr2=0x7f1b3c10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 --2- 172.21.15.90:0/3542543853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c10a6d0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009910 tx=0x7f1b30009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 -- 172.21.15.90:0/3542543853 shutdown_connections 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 --2- 172.21.15.90:0/3542543853 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c10a6d0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 --2- 172.21.15.90:0/3542543853 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b3c0ff350 0x7f1b3c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 -- 172.21.15.90:0/3542543853 >> 172.21.15.90:0/3542543853 conn(0x7f1b3c0748b0 msgr2=0x7f1b3c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 -- 172.21.15.90:0/3542543853 shutdown_connections 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.987+0000 7f1b425f2700 1 -- 172.21.15.90:0/3542543853 wait complete. 2024-04-26T04:46:28.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b425f2700 1 Processor -- start 2024-04-26T04:46:28.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b425f2700 1 -- start start 2024-04-26T04:46:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b425f2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b3c0ff350 0x7f1b3c100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b425f2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b425f2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c100fb0 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b425f2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3c1010f0 con 0x7f1b3c0ff350 2024-04-26T04:46:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b3b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b3b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42204/0 (socket says 172.21.15.90:42204) 2024-04-26T04:46:28.991 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.988+0000 7f1b3b7fe700 1 -- 172.21.15.90:0/4037368483 learned_addr learned my addr 172.21.15.90:0/4037368483 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b3c0ff350 0x7f1b3c100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b3b7fe700 1 -- 172.21.15.90:0/4037368483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b3c0ff350 msgr2=0x7f1b3c100530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b3c0ff350 0x7f1b3c100530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b3b7fe700 1 -- 172.21.15.90:0/4037368483 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300095f0 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b3b7fe700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c100a70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1b30006010 tx=0x7f1b3000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b397fa700 1 -- 172.21.15.90:0/4037368483 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b3001d070 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b397fa700 1 -- 172.21.15.90:0/4037368483 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b30005dc0 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3c101370 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b397fa700 1 -- 172.21.15.90:0/4037368483 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b30017400 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.989+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b3c1a23e0 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.992 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.990+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3c107dd0 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.991+0000 7f1b397fa700 1 -- 172.21.15.90:0/4037368483 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f1b30003a60 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.991+0000 7f1b397fa700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f1b2406de80 0x7f1b24070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:28.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.991+0000 7f1b397fa700 1 -- 172.21.15.90:0/4037368483 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f1b3008f030 con 0x7f1b3c0ffc60 2024-04-26T04:46:28.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.991+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f1b2406de80 0x7f1b24070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:28.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.992+0000 7f1b3bfff700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f1b2406de80 0x7f1b24070330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c06a640 tx=0x7f1b2c00b410 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:28.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:28.996+0000 7f1b397fa700 1 -- 172.21.15.90:0/4037368483 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f1b30058840 con 0x7f1b3c0ffc60 2024-04-26T04:46:29.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:28 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:46:29.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.193+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}] v 0) v1 -- 0x7f1b3c101a90 con 0x7f1b3c0ffc60 2024-04-26T04:46:29.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.193+0000 7f1b397fa700 1 -- 172.21.15.90:0/4037368483 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim}]=0 v46)=0 v46) v1 ==== 155+0+0 (secure 0 0 0) 0x7f1b3005bf00 con 0x7f1b3c0ffc60 2024-04-26T04:46:29.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.196+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f1b2406de80 msgr2=0x7f1b24070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:29.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.196+0000 7f1b425f2700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f1b2406de80 0x7f1b24070330 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1b3c06a640 tx=0x7f1b2c00b410 comp rx=0 tx=0).stop 2024-04-26T04:46:29.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.196+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 msgr2=0x7f1b3c100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:29.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.196+0000 7f1b425f2700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c100a70 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f1b30006010 tx=0x7f1b3000bd70 comp rx=0 tx=0).stop 2024-04-26T04:46:29.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.197+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 shutdown_connections 2024-04-26T04:46:29.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.197+0000 7f1b425f2700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f1b2406de80 0x7f1b24070330 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:29.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.197+0000 7f1b425f2700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b3c0ff350 0x7f1b3c100530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:29.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.197+0000 7f1b425f2700 1 --2- 172.21.15.90:0/4037368483 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b3c0ffc60 0x7f1b3c100a70 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:29.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.197+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 >> 172.21.15.90:0/4037368483 conn(0x7f1b3c0748b0 msgr2=0x7f1b3c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:29.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.197+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 shutdown_connections 2024-04-26T04:46:29.199 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:29.197+0000 7f1b425f2700 1 -- 172.21.15.90:0/4037368483 wait complete. 2024-04-26T04:46:29.775 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:29 smithi090 ceph-mon[26648]: pgmap v77: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:29.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set mon mon_warn_on_insecure_global_id_reclaim_allowed false --force' 2024-04-26T04:46:29.982 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:30.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:29 smithi104 ceph-mon[32369]: pgmap v77: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:30 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:30 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:31.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 -- 172.21.15.90:0/3047300980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 msgr2=0x7f5950104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:31.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 --2- 172.21.15.90:0/3047300980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950104e50 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f5940009910 tx=0x7f5940009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:31.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 -- 172.21.15.90:0/3047300980 shutdown_connections 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 --2- 172.21.15.90:0/3047300980 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950104e50 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 --2- 172.21.15.90:0/3047300980 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59501004e0 0x7f59501008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 -- 172.21.15.90:0/3047300980 >> 172.21.15.90:0/3047300980 conn(0x7f59500fbda0 msgr2=0x7f59500fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 -- 172.21.15.90:0/3047300980 shutdown_connections 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.575+0000 7f5957e1f700 1 -- 172.21.15.90:0/3047300980 wait complete. 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f5957e1f700 1 Processor -- start 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f5957e1f700 1 -- start start 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f5957e1f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59501004e0 0x7f5950073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f5957e1f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f5957e1f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950073ad0 con 0x7f5950100e80 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f5957e1f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5950073c10 con 0x7f59501004e0 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f59553ba700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f59553ba700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42228/0 (socket says 172.21.15.90:42228) 2024-04-26T04:46:31.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f59553ba700 1 -- 172.21.15.90:0/1183960443 learned_addr learned my addr 172.21.15.90:0/1183960443 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f5955bbb700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59501004e0 0x7f5950073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.576+0000 7f59553ba700 1 -- 172.21.15.90:0/1183960443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59501004e0 msgr2=0x7f5950073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f59553ba700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59501004e0 0x7f5950073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f59553ba700 1 -- 172.21.15.90:0/1183960443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59400095f0 con 0x7f5950100e80 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f59553ba700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950073590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f594000bda0 tx=0x7f594000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f5946ffd700 1 -- 172.21.15.90:0/1183960443 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594001d070 con 0x7f5950100e80 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5950073e90 con 0x7f5950100e80 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f5946ffd700 1 -- 172.21.15.90:0/1183960443 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5940004fb0 con 0x7f5950100e80 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f5946ffd700 1 -- 172.21.15.90:0/1183960443 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f594000f480 con 0x7f5950100e80 2024-04-26T04:46:31.580 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.577+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59501a2500 con 0x7f5950100e80 2024-04-26T04:46:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.578+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59501087a0 con 0x7f5950100e80 2024-04-26T04:46:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.578+0000 7f5946ffd700 1 -- 172.21.15.90:0/1183960443 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f5940022870 con 0x7f5950100e80 2024-04-26T04:46:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.579+0000 7f5946ffd700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f593c06de80 0x7f593c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.579+0000 7f5946ffd700 1 -- 172.21.15.90:0/1183960443 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f594008f050 con 0x7f5950100e80 2024-04-26T04:46:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.579+0000 7f5955bbb700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f593c06de80 0x7f593c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:31.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.579+0000 7f5955bbb700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f593c06de80 0x7f593c070330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5948006fd0 tx=0x7f5948008040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:31.584 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.583+0000 7f5946ffd700 1 -- 172.21.15.90:0/1183960443 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f5940058ab0 con 0x7f5950100e80 2024-04-26T04:46:31.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.797+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}] v 0) v1 -- 0x7f5950074620 con 0x7f5950100e80 2024-04-26T04:46:31.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.798+0000 7f5946ffd700 1 -- 172.21.15.90:0/1183960443 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_insecure_global_id_reclaim_allowed}]=0 v46)=0 v46) v1 ==== 163+0+0 (secure 0 0 0) 0x7f594005c170 con 0x7f5950100e80 2024-04-26T04:46:31.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f593c06de80 msgr2=0x7f593c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:31.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f593c06de80 0x7f593c070330 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5948006fd0 tx=0x7f5948008040 comp rx=0 tx=0).stop 2024-04-26T04:46:31.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 msgr2=0x7f5950073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950073590 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f594000bda0 tx=0x7f594000be80 comp rx=0 tx=0).stop 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 shutdown_connections 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f593c06de80 0x7f593c070330 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f59501004e0 0x7f5950073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 --2- 172.21.15.90:0/1183960443 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5950100e80 0x7f5950073590 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 >> 172.21.15.90:0/1183960443 conn(0x7f59500fbda0 msgr2=0x7f59500fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 shutdown_connections 2024-04-26T04:46:31.802 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:31.800+0000 7f5957e1f700 1 -- 172.21.15.90:0/1183960443 wait complete. 2024-04-26T04:46:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:31 smithi090 ceph-mon[26648]: pgmap v78: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:31 smithi104 ceph-mon[32369]: pgmap v78: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:32.417 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph config set global log_to_journald false --force' 2024-04-26T04:46:32.628 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:32.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:32 smithi090 ceph-mon[26648]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:46:32.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:32 smithi090 ceph-mon[26648]: pgmap v79: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:33.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:32 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:46:33.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:32 smithi104 ceph-mon[32369]: pgmap v79: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 -- 172.21.15.90:0/2235231570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a41004e0 msgr2=0x7fb2a41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/2235231570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a41004e0 0x7fb2a41008b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fb294009910 tx=0x7fb294009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 -- 172.21.15.90:0/2235231570 shutdown_connections 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/2235231570 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a4100e80 0x7fb2a4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/2235231570 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a41004e0 0x7fb2a41008b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 -- 172.21.15.90:0/2235231570 >> 172.21.15.90:0/2235231570 conn(0x7fb2a40fbda0 msgr2=0x7fb2a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 -- 172.21.15.90:0/2235231570 shutdown_connections 2024-04-26T04:46:34.190 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.188+0000 7fb2aa98a700 1 -- 172.21.15.90:0/2235231570 wait complete. 2024-04-26T04:46:34.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2aa98a700 1 Processor -- start 2024-04-26T04:46:34.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2aa98a700 1 -- start start 2024-04-26T04:46:34.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2aa98a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41004e0 0x7fb2a4195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2aa98a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a4100e80 0x7fb2a41964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2aa98a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a4196bd0 con 0x7fb2a4100e80 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2aa98a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a419a960 con 0x7fb2a41004e0 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2a37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a4100e80 0x7fb2a41964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2a37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a4100e80 0x7fb2a41964f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42248/0 (socket says 172.21.15.90:42248) 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2a37fe700 1 -- 172.21.15.90:0/207721880 learned_addr learned my addr 172.21.15.90:0/207721880 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.189+0000 7fb2a3fff700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41004e0 0x7fb2a4195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a37fe700 1 -- 172.21.15.90:0/207721880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41004e0 msgr2=0x7fb2a4195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a37fe700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41004e0 0x7fb2a4195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a37fe700 1 -- 172.21.15.90:0/207721880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2940095f0 con 0x7fb2a4100e80 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a3fff700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41004e0 0x7fb2a4195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a37fe700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a4100e80 0x7fb2a41964f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb29800b480 tx=0x7fb29800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:34.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a17fa700 1 -- 172.21.15.90:0/207721880 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb2980118c0 con 0x7fb2a4100e80 2024-04-26T04:46:34.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2a419ac40 con 0x7fb2a4100e80 2024-04-26T04:46:34.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a17fa700 1 -- 172.21.15.90:0/207721880 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb298010380 con 0x7fb2a4100e80 2024-04-26T04:46:34.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2a419b190 con 0x7fb2a4100e80 2024-04-26T04:46:34.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.190+0000 7fb2a17fa700 1 -- 172.21.15.90:0/207721880 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb29800f4f0 con 0x7fb2a4100e80 2024-04-26T04:46:34.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.191+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2a41087f0 con 0x7fb2a4100e80 2024-04-26T04:46:34.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.192+0000 7fb2a17fa700 1 -- 172.21.15.90:0/207721880 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fb2980104f0 con 0x7fb2a4100e80 2024-04-26T04:46:34.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.192+0000 7fb2a17fa700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb28c06de80 0x7fb28c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:34.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.192+0000 7fb2a17fa700 1 -- 172.21.15.90:0/207721880 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb29808d820 con 0x7fb2a4100e80 2024-04-26T04:46:34.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.192+0000 7fb2a3fff700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb28c06de80 0x7fb28c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:34.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.192+0000 7fb2a3fff700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb28c06de80 0x7fb28c070330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2940059d0 tx=0x7fb29401a040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:34.198 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.197+0000 7fb2a17fa700 1 -- 172.21.15.90:0/207721880 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fb298057030 con 0x7fb2a4100e80 2024-04-26T04:46:34.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.396+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command([{prefix=config set, name=log_to_journald}] v 0) v1 -- 0x7fb2a404f2a0 con 0x7fb2a4100e80 2024-04-26T04:46:34.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.396+0000 7fb2a17fa700 1 -- 172.21.15.90:0/207721880 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{prefix=config set, name=log_to_journald}]=0 v46)=0 v46) v1 ==== 135+0+0 (secure 0 0 0) 0x7fb29805a6f0 con 0x7fb2a4100e80 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb28c06de80 msgr2=0x7fb28c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb28c06de80 0x7fb28c070330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb2940059d0 tx=0x7fb29401a040 comp rx=0 tx=0).stop 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a4100e80 msgr2=0x7fb2a41964f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a4100e80 0x7fb2a41964f0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fb29800b480 tx=0x7fb29800b840 comp rx=0 tx=0).stop 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 shutdown_connections 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb28c06de80 0x7fb28c070330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb2a41004e0 0x7fb2a4195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 --2- 172.21.15.90:0/207721880 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb2a4100e80 0x7fb2a41964f0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 >> 172.21.15.90:0/207721880 conn(0x7fb2a40fbda0 msgr2=0x7fb2a40fd480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:34.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 shutdown_connections 2024-04-26T04:46:34.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:34.399+0000 7fb2aa98a700 1 -- 172.21.15.90:0/207721880 wait complete. 2024-04-26T04:46:35.028 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch upgrade start --image quay.ceph.io/ceph-ci/ceph:$sha1' 2024-04-26T04:46:35.234 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:35.586 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:35 smithi090 ceph-mon[26648]: pgmap v80: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:35.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:35 smithi104 ceph-mon[32369]: pgmap v80: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:36.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 -- 172.21.15.90:0/3630373356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 msgr2=0x7f81a8104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:36.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 --2- 172.21.15.90:0/3630373356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8104500 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f81a4009910 tx=0x7f81a4009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:36.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 -- 172.21.15.90:0/3630373356 shutdown_connections 2024-04-26T04:46:36.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 --2- 172.21.15.90:0/3630373356 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a80ffbd0 0x7f81a8100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:36.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 --2- 172.21.15.90:0/3630373356 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8104500 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 -- 172.21.15.90:0/3630373356 >> 172.21.15.90:0/3630373356 conn(0x7f81a8075130 msgr2=0x7f81a8075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 -- 172.21.15.90:0/3630373356 shutdown_connections 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.841+0000 7f81af8a8700 1 -- 172.21.15.90:0/3630373356 wait complete. 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81af8a8700 1 Processor -- start 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81af8a8700 1 -- start start 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81af8a8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a80ffbd0 0x7f81a8196c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81af8a8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8197160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81af8a8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a8197840 con 0x7f81a8104130 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81af8a8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81a8197980 con 0x7f81a80ffbd0 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ace43700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8197160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ace43700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8197160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:42252/0 (socket says 172.21.15.90:42252) 2024-04-26T04:46:36.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ace43700 1 -- 172.21.15.90:0/2323208149 learned_addr learned my addr 172.21.15.90:0/2323208149 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ad644700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a80ffbd0 0x7f81a8196c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ace43700 1 -- 172.21.15.90:0/2323208149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a80ffbd0 msgr2=0x7f81a8196c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ace43700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a80ffbd0 0x7f81a8196c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ace43700 1 -- 172.21.15.90:0/2323208149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81a40095f0 con 0x7f81a8104130 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.842+0000 7f81ad644700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a80ffbd0 0x7f81a8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.843+0000 7f81ace43700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8197160 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f819800d900 tx=0x7f819800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.843+0000 7f819e7fc700 1 -- 172.21.15.90:0/2323208149 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f81980041f0 con 0x7f81a8104130 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.843+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81a819fc90 con 0x7f81a8104130 2024-04-26T04:46:36.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.843+0000 7f819e7fc700 1 -- 172.21.15.90:0/2323208149 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8198004cf0 con 0x7f81a8104130 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.843+0000 7f819e7fc700 1 -- 172.21.15.90:0/2323208149 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f819800b750 con 0x7f81a8104130 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.843+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81a81a01b0 con 0x7f81a8104130 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.844+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81a804f2a0 con 0x7f81a8104130 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.844+0000 7f819e7fc700 1 -- 172.21.15.90:0/2323208149 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f8198004350 con 0x7f81a8104130 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.845+0000 7f819e7fc700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f819406de80 0x7f8194070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.845+0000 7f819e7fc700 1 -- 172.21.15.90:0/2323208149 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f819808dae0 con 0x7f81a8104130 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.845+0000 7f81ad644700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f819406de80 0x7f8194070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:36.847 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.845+0000 7f81ad644700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f819406de80 0x7f8194070330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005f50 tx=0x7f81a4005dc0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:36.850 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:36.849+0000 7f819e7fc700 1 -- 172.21.15.90:0/2323208149 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f8198057310 con 0x7f81a8104130 2024-04-26T04:46:37.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.070+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}) v1 -- 0x7f81a80ff3a0 con 0x7f819406de80 2024-04-26T04:46:37.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.075+0000 7f819e7fc700 1 -- 172.21.15.90:0/2323208149 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+89 (secure 0 0 0) 0x7f81a80ff3a0 con 0x7f819406de80 2024-04-26T04:46:37.077 INFO:teuthology.orchestra.run.smithi090.stdout:Initiating upgrade to quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.077+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f819406de80 msgr2=0x7f8194070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f819406de80 0x7f8194070330 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f81a4005f50 tx=0x7f81a4005dc0 comp rx=0 tx=0).stop 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 msgr2=0x7f81a8197160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8197160 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f819800d900 tx=0x7f819800dcc0 comp rx=0 tx=0).stop 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 shutdown_connections 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f819406de80 0x7f8194070330 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f81a80ffbd0 0x7f81a8196c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:37.079 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 --2- 172.21.15.90:0/2323208149 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f81a8104130 0x7f81a8197160 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:37.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 >> 172.21.15.90:0/2323208149 conn(0x7f81a8075130 msgr2=0x7f81a80febf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:37.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 shutdown_connections 2024-04-26T04:46:37.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:37.078+0000 7f81af8a8700 1 -- 172.21.15.90:0/2323208149 wait complete. 2024-04-26T04:46:37.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:37 smithi090 ceph-mon[26648]: pgmap v81: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:37.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:37 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:37.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:37 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:37.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:37 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:37.427 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:37 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:37.428 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:37 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:37.563 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:37 smithi104 ceph-mon[32369]: pgmap v81: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:37.564 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:37 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:37.564 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:37 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:37.564 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:37 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:37.564 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:37 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:46:37.564 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:37 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:37.751 INFO:teuthology.task.sequential:In sequential, running task cephadm.shell... 2024-04-26T04:46:37.751 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:46:37.751 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'while ceph orch upgrade status | jq '"'"'.in_progress'"'"' | grep true && ! ceph orch upgrade status | jq '"'"'.message'"'"' | grep Error ; do ceph orch ps ; ceph versions ; ceph fs dump; ceph orch upgrade status ; ceph health detail ; sleep 30 ; done' 2024-04-26T04:46:37.979 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:46:38.380 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:38 smithi090 ceph-mon[26648]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:38.381 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:38 smithi090 ceph-mon[26648]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:46:38.381 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:38 smithi090 ceph-mon[26648]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:46:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:38 smithi104 ceph-mon[32369]: from='client.14842 -' entity='client.admin' cmd=[{"prefix": "orch upgrade start", "image": "quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:38 smithi104 ceph-mon[32369]: Upgrade: Started with target quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:46:38.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:38 smithi104 ceph-mon[32369]: Upgrade: First pull of quay.ceph.io/ceph-ci/ceph:b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-26T04:46:39.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:39 smithi090 ceph-mon[26648]: pgmap v82: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:39.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:39 smithi104 ceph-mon[32369]: pgmap v82: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 -- 172.21.15.90:0/4054991883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 msgr2=0x7f8dbc10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 --2- 172.21.15.90:0/4054991883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc10a6d0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8db0009910 tx=0x7f8db0009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 -- 172.21.15.90:0/4054991883 shutdown_connections 2024-04-26T04:46:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 --2- 172.21.15.90:0/4054991883 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc10a6d0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 --2- 172.21.15.90:0/4054991883 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0ff350 0x7f8dbc0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 -- 172.21.15.90:0/4054991883 >> 172.21.15.90:0/4054991883 conn(0x7f8dbc0748b0 msgr2=0x7f8dbc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 -- 172.21.15.90:0/4054991883 shutdown_connections 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.855+0000 7f8dc120d700 1 -- 172.21.15.90:0/4054991883 wait complete. 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.856+0000 7f8dc120d700 1 Processor -- start 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.856+0000 7f8dc120d700 1 -- start start 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.856+0000 7f8dc120d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0ff350 0x7f8dbc195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.856+0000 7f8dc120d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.856+0000 7f8dc120d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc196ad0 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.856+0000 7f8dc120d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dbc19a860 con 0x7f8dbc0ff350 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.856+0000 7f8dba59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dba59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc1963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57514/0 (socket says 172.21.15.90:57514) 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dba59c700 1 -- 172.21.15.90:0/2497038848 learned_addr learned my addr 172.21.15.90:0/2497038848 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dbad9d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0ff350 0x7f8dbc195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dba59c700 1 -- 172.21.15.90:0/2497038848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0ff350 msgr2=0x7f8dbc195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dba59c700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0ff350 0x7f8dbc195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dba59c700 1 -- 172.21.15.90:0/2497038848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00095f0 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dba59c700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc1963f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006010 tx=0x7f8db000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8da3fff700 1 -- 172.21.15.90:0/2497038848 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db001d070 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dbc19aae0 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8da3fff700 1 -- 172.21.15.90:0/2497038848 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8db0005dc0 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8da3fff700 1 -- 172.21.15.90:0/2497038848 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8db0017400 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.857+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dbc19afd0 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.858+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dbc107dd0 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.859+0000 7f8da3fff700 1 -- 172.21.15.90:0/2497038848 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f8db0003a60 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.859+0000 7f8da3fff700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8da406de80 0x7f8da4070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.859+0000 7f8da3fff700 1 -- 172.21.15.90:0/2497038848 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f8db008f040 con 0x7f8dbc0ffc60 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.859+0000 7f8dbad9d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8da406de80 0x7f8da4070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:40.878 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.860+0000 7f8dbad9d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8da406de80 0x7f8da4070330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc06a640 tx=0x7f8dac00b410 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:40.864+0000 7f8da3fff700 1 -- 172.21.15.90:0/2497038848 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f8db0058850 con 0x7f8dbc0ffc60 2024-04-26T04:46:41.097 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.095+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dbc02cca0 con 0x7f8da406de80 2024-04-26T04:46:41.098 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.096+0000 7f8da3fff700 1 -- 172.21.15.90:0/2497038848 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f8dbc02cca0 con 0x7f8da406de80 2024-04-26T04:46:41.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8da406de80 msgr2=0x7f8da4070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8da406de80 0x7f8da4070330 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc06a640 tx=0x7f8dac00b410 comp rx=0 tx=0).stop 2024-04-26T04:46:41.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 msgr2=0x7f8dbc1963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc1963f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f8db0006010 tx=0x7f8db000bd70 comp rx=0 tx=0).stop 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 shutdown_connections 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f8da406de80 0x7f8da4070330 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8dbc0ff350 0x7f8dbc195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 --2- 172.21.15.90:0/2497038848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8dbc0ffc60 0x7f8dbc1963f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 >> 172.21.15.90:0/2497038848 conn(0x7f8dbc0748b0 msgr2=0x7f8dbc104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 shutdown_connections 2024-04-26T04:46:41.100 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.098+0000 7f8dc120d700 1 -- 172.21.15.90:0/2497038848 wait complete. 2024-04-26T04:46:41.110 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/2839599598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 msgr2=0x7fbb54100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/2839599598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54100830 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fbb50009910 tx=0x7fbb50009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/2839599598 shutdown_connections 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/2839599598 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb54100d70 0x7fbb541094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/2839599598 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54100830 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/2839599598 >> 172.21.15.90:0/2839599598 conn(0x7fbb540748b0 msgr2=0x7fbb54074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/2839599598 shutdown_connections 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.208+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/2839599598 wait complete. 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb5bcc4700 1 Processor -- start 2024-04-26T04:46:41.212 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb5bcc4700 1 -- start start 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb5bcc4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb5bcc4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb54100d70 0x7fbb54101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb5bcc4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54101fc0 con 0x7fbb54100460 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb5bcc4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb54102100 con 0x7fbb54100d70 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb59a60700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb59a60700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57532/0 (socket says 172.21.15.90:57532) 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.209+0000 7fbb59a60700 1 -- 172.21.15.90:0/4266111260 learned_addr learned my addr 172.21.15.90:0/4266111260 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb5925f700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb54100d70 0x7fbb54101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb59a60700 1 -- 172.21.15.90:0/4266111260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb54100d70 msgr2=0x7fbb54101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.213 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb59a60700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb54100d70 0x7fbb54101a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb59a60700 1 -- 172.21.15.90:0/4266111260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500095f0 con 0x7fbb54100460 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb5925f700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb54100d70 0x7fbb54101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb59a60700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54101540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbb50005f50 tx=0x7fbb5000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb4affd700 1 -- 172.21.15.90:0/4266111260 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb5001d070 con 0x7fbb54100460 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb4affd700 1 -- 172.21.15.90:0/4266111260 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb5000b610 con 0x7fbb54100460 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb54102380 con 0x7fbb54100460 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb4affd700 1 -- 172.21.15.90:0/4266111260 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb50022470 con 0x7fbb54100460 2024-04-26T04:46:41.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.210+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb541a2310 con 0x7fbb54100460 2024-04-26T04:46:41.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.211+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb54107e40 con 0x7fbb54100460 2024-04-26T04:46:41.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.212+0000 7fbb4affd700 1 -- 172.21.15.90:0/4266111260 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fbb5000fca0 con 0x7fbb54100460 2024-04-26T04:46:41.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.212+0000 7fbb4affd700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbb4006de80 0x7fbb40070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.215 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.212+0000 7fbb5925f700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbb4006de80 0x7fbb40070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.212+0000 7fbb4affd700 1 -- 172.21.15.90:0/4266111260 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fbb5008f090 con 0x7fbb54100460 2024-04-26T04:46:41.216 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.213+0000 7fbb5925f700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbb4006de80 0x7fbb40070330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb54102d30 tx=0x7fbb44009260 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:41.218 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.217+0000 7fbb4affd700 1 -- 172.21.15.90:0/4266111260 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fbb500588a0 con 0x7fbb54100460 2024-04-26T04:46:41.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.443+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb54061190 con 0x7fbb4006de80 2024-04-26T04:46:41.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.444+0000 7fbb4affd700 1 -- 172.21.15.90:0/4266111260 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fbb54061190 con 0x7fbb4006de80 2024-04-26T04:46:41.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbb4006de80 msgr2=0x7fbb40070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbb4006de80 0x7fbb40070330 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbb54102d30 tx=0x7fbb44009260 comp rx=0 tx=0).stop 2024-04-26T04:46:41.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 msgr2=0x7fbb54101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54101540 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fbb50005f50 tx=0x7fbb5000f740 comp rx=0 tx=0).stop 2024-04-26T04:46:41.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 shutdown_connections 2024-04-26T04:46:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fbb4006de80 0x7fbb40070330 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb54100460 0x7fbb54101540 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 --2- 172.21.15.90:0/4266111260 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb54100d70 0x7fbb54101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 >> 172.21.15.90:0/4266111260 conn(0x7fbb540748b0 msgr2=0x7fbb540ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 shutdown_connections 2024-04-26T04:46:41.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.446+0000 7fbb5bcc4700 1 -- 172.21.15.90:0/4266111260 wait complete. 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.549+0000 7f5c38161700 1 -- 172.21.15.90:0/3261136885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ff3c0 msgr2=0x7f5c300ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.549+0000 7f5c38161700 1 --2- 172.21.15.90:0/3261136885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ff3c0 0x7f5c300ff790 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009910 tx=0x7f5c2c009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 -- 172.21.15.90:0/3261136885 shutdown_connections 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 --2- 172.21.15.90:0/3261136885 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c300ffd60 0x7f5c30103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 --2- 172.21.15.90:0/3261136885 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ff3c0 0x7f5c300ff790 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 -- 172.21.15.90:0/3261136885 >> 172.21.15.90:0/3261136885 conn(0x7f5c300fac80 msgr2=0x7f5c300fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 -- 172.21.15.90:0/3261136885 shutdown_connections 2024-04-26T04:46:41.551 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 -- 172.21.15.90:0/3261136885 wait complete. 2024-04-26T04:46:41.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 Processor -- start 2024-04-26T04:46:41.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.550+0000 7f5c38161700 1 -- start start 2024-04-26T04:46:41.553 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c38161700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c300ff3c0 0x7f5c30196cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c38161700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ffd60 0x7f5c30197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c38161700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c301977a0 con 0x7f5c300ffd60 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c38161700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c30197910 con 0x7f5c300ff3c0 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c356fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ffd60 0x7f5c30197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c356fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ffd60 0x7f5c30197210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57550/0 (socket says 172.21.15.90:57550) 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c356fc700 1 -- 172.21.15.90:0/113534103 learned_addr learned my addr 172.21.15.90:0/113534103 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c35efd700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c300ff3c0 0x7f5c30196cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c356fc700 1 -- 172.21.15.90:0/113534103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c300ff3c0 msgr2=0x7f5c30196cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c356fc700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c300ff3c0 0x7f5c30196cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c356fc700 1 -- 172.21.15.90:0/113534103 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c2c0095f0 con 0x7f5c300ffd60 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c35efd700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c300ff3c0 0x7f5c30196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:46:41.554 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c356fc700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ffd60 0x7f5c30197210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000c960 tx=0x7f5c2000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:41.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c26ffd700 1 -- 172.21.15.90:0/113534103 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c20007a70 con 0x7f5c300ffd60 2024-04-26T04:46:41.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c26ffd700 1 -- 172.21.15.90:0/113534103 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5c20007bd0 con 0x7f5c300ffd60 2024-04-26T04:46:41.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3019fd40 con 0x7f5c300ffd60 2024-04-26T04:46:41.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.551+0000 7f5c26ffd700 1 -- 172.21.15.90:0/113534103 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5c200186c0 con 0x7f5c300ffd60 2024-04-26T04:46:41.555 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.552+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c301a01b0 con 0x7f5c300ffd60 2024-04-26T04:46:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.553+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c301076d0 con 0x7f5c300ffd60 2024-04-26T04:46:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.553+0000 7f5c26ffd700 1 -- 172.21.15.90:0/113534103 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f5c2000faa0 con 0x7f5c300ffd60 2024-04-26T04:46:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.553+0000 7f5c26ffd700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f5c1c06de80 0x7f5c1c070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.554+0000 7f5c35efd700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f5c1c06de80 0x7f5c1c070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.554+0000 7f5c26ffd700 1 -- 172.21.15.90:0/113534103 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f5c2008dc50 con 0x7f5c300ffd60 2024-04-26T04:46:41.556 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.554+0000 7f5c35efd700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f5c1c06de80 0x7f5c1c070330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0059d0 tx=0x7f5c2c01a040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:41.559 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.558+0000 7f5c26ffd700 1 -- 172.21.15.90:0/113534103 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f5c2005af90 con 0x7f5c300ffd60 2024-04-26T04:46:41.559 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: pgmap v83: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: Upgrade: Target is version 19.0.0-2447-gb22e2ebd (squid) 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6'] 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: Upgrade: Setting container_image for all mgr 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:41.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:41 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:46:41.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: pgmap v83: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:41.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:41.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: Upgrade: Target is version 19.0.0-2447-gb22e2ebd (squid) 2024-04-26T04:46:41.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: Upgrade: Target container is quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, digests ['quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6'] 2024-04-26T04:46:41.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:41.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:41.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: Upgrade: Setting container_image for all mgr 2024-04-26T04:46:41.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:41.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:41 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:46:41.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.778+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c3002cca0 con 0x7f5c1c06de80 2024-04-26T04:46:41.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.789+0000 7f5c26ffd700 1 -- 172.21.15.90:0/113534103 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7f5c3002cca0 con 0x7f5c1c06de80 2024-04-26T04:46:41.790 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (90s) 60s ago 12m 17.0M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (12m) 60s ago 12m 7906k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (11m) 2m ago 11m 7742k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (12m) 60s ago 12m 6665k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (11m) 2m ago 11m 6660k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (66s) 60s ago 11m 52.8M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (6m) 60s ago 6m 130M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (6m) 60s ago 6m 14.7M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:46:41.791 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (6m) 2m ago 6m 15.0M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:46:41.792 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (6m) 2m ago 6m 131M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:46:41.792 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (2m) 60s ago 13m 508M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:46:41.792 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (2m) 2m ago 11m 447M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:46:41.792 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (13m) 60s ago 13m 61.6M 2048M 18.2.0 dc2bc1663786 cb4c73ccf4d5 2024-04-26T04:46:41.792 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (11m) 2m ago 11m 42.5M 2048M 18.2.0 dc2bc1663786 0ba8ae453659 2024-04-26T04:46:41.792 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (2m) 60s ago 12m 8484k - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:46:41.792 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (2m) 2m ago 11m 3481k - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:46:41.793 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (10m) 60s ago 10m 229M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:46:41.793 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (9m) 60s ago 9m 244M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:46:41.793 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (9m) 60s ago 9m 201M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:46:41.793 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (9m) 2m ago 9m 269M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:46:41.793 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (8m) 2m ago 8m 234M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:46:41.793 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (8m) 2m ago 8m 239M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:46:41.794 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (107s) 60s ago 11m 51.4M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f5c1c06de80 msgr2=0x7f5c1c070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f5c1c06de80 0x7f5c1c070330 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c0059d0 tx=0x7f5c2c01a040 comp rx=0 tx=0).stop 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ffd60 msgr2=0x7f5c30197210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ffd60 0x7f5c30197210 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f5c2000c960 tx=0x7f5c2000cd20 comp rx=0 tx=0).stop 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 shutdown_connections 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f5c1c06de80 0x7f5c1c070330 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5c300ff3c0 0x7f5c30196cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 --2- 172.21.15.90:0/113534103 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5c300ffd60 0x7f5c30197210 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 >> 172.21.15.90:0/113534103 conn(0x7f5c300fac80 msgr2=0x7f5c300fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:41.795 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 shutdown_connections 2024-04-26T04:46:41.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.791+0000 7f5c38161700 1 -- 172.21.15.90:0/113534103 wait complete. 2024-04-26T04:46:41.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 -- 172.21.15.90:0/4141263301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 msgr2=0x7f648810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 --2- 172.21.15.90:0/4141263301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f648810a6d0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f6470009910 tx=0x7f6470009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:41.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 -- 172.21.15.90:0/4141263301 shutdown_connections 2024-04-26T04:46:41.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 --2- 172.21.15.90:0/4141263301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f648810a6d0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 --2- 172.21.15.90:0/4141263301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64880ff350 0x7f64880ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 -- 172.21.15.90:0/4141263301 >> 172.21.15.90:0/4141263301 conn(0x7f64880748b0 msgr2=0x7f6488074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:41.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 -- 172.21.15.90:0/4141263301 shutdown_connections 2024-04-26T04:46:41.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 -- 172.21.15.90:0/4141263301 wait complete. 2024-04-26T04:46:41.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.895+0000 7f648e13d700 1 Processor -- start 2024-04-26T04:46:41.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f648e13d700 1 -- start start 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f648e13d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64880ff350 0x7f6488195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f648e13d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f64881963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f648e13d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6488196a80 con 0x7f64880ffc60 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f648e13d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648819a810 con 0x7f64880ff350 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f6486ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f64881963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f6486ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f64881963a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57572/0 (socket says 172.21.15.90:57572) 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f6486ffd700 1 -- 172.21.15.90:0/3248051916 learned_addr learned my addr 172.21.15.90:0/3248051916 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f64877fe700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64880ff350 0x7f6488195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f6486ffd700 1 -- 172.21.15.90:0/3248051916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64880ff350 msgr2=0x7f6488195e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f6486ffd700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64880ff350 0x7f6488195e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:41.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.896+0000 7f6486ffd700 1 -- 172.21.15.90:0/3248051916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64700095f0 con 0x7f64880ffc60 2024-04-26T04:46:41.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.897+0000 7f6486ffd700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f64881963a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6470006010 tx=0x7f647000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:41.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.897+0000 7f6484ff9700 1 -- 172.21.15.90:0/3248051916 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f647001d070 con 0x7f64880ffc60 2024-04-26T04:46:41.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.897+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648819aa90 con 0x7f64880ffc60 2024-04-26T04:46:41.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.897+0000 7f6484ff9700 1 -- 172.21.15.90:0/3248051916 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f647000f460 con 0x7f64880ffc60 2024-04-26T04:46:41.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.897+0000 7f6484ff9700 1 -- 172.21.15.90:0/3248051916 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6470017630 con 0x7f64880ffc60 2024-04-26T04:46:41.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.897+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648819af80 con 0x7f64880ffc60 2024-04-26T04:46:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.898+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6488107dd0 con 0x7f64880ffc60 2024-04-26T04:46:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.898+0000 7f6484ff9700 1 -- 172.21.15.90:0/3248051916 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f647000f5d0 con 0x7f64880ffc60 2024-04-26T04:46:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.899+0000 7f6484ff9700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f647406de80 0x7f6474070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.899+0000 7f6484ff9700 1 -- 172.21.15.90:0/3248051916 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f647008f060 con 0x7f64880ffc60 2024-04-26T04:46:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.899+0000 7f64877fe700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f647406de80 0x7f6474070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.899+0000 7f64877fe700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f647406de80 0x7f6474070330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6480009d10 tx=0x7f6480009040 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:41.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:41.903+0000 7f6484ff9700 1 -- 172.21.15.90:0/3248051916 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f6470058870 con 0x7f64880ffc60 2024-04-26T04:46:42.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.201+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f648804f2a0 con 0x7f64880ffc60 2024-04-26T04:46:42.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.201+0000 7f6484ff9700 1 -- 172.21.15.90:0/3248051916 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+675 (secure 0 0 0) 0x7f647005bf30 con 0x7f64880ffc60 2024-04-26T04:46:42.203 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:46:42.203 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:46:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2 2024-04-26T04:46:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:46:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:46:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:46:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:46:42.204 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 12, 2024-04-26T04:46:42.205 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:46:42.206 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:46:42.206 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:46:42.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f647406de80 msgr2=0x7f6474070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f647406de80 0x7f6474070330 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6480009d10 tx=0x7f6480009040 comp rx=0 tx=0).stop 2024-04-26T04:46:42.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 msgr2=0x7f64881963a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f64881963a0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f6470006010 tx=0x7f647000be20 comp rx=0 tx=0).stop 2024-04-26T04:46:42.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 shutdown_connections 2024-04-26T04:46:42.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f647406de80 0x7f6474070330 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f64880ff350 0x7f6488195e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 --2- 172.21.15.90:0/3248051916 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f64880ffc60 0x7f64881963a0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 >> 172.21.15.90:0/3248051916 conn(0x7f64880748b0 msgr2=0x7f6488104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:42.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 shutdown_connections 2024-04-26T04:46:42.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.205+0000 7f648e13d700 1 -- 172.21.15.90:0/3248051916 wait complete. 2024-04-26T04:46:42.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 -- 172.21.15.90:0/3577720295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf01004e0 msgr2=0x7f3cf01008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 --2- 172.21.15.90:0/3577720295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf01004e0 0x7f3cf01008b0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f3cec009910 tx=0x7f3cec009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:42.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 -- 172.21.15.90:0/3577720295 shutdown_connections 2024-04-26T04:46:42.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 --2- 172.21.15.90:0/3577720295 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cf0100e80 0x7f3cf0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 --2- 172.21.15.90:0/3577720295 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf01004e0 0x7f3cf01008b0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 -- 172.21.15.90:0/3577720295 >> 172.21.15.90:0/3577720295 conn(0x7f3cf00fbda0 msgr2=0x7f3cf00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:42.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 -- 172.21.15.90:0/3577720295 shutdown_connections 2024-04-26T04:46:42.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.316+0000 7f3cf805e700 1 -- 172.21.15.90:0/3577720295 wait complete. 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf805e700 1 Processor -- start 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf805e700 1 -- start start 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf805e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cf01004e0 0x7f3cf0073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf805e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf0100e80 0x7f3cf0073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf805e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf0073ad0 con 0x7f3cf0100e80 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf805e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cf0073c10 con 0x7f3cf01004e0 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf55f9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf0100e80 0x7f3cf0073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf55f9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf0100e80 0x7f3cf0073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57602/0 (socket says 172.21.15.90:57602) 2024-04-26T04:46:42.321 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf55f9700 1 -- 172.21.15.90:0/391575923 learned_addr learned my addr 172.21.15.90:0/391575923 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf5dfa700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cf01004e0 0x7f3cf0073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf55f9700 1 -- 172.21.15.90:0/391575923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cf01004e0 msgr2=0x7f3cf0073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf55f9700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cf01004e0 0x7f3cf0073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.317+0000 7f3cf55f9700 1 -- 172.21.15.90:0/391575923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cec0095f0 con 0x7f3cf0100e80 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.318+0000 7f3cf5dfa700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cf01004e0 0x7f3cf0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.318+0000 7f3cf55f9700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf0100e80 0x7f3cf0073590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d900 tx=0x7f3ce000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.318+0000 7f3ce6ffd700 1 -- 172.21.15.90:0/391575923 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce0009900 con 0x7f3cf0100e80 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.318+0000 7f3ce6ffd700 1 -- 172.21.15.90:0/391575923 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3ce0010460 con 0x7f3cf0100e80 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.318+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cf01a20c0 con 0x7f3cf0100e80 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.318+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cf01a24d0 con 0x7f3cf0100e80 2024-04-26T04:46:42.322 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.318+0000 7f3ce6ffd700 1 -- 172.21.15.90:0/391575923 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3ce000f5f0 con 0x7f3cf0100e80 2024-04-26T04:46:42.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.319+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cd4005320 con 0x7f3cf0100e80 2024-04-26T04:46:42.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.319+0000 7f3ce6ffd700 1 -- 172.21.15.90:0/391575923 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7f3ce0009a60 con 0x7f3cf0100e80 2024-04-26T04:46:42.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.320+0000 7f3ce6ffd700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f3cdc06de80 0x7f3cdc070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:42.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.320+0000 7f3ce6ffd700 1 -- 172.21.15.90:0/391575923 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f3ce008d750 con 0x7f3cf0100e80 2024-04-26T04:46:42.323 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.320+0000 7f3cf5dfa700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f3cdc06de80 0x7f3cdc070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:42.324 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.320+0000 7f3cf5dfa700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f3cdc06de80 0x7f3cdc070330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b5c0 tx=0x7f3cec0053a0 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:42.325 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.324+0000 7f3ce6ffd700 1 -- 172.21.15.90:0/391575923 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7f3ce005aa90 con 0x7f3cf0100e80 2024-04-26T04:46:42.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:42 smithi090 ceph-mon[26648]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:42.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:42 smithi090 ceph-mon[26648]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:42.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:42 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:42.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:42 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:46:42.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:42 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:46:42.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:42 smithi090 ceph-mon[26648]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:42.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:42 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/3248051916' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:42.586 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.585+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f3cd4006200 con 0x7f3cf0100e80 2024-04-26T04:46:42.587 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.585+0000 7f3ce6ffd700 1 -- 172.21.15.90:0/391575923 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1833 (secure 0 0 0) 0x7f3ce0017740 con 0x7f3cf0100e80 2024-04-26T04:46:42.587 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:46:42.587 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:46:42.587 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:46:42.587 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:46:42.587 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:46:42.588 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:46:42.589 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:46:42.589 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:46:42.589 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:46:42.589 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:46:42.589 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:46:42.590 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:46:42.590 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:46:42.590 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:46:42.590 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:46:42.590 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:46:42.590 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:46:42.591 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:46:42.591 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:46:42.591 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:46:42.591 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:46:42.591 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:46:42.591 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:46:42.591 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:42.592 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.587+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f3cdc06de80 msgr2=0x7f3cdc070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.587+0000 7f3cf805e700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f3cdc06de80 0x7f3cdc070330 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3cec00b5c0 tx=0x7f3cec0053a0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.587+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf0100e80 msgr2=0x7f3cf0073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.587+0000 7f3cf805e700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf0100e80 0x7f3cf0073590 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f3ce000d900 tx=0x7f3ce000dcc0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.588+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 shutdown_connections 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.588+0000 7f3cf805e700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7f3cdc06de80 0x7f3cdc070330 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.588+0000 7f3cf805e700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3cf01004e0 0x7f3cf0073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.588+0000 7f3cf805e700 1 --2- 172.21.15.90:0/391575923 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3cf0100e80 0x7f3cf0073590 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.588+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 >> 172.21.15.90:0/391575923 conn(0x7f3cf00fbda0 msgr2=0x7f3cf00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:42.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.588+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 shutdown_connections 2024-04-26T04:46:42.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.588+0000 7f3cf805e700 1 -- 172.21.15.90:0/391575923 wait complete. 2024-04-26T04:46:42.595 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:46:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 -- 172.21.15.90:0/2307911210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 msgr2=0x7fc52c103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 --2- 172.21.15.90:0/2307911210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c103d30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7fc520009910 tx=0x7fc520009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 -- 172.21.15.90:0/2307911210 shutdown_connections 2024-04-26T04:46:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 --2- 172.21.15.90:0/2307911210 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c103d30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 --2- 172.21.15.90:0/2307911210 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc52c0ff3c0 0x7fc52c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 -- 172.21.15.90:0/2307911210 >> 172.21.15.90:0/2307911210 conn(0x7fc52c0fac80 msgr2=0x7fc52c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:42.697 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 -- 172.21.15.90:0/2307911210 shutdown_connections 2024-04-26T04:46:42.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.695+0000 7fc531260700 1 -- 172.21.15.90:0/2307911210 wait complete. 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.696+0000 7fc531260700 1 Processor -- start 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.696+0000 7fc531260700 1 -- start start 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.696+0000 7fc531260700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc52c0ff3c0 0x7fc52c196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.696+0000 7fc531260700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.696+0000 7fc531260700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c196c20 con 0x7fc52c0ffd60 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.696+0000 7fc531260700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc52c19a940 con 0x7fc52c0ff3c0 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.696+0000 7fc52a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:42.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc52a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57618/0 (socket says 172.21.15.90:57618) 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc52a59c700 1 -- 172.21.15.90:0/1972181013 learned_addr learned my addr 172.21.15.90:0/1972181013 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc52ad9d700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc52c0ff3c0 0x7fc52c196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc52a59c700 1 -- 172.21.15.90:0/1972181013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc52c0ff3c0 msgr2=0x7fc52c196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc52a59c700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc52c0ff3c0 0x7fc52c196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc52a59c700 1 -- 172.21.15.90:0/1972181013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5200095f0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc52a59c700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c196540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc5200092e0 tx=0x7fc52000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc513fff700 1 -- 172.21.15.90:0/1972181013 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc52001d070 con 0x7fc52c0ffd60 2024-04-26T04:46:42.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc52c19abc0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc513fff700 1 -- 172.21.15.90:0/1972181013 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc52000f460 con 0x7fc52c0ffd60 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc513fff700 1 -- 172.21.15.90:0/1972181013 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc5200175d0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.697+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc52c19b0b0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.698+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc52c04f2a0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.699+0000 7fc513fff700 1 -- 172.21.15.90:0/1972181013 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fc52000f5d0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.699+0000 7fc513fff700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc51406de80 0x7fc514070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.699+0000 7fc513fff700 1 -- 172.21.15.90:0/1972181013 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fc52008f0d0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.699+0000 7fc52ad9d700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc51406de80 0x7fc514070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:42.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.700+0000 7fc52ad9d700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc51406de80 0x7fc514070330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009a10 tx=0x7fc51c009190 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:42.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.704+0000 7fc513fff700 1 -- 172.21.15.90:0/1972181013 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fc5200588e0 con 0x7fc52c0ffd60 2024-04-26T04:46:42.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:42 smithi104 ceph-mon[32369]: from='client.14846 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:42.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:42 smithi104 ceph-mon[32369]: from='client.14850 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:42.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:42 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:46:42.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:42 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:46:42.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:42 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:46:42.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:42 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:46:42.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:42 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/3248051916' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:46:42.938 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.937+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 --> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc52c19b340 con 0x7fc51406de80 2024-04-26T04:46:42.939 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.938+0000 7fc513fff700 1 -- 172.21.15.90:0/1972181013 <== mgr.14688 v2:172.21.15.90:6800/3458990238 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7fc52c19b340 con 0x7fc51406de80 2024-04-26T04:46:42.939 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:46:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:46:42.939 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr" 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "2/23 daemons upgraded", 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mon daemons", 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:46:42.940 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:46:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.940+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc51406de80 msgr2=0x7fc514070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.940+0000 7fc531260700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc51406de80 0x7fc514070330 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fc51c009a10 tx=0x7fc51c009190 comp rx=0 tx=0).stop 2024-04-26T04:46:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 msgr2=0x7fc52c196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c196540 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fc5200092e0 tx=0x7fc52000bd70 comp rx=0 tx=0).stop 2024-04-26T04:46:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 shutdown_connections 2024-04-26T04:46:42.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fc51406de80 0x7fc514070330 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc52c0ff3c0 0x7fc52c196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 --2- 172.21.15.90:0/1972181013 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc52c0ffd60 0x7fc52c196540 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:42.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 >> 172.21.15.90:0/1972181013 conn(0x7fc52c0fac80 msgr2=0x7fc52c1034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:42.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 shutdown_connections 2024-04-26T04:46:42.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:42.941+0000 7fc531260700 1 -- 172.21.15.90:0/1972181013 wait complete. 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 -- 172.21.15.90:0/3243341996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9940ffbd0 msgr2=0x7fb994100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 --2- 172.21.15.90:0/3243341996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9940ffbd0 0x7fb994100040 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fb980009910 tx=0x7fb980009c20 comp rx=0 tx=0).stop 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 -- 172.21.15.90:0/3243341996 shutdown_connections 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 --2- 172.21.15.90:0/3243341996 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb9940ffbd0 0x7fb994100040 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 --2- 172.21.15.90:0/3243341996 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb994104130 0x7fb994104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 -- 172.21.15.90:0/3243341996 >> 172.21.15.90:0/3243341996 conn(0x7fb994075130 msgr2=0x7fb994075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 -- 172.21.15.90:0/3243341996 shutdown_connections 2024-04-26T04:46:43.049 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 -- 172.21.15.90:0/3243341996 wait complete. 2024-04-26T04:46:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 Processor -- start 2024-04-26T04:46:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.046+0000 7fb99ac54700 1 -- start start 2024-04-26T04:46:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb99ac54700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9940ffbd0 0x7fb994195e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb99ac54700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb994104130 0x7fb994196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb99ac54700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb994196a20 con 0x7fb994104130 2024-04-26T04:46:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb99ac54700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb99419a740 con 0x7fb9940ffbd0 2024-04-26T04:46:43.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb993fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb994104130 0x7fb994196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb993fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb994104130 0x7fb994196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:57636/0 (socket says 172.21.15.90:57636) 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb993fff700 1 -- 172.21.15.90:0/1212477056 learned_addr learned my addr 172.21.15.90:0/1212477056 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb9989f0700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9940ffbd0 0x7fb994195e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb993fff700 1 -- 172.21.15.90:0/1212477056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9940ffbd0 msgr2=0x7fb994195e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb993fff700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9940ffbd0 0x7fb994195e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb993fff700 1 -- 172.21.15.90:0/1212477056 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9800095f0 con 0x7fb994104130 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb993fff700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb994104130 0x7fb994196340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb980000c00 tx=0x7fb98000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb991ffb700 1 -- 172.21.15.90:0/1212477056 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb98001d070 con 0x7fb994104130 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb99419a9c0 con 0x7fb994104130 2024-04-26T04:46:43.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb991ffb700 1 -- 172.21.15.90:0/1212477056 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb98000f460 con 0x7fb994104130 2024-04-26T04:46:43.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.047+0000 7fb991ffb700 1 -- 172.21.15.90:0/1212477056 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb980005e20 con 0x7fb994104130 2024-04-26T04:46:43.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.048+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb99419aeb0 con 0x7fb994104130 2024-04-26T04:46:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.049+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb99404f2a0 con 0x7fb994104130 2024-04-26T04:46:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.049+0000 7fb991ffb700 1 -- 172.21.15.90:0/1212477056 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 32) v1 ==== 92056+0+0 (secure 0 0 0) 0x7fb98000f5d0 con 0x7fb994104130 2024-04-26T04:46:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.049+0000 7fb991ffb700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb98406de80 0x7fb984070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:46:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.050+0000 7fb991ffb700 1 -- 172.21.15.90:0/1212477056 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fb98008f020 con 0x7fb994104130 2024-04-26T04:46:43.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.050+0000 7fb9989f0700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb98406de80 0x7fb984070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:46:43.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.050+0000 7fb9989f0700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb98406de80 0x7fb984070330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb988005950 tx=0x7fb98800b410 comp rx=0 tx=0).ready entity=mgr.14688 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:46:43.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.054+0000 7fb991ffb700 1 -- 172.21.15.90:0/1212477056 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+185428 (secure 0 0 0) 0x7fb9800588f0 con 0x7fb994104130 2024-04-26T04:46:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.345+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fb99419b190 con 0x7fb994104130 2024-04-26T04:46:43.346 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.345+0000 7fb991ffb700 1 -- 172.21.15.90:0/1212477056 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fb98005bfb0 con 0x7fb994104130 2024-04-26T04:46:43.346 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_OK 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb98406de80 msgr2=0x7fb984070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb98406de80 0x7fb984070330 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fb988005950 tx=0x7fb98800b410 comp rx=0 tx=0).stop 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb994104130 msgr2=0x7fb994196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb994104130 0x7fb994196340 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fb980000c00 tx=0x7fb98000bd70 comp rx=0 tx=0).stop 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 shutdown_connections 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:6800/3458990238,v1:172.21.15.90:6801/3458990238] conn(0x7fb98406de80 0x7fb984070330 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb9940ffbd0 0x7fb994195e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 --2- 172.21.15.90:0/1212477056 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb994104130 0x7fb994196340 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 >> 172.21.15.90:0/1212477056 conn(0x7fb994075130 msgr2=0x7fb9940fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:46:43.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 shutdown_connections 2024-04-26T04:46:43.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:46:43.347+0000 7fb99ac54700 1 -- 172.21.15.90:0/1212477056 wait complete. 2024-04-26T04:46:43.605 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:43 smithi090 ceph-mon[26648]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:43.605 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:43 smithi090 ceph-mon[26648]: Upgrade: Updating mon.smithi090 2024-04-26T04:46:43.605 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:43 smithi090 ceph-mon[26648]: Deploying daemon mon.smithi090 on smithi090 2024-04-26T04:46:43.605 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:43 smithi090 ceph-mon[26648]: pgmap v84: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:43.606 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:43 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/391575923' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:46:43.606 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:43 smithi090 ceph-mon[26648]: from='client.? 172.21.15.90:0/1212477056' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:46:43.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:43 smithi104 ceph-mon[32369]: from='client.14854 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:46:43.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:43 smithi104 ceph-mon[32369]: Upgrade: Updating mon.smithi090 2024-04-26T04:46:43.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:43 smithi104 ceph-mon[32369]: Deploying daemon mon.smithi090 on smithi090 2024-04-26T04:46:43.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:43 smithi104 ceph-mon[32369]: pgmap v84: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:43.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:43 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/391575923' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:46:43.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:43 smithi104 ceph-mon[32369]: from='client.? 172.21.15.90:0/1212477056' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:46:44.146 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:43 smithi090 systemd[1]: Stopping Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:46:44.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:44 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[26644]: 2024-04-26T04:46:44.144+0000 7fd6365dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi090 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:46:44.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:44 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[26644]: 2024-04-26T04:46:44.144+0000 7fd6365dd700 -1 mon.smithi090@0(leader) e2 *** Got Signal Terminated *** 2024-04-26T04:46:44.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:44 smithi090 podman[86376]: 2024-04-26 04:46:44.226293577 +0000 UTC m=+0.194490009 container died cb4c73ccf4d5b04e1b37736e3510bcbcdade8f89a3bdaa1dbb0b6a018b81d457 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=-18.2.0) 2024-04-26T04:46:44.819 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:44 smithi090 podman[86376]: 2024-04-26 04:46:44.578709754 +0000 UTC m=+0.546906198 container cleanup cb4c73ccf4d5b04e1b37736e3510bcbcdade8f89a3bdaa1dbb0b6a018b81d457 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T04:46:44.819 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:44 smithi090 bash[86376]: ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090 2024-04-26T04:46:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:44 smithi090 podman[86401]: 2024-04-26 04:46:44.905509726 +0000 UTC m=+0.672001389 container remove cb4c73ccf4d5b04e1b37736e3510bcbcdade8f89a3bdaa1dbb0b6a018b81d457 (image=quay.io/ceph/ceph:v18.2.0, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.license=GPLv2) 2024-04-26T04:46:45.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service: Deactivated successfully. 2024-04-26T04:46:45.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 systemd[1]: Stopped Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:46:45.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service: Consumed 13.592s CPU time. 2024-04-26T04:46:45.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 systemd[1]: Starting Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:46:45.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 podman[86531]: 2024-04-26 04:46:45.35112077 +0000 UTC m=+0.018781276 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:46:45.560 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 podman[86531]: 2024-04-26 04:46:45.448389503 +0000 UTC m=+0.116049990 container create e31d5b589b5cd1b13a519542b3872de1913c6a1d218cfeea84ad598fb1fdc988 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, GIT_CLEAN=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-04-26T04:46:45.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 podman[86531]: 2024-04-26 04:46:45.669455928 +0000 UTC m=+0.337116418 container init e31d5b589b5cd1b13a519542b3872de1913c6a1d218cfeea84ad598fb1fdc988 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 podman[86531]: 2024-04-26 04:46:45.673743879 +0000 UTC m=+0.341404372 container start e31d5b589b5cd1b13a519542b3872de1913c6a1d218cfeea84ad598fb1fdc988 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5) 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev), process ceph-mon, pid 2 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: pidfile_write: ignore empty --pid-file 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: load: jerasure load: lrc 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: RocksDB version: 7.9.2 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Git sha 0 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Compile date 2024-04-25 16:43:49 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: DB SUMMARY 2024-04-26T04:46:45.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: DB Session ID: J40131PHMA7KHC9RE299 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: CURRENT file: CURRENT 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: MANIFEST file: MANIFEST-000015 size: 1509 Bytes 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi090/store.db dir, Total Num: 1, files: 000032.sst 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi090/store.db: 000030.log size: 3986485 ; 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.error_if_exists: 0 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.create_if_missing: 0 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.paranoid_checks: 1 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T04:46:45.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.env: 0x5617205d4c60 2024-04-26T04:46:45.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T04:46:45.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.info_log: 0x56172211b960 2024-04-26T04:46:45.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T04:46:45.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.statistics: (nil) 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.use_fsync: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_log_file_size: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.allow_fallocate: 1 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.use_direct_reads: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T04:46:45.976 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.db_log_dir: 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.wal_dir: 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.write_buffer_manager: 0x5617220ee1e0 2024-04-26T04:46:45.977 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.unordered_write: 0 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T04:46:45.978 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.row_cache: None 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.wal_filter: None 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.two_write_queues: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.wal_compression: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.atomic_flush: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.log_readahead_size: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T04:46:45.979 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_background_jobs: 2 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_background_compactions: -1 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_subcompactions: 1 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T04:46:45.980 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_open_files: -1 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_background_flushes: -1 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Compression algorithms supported: 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kZSTD supported: 0 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kXpressCompression supported: 0 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T04:46:45.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kZlibCompression supported: 1 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kSnappyCompression supported: 1 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kLZ4Compression supported: 1 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: kBZip2Compression supported: 0 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000015 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.merge_operator: 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_filter: None 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T04:46:45.982 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x56172211ba60) 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks: 1 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: pin_top_level_index_and_filter: 1 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_type: 0 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_index_type: 0 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_shortening: 1 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: checksum: 4 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: no_block_cache: 0 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache: 0x56172211f090 2024-04-26T04:46:45.983 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_name: BinnedLRUCache 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_options: 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: capacity : 536870912 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_shard_bits : 4 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: strict_capacity_limit : 0 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: high_pri_pool_ratio: 0.000 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_cache_compressed: (nil) 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: persistent_cache: (nil) 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size: 4096 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_size_deviation: 10 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_restart_interval: 16 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: index_block_restart_interval: 1 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: metadata_block_size: 4096 2024-04-26T04:46:45.984 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: partition_filters: 0 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: use_delta_encoding: 1 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: filter_policy: bloomfilter 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: whole_key_filtering: 1 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: verify_compression: 0 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: read_amp_bytes_per_bit: 0 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: format_version: 5 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_index_compression: 1 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: block_align: 0 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_auto_readahead_size: 262144 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: prepopulate_block_cache: 0 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: initial_auto_readahead_size: 8192 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T04:46:45.985 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression: NoCompression 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.num_levels: 7 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:46:45.986 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T04:46:45.987 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T04:46:45.988 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T04:46:45.989 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.table_properties_collectors: 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.inplace_update_support: 0 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.bloom_locality: 0 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.max_successive_merges: 0 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T04:46:45.990 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.ttl: 2592000 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.enable_blob_files: false 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.min_blob_size: 0 2024-04-26T04:46:45.991 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi090/store.db/MANIFEST-000015 succeeded,manifest_file_number is 15, next_file_number is 34, last_sequence is 12123, log_number is 30,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 30 2024-04-26T04:46:45.992 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 30 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: b4a63d83-9182-4f50-9e8e-6355b5cea0f5 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106805702504, "job": 1, "event": "recovery_started", "wal_files": [30]} 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #30 mode 2 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106805713665, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 35, "file_size": 3466696, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12124, "largest_seqno": 15168, "table_properties": {"data_size": 3455113, "index_size": 7106, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3461, "raw_key_size": 30549, "raw_average_key_size": 22, "raw_value_size": 3428170, "raw_average_value_size": 2482, "num_data_blocks": 323, "num_entries": 1381, "num_filter_entries": 1381, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714106805, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "b4a63d83-9182-4f50-9e8e-6355b5cea0f5", "db_session_id": "J40131PHMA7KHC9RE299", "orig_file_number": 35, "seqno_to_time_mapping": "N/A"}} 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106805713770, "job": 1, "event": "recovery_finished"} 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/version_set.cc:5047] Creating manifest 37 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi090/store.db/000030.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x561722210000 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: DB pointer 0x5617221fa000 2024-04-26T04:46:45.993 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T04:46:45.996 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T04:46:45.996 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** DB Stats ** 2024-04-26T04:46:45.996 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:46:45.999 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T04:46:45.999 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:46:45.999 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:46:45.999 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T04:46:46.000 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:46:46.000 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:46:46.000 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.000 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-26T04:46:46.000 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:46:46.000 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T04:46:46.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: L0 1/0 3.31 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 655.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-26T04:46:46.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: L6 1/0 5.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-26T04:46:46.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Sum 2/0 8.73 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 655.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-26T04:46:46.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 655.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-26T04:46:46.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.001 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** Compaction Stats [default] ** 2024-04-26T04:46:46.002 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:46:46.002 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:46:46.002 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 655.8 0.01 0.00 1 0.005 0 0 0.0 0.0 2024-04-26T04:46:46.002 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.002 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T04:46:46.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:46:46.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Flush(GB): cumulative 0.003, interval 0.003 2024-04-26T04:46:46.003 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T04:46:46.004 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T04:46:46.005 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T04:46:46.005 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T04:46:46.005 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Cumulative compaction: 0.00 GB write, 219.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:46:46.006 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Interval compaction: 0.00 GB write, 219.86 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:46:46.006 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T04:46:46.006 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache BinnedLRUCache@0x56172211f090#2 capacity: 512.00 MB usage: 12.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.1e-05 secs_since: 0 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,8.08 KB,0.00154078%) Misc(1,0.00 KB,0%) 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: starting mon.smithi090 rank 0 at public addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] at bind addrs [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi090 fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???) e2 preinit fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???).mds e21 new map 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???).mds e21 print_map 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: e21 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:46:46.007 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: legacy client fscid: 1 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Filesystem 'cephfs' (1) 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: fs_name cephfs 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: epoch 21 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: tableserver 0 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: root 0 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: session_timeout 60 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: session_autoclose 300 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_file_size 1099511627776 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_xattr_size 65536 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: required_client_features {} 2024-04-26T04:46:46.008 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: last_failure 0 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: last_failure_osd_epoch 46 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: max_mds 2 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: in 0,1 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: up {0=14486,1=24273} 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: failed 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: damaged 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: stopped 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: data_pools [3] 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: metadata_pool 2 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: inline_data disabled 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: balancer 2024-04-26T04:46:46.009 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: bal_rank_mask -1 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: standby_count_wanted 1 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: qdb_cluster leader: 0 members: 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: Standby daemons: 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout: [mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???).osd e49 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???).osd e49 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:46:46.010 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 ceph-mon[86550]: mon.smithi090@-1(???).paxosservice(auth 1..23) refresh upgraded, format 0 -> 3 2024-04-26T04:46:46.011 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 bash[86531]: e31d5b589b5cd1b13a519542b3872de1913c6a1d218cfeea84ad598fb1fdc988 2024-04-26T04:46:46.011 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:45 smithi090 systemd[1]: Started Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:46:49.112 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:46:49.112 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:46:49.112 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: pgmap v85: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:49.112 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: pgmap v86: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: pgmap v87: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: mon.smithi090 calling monitor election 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: mon.smithi090 is new leader, mons smithi090,smithi104 in quorum (ranks 0,1) 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: monmap epoch 2 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: last_changed 2024-04-26T04:35:30.367475+0000 2024-04-26T04:46:49.113 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: created 2024-04-26T04:32:53.473219+0000 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: min_mon_release 18 (reef) 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: election_strategy: 1 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.smithi104 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: mgrmap e32: smithi090.iwrvwa(active, since 2m), standbys: smithi104.wwoewi 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: overall HEALTH_OK 2024-04-26T04:46:49.114 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: from='mgr.14688 ' entity='' 2024-04-26T04:46:49.115 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:48 smithi104 ceph-mon[32369]: mgrmap e33: smithi090.iwrvwa(active, since 2m), standbys: smithi104.wwoewi 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: pgmap v85: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: from='mgr.14688 172.21.15.90:0/2441032841' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: pgmap v86: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: pgmap v87: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: mon.smithi090 calling monitor election 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: mon.smithi090 is new leader, mons smithi090,smithi104 in quorum (ranks 0,1) 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: monmap epoch 2 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: last_changed 2024-04-26T04:35:30.367475+0000 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: created 2024-04-26T04:32:53.473219+0000 2024-04-26T04:46:49.119 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: min_mon_release 18 (reef) 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: election_strategy: 1 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.smithi104 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: osdmap e49: 6 total, 6 up, 6 in 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: mgrmap e32: smithi090.iwrvwa(active, since 2m), standbys: smithi104.wwoewi 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: overall HEALTH_OK 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: from='mgr.14688 ' entity='' 2024-04-26T04:46:49.120 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:48 smithi090 ceph-mon[86550]: mgrmap e33: smithi090.iwrvwa(active, since 2m), standbys: smithi104.wwoewi 2024-04-26T04:46:55.703 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: Standby manager daemon smithi104.wwoewi restarted 2024-04-26T04:46:55.703 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: Standby manager daemon smithi104.wwoewi started 2024-04-26T04:46:55.703 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:46:55.703 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: Active manager daemon smithi090.iwrvwa restarted 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: Activating manager daemon smithi090.iwrvwa 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: osdmap e50: 6 total, 6 up, 6 in 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: mgrmap e34: smithi090.iwrvwa(active, starting, since 0.00611314s), standbys: smithi104.wwoewi 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:46:55.704 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:46:55.705 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: Standby manager daemon smithi104.wwoewi restarted 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: Standby manager daemon smithi104.wwoewi started 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/crt"}]: dispatch 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/smithi104.wwoewi/key"}]: dispatch 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.? 172.21.15.104:0/2412212753' entity='mgr.smithi104.wwoewi' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: Active manager daemon smithi090.iwrvwa restarted 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: Activating manager daemon smithi090.iwrvwa 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: osdmap e50: 6 total, 6 up, 6 in 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: mgrmap e34: smithi090.iwrvwa(active, starting, since 0.00611314s), standbys: smithi104.wwoewi 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:46:56.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi090.iwrvwa", "id": "smithi090.iwrvwa"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr metadata", "who": "smithi104.wwoewi", "id": "smithi104.wwoewi"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-26T04:46:56.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:55 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-26T04:46:56.923 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:56 smithi090 ceph-mon[86550]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:46:56.923 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:56.923 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:56.923 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:46:56.923 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:46:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:56 smithi104 ceph-mon[32369]: Manager daemon smithi090.iwrvwa is now available 2024-04-26T04:46:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:56 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:46:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:56 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:46:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:56 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/mirror_snapshot_schedule"}]: dispatch 2024-04-26T04:46:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:56 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/smithi090.iwrvwa/trash_purge_schedule"}]: dispatch 2024-04-26T04:46:57.939 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:57 smithi090 ceph-mon[86550]: mgrmap e35: smithi090.iwrvwa(active, since 1.02807s), standbys: smithi104.wwoewi 2024-04-26T04:46:57.939 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:57 smithi090 ceph-mon[86550]: [26/Apr/2024:04:46:56] ENGINE Bus STARTING 2024-04-26T04:46:57.939 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:57 smithi090 ceph-mon[86550]: [26/Apr/2024:04:46:56] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T04:46:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:57 smithi104 ceph-mon[32369]: mgrmap e35: smithi090.iwrvwa(active, since 1.02807s), standbys: smithi104.wwoewi 2024-04-26T04:46:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:57 smithi104 ceph-mon[32369]: [26/Apr/2024:04:46:56] ENGINE Bus STARTING 2024-04-26T04:46:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:57 smithi104 ceph-mon[32369]: [26/Apr/2024:04:46:56] ENGINE Serving on https://172.21.15.90:7150 2024-04-26T04:46:58.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:58 smithi090 ceph-mon[86550]: [26/Apr/2024:04:46:56] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T04:46:58.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:58 smithi090 ceph-mon[86550]: [26/Apr/2024:04:46:56] ENGINE Bus STARTED 2024-04-26T04:46:58.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:58 smithi090 ceph-mon[86550]: pgmap v4: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:58.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:58 smithi090 ceph-mon[86550]: mgrmap e36: smithi090.iwrvwa(active, since 2s), standbys: smithi104.wwoewi 2024-04-26T04:46:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:58 smithi104 ceph-mon[32369]: [26/Apr/2024:04:46:56] ENGINE Serving on http://172.21.15.90:8765 2024-04-26T04:46:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:58 smithi104 ceph-mon[32369]: [26/Apr/2024:04:46:56] ENGINE Bus STARTED 2024-04-26T04:46:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:58 smithi104 ceph-mon[32369]: pgmap v4: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:46:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:58 smithi104 ceph-mon[32369]: mgrmap e36: smithi090.iwrvwa(active, since 2s), standbys: smithi104.wwoewi 2024-04-26T04:47:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:59 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:46:59 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:59 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:46:59 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:00.767 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:00 smithi090 ceph-mon[86550]: pgmap v5: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:00.767 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:00 smithi090 ceph-mon[86550]: mgrmap e37: smithi090.iwrvwa(active, since 4s), standbys: smithi104.wwoewi 2024-04-26T04:47:00.767 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:00.767 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:01.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:00 smithi104 ceph-mon[32369]: pgmap v5: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:01.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:00 smithi104 ceph-mon[32369]: mgrmap e37: smithi090.iwrvwa(active, since 4s), standbys: smithi104.wwoewi 2024-04-26T04:47:01.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:00 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:01.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:00 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:02 smithi090 ceph-mon[86550]: pgmap v6: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:02.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:02 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:02.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:02 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:02.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:02 smithi104 ceph-mon[32369]: pgmap v6: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:03.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:03.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:03.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:03 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:03.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:03 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:04 smithi090 ceph-mon[86550]: pgmap v7: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T04:47:04.599 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:04 smithi104 ceph-mon[32369]: pgmap v7: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 30 KiB/s rd, 0 B/s wr, 16 op/s 2024-04-26T04:47:06.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: pgmap v8: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: Detected new or changed devices on smithi090 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:47:06.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:06.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:06.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:06.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:06.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: pgmap v8: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 23 KiB/s rd, 0 B/s wr, 12 op/s 2024-04-26T04:47:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: Detected new or changed devices on smithi090 2024-04-26T04:47:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: Updating smithi090:/etc/ceph/ceph.conf 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: Updating smithi104:/etc/ceph/ceph.conf 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:06 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:07.780 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:07.781 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:07 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:47:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:47:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.conf 2024-04-26T04:47:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: Updating smithi090:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:47:07.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: Updating smithi104:/etc/ceph/ceph.client.admin.keyring 2024-04-26T04:47:07.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: Updating smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:47:07.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: Updating smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-26T04:47:07.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:07.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:07.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:07 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "quorum_status"}]: dispatch 2024-04-26T04:47:08.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:08 smithi090 ceph-mon[86550]: pgmap v9: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T04:47:08.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:08.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:47:08.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:47:08.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:08.999 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:08 smithi104 ceph-mon[32369]: pgmap v9: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 19 KiB/s rd, 0 B/s wr, 10 op/s 2024-04-26T04:47:08.999 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:08 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:08.999 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:08 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:47:09.000 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:08 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:47:09.000 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:08 smithi104 ceph-mon[32369]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:09.872 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:09 smithi104 ceph-mon[32369]: Upgrade: Updating mon.smithi104 2024-04-26T04:47:09.872 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:09 smithi104 ceph-mon[32369]: Deploying daemon mon.smithi104 on smithi104 2024-04-26T04:47:09.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:09 smithi090 ceph-mon[86550]: Upgrade: Updating mon.smithi104 2024-04-26T04:47:09.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:09 smithi090 ceph-mon[86550]: Deploying daemon mon.smithi104 on smithi104 2024-04-26T04:47:10.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:10 smithi104 systemd[1]: Stopping Ceph mon.smithi104 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:47:10.637 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:10 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104[32365]: 2024-04-26T04:47:10.310+0000 7fb3aaedf700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi104 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:47:10.637 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:10 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104[32365]: 2024-04-26T04:47:10.310+0000 7fb3aaedf700 -1 mon.smithi104@1(peon) e2 *** Got Signal Terminated *** 2024-04-26T04:47:10.637 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:10 smithi104 podman[64616]: 2024-04-26 04:47:10.382341152 +0000 UTC m=+0.188061246 container died 0ba8ae45365994e822f6a299f77b5720f835748a5e02e6e23330d0afc909193c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, GIT_CLEAN=True) 2024-04-26T04:47:10.987 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:10 smithi104 podman[64616]: 2024-04-26 04:47:10.735852339 +0000 UTC m=+0.541572441 container cleanup 0ba8ae45365994e822f6a299f77b5720f835748a5e02e6e23330d0afc909193c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True) 2024-04-26T04:47:10.987 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:10 smithi104 bash[64616]: ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104 2024-04-26T04:47:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:10 smithi104 podman[64630]: 2024-04-26 04:47:10.987091292 +0000 UTC m=+0.600191134 container remove 0ba8ae45365994e822f6a299f77b5720f835748a5e02e6e23330d0afc909193c (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20231212) 2024-04-26T04:47:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi104.service: Deactivated successfully. 2024-04-26T04:47:11.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 systemd[1]: Stopped Ceph mon.smithi104 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:47:11.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi104.service: Consumed 7.464s CPU time. 2024-04-26T04:47:11.644 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 systemd[1]: Starting Ceph mon.smithi104 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:47:11.644 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 podman[64759]: 2024-04-26 04:47:11.529956097 +0000 UTC m=+0.109989708 container create 11c4dd69e45404d95fa5efef1ddbaf1523edb2280db940cd517aa5ba4486da15 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2) 2024-04-26T04:47:11.644 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 podman[64759]: 2024-04-26 04:47:11.437807574 +0000 UTC m=+0.017841178 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:47:11.792 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 podman[64759]: 2024-04-26 04:47:11.75758427 +0000 UTC m=+0.337617874 container init 11c4dd69e45404d95fa5efef1ddbaf1523edb2280db940cd517aa5ba4486da15 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD) 2024-04-26T04:47:11.792 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 podman[64759]: 2024-04-26 04:47:11.76055001 +0000 UTC m=+0.340583613 container start 11c4dd69e45404d95fa5efef1ddbaf1523edb2280db940cd517aa5ba4486da15 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: set uid:gid to 167:167 (ceph:ceph) 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev), process ceph-mon, pid 2 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: pidfile_write: ignore empty --pid-file 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: load: jerasure load: lrc 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: RocksDB version: 7.9.2 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Git sha 0 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Compile date 2024-04-25 16:43:49 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: DB SUMMARY 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: DB Session ID: XJ6TTOLTDIMPQ6DRUPCF 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: CURRENT file: CURRENT 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: IDENTITY file: IDENTITY 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: MANIFEST file: MANIFEST-000010 size: 1403 Bytes 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: SST files in /var/lib/ceph/mon/ceph-smithi104/store.db dir, Total Num: 1, files: 000027.sst 2024-04-26T04:47:11.793 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-smithi104/store.db: 000025.log size: 8595188 ; 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.error_if_exists: 0 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.create_if_missing: 0 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.paranoid_checks: 1 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.env: 0x55f8eefbbc60 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.fs: PosixFileSystem 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.info_log: 0x55f8f060b960 2024-04-26T04:47:11.794 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_file_opening_threads: 16 2024-04-26T04:47:11.795 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.statistics: (nil) 2024-04-26T04:47:11.795 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.use_fsync: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_log_file_size: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.keep_log_file_num: 1000 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.recycle_log_file_num: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.allow_fallocate: 1 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.allow_mmap_reads: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.allow_mmap_writes: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.use_direct_reads: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.create_missing_column_families: 0 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.db_log_dir: 2024-04-26T04:47:11.796 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.wal_dir: 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.advise_random_on_open: 1 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.db_write_buffer_size: 0 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.write_buffer_manager: 0x55f8f05de1e0 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.rate_limiter: (nil) 2024-04-26T04:47:11.797 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.wal_recovery_mode: 2 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.enable_thread_tracking: 0 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.enable_pipelined_write: 0 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.unordered_write: 0 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.row_cache: None 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.wal_filter: None 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.allow_ingest_behind: 0 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.two_write_queues: 0 2024-04-26T04:47:11.798 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.manual_wal_flush: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.wal_compression: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.atomic_flush: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.log_readahead_size: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.best_efforts_recovery: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.allow_data_in_errors: 0 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.db_host_id: __hostname__ 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_background_jobs: 2 2024-04-26T04:47:11.799 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_background_compactions: -1 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_subcompactions: 1 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_total_wal_size: 0 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_open_files: -1 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bytes_per_sync: 0 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_readahead_size: 0 2024-04-26T04:47:11.800 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_background_flushes: -1 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Compression algorithms supported: 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kZSTD supported: 0 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kXpressCompression supported: 0 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kLZ4HCCompression supported: 1 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kZlibCompression supported: 1 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kSnappyCompression supported: 1 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kLZ4Compression supported: 1 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: kBZip2Compression supported: 0 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-smithi104/store.db/MANIFEST-000010 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-26T04:47:11.801 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.merge_operator: 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_filter: None 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_filter_factory: None 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.sst_partitioner_factory: None 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f8f060ba60) 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: cache_index_and_filter_blocks: 1 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: pin_top_level_index_and_filter: 1 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: index_type: 0 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: data_block_index_type: 0 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: index_shortening: 1 2024-04-26T04:47:11.802 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: checksum: 4 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: no_block_cache: 0 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_cache: 0x55f8f060f090 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_cache_name: BinnedLRUCache 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_cache_options: 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: capacity : 536870912 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: num_shard_bits : 4 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: strict_capacity_limit : 0 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: high_pri_pool_ratio: 0.000 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_cache_compressed: (nil) 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: persistent_cache: (nil) 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_size: 4096 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_size_deviation: 10 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_restart_interval: 16 2024-04-26T04:47:11.803 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: index_block_restart_interval: 1 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: metadata_block_size: 4096 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: partition_filters: 0 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: use_delta_encoding: 1 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: filter_policy: bloomfilter 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: whole_key_filtering: 1 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: verify_compression: 0 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: read_amp_bytes_per_bit: 0 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: format_version: 5 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: enable_index_compression: 1 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: block_align: 0 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: max_auto_readahead_size: 262144 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: prepopulate_block_cache: 0 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: initial_auto_readahead_size: 8192 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: num_file_reads_for_auto_readahead: 2 2024-04-26T04:47:11.804 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.write_buffer_size: 33554432 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_write_buffer_number: 2 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression: NoCompression 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression: Disabled 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.prefix_extractor: nullptr 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.num_levels: 7 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-26T04:47:11.805 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.level: 32767 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.strategy: 0 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.enabled: false 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-26T04:47:11.806 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.target_file_size_base: 67108864 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-26T04:47:11.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.arena_block_size: 1048576 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.disable_auto_compactions: 0 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-26T04:47:11.808 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.table_properties_collectors: 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.inplace_update_support: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.bloom_locality: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.max_successive_merges: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.paranoid_file_checks: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.force_consistency_checks: 1 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.report_bg_io_stats: 0 2024-04-26T04:47:11.809 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.ttl: 2592000 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.enable_blob_files: false 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.min_blob_size: 0 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.blob_file_size: 268435456 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.blob_file_starting_level: 0 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-26T04:47:11.810 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:47:11.811 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-smithi104/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 29, last_sequence is 12197, log_number is 25,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 25 2024-04-26T04:47:11.811 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 25 2024-04-26T04:47:11.811 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 6dc9523f-cb5c-44ee-a86b-97d926661c66 2024-04-26T04:47:11.811 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106831789825, "job": 1, "event": "recovery_started", "wal_files": [25]} 2024-04-26T04:47:11.811 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #25 mode 2 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106831810875, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 30, "file_size": 5099246, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 12202, "largest_seqno": 16041, "table_properties": {"data_size": 5085738, "index_size": 8519, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 3973, "raw_key_size": 35345, "raw_average_key_size": 22, "raw_value_size": 5054495, "raw_average_value_size": 3188, "num_data_blocks": 386, "num_entries": 1585, "num_filter_entries": 1585, "num_deletions": 259, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1714106831, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "6dc9523f-cb5c-44ee-a86b-97d926661c66", "db_session_id": "XJ6TTOLTDIMPQ6DRUPCF", "orig_file_number": 30, "seqno_to_time_mapping": "N/A"}} 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: EVENT_LOG_v1 {"time_micros": 1714106831810989, "job": 1, "event": "recovery_finished"} 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/version_set.cc:5047] Creating manifest 32 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/version_set.cc:4390] More existing levels in DB than needed. max_bytes_for_level_multiplier may not be guaranteed. 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-smithi104/store.db/000025.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55f8f0700000 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: DB pointer 0x55f8f06ea000 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-26T04:47:12.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: ** DB Stats ** 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: ** Compaction Stats [default] ** 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-26T04:47:12.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: L0 1/0 4.86 MB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 540.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: L6 1/0 5.43 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Sum 2/0 10.29 MB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 540.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 540.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: ** Compaction Stats [default] ** 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 540.9 0.01 0.00 1 0.009 0 0 0.0 0.0 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-26T04:47:12.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Flush(GB): cumulative 0.005, interval 0.005 2024-04-26T04:47:12.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-26T04:47:12.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-26T04:47:12.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-26T04:47:12.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-26T04:47:12.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Cumulative compaction: 0.00 GB write, 204.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:47:12.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Interval compaction: 0.00 GB write, 204.60 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-26T04:47:12.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Block cache BinnedLRUCache@0x55f8f060f090#2 capacity: 512.00 MB usage: 13.11 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.3e-05 secs_since: 0 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,4.03 KB,0.0007689%) IndexBlock(1,9.08 KB,0.00173151%) Misc(1,0.00 KB,0%) 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: starting mon.smithi104 rank 1 at public addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] at bind addrs [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon_data /var/lib/ceph/mon/ceph-smithi104 fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???) e2 preinit fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???).mds e21 new map 2024-04-26T04:47:12.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???).mds e21 print_map 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: e21 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: legacy client fscid: 1 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Filesystem 'cephfs' (1) 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: fs_name cephfs 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: epoch 21 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: tableserver 0 2024-04-26T04:47:12.017 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: root 0 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: session_timeout 60 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: session_autoclose 300 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: max_file_size 1099511627776 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: max_xattr_size 65536 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: required_client_features {} 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: last_failure 0 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: last_failure_osd_epoch 46 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: max_mds 2 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: in 0,1 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: up {0=14486,1=24273} 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: failed 2024-04-26T04:47:12.018 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: damaged 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: stopped 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: data_pools [3] 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: metadata_pool 2 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: inline_data disabled 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: balancer 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: bal_rank_mask -1 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: standby_count_wanted 1 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: qdb_cluster leader: 0 members: 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: [mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: [mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.019 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: Standby daemons: 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: [mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout: [mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???).osd e50 crush map has features 3314933000852226048, adjusting msgr requires 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???).osd e50 crush map has features 288514051259236352, adjusting msgr requires 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 ceph-mon[64778]: mon.smithi104@-1(???).paxosservice(auth 1..24) refresh upgraded, format 0 -> 3 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 bash[64759]: 11c4dd69e45404d95fa5efef1ddbaf1523edb2280db940cd517aa5ba4486da15 2024-04-26T04:47:12.020 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:11 smithi104 systemd[1]: Started Ceph mon.smithi104 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:47:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:12 smithi090 ceph-mon[86550]: pgmap v10: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:47:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:12 smithi104 ceph-mon[64778]: pgmap v10: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: osdmap e50: 6 total, 6 up, 6 in 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: mgrmap e37: smithi090.iwrvwa(active, since 16s), standbys: smithi104.wwoewi 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: overall HEALTH_OK 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: mon.smithi090 calling monitor election 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: mon.smithi104 calling monitor election 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: mon.smithi090 is new leader, mons smithi090,smithi104 in quorum (ranks 0,1) 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: monmap epoch 3 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: last_changed 2024-04-26T04:47:12.098138+0000 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: created 2024-04-26T04:32:53.473219+0000 2024-04-26T04:47:13.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: min_mon_release 19 (squid) 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: election_strategy: 1 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.smithi104 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: osdmap e50: 6 total, 6 up, 6 in 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: mgrmap e37: smithi090.iwrvwa(active, since 16s), standbys: smithi104.wwoewi 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: overall HEALTH_OK 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:13.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:13 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: osdmap e50: 6 total, 6 up, 6 in 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: mgrmap e37: smithi090.iwrvwa(active, since 16s), standbys: smithi104.wwoewi 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: overall HEALTH_OK 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: mon.smithi090 calling monitor election 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: mon.smithi104 calling monitor election 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi090"}]: dispatch 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: mon.smithi090 is new leader, mons smithi090,smithi104 in quorum (ranks 0,1) 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: monmap epoch 3 2024-04-26T04:47:13.470 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: fsid f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: last_changed 2024-04-26T04:47:12.098138+0000 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: created 2024-04-26T04:32:53.473219+0000 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: min_mon_release 19 (squid) 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: election_strategy: 1 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: 0: [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] mon.smithi090 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: 1: [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] mon.smithi104 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: fsmap cephfs:2 {0=cephfs.smithi090.dbkjuk=up:active,1=cephfs.smithi104.wpntdi=up:active} 2 up:standby 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: osdmap e50: 6 total, 6 up, 6 in 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: mgrmap e37: smithi090.iwrvwa(active, since 16s), standbys: smithi104.wwoewi 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: overall HEALTH_OK 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mon metadata", "id": "smithi104"}]: dispatch 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:13 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.466+0000 7f0b740a2700 1 -- 172.21.15.90:0/1617299074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c100e80 msgr2=0x7f0b6c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.466+0000 7f0b740a2700 1 --2- 172.21.15.90:0/1617299074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c100e80 0x7f0b6c104e50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0b68009910 tx=0x7f0b68009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.466+0000 7f0b740a2700 1 -- 172.21.15.90:0/1617299074 shutdown_connections 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.466+0000 7f0b740a2700 1 --2- 172.21.15.90:0/1617299074 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c100e80 0x7f0b6c104e50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.466+0000 7f0b740a2700 1 --2- 172.21.15.90:0/1617299074 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c1004e0 0x7f0b6c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.466+0000 7f0b740a2700 1 -- 172.21.15.90:0/1617299074 >> 172.21.15.90:0/1617299074 conn(0x7f0b6c0fbda0 msgr2=0x7f0b6c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.467+0000 7f0b740a2700 1 -- 172.21.15.90:0/1617299074 shutdown_connections 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.467+0000 7f0b740a2700 1 -- 172.21.15.90:0/1617299074 wait complete. 2024-04-26T04:47:13.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.467+0000 7f0b740a2700 1 Processor -- start 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.467+0000 7f0b740a2700 1 -- start start 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b740a2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c1004e0 0x7f0b6c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b740a2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 0x7f0b6c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b740a2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c196aa0 con 0x7f0b6c1004e0 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b740a2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b6c196be0 con 0x7f0b6c100e80 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b71e3e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c1004e0 0x7f0b6c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b71e3e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c1004e0 0x7f0b6c195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49922/0 (socket says 172.21.15.90:49922) 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b71e3e700 1 -- 172.21.15.90:0/2399753439 learned_addr learned my addr 172.21.15.90:0/2399753439 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b7163d700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 0x7f0b6c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b7163d700 1 -- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 msgr2=0x7f0b6c196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b7163d700 1 -- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 msgr2=0x7f0b6c196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b7163d700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 0x7f0b6c196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b71e3e700 1 -- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 msgr2=0x7f0b6c196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:13.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b7163d700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 0x7f0b6c196450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b71e3e700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 0x7f0b6c196450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b71e3e700 1 -- 172.21.15.90:0/2399753439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b680095f0 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b71e3e700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c1004e0 0x7f0b6c195f10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c930 tx=0x7f0b5c00ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b62ffd700 1 -- 172.21.15.90:0/2399753439 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c007ad0 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.468+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b6c19aa30 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.469+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b6c19af80 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.469+0000 7f0b62ffd700 1 -- 172.21.15.90:0/2399753439 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0b5c00a5f0 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.469+0000 7f0b62ffd700 1 -- 172.21.15.90:0/2399753439 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0b5c018630 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.470+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b50005320 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.470+0000 7f0b62ffd700 1 -- 172.21.15.90:0/2399753439 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0b5c018790 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.470+0000 7f0b62ffd700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0b5806e340 0x7f0b580707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.470+0000 7f0b62ffd700 1 -- 172.21.15.90:0/2399753439 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0b5c08e4d0 con 0x7f0b6c1004e0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.471+0000 7f0b7163d700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0b5806e340 0x7f0b580707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:13.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.471+0000 7f0b7163d700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0b5806e340 0x7f0b580707f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c197530 tx=0x7f0b6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:13.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.475+0000 7f0b62ffd700 1 -- 172.21.15.90:0/2399753439 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0b5c05b170 con 0x7f0b6c1004e0 2024-04-26T04:47:13.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.696+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b50000bf0 con 0x7f0b5806e340 2024-04-26T04:47:13.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.698+0000 7f0b62ffd700 1 -- 172.21.15.90:0/2399753439 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f0b50000bf0 con 0x7f0b5806e340 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0b5806e340 msgr2=0x7f0b580707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0b5806e340 0x7f0b580707f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c197530 tx=0x7f0b6801a040 comp rx=0 tx=0).stop 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c1004e0 msgr2=0x7f0b6c195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c1004e0 0x7f0b6c195f10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f0b5c00c930 tx=0x7f0b5c00ccf0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 shutdown_connections 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0b5806e340 0x7f0b580707f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0b6c1004e0 0x7f0b6c195f10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 --2- 172.21.15.90:0/2399753439 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0b6c100e80 0x7f0b6c196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 >> 172.21.15.90:0/2399753439 conn(0x7f0b6c0fbda0 msgr2=0x7f0b6c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 shutdown_connections 2024-04-26T04:47:13.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.700+0000 7f0b740a2700 1 -- 172.21.15.90:0/2399753439 wait complete. 2024-04-26T04:47:13.712 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 -- 172.21.15.90:0/961590154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ffc60 msgr2=0x7f0f4010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 --2- 172.21.15.90:0/961590154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ffc60 0x7f0f4010a6d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0f34009910 tx=0x7f0f34009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 -- 172.21.15.90:0/961590154 shutdown_connections 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 --2- 172.21.15.90:0/961590154 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ffc60 0x7f0f4010a6d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 --2- 172.21.15.90:0/961590154 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ff350 0x7f0f400ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 -- 172.21.15.90:0/961590154 >> 172.21.15.90:0/961590154 conn(0x7f0f400748b0 msgr2=0x7f0f40074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 -- 172.21.15.90:0/961590154 shutdown_connections 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.807+0000 7f0f46a13700 1 -- 172.21.15.90:0/961590154 wait complete. 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f46a13700 1 Processor -- start 2024-04-26T04:47:13.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f46a13700 1 -- start start 2024-04-26T04:47:13.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f46a13700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ff350 0x7f0f40196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:13.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f46a13700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 0x7f0f40196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:13.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f46a13700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f40196c20 con 0x7f0f400ff350 2024-04-26T04:47:13.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f46a13700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4019a9b0 con 0x7f0f400ffc60 2024-04-26T04:47:13.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ff350 0x7f0f40196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3ffff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ff350 0x7f0f40196000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49938/0 (socket says 172.21.15.90:49938) 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3ffff700 1 -- 172.21.15.90:0/1238428907 learned_addr learned my addr 172.21.15.90:0/1238428907 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3f7fe700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 0x7f0f40196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3f7fe700 1 -- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 msgr2=0x7f0f40196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3f7fe700 1 -- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 msgr2=0x7f0f40196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3f7fe700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 0x7f0f40196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.808+0000 7f0f3f7fe700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 0x7f0f40196540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f3ffff700 1 -- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 msgr2=0x7f0f40196540 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f3ffff700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 0x7f0f40196540 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f3ffff700 1 -- 172.21.15.90:0/1238428907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f340095f0 con 0x7f0f400ff350 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f3ffff700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ff350 0x7f0f40196000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b480 tx=0x7f0f3000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f3d7fa700 1 -- 172.21.15.90:0/1238428907 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f300118c0 con 0x7f0f400ff350 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f3d7fa700 1 -- 172.21.15.90:0/1238428907 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f30010380 con 0x7f0f400ff350 2024-04-26T04:47:13.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f4019ac90 con 0x7f0f400ff350 2024-04-26T04:47:13.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f3d7fa700 1 -- 172.21.15.90:0/1238428907 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f3000f4f0 con 0x7f0f400ff350 2024-04-26T04:47:13.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.809+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f4019b1e0 con 0x7f0f400ff350 2024-04-26T04:47:13.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.810+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f40107dd0 con 0x7f0f400ff350 2024-04-26T04:47:13.813 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.810+0000 7f0f3d7fa700 1 -- 172.21.15.90:0/1238428907 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0f300104f0 con 0x7f0f400ff350 2024-04-26T04:47:13.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.811+0000 7f0f3d7fa700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f2806e340 0x7f0f280707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:13.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.811+0000 7f0f3f7fe700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f2806e340 0x7f0f280707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:13.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.811+0000 7f0f3d7fa700 1 -- 172.21.15.90:0/1238428907 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0f3008deb0 con 0x7f0f400ff350 2024-04-26T04:47:13.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.811+0000 7f0f3f7fe700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f2806e340 0x7f0f280707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f340059d0 tx=0x7f0f3401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:13.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:13.816+0000 7f0f3d7fa700 1 -- 172.21.15.90:0/1238428907 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0f30057130 con 0x7f0f400ff350 2024-04-26T04:47:14.033 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.032+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f0f4019b460 con 0x7f0f2806e340 2024-04-26T04:47:14.034 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.034+0000 7f0f3d7fa700 1 -- 172.21.15.90:0/1238428907 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+368 (secure 0 0 0) 0x7f0f4019b460 con 0x7f0f2806e340 2024-04-26T04:47:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f2806e340 msgr2=0x7f0f280707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f2806e340 0x7f0f280707f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f0f340059d0 tx=0x7f0f3401a040 comp rx=0 tx=0).stop 2024-04-26T04:47:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ff350 msgr2=0x7f0f40196000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ff350 0x7f0f40196000 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0f3000b480 tx=0x7f0f3000b840 comp rx=0 tx=0).stop 2024-04-26T04:47:14.037 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 shutdown_connections 2024-04-26T04:47:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f2806e340 0x7f0f280707f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f400ff350 0x7f0f40196000 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 --2- 172.21.15.90:0/1238428907 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f400ffc60 0x7f0f40196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 >> 172.21.15.90:0/1238428907 conn(0x7f0f400748b0 msgr2=0x7f0f40104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 shutdown_connections 2024-04-26T04:47:14.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.036+0000 7f0f46a13700 1 -- 172.21.15.90:0/1238428907 wait complete. 2024-04-26T04:47:14.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 -- 172.21.15.90:0/130833166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ffd60 msgr2=0x7f01dc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/130833166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ffd60 0x7f01dc103d30 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f01c4009910 tx=0x7f01c4009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:14.144 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 -- 172.21.15.90:0/130833166 shutdown_connections 2024-04-26T04:47:14.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/130833166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ffd60 0x7f01dc103d30 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/130833166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01dc0ff3c0 0x7f01dc0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 -- 172.21.15.90:0/130833166 >> 172.21.15.90:0/130833166 conn(0x7f01dc0fac60 msgr2=0x7f01dc0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:14.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 -- 172.21.15.90:0/130833166 shutdown_connections 2024-04-26T04:47:14.145 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.143+0000 7f01e1e8c700 1 -- 172.21.15.90:0/130833166 wait complete. 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01e1e8c700 1 Processor -- start 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01e1e8c700 1 -- start start 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01e1e8c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ff3c0 0x7f01dc196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01e1e8c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01dc0ffd60 0x7f01dc197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01e1e8c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc197880 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01e1e8c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01dc1979c0 con 0x7f01dc0ffd60 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01db7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ff3c0 0x7f01dc196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01db7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ff3c0 0x7f01dc196d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49958/0 (socket says 172.21.15.90:49958) 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01db7fe700 1 -- 172.21.15.90:0/1704288015 learned_addr learned my addr 172.21.15.90:0/1704288015 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01d2ffd700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01dc0ffd60 0x7f01dc197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01db7fe700 1 -- 172.21.15.90:0/1704288015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01dc0ffd60 msgr2=0x7f01dc197260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01db7fe700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01dc0ffd60 0x7f01dc197260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.144+0000 7f01db7fe700 1 -- 172.21.15.90:0/1704288015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c40095f0 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.145+0000 7f01db7fe700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ff3c0 0x7f01dc196d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00d8d0 tx=0x7f01cc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.145+0000 7f01d97fa700 1 -- 172.21.15.90:0/1704288015 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01cc009940 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.145+0000 7f01d97fa700 1 -- 172.21.15.90:0/1704288015 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f01cc010460 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.145+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01dc19fd90 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.145+0000 7f01d97fa700 1 -- 172.21.15.90:0/1704288015 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f01cc00f5f0 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.145+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01dc1a0250 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.146+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01dc107730 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.146+0000 7f01d97fa700 1 -- 172.21.15.90:0/1704288015 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f01cc0105d0 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.147+0000 7f01d97fa700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f01c806e330 0x7f01c80707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.147+0000 7f01d97fa700 1 -- 172.21.15.90:0/1704288015 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f01cc08e2c0 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.147+0000 7f01d2ffd700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f01c806e330 0x7f01c80707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.149 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.147+0000 7f01d2ffd700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f01c806e330 0x7f01c80707e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01c4006010 tx=0x7f01c401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:14.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.151+0000 7f01d97fa700 1 -- 172.21.15.90:0/1704288015 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f01cc0574b0 con 0x7f01dc0ff3c0 2024-04-26T04:47:14.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.361+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f01dc0694d0 con 0x7f01c806e330 2024-04-26T04:47:14.362 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:14 smithi090 ceph-mon[86550]: pgmap v12: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:47:14.362 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:14 smithi090 ceph-mon[86550]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.373+0000 7f01d97fa700 1 -- 172.21.15.90:0/1704288015 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f01dc0694d0 con 0x7f01c806e330 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (2m) 13s ago 12m 17.0M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (13m) 13s ago 13m 8010k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (12m) 15s ago 11m 7952k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (12m) 13s ago 12m 6665k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (11m) 15s ago 11m 6665k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (99s) 13s ago 12m 74.1M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (7m) 13s ago 7m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:47:14.374 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (7m) 13s ago 7m 15.5M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (7m) 15s ago 7m 16.0M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (7m) 15s ago 7m 131M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (3m) 13s ago 14m 471M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (2m) 15s ago 11m 447M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (29s) 13s ago 14m 28.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 starting - - - 2048M 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (2m) 13s ago 12m 17.2M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (2m) 15s ago 11m 14.7M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (10m) 13s ago 10m 229M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (10m) 13s ago 10m 246M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (10m) 13s ago 10m 202M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (9m) 15s ago 9m 271M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (9m) 15s ago 9m 235M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (8m) 15s ago 8m 241M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:47:14.375 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (2m) 13s ago 12m 56.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:47:14.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f01c806e330 msgr2=0x7f01c80707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f01c806e330 0x7f01c80707e0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f01c4006010 tx=0x7f01c401a040 comp rx=0 tx=0).stop 2024-04-26T04:47:14.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ff3c0 msgr2=0x7f01dc196d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ff3c0 0x7f01dc196d20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01cc00d8d0 tx=0x7f01cc00dc90 comp rx=0 tx=0).stop 2024-04-26T04:47:14.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 shutdown_connections 2024-04-26T04:47:14.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f01c806e330 0x7f01c80707e0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f01dc0ff3c0 0x7f01dc196d20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 --2- 172.21.15.90:0/1704288015 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f01dc0ffd60 0x7f01dc197260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 >> 172.21.15.90:0/1704288015 conn(0x7f01dc0fac60 msgr2=0x7f01dc0fd060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:14.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 shutdown_connections 2024-04-26T04:47:14.377 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.375+0000 7f01e1e8c700 1 -- 172.21.15.90:0/1704288015 wait complete. 2024-04-26T04:47:14.385 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:14 smithi104 ceph-mon[64778]: pgmap v12: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 17 KiB/s rd, 0 B/s wr, 9 op/s 2024-04-26T04:47:14.385 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:14 smithi104 ceph-mon[64778]: from='client.34132 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:14.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 -- 172.21.15.90:0/166500778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 msgr2=0x7fda58100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/166500778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58100040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fda40009910 tx=0x7fda40009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 -- 172.21.15.90:0/166500778 shutdown_connections 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/166500778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58100040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/166500778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 0x7fda58104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 -- 172.21.15.90:0/166500778 >> 172.21.15.90:0/166500778 conn(0x7fda58075130 msgr2=0x7fda58075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 -- 172.21.15.90:0/166500778 shutdown_connections 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.478+0000 7fda5e1f0700 1 -- 172.21.15.90:0/166500778 wait complete. 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda5e1f0700 1 Processor -- start 2024-04-26T04:47:14.480 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda5e1f0700 1 -- start start 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda5e1f0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58194da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda5e1f0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 0x7fda581952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda5e1f0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda581959c0 con 0x7fda580ffbd0 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda5e1f0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda58199750 con 0x7fda58104130 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda577fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58194da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda577fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58194da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:49976/0 (socket says 172.21.15.90:49976) 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda577fe700 1 -- 172.21.15.90:0/2072751810 learned_addr learned my addr 172.21.15.90:0/2072751810 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda4effd700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 0x7fda581952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda4effd700 1 -- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 msgr2=0x7fda581952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda4effd700 1 -- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 msgr2=0x7fda581952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda4effd700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 0x7fda581952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda4effd700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 0x7fda581952e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda577fe700 1 -- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 msgr2=0x7fda581952e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda577fe700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 0x7fda581952e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.479+0000 7fda577fe700 1 -- 172.21.15.90:0/2072751810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400095f0 con 0x7fda580ffbd0 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.480+0000 7fda577fe700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58194da0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fda4800d8d0 tx=0x7fda4800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.480+0000 7fda557fa700 1 -- 172.21.15.90:0/2072751810 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda48004a00 con 0x7fda580ffbd0 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.480+0000 7fda557fa700 1 -- 172.21.15.90:0/2072751810 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fda480054e0 con 0x7fda580ffbd0 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.480+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda58199a30 con 0x7fda580ffbd0 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.480+0000 7fda557fa700 1 -- 172.21.15.90:0/2072751810 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fda48009d60 con 0x7fda580ffbd0 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.480+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda58199f80 con 0x7fda580ffbd0 2024-04-26T04:47:14.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.481+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda5804f2a0 con 0x7fda580ffbd0 2024-04-26T04:47:14.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.481+0000 7fda557fa700 1 -- 172.21.15.90:0/2072751810 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fda48004f50 con 0x7fda580ffbd0 2024-04-26T04:47:14.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.482+0000 7fda557fa700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fda4406e330 0x7fda440707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.482+0000 7fda557fa700 1 -- 172.21.15.90:0/2072751810 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fda4808df70 con 0x7fda580ffbd0 2024-04-26T04:47:14.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.482+0000 7fda4effd700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fda4406e330 0x7fda440707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.482+0000 7fda4effd700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fda4406e330 0x7fda440707e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda40005b40 tx=0x7fda4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:14.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.486+0000 7fda557fa700 1 -- 172.21.15.90:0/2072751810 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fda480c2740 con 0x7fda580ffbd0 2024-04-26T04:47:14.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.774+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fda5819a1f0 con 0x7fda580ffbd0 2024-04-26T04:47:14.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.774+0000 7fda557fa700 1 -- 172.21.15.90:0/2072751810 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7fda4801f2c0 con 0x7fda580ffbd0 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:47:14.776 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-04-26T04:47:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:47:14.777 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:47:14.777 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.776+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fda4406e330 msgr2=0x7fda440707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.776+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fda4406e330 0x7fda440707e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fda40005b40 tx=0x7fda4001a040 comp rx=0 tx=0).stop 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.776+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 msgr2=0x7fda58194da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.776+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58194da0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fda4800d8d0 tx=0x7fda4800dc90 comp rx=0 tx=0).stop 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.777+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 shutdown_connections 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.777+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fda4406e330 0x7fda440707e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.777+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fda580ffbd0 0x7fda58194da0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.777+0000 7fda5e1f0700 1 --2- 172.21.15.90:0/2072751810 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fda58104130 0x7fda581952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.777+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 >> 172.21.15.90:0/2072751810 conn(0x7fda58075130 msgr2=0x7fda580febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:14.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.777+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 shutdown_connections 2024-04-26T04:47:14.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.777+0000 7fda5e1f0700 1 -- 172.21.15.90:0/2072751810 wait complete. 2024-04-26T04:47:14.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 -- 172.21.15.90:0/2774848124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 msgr2=0x7f54a01017f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:14.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/2774848124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a01017f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5490009910 tx=0x7f5490009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:14.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 -- 172.21.15.90:0/2774848124 shutdown_connections 2024-04-26T04:47:14.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/2774848124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 0x7f54a010a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/2774848124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a01017f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 -- 172.21.15.90:0/2774848124 >> 172.21.15.90:0/2774848124 conn(0x7f54a00fac60 msgr2=0x7f54a00fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:14.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 -- 172.21.15.90:0/2774848124 shutdown_connections 2024-04-26T04:47:14.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 -- 172.21.15.90:0/2774848124 wait complete. 2024-04-26T04:47:14.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 Processor -- start 2024-04-26T04:47:14.883 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.881+0000 7f54a5b9d700 1 -- start start 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f54a5b9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a00ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f54a5b9d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 0x7f54a00ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f54a5b9d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a00ffb00 con 0x7f54a0101420 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f54a5b9d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54a00ffc40 con 0x7f54a0101d30 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a00ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a00ff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50006/0 (socket says 172.21.15.90:50006) 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549f7fe700 1 -- 172.21.15.90:0/1048114320 learned_addr learned my addr 172.21.15.90:0/1048114320 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:14.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f5496ffd700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 0x7f54a00ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f5496ffd700 1 -- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 msgr2=0x7f54a00ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f5496ffd700 1 -- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 msgr2=0x7f54a00ff5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f5496ffd700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 0x7f54a00ff5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549f7fe700 1 -- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 msgr2=0x7f54a00ff5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f5496ffd700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 0x7f54a00ff5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549f7fe700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 0x7f54a00ff5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549f7fe700 1 -- 172.21.15.90:0/1048114320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54900095f0 con 0x7f54a0101420 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549f7fe700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a00ff080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5490006010 tx=0x7f549000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549d7fa700 1 -- 172.21.15.90:0/1048114320 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f549001d070 con 0x7f54a0101420 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.882+0000 7f549d7fa700 1 -- 172.21.15.90:0/1048114320 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5490004fb0 con 0x7f54a0101420 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.883+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54a00ffec0 con 0x7f54a0101420 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.883+0000 7f549d7fa700 1 -- 172.21.15.90:0/1048114320 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5490022ea0 con 0x7f54a0101420 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.883+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54a01a2350 con 0x7f54a0101420 2024-04-26T04:47:14.886 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.883+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54a0107900 con 0x7f54a0101420 2024-04-26T04:47:14.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.884+0000 7f549d7fa700 1 -- 172.21.15.90:0/1048114320 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f5490022470 con 0x7f54a0101420 2024-04-26T04:47:14.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.884+0000 7f549d7fa700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f548c06e340 0x7f548c0707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:14.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.884+0000 7f549d7fa700 1 -- 172.21.15.90:0/1048114320 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f549008f8f0 con 0x7f54a0101420 2024-04-26T04:47:14.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.885+0000 7f5496ffd700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f548c06e340 0x7f548c0707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:14.887 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.885+0000 7f5496ffd700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f548c06e340 0x7f548c0707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54a0100870 tx=0x7f548800a5c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:14.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:14.889+0000 7f549d7fa700 1 -- 172.21.15.90:0/1048114320 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5490058b10 con 0x7f54a0101420 2024-04-26T04:47:15.147 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.146+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f54a004f2a0 con 0x7f54a0101420 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.146+0000 7f549d7fa700 1 -- 172.21.15.90:0/1048114320 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f5490022720 con 0x7f54a0101420 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:47:15.148 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:47:15.149 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:47:15.150 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:15.151 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:15.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.149+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f548c06e340 msgr2=0x7f548c0707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.149+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f548c06e340 0x7f548c0707f0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54a0100870 tx=0x7f548800a5c0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.149+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 msgr2=0x7f54a00ff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.149+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a00ff080 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f5490006010 tx=0x7f549000be20 comp rx=0 tx=0).stop 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.150+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 shutdown_connections 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.150+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f548c06e340 0x7f548c0707f0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.150+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f54a0101420 0x7f54a00ff080 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.150+0000 7f54a5b9d700 1 --2- 172.21.15.90:0/1048114320 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f54a0101d30 0x7f54a00ff5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.150+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 >> 172.21.15.90:0/1048114320 conn(0x7f54a00fac60 msgr2=0x7f54a0104a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.150+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 shutdown_connections 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.150+0000 7f54a5b9d700 1 -- 172.21.15.90:0/1048114320 wait complete. 2024-04-26T04:47:15.153 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:47:15.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.253+0000 7f4ba3618700 1 -- 172.21.15.90:0/1234442092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 msgr2=0x7f4b9c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.253+0000 7f4ba3618700 1 --2- 172.21.15.90:0/1234442092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1008b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4b98009910 tx=0x7f4b98009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:15.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 -- 172.21.15.90:0/1234442092 shutdown_connections 2024-04-26T04:47:15.255 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 --2- 172.21.15.90:0/1234442092 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 0x7f4b9c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 --2- 172.21.15.90:0/1234442092 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1008b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 -- 172.21.15.90:0/1234442092 >> 172.21.15.90:0/1234442092 conn(0x7f4b9c0fbda0 msgr2=0x7f4b9c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:15.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 -- 172.21.15.90:0/1234442092 shutdown_connections 2024-04-26T04:47:15.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 -- 172.21.15.90:0/1234442092 wait complete. 2024-04-26T04:47:15.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 Processor -- start 2024-04-26T04:47:15.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.254+0000 7f4ba3618700 1 -- start start 2024-04-26T04:47:15.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba3618700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1980f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:15.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba3618700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 0x7f4b9c198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:15.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba3618700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c198d10 con 0x7f4b9c1004e0 2024-04-26T04:47:15.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba3618700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b9c19caa0 con 0x7f4b9c100e80 2024-04-26T04:47:15.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba13b4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1980f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba13b4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1980f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50026/0 (socket says 172.21.15.90:50026) 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba13b4700 1 -- 172.21.15.90:0/3524874334 learned_addr learned my addr 172.21.15.90:0/3524874334 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba0bb3700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 0x7f4b9c198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba0bb3700 1 -- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 msgr2=0x7f4b9c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba0bb3700 1 -- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 msgr2=0x7f4b9c198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba0bb3700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 0x7f4b9c198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba0bb3700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 0x7f4b9c198630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba13b4700 1 -- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 msgr2=0x7f4b9c198630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba13b4700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 0x7f4b9c198630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba13b4700 1 -- 172.21.15.90:0/3524874334 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b980095f0 con 0x7f4b9c1004e0 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba13b4700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1980f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4b98006010 tx=0x7f4b9800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4b927fc700 1 -- 172.21.15.90:0/3524874334 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b9801d070 con 0x7f4b9c1004e0 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4b927fc700 1 -- 172.21.15.90:0/3524874334 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4b9800f460 con 0x7f4b9c1004e0 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4b927fc700 1 -- 172.21.15.90:0/3524874334 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4b98017630 con 0x7f4b9c1004e0 2024-04-26T04:47:15.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b9c19cd20 con 0x7f4b9c1004e0 2024-04-26T04:47:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.255+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b9c19d210 con 0x7f4b9c1004e0 2024-04-26T04:47:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.256+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b9c1045d0 con 0x7f4b9c1004e0 2024-04-26T04:47:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.257+0000 7f4b927fc700 1 -- 172.21.15.90:0/3524874334 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4b9800f5d0 con 0x7f4b9c1004e0 2024-04-26T04:47:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.257+0000 7f4b927fc700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4b8806e340 0x7f4b880707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.257+0000 7f4b927fc700 1 -- 172.21.15.90:0/3524874334 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4b9808f850 con 0x7f4b9c1004e0 2024-04-26T04:47:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.257+0000 7f4ba0bb3700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4b8806e340 0x7f4b880707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:15.260 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.258+0000 7f4ba0bb3700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4b8806e340 0x7f4b880707f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c005950 tx=0x7f4b8c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:15.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.262+0000 7f4b927fc700 1 -- 172.21.15.90:0/3524874334 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4b980589c0 con 0x7f4b9c1004e0 2024-04-26T04:47:15.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:15 smithi090 ceph-mon[86550]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:15.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:15 smithi090 ceph-mon[86550]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:15.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:15 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2072751810' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:15.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:15 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:15.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:15 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:15.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.482+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4b9c19d4f0 con 0x7f4b8806e340 2024-04-26T04:47:15.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.489+0000 7f4b927fc700 1 -- 172.21.15.90:0/3524874334 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f4b9c19d4f0 con 0x7f4b8806e340 2024-04-26T04:47:15.490 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "4/23 daemons upgraded", 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mon daemons", 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:47:15.491 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.491+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4b8806e340 msgr2=0x7f4b880707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.491+0000 7f4ba3618700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4b8806e340 0x7f4b880707f0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4b8c005950 tx=0x7f4b8c0058e0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.491+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 msgr2=0x7f4b9c1980f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.491+0000 7f4ba3618700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1980f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4b98006010 tx=0x7f4b9800be20 comp rx=0 tx=0).stop 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.492+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 shutdown_connections 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.492+0000 7f4ba3618700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4b8806e340 0x7f4b880707f0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.492+0000 7f4ba3618700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4b9c1004e0 0x7f4b9c1980f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.492+0000 7f4ba3618700 1 --2- 172.21.15.90:0/3524874334 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4b9c100e80 0x7f4b9c198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.492+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 >> 172.21.15.90:0/3524874334 conn(0x7f4b9c0fbda0 msgr2=0x7f4b9c0fe140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.492+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 shutdown_connections 2024-04-26T04:47:15.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.492+0000 7f4ba3618700 1 -- 172.21.15.90:0/3524874334 wait complete. 2024-04-26T04:47:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:15 smithi104 ceph-mon[64778]: from='client.34136 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:15 smithi104 ceph-mon[64778]: from='client.34140 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:15 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2072751810' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:15 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:15.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:15 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 -- 172.21.15.90:0/2286355050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 msgr2=0x7f6e280ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 --2- 172.21.15.90:0/2286355050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e280ff720 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e2000b0a0 tx=0x7f6e2000b3b0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 -- 172.21.15.90:0/2286355050 shutdown_connections 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 --2- 172.21.15.90:0/2286355050 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 0x7f6e2810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 --2- 172.21.15.90:0/2286355050 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e280ff720 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 -- 172.21.15.90:0/2286355050 >> 172.21.15.90:0/2286355050 conn(0x7f6e280748b0 msgr2=0x7f6e28074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 -- 172.21.15.90:0/2286355050 shutdown_connections 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 -- 172.21.15.90:0/2286355050 wait complete. 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.597+0000 7f6e2c852700 1 Processor -- start 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2c852700 1 -- start start 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2c852700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e28195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2c852700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 0x7f6e281964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:15.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2c852700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e28196b80 con 0x7f6e280ff350 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2c852700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2819a910 con 0x7f6e280ffc60 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e28195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e28195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50054/0 (socket says 172.21.15.90:50054) 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2659c700 1 -- 172.21.15.90:0/3642734992 learned_addr learned my addr 172.21.15.90:0/3642734992 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e25d9b700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 0x7f6e281964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e25d9b700 1 -- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 msgr2=0x7f6e281964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e25d9b700 1 -- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 msgr2=0x7f6e281964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e25d9b700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 0x7f6e281964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2659c700 1 -- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 msgr2=0x7f6e281964a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e25d9b700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 0x7f6e281964a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2659c700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 0x7f6e281964a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2659c700 1 -- 172.21.15.90:0/3642734992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e20009d00 con 0x7f6e280ff350 2024-04-26T04:47:15.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2659c700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e28195f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009fd0 tx=0x7f6e2000cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:15.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e177fe700 1 -- 172.21.15.90:0/3642734992 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e2000e040 con 0x7f6e280ff350 2024-04-26T04:47:15.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e177fe700 1 -- 172.21.15.90:0/3642734992 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e20003de0 con 0x7f6e280ff350 2024-04-26T04:47:15.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e2819ab90 con 0x7f6e280ff350 2024-04-26T04:47:15.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.598+0000 7f6e177fe700 1 -- 172.21.15.90:0/3642734992 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e2001d780 con 0x7f6e280ff350 2024-04-26T04:47:15.602 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.599+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e2819b080 con 0x7f6e280ff350 2024-04-26T04:47:15.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.600+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e28107dd0 con 0x7f6e280ff350 2024-04-26T04:47:15.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.600+0000 7f6e177fe700 1 -- 172.21.15.90:0/3642734992 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f6e20004040 con 0x7f6e280ff350 2024-04-26T04:47:15.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.600+0000 7f6e177fe700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e1006e330 0x7f6e100707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:15.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.600+0000 7f6e177fe700 1 -- 172.21.15.90:0/3642734992 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f6e2008fbb0 con 0x7f6e280ff350 2024-04-26T04:47:15.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.600+0000 7f6e25d9b700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e1006e330 0x7f6e100707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:15.603 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.601+0000 7f6e25d9b700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e1006e330 0x7f6e100707e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005950 tx=0x7f6e1c0058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:15.606 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.605+0000 7f6e177fe700 1 -- 172.21.15.90:0/3642734992 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6e20096080 con 0x7f6e280ff350 2024-04-26T04:47:15.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.888+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6e2804f2a0 con 0x7f6e280ff350 2024-04-26T04:47:15.890 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.888+0000 7f6e177fe700 1 -- 172.21.15.90:0/3642734992 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f6e20058d20 con 0x7f6e280ff350 2024-04-26T04:47:15.890 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_OK 2024-04-26T04:47:15.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e1006e330 msgr2=0x7f6e100707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e1006e330 0x7f6e100707e0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6e1c005950 tx=0x7f6e1c0058e0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 msgr2=0x7f6e28195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e28195f60 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e20009fd0 tx=0x7f6e2000cdd0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 shutdown_connections 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e1006e330 0x7f6e100707e0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e280ff350 0x7f6e28195f60 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 --2- 172.21.15.90:0/3642734992 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e280ffc60 0x7f6e281964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 >> 172.21.15.90:0/3642734992 conn(0x7f6e280748b0 msgr2=0x7f6e28104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 shutdown_connections 2024-04-26T04:47:15.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:15.891+0000 7f6e2c852700 1 -- 172.21.15.90:0/3642734992 wait complete. 2024-04-26T04:47:16.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:16 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1048114320' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:47:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:16 smithi090 ceph-mon[86550]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:16 smithi090 ceph-mon[86550]: pgmap v13: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:16 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3642734992' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:47:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:16 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1048114320' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:47:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:16 smithi104 ceph-mon[64778]: from='client.34152 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:16 smithi104 ceph-mon[64778]: pgmap v13: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:16 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3642734992' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:47:16.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:16.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:18.202 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:17 smithi104 ceph-mon[64778]: pgmap v14: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:18.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:17 smithi090 ceph-mon[86550]: pgmap v14: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: Detected new or changed devices on smithi104 2024-04-26T04:47:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:20.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:20.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: Reconfiguring mon.smithi090 (monmap changed)... 2024-04-26T04:47:20.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:47:20.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:47:20.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:20.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-26T04:47:20.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:20 smithi090 ceph-mon[86550]: pgmap v15: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:20.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: Detected new or changed devices on smithi104 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: Reconfiguring mon.smithi090 (monmap changed)... 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: Reconfiguring daemon mon.smithi090 on smithi090 2024-04-26T04:47:20.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:20 smithi104 ceph-mon[64778]: pgmap v15: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:22.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:22.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:22.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: Reconfiguring mgr.smithi090.iwrvwa (monmap changed)... 2024-04-26T04:47:22.341 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:47:22.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:47:22.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:22.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: Reconfiguring daemon mgr.smithi090.iwrvwa on smithi090 2024-04-26T04:47:22.342 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:22 smithi090 ceph-mon[86550]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: Reconfiguring mgr.smithi090.iwrvwa (monmap changed)... 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi090.iwrvwa", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: Reconfiguring daemon mgr.smithi090.iwrvwa on smithi090 2024-04-26T04:47:22.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:22 smithi104 ceph-mon[64778]: pgmap v16: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:23.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: Unable to update caps for client.ceph-exporter.smithi090 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:23.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:23 smithi090 ceph-mon[86550]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: Reconfiguring ceph-exporter.smithi090 (monmap changed)... 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: Unable to update caps for client.ceph-exporter.smithi090 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-26T04:47:24.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:24.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:23 smithi104 ceph-mon[64778]: Reconfiguring daemon ceph-exporter.smithi090 on smithi090 2024-04-26T04:47:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:24 smithi090 ceph-mon[86550]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:25.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:24 smithi104 ceph-mon[64778]: pgmap v17: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:25.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:25.871 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:25 smithi090 ceph-mon[86550]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-26T04:47:25.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:25 smithi090 ceph-mon[86550]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-26T04:47:25.872 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:25 smithi104 ceph-mon[64778]: Reconfiguring crash.smithi090 (monmap changed)... 2024-04-26T04:47:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:25 smithi104 ceph-mon[64778]: Reconfiguring daemon crash.smithi090 on smithi090 2024-04-26T04:47:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:26.768 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:26 smithi090 ceph-mon[86550]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:26.768 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:26.768 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:26.768 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:47:26.768 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:26 smithi104 ceph-mon[64778]: pgmap v18: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:47:27.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:27.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:27 smithi090 ceph-mon[86550]: Reconfiguring osd.0 (monmap changed)... 2024-04-26T04:47:27.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:27 smithi090 ceph-mon[86550]: Reconfiguring daemon osd.0 on smithi090 2024-04-26T04:47:27.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:27 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:27.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:27 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:27.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:27 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:47:27.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:27 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:28.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:27 smithi104 ceph-mon[64778]: Reconfiguring osd.0 (monmap changed)... 2024-04-26T04:47:28.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:27 smithi104 ceph-mon[64778]: Reconfiguring daemon osd.0 on smithi090 2024-04-26T04:47:28.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:27 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:28.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:27 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:28.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:27 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:47:28.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:27 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:28 smithi104 ceph-mon[64778]: Reconfiguring osd.1 (monmap changed)... 2024-04-26T04:47:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:28 smithi104 ceph-mon[64778]: Reconfiguring daemon osd.1 on smithi090 2024-04-26T04:47:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:28 smithi104 ceph-mon[64778]: pgmap v19: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:29.041 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:28 smithi090 ceph-mon[86550]: Reconfiguring osd.1 (monmap changed)... 2024-04-26T04:47:29.041 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:28 smithi090 ceph-mon[86550]: Reconfiguring daemon osd.1 on smithi090 2024-04-26T04:47:29.041 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:28 smithi090 ceph-mon[86550]: pgmap v19: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:30.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:30.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:30.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:30 smithi090 ceph-mon[86550]: Reconfiguring osd.2 (monmap changed)... 2024-04-26T04:47:30.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:47:30.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:30.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:30 smithi090 ceph-mon[86550]: Reconfiguring daemon osd.2 on smithi090 2024-04-26T04:47:30.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:30 smithi090 ceph-mon[86550]: pgmap v20: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:30.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:30.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:30.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:30 smithi104 ceph-mon[64778]: Reconfiguring osd.2 (monmap changed)... 2024-04-26T04:47:30.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:47:30.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:30.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:30 smithi104 ceph-mon[64778]: Reconfiguring daemon osd.2 on smithi090 2024-04-26T04:47:30.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:30 smithi104 ceph-mon[64778]: pgmap v20: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:32.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:31 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:32.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:31 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:32.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:31 smithi090 ceph-mon[86550]: Reconfiguring mds.cephfs.smithi090.ehputv (monmap changed)... 2024-04-26T04:47:32.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:31 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:32.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:31 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:32.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:31 smithi090 ceph-mon[86550]: Reconfiguring daemon mds.cephfs.smithi090.ehputv on smithi090 2024-04-26T04:47:32.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:31 smithi090 ceph-mon[86550]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:32.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:31 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:32.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:31 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:32.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:31 smithi104 ceph-mon[64778]: Reconfiguring mds.cephfs.smithi090.ehputv (monmap changed)... 2024-04-26T04:47:32.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:31 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:32.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:31 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:32.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:31 smithi104 ceph-mon[64778]: Reconfiguring daemon mds.cephfs.smithi090.ehputv on smithi090 2024-04-26T04:47:32.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:31 smithi104 ceph-mon[64778]: pgmap v21: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:33.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:33.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:33.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:33 smithi090 ceph-mon[86550]: Reconfiguring mds.cephfs.smithi090.dbkjuk (monmap changed)... 2024-04-26T04:47:33.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:33.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:33.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:33 smithi090 ceph-mon[86550]: Reconfiguring daemon mds.cephfs.smithi090.dbkjuk on smithi090 2024-04-26T04:47:34.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:34.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:34.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:33 smithi104 ceph-mon[64778]: Reconfiguring mds.cephfs.smithi090.dbkjuk (monmap changed)... 2024-04-26T04:47:34.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:34.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:34.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:33 smithi104 ceph-mon[64778]: Reconfiguring daemon mds.cephfs.smithi090.dbkjuk on smithi090 2024-04-26T04:47:34.874 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: pgmap v22: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:34.875 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:34.875 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:34.875 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:34.875 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:34.875 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-26T04:47:34.875 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi104"}]: dispatch 2024-04-26T04:47:34.875 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: pgmap v22: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:34.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:34.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:34.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:47:34.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "auth caps", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]': finished 2024-04-26T04:47:34.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.ceph-exporter.smithi104"}]: dispatch 2024-04-26T04:47:34.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:35.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:35 smithi090 ceph-mon[86550]: Reconfiguring ceph-exporter.smithi104 (monmap changed)... 2024-04-26T04:47:35.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:35 smithi090 ceph-mon[86550]: Unable to update caps for client.ceph-exporter.smithi104 2024-04-26T04:47:35.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:35 smithi090 ceph-mon[86550]: Reconfiguring daemon ceph-exporter.smithi104 on smithi104 2024-04-26T04:47:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:35 smithi104 ceph-mon[64778]: Reconfiguring ceph-exporter.smithi104 (monmap changed)... 2024-04-26T04:47:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:35 smithi104 ceph-mon[64778]: Unable to update caps for client.ceph-exporter.smithi104 2024-04-26T04:47:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:35 smithi104 ceph-mon[64778]: Reconfiguring daemon ceph-exporter.smithi104 on smithi104 2024-04-26T04:47:36.913 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:36 smithi104 ceph-mon[64778]: pgmap v23: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:36.913 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:36 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:36.913 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:36 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:36.913 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:36 smithi104 ceph-mon[64778]: Reconfiguring crash.smithi104 (monmap changed)... 2024-04-26T04:47:36.913 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:36 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:36.913 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:36 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:36.913 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:36 smithi104 ceph-mon[64778]: Reconfiguring daemon crash.smithi104 on smithi104 2024-04-26T04:47:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:36 smithi090 ceph-mon[86550]: pgmap v23: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:36 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:36 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:36.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:36 smithi090 ceph-mon[86550]: Reconfiguring crash.smithi104 (monmap changed)... 2024-04-26T04:47:36.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:36 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:36.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:36 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:36.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:36 smithi090 ceph-mon[86550]: Reconfiguring daemon crash.smithi104 on smithi104 2024-04-26T04:47:38.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:38.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: Reconfiguring mgr.smithi104.wwoewi (monmap changed)... 2024-04-26T04:47:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:47:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:47:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: Reconfiguring daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:47:38.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:38 smithi104 ceph-mon[64778]: pgmap v24: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:38.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:38.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:38.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: Reconfiguring mgr.smithi104.wwoewi (monmap changed)... 2024-04-26T04:47:38.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.smithi104.wwoewi", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-26T04:47:38.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-26T04:47:38.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:38.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: Reconfiguring daemon mgr.smithi104.wwoewi on smithi104 2024-04-26T04:47:38.723 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:38 smithi090 ceph-mon[86550]: pgmap v24: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:39.910 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:39 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:39.911 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:39 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:39.911 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:39 smithi104 ceph-mon[64778]: Reconfiguring mon.smithi104 (monmap changed)... 2024-04-26T04:47:39.911 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:39 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:47:39.911 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:39 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:47:39.911 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:39 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:39.911 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:39 smithi104 ceph-mon[64778]: Reconfiguring daemon mon.smithi104 on smithi104 2024-04-26T04:47:40.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:39 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:40.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:39 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:40.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:39 smithi090 ceph-mon[86550]: Reconfiguring mon.smithi104 (monmap changed)... 2024-04-26T04:47:40.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:39 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-26T04:47:40.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:39 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-26T04:47:40.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:39 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:40.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:39 smithi090 ceph-mon[86550]: Reconfiguring daemon mon.smithi104 on smithi104 2024-04-26T04:47:41.142 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:40 smithi104 ceph-mon[64778]: pgmap v25: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:41.142 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:41.142 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:41.142 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:47:41.142 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:41.143 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:40 smithi090 ceph-mon[86550]: pgmap v25: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:47:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:41.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:41 smithi104 ceph-mon[64778]: Reconfiguring osd.3 (monmap changed)... 2024-04-26T04:47:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:41 smithi104 ceph-mon[64778]: Reconfiguring daemon osd.3 on smithi104 2024-04-26T04:47:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:41 smithi090 ceph-mon[86550]: Reconfiguring osd.3 (monmap changed)... 2024-04-26T04:47:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:41 smithi090 ceph-mon[86550]: Reconfiguring daemon osd.3 on smithi104 2024-04-26T04:47:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:42 smithi090 ceph-mon[86550]: pgmap v26: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:42 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:42 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:42 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:47:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:42 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:42 smithi104 ceph-mon[64778]: pgmap v26: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:42 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:42 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:42 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:47:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:42 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:44.075 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:43 smithi104 ceph-mon[64778]: Reconfiguring osd.4 (monmap changed)... 2024-04-26T04:47:44.075 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:43 smithi104 ceph-mon[64778]: Reconfiguring daemon osd.4 on smithi104 2024-04-26T04:47:44.075 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:43 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:44.075 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:43 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:44.075 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:43 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:47:44.075 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:43 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:43 smithi090 ceph-mon[86550]: Reconfiguring osd.4 (monmap changed)... 2024-04-26T04:47:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:43 smithi090 ceph-mon[86550]: Reconfiguring daemon osd.4 on smithi104 2024-04-26T04:47:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:43 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:43 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:43 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:47:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:43 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:44.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:44 smithi090 ceph-mon[86550]: Reconfiguring osd.5 (monmap changed)... 2024-04-26T04:47:44.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:44 smithi090 ceph-mon[86550]: Reconfiguring daemon osd.5 on smithi104 2024-04-26T04:47:44.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:44 smithi090 ceph-mon[86550]: pgmap v27: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:44.972 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:44 smithi104 ceph-mon[64778]: Reconfiguring osd.5 (monmap changed)... 2024-04-26T04:47:44.973 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:44 smithi104 ceph-mon[64778]: Reconfiguring daemon osd.5 on smithi104 2024-04-26T04:47:44.973 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:44 smithi104 ceph-mon[64778]: pgmap v27: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:46.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.009+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/702966976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ffd60 msgr2=0x7fa3a8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.009+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/702966976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ffd60 0x7fa3a8103d30 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa39c009910 tx=0x7fa39c009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:46.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.009+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/702966976 shutdown_connections 2024-04-26T04:47:46.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.010+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/702966976 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ffd60 0x7fa3a8103d30 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.010+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/702966976 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a80ff3c0 0x7fa3a80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.013 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.010+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/702966976 >> 172.21.15.90:0/702966976 conn(0x7fa3a80fac80 msgr2=0x7fa3a80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.010+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/702966976 shutdown_connections 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.010+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/702966976 wait complete. 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.010+0000 7fa3ad3fd700 1 Processor -- start 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.010+0000 7fa3ad3fd700 1 -- start start 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3ad3fd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ff3c0 0x7fa3a8100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3ad3fd700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a80ffd60 0x7fa3a8101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3ad3fd700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a8101850 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3ad3fd700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3a8101990 con 0x7fa3a80ffd60 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ff3c0 0x7fa3a8100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a6ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ff3c0 0x7fa3a8100dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37510/0 (socket says 172.21.15.90:37510) 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a6ffd700 1 -- 172.21.15.90:0/2370201186 learned_addr learned my addr 172.21.15.90:0/2370201186 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a67fc700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a80ffd60 0x7fa3a8101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a6ffd700 1 -- 172.21.15.90:0/2370201186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a80ffd60 msgr2=0x7fa3a8101310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a6ffd700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a80ffd60 0x7fa3a8101310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a6ffd700 1 -- 172.21.15.90:0/2370201186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa39c0095f0 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3a6ffd700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ff3c0 0x7fa3a8100dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa39800d8d0 tx=0x7fa39800dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa38ffff700 1 -- 172.21.15.90:0/2370201186 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa3980098a0 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3a81a2060 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa38ffff700 1 -- 172.21.15.90:0/2370201186 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa398010460 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa38ffff700 1 -- 172.21.15.90:0/2370201186 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa39800f5f0 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.011+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3a81a2470 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.012+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3a8108740 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.013+0000 7fa38ffff700 1 -- 172.21.15.90:0/2370201186 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fa398009a00 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.013+0000 7fa38ffff700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa39006e340 0x7fa3900707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.013+0000 7fa38ffff700 1 -- 172.21.15.90:0/2370201186 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa39808e250 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.013+0000 7fa3a67fc700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa39006e340 0x7fa3900707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.014+0000 7fa3a67fc700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa39006e340 0x7fa3900707f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa3a81025c0 tx=0x7fa39c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:46.020 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.018+0000 7fa38ffff700 1 -- 172.21.15.90:0/2370201186 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa3980573c0 con 0x7fa3a80ff3c0 2024-04-26T04:47:46.239 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.238+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3a8061190 con 0x7fa39006e340 2024-04-26T04:47:46.240 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.240+0000 7fa38ffff700 1 -- 172.21.15.90:0/2370201186 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7fa3a8061190 con 0x7fa39006e340 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa39006e340 msgr2=0x7fa3900707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa39006e340 0x7fa3900707f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa3a81025c0 tx=0x7fa39c01a040 comp rx=0 tx=0).stop 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ff3c0 msgr2=0x7fa3a8100dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ff3c0 0x7fa3a8100dd0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa39800d8d0 tx=0x7fa39800dbe0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 shutdown_connections 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa39006e340 0x7fa3900707f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa3a80ff3c0 0x7fa3a8100dd0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 --2- 172.21.15.90:0/2370201186 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa3a80ffd60 0x7fa3a8101310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.243 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 >> 172.21.15.90:0/2370201186 conn(0x7fa3a80fac80 msgr2=0x7fa3a80690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:46.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 shutdown_connections 2024-04-26T04:47:46.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.242+0000 7fa3ad3fd700 1 -- 172.21.15.90:0/2370201186 wait complete. 2024-04-26T04:47:46.254 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:47:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:46 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:46 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:46 smithi104 ceph-mon[64778]: Reconfiguring mds.cephfs.smithi104.wpntdi (monmap changed)... 2024-04-26T04:47:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:46 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:46 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:46 smithi104 ceph-mon[64778]: Reconfiguring daemon mds.cephfs.smithi104.wpntdi on smithi104 2024-04-26T04:47:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:46 smithi104 ceph-mon[64778]: pgmap v28: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 -- 172.21.15.90:0/2982103835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 msgr2=0x7f78a41008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 --2- 172.21.15.90:0/2982103835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a41008b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7898009910 tx=0x7f7898009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 -- 172.21.15.90:0/2982103835 shutdown_connections 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 --2- 172.21.15.90:0/2982103835 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a4100e80 0x7f78a4104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 --2- 172.21.15.90:0/2982103835 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a41008b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 -- 172.21.15.90:0/2982103835 >> 172.21.15.90:0/2982103835 conn(0x7f78a40fbda0 msgr2=0x7f78a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 -- 172.21.15.90:0/2982103835 shutdown_connections 2024-04-26T04:47:46.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.346+0000 7f78aa32b700 1 -- 172.21.15.90:0/2982103835 wait complete. 2024-04-26T04:47:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:46 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:46 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:46 smithi090 ceph-mon[86550]: Reconfiguring mds.cephfs.smithi104.wpntdi (monmap changed)... 2024-04-26T04:47:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:46 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:46 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:46 smithi090 ceph-mon[86550]: Reconfiguring daemon mds.cephfs.smithi104.wpntdi on smithi104 2024-04-26T04:47:46.349 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:46 smithi090 ceph-mon[86550]: pgmap v28: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.347+0000 7f78aa32b700 1 Processor -- start 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.347+0000 7f78aa32b700 1 -- start start 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78aa32b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a4073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78aa32b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a4100e80 0x7f78a4073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78aa32b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4073ad0 con 0x7f78a41004e0 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78aa32b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a4073c10 con 0x7f78a4100e80 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a4073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a4073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37530/0 (socket says 172.21.15.90:37530) 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a3fff700 1 -- 172.21.15.90:0/3715446022 learned_addr learned my addr 172.21.15.90:0/3715446022 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:46.351 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a37fe700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a4100e80 0x7f78a4073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a3fff700 1 -- 172.21.15.90:0/3715446022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a4100e80 msgr2=0x7f78a4073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a3fff700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a4100e80 0x7f78a4073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a3fff700 1 -- 172.21.15.90:0/3715446022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980095f0 con 0x7f78a41004e0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a3fff700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a4073050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f78980092e0 tx=0x7f789800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a17fa700 1 -- 172.21.15.90:0/3715446022 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f789801d070 con 0x7f78a41004e0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a17fa700 1 -- 172.21.15.90:0/3715446022 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7898005e00 con 0x7f78a41004e0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a4073e90 con 0x7f78a41004e0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78a17fa700 1 -- 172.21.15.90:0/3715446022 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7898017400 con 0x7f78a41004e0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.348+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a41a23f0 con 0x7f78a41004e0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.349+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a41087f0 con 0x7f78a41004e0 2024-04-26T04:47:46.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.350+0000 7f78a17fa700 1 -- 172.21.15.90:0/3715446022 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f7898003a60 con 0x7f78a41004e0 2024-04-26T04:47:46.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.350+0000 7f78a17fa700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f788c06e330 0x7f788c0707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.350+0000 7f78a17fa700 1 -- 172.21.15.90:0/3715446022 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f789808f7d0 con 0x7f78a41004e0 2024-04-26T04:47:46.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.350+0000 7f78a37fe700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f788c06e330 0x7f788c0707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.351+0000 7f78a37fe700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f788c06e330 0x7f788c0707e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f78a4074840 tx=0x7f78940073d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:46.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.355+0000 7f78a17fa700 1 -- 172.21.15.90:0/3715446022 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7898058940 con 0x7f78a41004e0 2024-04-26T04:47:46.574 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.573+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f78a4074620 con 0x7f788c06e330 2024-04-26T04:47:46.575 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.574+0000 7f78a17fa700 1 -- 172.21.15.90:0/3715446022 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7f78a4074620 con 0x7f788c06e330 2024-04-26T04:47:46.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f788c06e330 msgr2=0x7f788c0707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f788c06e330 0x7f788c0707e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f78a4074840 tx=0x7f78940073d0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 msgr2=0x7f78a4073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a4073050 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f78980092e0 tx=0x7f789800bd70 comp rx=0 tx=0).stop 2024-04-26T04:47:46.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 shutdown_connections 2024-04-26T04:47:46.578 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f788c06e330 0x7f788c0707e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78a41004e0 0x7f78a4073050 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 --2- 172.21.15.90:0/3715446022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78a4100e80 0x7f78a4073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 >> 172.21.15.90:0/3715446022 conn(0x7f78a40fbda0 msgr2=0x7f78a40fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 shutdown_connections 2024-04-26T04:47:46.579 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.577+0000 7f78aa32b700 1 -- 172.21.15.90:0/3715446022 wait complete. 2024-04-26T04:47:46.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 -- 172.21.15.90:0/1684416515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ff350 msgr2=0x7f0c740ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/1684416515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ff350 0x7f0c740ff720 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0c68009910 tx=0x7f0c68009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:46.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 -- 172.21.15.90:0/1684416515 shutdown_connections 2024-04-26T04:47:46.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/1684416515 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ffc60 0x7f0c7410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/1684416515 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ff350 0x7f0c740ff720 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.687 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 -- 172.21.15.90:0/1684416515 >> 172.21.15.90:0/1684416515 conn(0x7f0c740748b0 msgr2=0x7f0c74074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 -- 172.21.15.90:0/1684416515 shutdown_connections 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.683+0000 7f0c7ab64700 1 -- 172.21.15.90:0/1684416515 wait complete. 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.684+0000 7f0c7ab64700 1 Processor -- start 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.684+0000 7f0c7ab64700 1 -- start start 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.684+0000 7f0c7ab64700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ff350 0x7f0c74195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.684+0000 7f0c7ab64700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ffc60 0x7f0c74196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.684+0000 7f0c7ab64700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c74196ae0 con 0x7f0c740ffc60 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.684+0000 7f0c7ab64700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c7419a870 con 0x7f0c740ff350 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c73fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ffc60 0x7f0c74196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c78900700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ff350 0x7f0c74195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c73fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ffc60 0x7f0c74196400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37556/0 (socket says 172.21.15.90:37556) 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c78900700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ff350 0x7f0c74195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.104:3300/0 says I am v2:172.21.15.90:46700/0 (socket says 172.21.15.90:46700) 2024-04-26T04:47:46.688 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c73fff700 1 -- 172.21.15.90:0/2610376924 learned_addr learned my addr 172.21.15.90:0/2610376924 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:46.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c73fff700 1 -- 172.21.15.90:0/2610376924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ff350 msgr2=0x7f0c74195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c73fff700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ff350 0x7f0c74195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c73fff700 1 -- 172.21.15.90:0/2610376924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c680095f0 con 0x7f0c740ffc60 2024-04-26T04:47:46.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c78900700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ff350 0x7f0c74195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:47:46.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c73fff700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ffc60 0x7f0c74196400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000d900 tx=0x7f0c6000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:46.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c71ffb700 1 -- 172.21.15.90:0/2610376924 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c600041f0 con 0x7f0c740ffc60 2024-04-26T04:47:46.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c71ffb700 1 -- 172.21.15.90:0/2610376924 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0c60004cf0 con 0x7f0c740ffc60 2024-04-26T04:47:46.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c7419ab50 con 0x7f0c740ffc60 2024-04-26T04:47:46.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.685+0000 7f0c71ffb700 1 -- 172.21.15.90:0/2610376924 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0c6000b750 con 0x7f0c740ffc60 2024-04-26T04:47:46.690 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.686+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c7419b0a0 con 0x7f0c740ffc60 2024-04-26T04:47:46.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.687+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c74107dd0 con 0x7f0c740ffc60 2024-04-26T04:47:46.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.687+0000 7f0c71ffb700 1 -- 172.21.15.90:0/2610376924 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f0c60004750 con 0x7f0c740ffc60 2024-04-26T04:47:46.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.688+0000 7f0c71ffb700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0c6406e330 0x7f0c640707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:46.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.688+0000 7f0c78900700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0c6406e330 0x7f0c640707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:46.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.688+0000 7f0c71ffb700 1 -- 172.21.15.90:0/2610376924 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0c6008df50 con 0x7f0c740ffc60 2024-04-26T04:47:46.691 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.688+0000 7f0c78900700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0c6406e330 0x7f0c640707e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006010 tx=0x7f0c6801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:46.693 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.692+0000 7f0c71ffb700 1 -- 172.21.15.90:0/2610376924 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0c600c2740 con 0x7f0c740ffc60 2024-04-26T04:47:46.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.903+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c7419b350 con 0x7f0c6406e330 2024-04-26T04:47:46.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.914+0000 7f0c71ffb700 1 -- 172.21.15.90:0/2610376924 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f0c7419b350 con 0x7f0c6406e330 2024-04-26T04:47:46.915 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:47:46.915 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (2m) 45s ago 13m 17.0M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:47:46.915 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (13m) 45s ago 13m 8010k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:47:46.915 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (12m) 31s ago 12m 7972k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:47:46.915 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (13m) 45s ago 13m 6665k - 18.2.0 dc2bc1663786 d7f38f7278e5 2024-04-26T04:47:46.915 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (12m) 31s ago 12m 6665k - 18.2.0 dc2bc1663786 b4f5e4086253 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (2m) 45s ago 13m 74.1M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (7m) 45s ago 7m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (7m) 45s ago 7m 15.5M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (7m) 31s ago 7m 16.3M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (7m) 31s ago 7m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (3m) 45s ago 14m 471M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (3m) 31s ago 12m 448M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (61s) 45s ago 14m 28.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (35s) 31s ago 12m 28.7M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (3m) 45s ago 13m 17.2M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (3m) 31s ago 12m 14.7M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (11m) 45s ago 11m 229M 1216M 18.2.0 dc2bc1663786 741fe93cef0c 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (11m) 45s ago 11m 246M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (10m) 45s ago 10m 202M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:47:46.916 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (10m) 31s ago 10m 271M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:47:46.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (9m) 31s ago 9m 235M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:47:46.917 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (9m) 31s ago 9m 241M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:47:46.917 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (2m) 45s ago 12m 56.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:47:46.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0c6406e330 msgr2=0x7f0c640707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0c6406e330 0x7f0c640707e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0c68006010 tx=0x7f0c6801a040 comp rx=0 tx=0).stop 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ffc60 msgr2=0x7f0c74196400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ffc60 0x7f0c74196400 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0c6000d900 tx=0x7f0c6000dc10 comp rx=0 tx=0).stop 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 shutdown_connections 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0c6406e330 0x7f0c640707e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0c740ff350 0x7f0c74195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 --2- 172.21.15.90:0/2610376924 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0c740ffc60 0x7f0c74196400 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 >> 172.21.15.90:0/2610376924 conn(0x7f0c740748b0 msgr2=0x7f0c74104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 shutdown_connections 2024-04-26T04:47:46.918 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:46.916+0000 7f0c7ab64700 1 -- 172.21.15.90:0/2610376924 wait complete. 2024-04-26T04:47:47.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 -- 172.21.15.90:0/1943604296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ffd60 msgr2=0x7f3998103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1943604296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ffd60 0x7f3998103d30 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3988009910 tx=0x7f3988009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 -- 172.21.15.90:0/1943604296 shutdown_connections 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1943604296 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ffd60 0x7f3998103d30 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1943604296 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39980ff3c0 0x7f39980ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 -- 172.21.15.90:0/1943604296 >> 172.21.15.90:0/1943604296 conn(0x7f39980fac80 msgr2=0x7f39980fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 -- 172.21.15.90:0/1943604296 shutdown_connections 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.019+0000 7f399e7e4700 1 -- 172.21.15.90:0/1943604296 wait complete. 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.020+0000 7f399e7e4700 1 Processor -- start 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.020+0000 7f399e7e4700 1 -- start start 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.020+0000 7f399e7e4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ff3c0 0x7f39981980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.020+0000 7f399e7e4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39980ffd60 0x7f39981985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.020+0000 7f399e7e4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3998198cd0 con 0x7f39980ff3c0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.020+0000 7f399e7e4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f399819ca60 con 0x7f39980ffd60 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f3997fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ff3c0 0x7f39981980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f3997fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ff3c0 0x7f39981980b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37576/0 (socket says 172.21.15.90:37576) 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f3997fff700 1 -- 172.21.15.90:0/1546738468 learned_addr learned my addr 172.21.15.90:0/1546738468 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f39977fe700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39980ffd60 0x7f39981985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f3997fff700 1 -- 172.21.15.90:0/1546738468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39980ffd60 msgr2=0x7f39981985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f3997fff700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39980ffd60 0x7f39981985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f3997fff700 1 -- 172.21.15.90:0/1546738468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39880095f0 con 0x7f39980ff3c0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f3997fff700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ff3c0 0x7f39981980b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f398c00d8d0 tx=0x7f398c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f39957fa700 1 -- 172.21.15.90:0/1546738468 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f398c0098a0 con 0x7f39980ff3c0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f39957fa700 1 -- 172.21.15.90:0/1546738468 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f398c010460 con 0x7f39980ff3c0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f399819cd40 con 0x7f39980ff3c0 2024-04-26T04:47:47.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f399819d290 con 0x7f39980ff3c0 2024-04-26T04:47:47.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.021+0000 7f39957fa700 1 -- 172.21.15.90:0/1546738468 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f398c00f5f0 con 0x7f39980ff3c0 2024-04-26T04:47:47.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.022+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f399804f2a0 con 0x7f39980ff3c0 2024-04-26T04:47:47.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.023+0000 7f39957fa700 1 -- 172.21.15.90:0/1546738468 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f398c009a00 con 0x7f39980ff3c0 2024-04-26T04:47:47.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.023+0000 7f39957fa700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f398006e340 0x7f39800707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.023+0000 7f39957fa700 1 -- 172.21.15.90:0/1546738468 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f398c08e230 con 0x7f39980ff3c0 2024-04-26T04:47:47.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.023+0000 7f39977fe700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f398006e340 0x7f39800707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.026 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.023+0000 7f39977fe700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f398006e340 0x7f39800707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39981996d0 tx=0x7f398801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.028 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.028+0000 7f39957fa700 1 -- 172.21.15.90:0/1546738468 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f398c0573a0 con 0x7f39980ff3c0 2024-04-26T04:47:47.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.309+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f399819d5a0 con 0x7f39980ff3c0 2024-04-26T04:47:47.310 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.309+0000 7f39957fa700 1 -- 172.21.15.90:0/1546738468 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+688 (secure 0 0 0) 0x7f398c05aa60 con 0x7f39980ff3c0 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6 2024-04-26T04:47:47.311 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 10, 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:47:47.312 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f398006e340 msgr2=0x7f39800707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f398006e340 0x7f39800707f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f39981996d0 tx=0x7f398801a040 comp rx=0 tx=0).stop 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ff3c0 msgr2=0x7f39981980b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ff3c0 0x7f39981980b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f398c00d8d0 tx=0x7f398c00dbe0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 shutdown_connections 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f398006e340 0x7f39800707f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f39980ff3c0 0x7f39981980b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 --2- 172.21.15.90:0/1546738468 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f39980ffd60 0x7f39981985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.313 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.311+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 >> 172.21.15.90:0/1546738468 conn(0x7f39980fac80 msgr2=0x7f39980696e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:47.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.312+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 shutdown_connections 2024-04-26T04:47:47.314 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.312+0000 7f399e7e4700 1 -- 172.21.15.90:0/1546738468 wait complete. 2024-04-26T04:47:47.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 -- 172.21.15.90:0/2285796766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 msgr2=0x7f5b340ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/2285796766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b340ff780 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00b0a0 tx=0x7f5b2c00b3b0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 -- 172.21.15.90:0/2285796766 shutdown_connections 2024-04-26T04:47:47.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/2285796766 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/2285796766 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b340ff780 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 -- 172.21.15.90:0/2285796766 >> 172.21.15.90:0/2285796766 conn(0x7f5b340fac90 msgr2=0x7f5b340fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 -- 172.21.15.90:0/2285796766 shutdown_connections 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 -- 172.21.15.90:0/2285796766 wait complete. 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.416+0000 7f5b3aa98700 1 Processor -- start 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b3aa98700 1 -- start start 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b3aa98700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b3aa98700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b3aa98700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b341969e0 con 0x7f5b340ff3b0 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b3aa98700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34196b20 con 0x7f5b340ffd50 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b38834700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b38834700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:37584/0 (socket says 172.21.15.90:37584) 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b38834700 1 -- 172.21.15.90:0/1098899899 learned_addr learned my addr 172.21.15.90:0/1098899899 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:47.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b38834700 1 -- 172.21.15.90:0/1098899899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 msgr2=0x7f5b34196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b33fff700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b38834700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b38834700 1 -- 172.21.15.90:0/1098899899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2c009d00 con 0x7f5b340ff3b0 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b38834700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34195e50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f80 tx=0x7f5b2c00cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b31ffb700 1 -- 172.21.15.90:0/1098899899 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2c00e040 con 0x7f5b340ff3b0 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.417+0000 7f5b31ffb700 1 -- 172.21.15.90:0/1098899899 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b2c003e60 con 0x7f5b340ff3b0 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.418+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b3419a910 con 0x7f5b340ff3b0 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.418+0000 7f5b31ffb700 1 -- 172.21.15.90:0/1098899899 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b2c01d910 con 0x7f5b340ff3b0 2024-04-26T04:47:47.421 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.418+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b3419ae00 con 0x7f5b340ff3b0 2024-04-26T04:47:47.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.419+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b341086d0 con 0x7f5b340ff3b0 2024-04-26T04:47:47.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.419+0000 7f5b31ffb700 1 -- 172.21.15.90:0/1098899899 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f5b2c004820 con 0x7f5b340ff3b0 2024-04-26T04:47:47.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.419+0000 7f5b31ffb700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2006e340 0x7f5b200707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.420+0000 7f5b31ffb700 1 -- 172.21.15.90:0/1098899899 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5b2c08fb70 con 0x7f5b340ff3b0 2024-04-26T04:47:47.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.420+0000 7f5b33fff700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2006e340 0x7f5b200707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.420+0000 7f5b33fff700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2006e340 0x7f5b200707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5b34197470 tx=0x7f5b24005d50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.424+0000 7f5b31ffb700 1 -- 172.21.15.90:0/1098899899 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5b2c058d60 con 0x7f5b340ff3b0 2024-04-26T04:47:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.672+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b3402cc30 con 0x7f5b340ff3b0 2024-04-26T04:47:47.673 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.672+0000 7f5b31ffb700 1 -- 172.21.15.90:0/1098899899 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f5b2c05c420 con 0x7f5b340ff3b0 2024-04-26T04:47:47.673 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:47:47.674 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:47:47.675 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:47.676 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2006e340 msgr2=0x7f5b200707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2006e340 0x7f5b200707f0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5b34197470 tx=0x7f5b24005d50 comp rx=0 tx=0).stop 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 msgr2=0x7f5b34195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34195e50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c000f80 tx=0x7f5b2c00cdd0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 shutdown_connections 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2006e340 0x7f5b200707f0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34195e50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.677 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 --2- 172.21.15.90:0/1098899899 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 >> 172.21.15.90:0/1098899899 conn(0x7f5b340fac90 msgr2=0x7f5b34069130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 shutdown_connections 2024-04-26T04:47:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.675+0000 7f5b3aa98700 1 -- 172.21.15.90:0/1098899899 wait complete. 2024-04-26T04:47:47.678 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:47:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: Reconfiguring mds.cephfs.smithi104.ndiusl (monmap changed)... 2024-04-26T04:47:47.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:47.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:47.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: Reconfiguring daemon mds.cephfs.smithi104.ndiusl on smithi104 2024-04-26T04:47:47.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:47 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1546738468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.780+0000 7ff58acd1700 1 -- 172.21.15.90:0/1137539713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100d70 msgr2=0x7ff5841094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.780+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1137539713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100d70 0x7ff5841094d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff570009910 tx=0x7ff570009c20 comp rx=0 tx=0).stop 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.780+0000 7ff58acd1700 1 -- 172.21.15.90:0/1137539713 shutdown_connections 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.780+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1137539713 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100d70 0x7ff5841094d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.780+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1137539713 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff584100460 0x7ff584100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.780+0000 7ff58acd1700 1 -- 172.21.15.90:0/1137539713 >> 172.21.15.90:0/1137539713 conn(0x7ff5840748b0 msgr2=0x7ff584074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.780+0000 7ff58acd1700 1 -- 172.21.15.90:0/1137539713 shutdown_connections 2024-04-26T04:47:47.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff58acd1700 1 -- 172.21.15.90:0/1137539713 wait complete. 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff58acd1700 1 Processor -- start 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff58acd1700 1 -- start start 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff58acd1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100460 0x7ff584196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff58acd1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff584100d70 0x7ff584197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff58acd1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5841977a0 con 0x7ff584100460 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff58acd1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff584197910 con 0x7ff584100d70 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff588a6d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100460 0x7ff584196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff588a6d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100460 0x7ff584196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55160/0 (socket says 172.21.15.90:55160) 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.781+0000 7ff588a6d700 1 -- 172.21.15.90:0/1707158035 learned_addr learned my addr 172.21.15.90:0/1707158035 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff583fff700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff584100d70 0x7ff584197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff588a6d700 1 -- 172.21.15.90:0/1707158035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff584100d70 msgr2=0x7ff584197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff588a6d700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff584100d70 0x7ff584197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff588a6d700 1 -- 172.21.15.90:0/1707158035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5700095f0 con 0x7ff584100460 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff583fff700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff584100d70 0x7ff584197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff588a6d700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100460 0x7ff584196c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff57800d900 tx=0x7ff57800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff581ffb700 1 -- 172.21.15.90:0/1707158035 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff5780041f0 con 0x7ff584100460 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff581ffb700 1 -- 172.21.15.90:0/1707158035 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff578004cf0 con 0x7ff584100460 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58419fcb0 con 0x7ff584100460 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff581ffb700 1 -- 172.21.15.90:0/1707158035 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff57800b750 con 0x7ff584100460 2024-04-26T04:47:47.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.782+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5841a01a0 con 0x7ff584100460 2024-04-26T04:47:47.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.783+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff584106bd0 con 0x7ff584100460 2024-04-26T04:47:47.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.784+0000 7ff581ffb700 1 -- 172.21.15.90:0/1707158035 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7ff578004350 con 0x7ff584100460 2024-04-26T04:47:47.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.784+0000 7ff581ffb700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff57406e330 0x7ff5740707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:47.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.784+0000 7ff581ffb700 1 -- 172.21.15.90:0/1707158035 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff57808e4b0 con 0x7ff584100460 2024-04-26T04:47:47.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.784+0000 7ff583fff700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff57406e330 0x7ff5740707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:47.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.785+0000 7ff583fff700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff57406e330 0x7ff5740707e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff570005b40 tx=0x7ff570005ab0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:47.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:47.789+0000 7ff581ffb700 1 -- 172.21.15.90:0/1707158035 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff578057620 con 0x7ff584100460 2024-04-26T04:47:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: from='client.34160 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: from='client.34164 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: Reconfiguring mds.cephfs.smithi104.ndiusl (monmap changed)... 2024-04-26T04:47:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:47:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: Reconfiguring daemon mds.cephfs.smithi104.ndiusl on smithi104 2024-04-26T04:47:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:47 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1546738468' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:48.010 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.009+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff58402cca0 con 0x7ff57406e330 2024-04-26T04:47:48.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.010+0000 7ff581ffb700 1 -- 172.21.15.90:0/1707158035 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+383 (secure 0 0 0) 0x7ff58402cca0 con 0x7ff57406e330 2024-04-26T04:47:48.011 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:47:48.011 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:47:48.011 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "4/23 daemons upgraded", 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mon daemons", 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:47:48.012 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff57406e330 msgr2=0x7ff5740707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff57406e330 0x7ff5740707e0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff570005b40 tx=0x7ff570005ab0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100460 msgr2=0x7ff584196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100460 0x7ff584196c40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff57800d900 tx=0x7ff57800dcc0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 shutdown_connections 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff57406e330 0x7ff5740707e0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff584100460 0x7ff584196c40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 --2- 172.21.15.90:0/1707158035 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff584100d70 0x7ff584197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.014 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.012+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 >> 172.21.15.90:0/1707158035 conn(0x7ff5840748b0 msgr2=0x7ff584103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:48.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.013+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 shutdown_connections 2024-04-26T04:47:48.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.013+0000 7ff58acd1700 1 -- 172.21.15.90:0/1707158035 wait complete. 2024-04-26T04:47:48.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 -- 172.21.15.90:0/3149558406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168104130 msgr2=0x7f9168104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:48.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 --2- 172.21.15.90:0/3149558406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168104130 0x7f9168104500 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f916000b0a0 tx=0x7f916000b3b0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 -- 172.21.15.90:0/3149558406 shutdown_connections 2024-04-26T04:47:48.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 --2- 172.21.15.90:0/3149558406 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f91680ffbd0 0x7f9168100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 --2- 172.21.15.90:0/3149558406 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9168104130 0x7f9168104500 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 -- 172.21.15.90:0/3149558406 >> 172.21.15.90:0/3149558406 conn(0x7f9168075130 msgr2=0x7f9168075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:48.117 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 -- 172.21.15.90:0/3149558406 shutdown_connections 2024-04-26T04:47:48.118 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 -- 172.21.15.90:0/3149558406 wait complete. 2024-04-26T04:47:48.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 Processor -- start 2024-04-26T04:47:48.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.116+0000 7f916f0b5700 1 -- start start 2024-04-26T04:47:48.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916f0b5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91680ffbd0 0x7f9168195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:48.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916f0b5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168104130 0x7f9168196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:48.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916f0b5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9168196a70 con 0x7f91680ffbd0 2024-04-26T04:47:48.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916f0b5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916819a800 con 0x7f9168104130 2024-04-26T04:47:48.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916ce51700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91680ffbd0 0x7f9168195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:48.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916ce51700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91680ffbd0 0x7f9168195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55182/0 (socket says 172.21.15.90:55182) 2024-04-26T04:47:48.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916ce51700 1 -- 172.21.15.90:0/103122422 learned_addr learned my addr 172.21.15.90:0/103122422 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:47:48.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f9167fff700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168104130 0x7f9168196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916ce51700 1 -- 172.21.15.90:0/103122422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168104130 msgr2=0x7f9168196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916ce51700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168104130 0x7f9168196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916ce51700 1 -- 172.21.15.90:0/103122422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9160009d00 con 0x7f91680ffbd0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f9167fff700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168104130 0x7f9168196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916ce51700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91680ffbd0 0x7f9168195e50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9160000f80 tx=0x7f916000bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f9165ffb700 1 -- 172.21.15.90:0/103122422 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f916000e040 con 0x7f91680ffbd0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f916819aa80 con 0x7f91680ffbd0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f9165ffb700 1 -- 172.21.15.90:0/103122422 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9160003c10 con 0x7f91680ffbd0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.117+0000 7f9165ffb700 1 -- 172.21.15.90:0/103122422 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f916001d730 con 0x7f91680ffbd0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.118+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f916819af70 con 0x7f91680ffbd0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.118+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91681086a0 con 0x7f91680ffbd0 2024-04-26T04:47:48.126 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.119+0000 7f9165ffb700 1 -- 172.21.15.90:0/103122422 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f9160019070 con 0x7f91680ffbd0 2024-04-26T04:47:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.119+0000 7f9165ffb700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f915006e340 0x7f91500707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:47:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.119+0000 7f9165ffb700 1 -- 172.21.15.90:0/103122422 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f916008f920 con 0x7f91680ffbd0 2024-04-26T04:47:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.119+0000 7f9167fff700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f915006e340 0x7f91500707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:47:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.120+0000 7f9167fff700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f915006e340 0x7f91500707f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9168197470 tx=0x7f915c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:47:48.127 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.124+0000 7f9165ffb700 1 -- 172.21.15.90:0/103122422 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9160058a90 con 0x7f91680ffbd0 2024-04-26T04:47:48.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.424+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f916804f2a0 con 0x7f91680ffbd0 2024-04-26T04:47:48.425 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.424+0000 7f9165ffb700 1 -- 172.21.15.90:0/103122422 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f916005c150 con 0x7f91680ffbd0 2024-04-26T04:47:48.425 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_OK 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.427+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f915006e340 msgr2=0x7f91500707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.427+0000 7f916f0b5700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f915006e340 0x7f91500707f0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9168197470 tx=0x7f915c009260 comp rx=0 tx=0).stop 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.427+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91680ffbd0 msgr2=0x7f9168195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.427+0000 7f916f0b5700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91680ffbd0 0x7f9168195e50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9160000f80 tx=0x7f916000bd10 comp rx=0 tx=0).stop 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.428+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 shutdown_connections 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.428+0000 7f916f0b5700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f915006e340 0x7f91500707f0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.428+0000 7f916f0b5700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f91680ffbd0 0x7f9168195e50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.428+0000 7f916f0b5700 1 --2- 172.21.15.90:0/103122422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9168104130 0x7f9168196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.428+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 >> 172.21.15.90:0/103122422 conn(0x7f9168075130 msgr2=0x7f91680fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:47:48.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.428+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 shutdown_connections 2024-04-26T04:47:48.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:47:48.428+0000 7f916f0b5700 1 -- 172.21.15.90:0/103122422 wait complete. 2024-04-26T04:47:48.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1098899899' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]: dispatch 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]': finished 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi104"}]: dispatch 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi104"}]': finished 2024-04-26T04:47:48.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:48 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/103122422' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='client.34168 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: pgmap v29: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1098899899' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]: dispatch 2024-04-26T04:47:49.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi090"}]': finished 2024-04-26T04:47:49.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi104"}]: dispatch 2024-04-26T04:47:49.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon.smithi104"}]': finished 2024-04-26T04:47:49.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:48 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/103122422' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:47:49.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:49 smithi104 ceph-mon[64778]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:49.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:49 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all mon 2024-04-26T04:47:49.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:49 smithi090 ceph-mon[86550]: from='client.34180 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:47:49.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:49 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all mon 2024-04-26T04:47:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:50 smithi090 ceph-mon[86550]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:50 smithi090 ceph-mon[86550]: Upgrade: Updating crash.smithi090 (1/2) 2024-04-26T04:47:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:50 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:50 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:50 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:50.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:50 smithi090 ceph-mon[86550]: Deploying daemon crash.smithi090 on smithi090 2024-04-26T04:47:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:50 smithi104 ceph-mon[64778]: pgmap v30: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:50 smithi104 ceph-mon[64778]: Upgrade: Updating crash.smithi090 (1/2) 2024-04-26T04:47:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:50 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:51.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:50 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi090", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:51.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:50 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:51.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:50 smithi104 ceph-mon[64778]: Deploying daemon crash.smithi090 on smithi090 2024-04-26T04:47:52.876 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:52 smithi090 ceph-mon[86550]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:52 smithi104 ceph-mon[64778]: pgmap v31: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:54.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:54 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:54.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:54 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:54.696 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:54 smithi090 ceph-mon[86550]: pgmap v32: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:54 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:54 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:54.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:54 smithi104 ceph-mon[64778]: pgmap v32: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:56 smithi104 ceph-mon[64778]: Upgrade: Updating crash.smithi104 (2/2) 2024-04-26T04:47:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:56 smithi104 ceph-mon[64778]: Deploying daemon crash.smithi104 on smithi104 2024-04-26T04:47:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:56 smithi104 ceph-mon[64778]: pgmap v33: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:56 smithi090 ceph-mon[86550]: Upgrade: Updating crash.smithi104 (2/2) 2024-04-26T04:47:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.crash.smithi104", "caps": ["mon", "profile crash", "mgr", "profile crash"]}]: dispatch 2024-04-26T04:47:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:47:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:56 smithi090 ceph-mon[86550]: Deploying daemon crash.smithi104 on smithi104 2024-04-26T04:47:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:56 smithi090 ceph-mon[86550]: pgmap v33: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:56.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:47:58.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:58 smithi104 ceph-mon[64778]: pgmap v34: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:58.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:58.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:58.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:47:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:47:58.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:58 smithi090 ceph-mon[86550]: pgmap v34: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:47:58.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:58.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:47:58.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:47:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:00.946 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:00 smithi090 ceph-mon[86550]: pgmap v35: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:01.007 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:00 smithi104 ceph-mon[64778]: pgmap v35: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:02.954 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:02.954 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:02.954 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:02 smithi090 ceph-mon[86550]: pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:03.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:02 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:03.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:02 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:03.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:02 smithi104 ceph-mon[64778]: pgmap v36: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:03.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:03.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:03.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:03.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:04.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:03 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:04.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:03 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:04.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:03 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:04.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:03 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:04.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:04 smithi090 ceph-mon[86550]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:05.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:04 smithi104 ceph-mon[64778]: pgmap v37: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:05.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:05.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:05.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:06.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:06 smithi090 ceph-mon[86550]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:06.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:06.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:06 smithi104 ceph-mon[64778]: pgmap v38: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: pgmap v39: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:09.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:09.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:09.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]: dispatch 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]': finished 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi104"}]: dispatch 2024-04-26T04:48:09.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi104"}]': finished 2024-04-26T04:48:09.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:08 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:48:09.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: pgmap v39: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:09.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.059 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:09.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]: dispatch 2024-04-26T04:48:09.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi090"}]': finished 2024-04-26T04:48:09.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi104"}]: dispatch 2024-04-26T04:48:09.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash.smithi104"}]': finished 2024-04-26T04:48:09.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:08 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:48:09.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:09 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all crash 2024-04-26T04:48:09.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:09 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:48:09.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:09 smithi090 ceph-mon[86550]: Upgrade: osd.0 is safe to restart 2024-04-26T04:48:10.015 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:09 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all crash 2024-04-26T04:48:10.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:09 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["0"], "max": 16}]: dispatch 2024-04-26T04:48:10.016 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:09 smithi104 ceph-mon[64778]: Upgrade: osd.0 is safe to restart 2024-04-26T04:48:10.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:10 smithi090 ceph-mon[86550]: pgmap v40: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:10.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:10 smithi090 ceph-mon[86550]: Upgrade: Updating osd.0 2024-04-26T04:48:10.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:10.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:48:10.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:10.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:10 smithi090 ceph-mon[86550]: Deploying daemon osd.0 on smithi090 2024-04-26T04:48:10.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:10 smithi104 ceph-mon[64778]: pgmap v40: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:10 smithi104 ceph-mon[64778]: Upgrade: Updating osd.0 2024-04-26T04:48:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-26T04:48:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:10 smithi104 ceph-mon[64778]: Deploying daemon osd.0 on smithi090 2024-04-26T04:48:11.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:12.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:12 smithi090 ceph-mon[86550]: pgmap v41: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:13.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:12 smithi104 ceph-mon[64778]: pgmap v41: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:13.183 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:12 smithi090 systemd[1]: Stopping Ceph osd.0 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:48:13.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:13 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[41569]: 2024-04-26T04:48:13.181+0000 7fdf59ecc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:48:13.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:13 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[41569]: 2024-04-26T04:48:13.181+0000 7fdf59ecc700 -1 osd.0 50 *** Got signal Terminated *** 2024-04-26T04:48:13.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:13 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[41569]: 2024-04-26T04:48:13.181+0000 7fdf59ecc700 -1 osd.0 50 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:48:14.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:13 smithi104 ceph-mon[64778]: osd.0 marked itself down and dead 2024-04-26T04:48:14.175 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:13 smithi090 ceph-mon[86550]: osd.0 marked itself down and dead 2024-04-26T04:48:14.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:14 smithi090 podman[97638]: 2024-04-26 04:48:14.17429517 +0000 UTC m=+1.107593683 container died 741fe93cef0c862d912e03afb25b8b61edc913adbb0f4b1bcbeb1365d0288b40 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.29.1, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308) 2024-04-26T04:48:14.758 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:14 smithi090 podman[97638]: 2024-04-26 04:48:14.492234826 +0000 UTC m=+1.425533336 container cleanup 741fe93cef0c862d912e03afb25b8b61edc913adbb0f4b1bcbeb1365d0288b40 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T04:48:14.758 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:14 smithi090 bash[97638]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0 2024-04-26T04:48:14.758 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:14 smithi090 podman[97658]: 2024-04-26 04:48:14.694076931 +0000 UTC m=+0.522594230 container remove 741fe93cef0c862d912e03afb25b8b61edc913adbb0f4b1bcbeb1365d0288b40 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, ceph=True, org.label-schema.build-date=20231212, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-26T04:48:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:14 smithi104 ceph-mon[64778]: pgmap v42: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:14 smithi104 ceph-mon[64778]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:48:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:14 smithi104 ceph-mon[64778]: osdmap e51: 6 total, 5 up, 6 in 2024-04-26T04:48:15.112 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:14 smithi090 ceph-mon[86550]: pgmap v42: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:15.112 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:14 smithi090 ceph-mon[86550]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:48:15.112 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:14 smithi090 ceph-mon[86550]: osdmap e51: 6 total, 5 up, 6 in 2024-04-26T04:48:15.423 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:15 smithi090 podman[97750]: 2024-04-26 04:48:15.111780984 +0000 UTC m=+0.109714492 container create 4277a4fdd804aa107193a587f6150785e3675db4f9e3ecc4e3ef2f32c7f14432 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:48:15.423 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:15 smithi090 podman[97750]: 2024-04-26 04:48:15.020306625 +0000 UTC m=+0.018240144 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:48:15.424 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:15 smithi090 podman[97750]: 2024-04-26 04:48:15.323229625 +0000 UTC m=+0.321163130 container init 4277a4fdd804aa107193a587f6150785e3675db4f9e3ecc4e3ef2f32c7f14432 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T04:48:15.424 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:15 smithi090 podman[97750]: 2024-04-26 04:48:15.326406795 +0000 UTC m=+0.324340304 container start 4277a4fdd804aa107193a587f6150785e3675db4f9e3ecc4e3ef2f32c7f14432 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T04:48:15.424 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:15 smithi090 podman[97750]: 2024-04-26 04:48:15.423348912 +0000 UTC m=+0.421282427 container attach 4277a4fdd804aa107193a587f6150785e3675db4f9e3ecc4e3ef2f32c7f14432 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-04-26T04:48:15.684 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:15 smithi090 podman[97776]: 2024-04-26 04:48:15.651787322 +0000 UTC m=+0.018894989 container died 4277a4fdd804aa107193a587f6150785e3675db4f9e3ecc4e3ef2f32c7f14432 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b22e2eb) 2024-04-26T04:48:15.962 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:15 smithi090 ceph-mon[86550]: osdmap e52: 6 total, 5 up, 6 in 2024-04-26T04:48:16.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:15 smithi104 ceph-mon[64778]: osdmap e52: 6 total, 5 up, 6 in 2024-04-26T04:48:16.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 podman[97776]: 2024-04-26 04:48:16.213167606 +0000 UTC m=+0.580275319 container remove 4277a4fdd804aa107193a587f6150785e3675db4f9e3ecc4e3ef2f32c7f14432 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:48:16.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.0.service: Deactivated successfully. 2024-04-26T04:48:16.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 systemd[1]: Stopped Ceph osd.0 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:48:16.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.0.service: Consumed 56.295s CPU time. 2024-04-26T04:48:16.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 systemd[1]: Starting Ceph osd.0 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:48:16.905 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:16 smithi090 ceph-mon[86550]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:16.905 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 podman[97880]: 2024-04-26 04:48:16.557369018 +0000 UTC m=+0.018455445 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:48:16.905 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 podman[97880]: 2024-04-26 04:48:16.65592145 +0000 UTC m=+0.117007861 container create fa5d0ef4a255e9a3df4d3c4f1f0eed44063fcee0893d693d12c9533345aabcc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0) 2024-04-26T04:48:16.905 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 podman[97880]: 2024-04-26 04:48:16.901050171 +0000 UTC m=+0.362136581 container init fa5d0ef4a255e9a3df4d3c4f1f0eed44063fcee0893d693d12c9533345aabcc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-26T04:48:16.905 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:16 smithi090 podman[97880]: 2024-04-26 04:48:16.904861552 +0000 UTC m=+0.365947963 container start fa5d0ef4a255e9a3df4d3c4f1f0eed44063fcee0893d693d12c9533345aabcc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-b22e2eb, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-04-26T04:48:17.061 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:16 smithi104 ceph-mon[64778]: pgmap v45: 65 pgs: 9 stale+active+clean, 56 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:17.204 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 podman[97880]: 2024-04-26 04:48:17.002424119 +0000 UTC m=+0.463510536 container attach fa5d0ef4a255e9a3df4d3c4f1f0eed44063fcee0893d693d12c9533345aabcc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate[97894]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate[97894]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 bash[97880]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 bash[97880]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-0 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 bash[97880]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 bash[97880]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 bash[97880]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 bash[97880]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 bash[97880]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate[97894]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:48:17.472 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate[97894]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:48:17.473 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate[97894]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-0/block 2024-04-26T04:48:17.473 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate[97894]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-26T04:48:17.473 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate[97894]: --> ceph-volume raw activate successful for osd ID: 0 2024-04-26T04:48:17.473 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:17 smithi090 podman[97880]: 2024-04-26 04:48:17.369846728 +0000 UTC m=+0.830933140 container died fa5d0ef4a255e9a3df4d3c4f1f0eed44063fcee0893d693d12c9533345aabcc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T04:48:17.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:17 smithi090 ceph-mon[86550]: Health check failed: Degraded data redundancy: 68/348 objects degraded (19.540%), 20 pgs degraded (PG_DEGRADED) 2024-04-26T04:48:18.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:17 smithi104 ceph-mon[64778]: Health check failed: Degraded data redundancy: 68/348 objects degraded (19.540%), 20 pgs degraded (PG_DEGRADED) 2024-04-26T04:48:18.359 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 podman[98020]: 2024-04-26 04:48:18.023982866 +0000 UTC m=+0.648659870 container remove fa5d0ef4a255e9a3df4d3c4f1f0eed44063fcee0893d693d12c9533345aabcc9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-activate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:48:18.359 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 podman[98082]: 2024-04-26 04:48:18.143265985 +0000 UTC m=+0.018417605 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:48:18.359 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 podman[98082]: 2024-04-26 04:48:18.250206095 +0000 UTC m=+0.125357683 container create 514106f42deebff6c6df4f5203e16f39fb2448e19863565483f4a67bac43851d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , ceph=True) 2024-04-26T04:48:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1642894282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be80ff350 msgr2=0x7f4be80ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:18.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1642894282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be80ff350 0x7f4be80ff7c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc00aa20 tx=0x7f4bdc00ad30 comp rx=0 tx=0).stop 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1642894282 shutdown_connections 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1642894282 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be80ff350 0x7f4be80ff7c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1642894282 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be81038b0 0x7f4be8103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1642894282 >> 172.21.15.90:0/1642894282 conn(0x7f4be80748b0 msgr2=0x7f4be8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1642894282 shutdown_connections 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1642894282 wait complete. 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 Processor -- start 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.538+0000 7f4bee2d1700 1 -- start start 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4bee2d1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be80ff350 0x7f4be8199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4bee2d1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be81038b0 0x7f4be8199f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4bee2d1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8193a20 con 0x7f4be81038b0 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4bee2d1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4be8193b90 con 0x7f4be80ff350 2024-04-26T04:48:18.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4be77fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be81038b0 0x7f4be8199f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4be77fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be81038b0 0x7f4be8199f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45842/0 (socket says 172.21.15.90:45842) 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4be77fe700 1 -- 172.21.15.90:0/1512930726 learned_addr learned my addr 172.21.15.90:0/1512930726 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4be7fff700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be80ff350 0x7f4be8199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4be77fe700 1 -- 172.21.15.90:0/1512930726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be80ff350 msgr2=0x7f4be8199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4be77fe700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be80ff350 0x7f4be8199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.539+0000 7f4be77fe700 1 -- 172.21.15.90:0/1512930726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bdc00a700 con 0x7f4be81038b0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.540+0000 7f4be77fe700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be81038b0 0x7f4be8199f70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.540+0000 7f4be57fa700 1 -- 172.21.15.90:0/1512930726 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bdc004c10 con 0x7f4be81038b0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.540+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4be8193e10 con 0x7f4be81038b0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.540+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4be8194300 con 0x7f4be81038b0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.540+0000 7f4be57fa700 1 -- 172.21.15.90:0/1512930726 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4bdc004d70 con 0x7f4be81038b0 2024-04-26T04:48:18.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.540+0000 7f4be57fa700 1 -- 172.21.15.90:0/1512930726 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4bdc016800 con 0x7f4be81038b0 2024-04-26T04:48:18.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.541+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4be804f2a0 con 0x7f4be81038b0 2024-04-26T04:48:18.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.541+0000 7f4be57fa700 1 -- 172.21.15.90:0/1512930726 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f4bdc00e450 con 0x7f4be81038b0 2024-04-26T04:48:18.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.542+0000 7f4be57fa700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4bd006e330 0x7f4bd00707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:18.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.542+0000 7f4be57fa700 1 -- 172.21.15.90:0/1512930726 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f4bdc012070 con 0x7f4be81038b0 2024-04-26T04:48:18.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.542+0000 7f4be7fff700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4bd006e330 0x7f4bd00707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:18.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.542+0000 7f4be7fff700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4bd006e330 0x7f4bd00707e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8005fd0 tx=0x7f4bd8005f60 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:18.548 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.547+0000 7f4be57fa700 1 -- 172.21.15.90:0/1512930726 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4bdc05a620 con 0x7f4be81038b0 2024-04-26T04:48:18.721 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 podman[98082]: 2024-04-26 04:48:18.512783257 +0000 UTC m=+0.387934854 container init 514106f42deebff6c6df4f5203e16f39fb2448e19863565483f4a67bac43851d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:48:18.721 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 podman[98082]: 2024-04-26 04:48:18.516367569 +0000 UTC m=+0.391519166 container start 514106f42deebff6c6df4f5203e16f39fb2448e19863565483f4a67bac43851d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb) 2024-04-26T04:48:18.722 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 ceph-osd[98104]: -- 172.21.15.90:0/1944660713 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556bfd4e1680 con 0x556bfd4aac00 2024-04-26T04:48:18.722 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 bash[98082]: 514106f42deebff6c6df4f5203e16f39fb2448e19863565483f4a67bac43851d 2024-04-26T04:48:18.722 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:18 smithi090 systemd[1]: Started Ceph osd.0 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:48:18.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.776+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f4be81945e0 con 0x7f4bd006e330 2024-04-26T04:48:18.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.777+0000 7f4be57fa700 1 -- 172.21.15.90:0/1512930726 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f4be81945e0 con 0x7f4bd006e330 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.779+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4bd006e330 msgr2=0x7f4bd00707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.779+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4bd006e330 0x7f4bd00707e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f4bd8005fd0 tx=0x7f4bd8005f60 comp rx=0 tx=0).stop 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.779+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be81038b0 msgr2=0x7f4be8199f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.779+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be81038b0 0x7f4be8199f70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f4bdc000c00 tx=0x7f4bdc004970 comp rx=0 tx=0).stop 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.780+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 shutdown_connections 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.780+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4bd006e330 0x7f4bd00707e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.780+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4be80ff350 0x7f4be8199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.780+0000 7f4bee2d1700 1 --2- 172.21.15.90:0/1512930726 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4be81038b0 0x7f4be8199f70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.780+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 >> 172.21.15.90:0/1512930726 conn(0x7f4be80748b0 msgr2=0x7f4be80fe330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:18.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.780+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 shutdown_connections 2024-04-26T04:48:18.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.780+0000 7f4bee2d1700 1 -- 172.21.15.90:0/1512930726 wait complete. 2024-04-26T04:48:18.792 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:48:18.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 -- 172.21.15.90:0/4199023525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 msgr2=0x7fb84c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:18.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 --2- 172.21.15.90:0/4199023525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c0ff5f0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb83c009950 tx=0x7fb83c009c60 comp rx=0 tx=0).stop 2024-04-26T04:48:18.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 -- 172.21.15.90:0/4199023525 shutdown_connections 2024-04-26T04:48:18.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 --2- 172.21.15.90:0/4199023525 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84c0ffb30 0x7fb84c10a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 --2- 172.21.15.90:0/4199023525 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c0ff5f0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 -- 172.21.15.90:0/4199023525 >> 172.21.15.90:0/4199023525 conn(0x7fb84c0747e0 msgr2=0x7fb84c074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:18.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 -- 172.21.15.90:0/4199023525 shutdown_connections 2024-04-26T04:48:18.893 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.891+0000 7fb8508ec700 1 -- 172.21.15.90:0/4199023525 wait complete. 2024-04-26T04:48:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb8508ec700 1 Processor -- start 2024-04-26T04:48:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb8508ec700 1 -- start start 2024-04-26T04:48:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb8508ec700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c100400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb8508ec700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84c0ffb30 0x7fb84c100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb8508ec700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c100e80 con 0x7fb84c0ff220 2024-04-26T04:48:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb8508ec700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb84c100fc0 con 0x7fb84c0ffb30 2024-04-26T04:48:18.895 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb84a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c100400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb84a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c100400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45862/0 (socket says 172.21.15.90:45862) 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb84a59c700 1 -- 172.21.15.90:0/3972311813 learned_addr learned my addr 172.21.15.90:0/3972311813 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.892+0000 7fb849d9b700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84c0ffb30 0x7fb84c100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb84a59c700 1 -- 172.21.15.90:0/3972311813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84c0ffb30 msgr2=0x7fb84c100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb84a59c700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84c0ffb30 0x7fb84c100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb84a59c700 1 -- 172.21.15.90:0/3972311813 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb83c009580 con 0x7fb84c0ff220 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb84a59c700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c100400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c00 tx=0x7fb83c00f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb8437fe700 1 -- 172.21.15.90:0/3972311813 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb83c01d070 con 0x7fb84c0ff220 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb84c101240 con 0x7fb84c0ff220 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb8437fe700 1 -- 172.21.15.90:0/3972311813 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb83c005da0 con 0x7fb84c0ff220 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb8437fe700 1 -- 172.21.15.90:0/3972311813 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb83c0176f0 con 0x7fb84c0ff220 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.893+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb84c1a21e0 con 0x7fb84c0ff220 2024-04-26T04:48:18.896 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.894+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb84c107ca0 con 0x7fb84c0ff220 2024-04-26T04:48:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.895+0000 7fb8437fe700 1 -- 172.21.15.90:0/3972311813 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7fb83c00fca0 con 0x7fb84c0ff220 2024-04-26T04:48:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.895+0000 7fb8437fe700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb83806e340 0x7fb8380707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.895+0000 7fb8437fe700 1 -- 172.21.15.90:0/3972311813 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fb83c08f6f0 con 0x7fb84c0ff220 2024-04-26T04:48:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.895+0000 7fb849d9b700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb83806e340 0x7fb8380707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:18.897 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.896+0000 7fb849d9b700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb83806e340 0x7fb8380707f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb84c101bf0 tx=0x7fb83400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:18.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:18.900+0000 7fb8437fe700 1 -- 172.21.15.90:0/3972311813 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb83c058910 con 0x7fb84c0ff220 2024-04-26T04:48:19.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:18 smithi090 ceph-mon[86550]: pgmap v46: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:19.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:18 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:19.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:18 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:19.061 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:18 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:19.131 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.130+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb84c02cd10 con 0x7fb83806e340 2024-04-26T04:48:19.132 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.131+0000 7fb8437fe700 1 -- 172.21.15.90:0/3972311813 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fb84c02cd10 con 0x7fb83806e340 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb83806e340 msgr2=0x7fb8380707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb83806e340 0x7fb8380707f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fb84c101bf0 tx=0x7fb83400b410 comp rx=0 tx=0).stop 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 msgr2=0x7fb84c100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c100400 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb83c000c00 tx=0x7fb83c00f670 comp rx=0 tx=0).stop 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 shutdown_connections 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb83806e340 0x7fb8380707f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb84c0ff220 0x7fb84c100400 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.135 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 --2- 172.21.15.90:0/3972311813 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb84c0ffb30 0x7fb84c100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 >> 172.21.15.90:0/3972311813 conn(0x7fb84c0747e0 msgr2=0x7fb84c104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:19.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 shutdown_connections 2024-04-26T04:48:19.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.134+0000 7fb8508ec700 1 -- 172.21.15.90:0/3972311813 wait complete. 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 -- 172.21.15.90:0/2258200349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c100e80 msgr2=0x7f576c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 --2- 172.21.15.90:0/2258200349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c100e80 0x7f576c104e50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f5760009910 tx=0x7f5760009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 -- 172.21.15.90:0/2258200349 shutdown_connections 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 --2- 172.21.15.90:0/2258200349 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c100e80 0x7f576c104e50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 --2- 172.21.15.90:0/2258200349 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f576c1004e0 0x7f576c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 -- 172.21.15.90:0/2258200349 >> 172.21.15.90:0/2258200349 conn(0x7f576c0fbda0 msgr2=0x7f576c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 -- 172.21.15.90:0/2258200349 shutdown_connections 2024-04-26T04:48:19.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.252+0000 7f577114a700 1 -- 172.21.15.90:0/2258200349 wait complete. 2024-04-26T04:48:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.253+0000 7f577114a700 1 Processor -- start 2024-04-26T04:48:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f577114a700 1 -- start start 2024-04-26T04:48:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f577114a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c1004e0 0x7f576c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f577114a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f576c100e80 0x7f576c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f577114a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c073ad0 con 0x7f576c1004e0 2024-04-26T04:48:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f577114a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576c073c10 con 0x7f576c100e80 2024-04-26T04:48:19.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c1004e0 0x7f576c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576ad9d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c1004e0 0x7f576c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45876/0 (socket says 172.21.15.90:45876) 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576ad9d700 1 -- 172.21.15.90:0/1521721893 learned_addr learned my addr 172.21.15.90:0/1521721893 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576a59c700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f576c100e80 0x7f576c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576ad9d700 1 -- 172.21.15.90:0/1521721893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f576c100e80 msgr2=0x7f576c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576ad9d700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f576c100e80 0x7f576c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576ad9d700 1 -- 172.21.15.90:0/1521721893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57600095f0 con 0x7f576c1004e0 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.254+0000 7f576ad9d700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c1004e0 0x7f576c073050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f575c00d8d0 tx=0x7f575c00dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.255+0000 7f5753fff700 1 -- 172.21.15.90:0/1521721893 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c0098a0 con 0x7f576c1004e0 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.255+0000 7f5753fff700 1 -- 172.21.15.90:0/1521721893 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f575c010460 con 0x7f576c1004e0 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.255+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f576c1a2100 con 0x7f576c1004e0 2024-04-26T04:48:19.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.255+0000 7f5753fff700 1 -- 172.21.15.90:0/1521721893 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f575c00f5f0 con 0x7f576c1004e0 2024-04-26T04:48:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.255+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f576c1a24b0 con 0x7f576c1004e0 2024-04-26T04:48:19.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.256+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576c108850 con 0x7f576c1004e0 2024-04-26T04:48:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.256+0000 7f5753fff700 1 -- 172.21.15.90:0/1521721893 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f575c00f750 con 0x7f576c1004e0 2024-04-26T04:48:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.257+0000 7f5753fff700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f575406e330 0x7f57540707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.257+0000 7f576a59c700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f575406e330 0x7f57540707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.257+0000 7f5753fff700 1 -- 172.21.15.90:0/1521721893 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f575c08e1a0 con 0x7f576c1004e0 2024-04-26T04:48:19.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.257+0000 7f576a59c700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f575406e330 0x7f57540707e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5760006010 tx=0x7f576001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:18 smithi104 ceph-mon[64778]: pgmap v46: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:18 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:18 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:18 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:19.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.262+0000 7f5753fff700 1 -- 172.21.15.90:0/1521721893 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f575c057400 con 0x7f576c1004e0 2024-04-26T04:48:19.471 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:19 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[98097]: 2024-04-26T04:48:19.059+0000 7fcd1dd517c0 -1 Falling back to public interface 2024-04-26T04:48:19.497 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.496+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f576c061190 con 0x7f575406e330 2024-04-26T04:48:19.507 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.506+0000 7f5753fff700 1 -- 172.21.15.90:0/1521721893 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f576c061190 con 0x7f575406e330 2024-04-26T04:48:19.507 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (3m) 15s ago 13m 21.7M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (14m) 15s ago 14m 8094k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (13m) 17s ago 13m 8032k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (26s) 15s ago 14m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (21s) 17s ago 13m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (2m) 15s ago 13m 76.4M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (8m) 15s ago 8m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (8m) 15s ago 8m 16.2M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (8m) 17s ago 8m 16.6M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (8m) 17s ago 8m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (4m) 15s ago 15m 496M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (3m) 17s ago 12m 448M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:48:19.508 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (94s) 15s ago 15m 42.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (68s) 17s ago 12m 38.1M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (3m) 15s ago 13m 16.4M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (3m) 17s ago 12m 14.8M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 starting - - - 1216M 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (11m) 15s ago 11m 246M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (11m) 15s ago 11m 202M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (10m) 17s ago 10m 271M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (10m) 17s ago 10m 236M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (9m) 17s ago 9m 241M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:48:19.509 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (3m) 15s ago 13m 68.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:48:19.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f575406e330 msgr2=0x7f57540707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f575406e330 0x7f57540707e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5760006010 tx=0x7f576001a040 comp rx=0 tx=0).stop 2024-04-26T04:48:19.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c1004e0 msgr2=0x7f576c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c1004e0 0x7f576c073050 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f575c00d8d0 tx=0x7f575c00dbe0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 shutdown_connections 2024-04-26T04:48:19.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f575406e330 0x7f57540707e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f576c1004e0 0x7f576c073050 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.508+0000 7f577114a700 1 --2- 172.21.15.90:0/1521721893 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f576c100e80 0x7f576c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.509+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 >> 172.21.15.90:0/1521721893 conn(0x7f576c0fbda0 msgr2=0x7f576c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:19.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.509+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 shutdown_connections 2024-04-26T04:48:19.511 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.509+0000 7f577114a700 1 -- 172.21.15.90:0/1521721893 wait complete. 2024-04-26T04:48:19.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 -- 172.21.15.90:0/1510867927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 msgr2=0x7f7808103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 --2- 172.21.15.90:0/1510867927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f7808103d30 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f77fc009910 tx=0x7f77fc009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:19.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 -- 172.21.15.90:0/1510867927 shutdown_connections 2024-04-26T04:48:19.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 --2- 172.21.15.90:0/1510867927 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f7808103d30 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 --2- 172.21.15.90:0/1510867927 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78080ff3c0 0x7f78080ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 -- 172.21.15.90:0/1510867927 >> 172.21.15.90:0/1510867927 conn(0x7f78080fac80 msgr2=0x7f78080fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:19.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 -- 172.21.15.90:0/1510867927 shutdown_connections 2024-04-26T04:48:19.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.618+0000 7f780e108700 1 -- 172.21.15.90:0/1510867927 wait complete. 2024-04-26T04:48:19.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.619+0000 7f780e108700 1 Processor -- start 2024-04-26T04:48:19.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.619+0000 7f780e108700 1 -- start start 2024-04-26T04:48:19.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.619+0000 7f780e108700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78080ff3c0 0x7f7808195e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:19.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.619+0000 7f780e108700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f78081963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:19.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.619+0000 7f780e108700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7808196aa0 con 0x7f78080ffd60 2024-04-26T04:48:19.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.619+0000 7f780e108700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f780819a830 con 0x7f78080ff3c0 2024-04-26T04:48:19.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f78081963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7806ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f78081963c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45886/0 (socket says 172.21.15.90:45886) 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7806ffd700 1 -- 172.21.15.90:0/518284469 learned_addr learned my addr 172.21.15.90:0/518284469 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f78077fe700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78080ff3c0 0x7f7808195e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7806ffd700 1 -- 172.21.15.90:0/518284469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78080ff3c0 msgr2=0x7f7808195e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7806ffd700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78080ff3c0 0x7f7808195e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7806ffd700 1 -- 172.21.15.90:0/518284469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77fc0095f0 con 0x7f78080ffd60 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7806ffd700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f78081963c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0092e0 tx=0x7f77fc00bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7804ff9700 1 -- 172.21.15.90:0/518284469 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77fc01d070 con 0x7f78080ffd60 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f780819aab0 con 0x7f78080ffd60 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f7804ff9700 1 -- 172.21.15.90:0/518284469 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f77fc005dc0 con 0x7f78080ffd60 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.620+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f780819afa0 con 0x7f78080ffd60 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.621+0000 7f7804ff9700 1 -- 172.21.15.90:0/518284469 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f77fc017400 con 0x7f78080ffd60 2024-04-26T04:48:19.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.621+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f780804f2a0 con 0x7f78080ffd60 2024-04-26T04:48:19.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.622+0000 7f7804ff9700 1 -- 172.21.15.90:0/518284469 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f77fc00f6f0 con 0x7f78080ffd60 2024-04-26T04:48:19.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.622+0000 7f7804ff9700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f77f006e340 0x7f77f00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:19.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.623+0000 7f78077fe700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f77f006e340 0x7f77f00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:19.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.623+0000 7f7804ff9700 1 -- 172.21.15.90:0/518284469 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f77fc08f880 con 0x7f78080ffd60 2024-04-26T04:48:19.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.623+0000 7f78077fe700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f77f006e340 0x7f77f00707f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005950 tx=0x7f77f800b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:19.628 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.627+0000 7f7804ff9700 1 -- 172.21.15.90:0/518284469 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f77fc058b10 con 0x7f78080ffd60 2024-04-26T04:48:19.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.925+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f78081971e0 con 0x7f78080ffd60 2024-04-26T04:48:19.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.925+0000 7f7804ff9700 1 -- 172.21.15.90:0/518284469 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+687 (secure 0 0 0) 0x7f77fc05c1d0 con 0x7f78080ffd60 2024-04-26T04:48:19.927 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:48:19.927 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:48:19.927 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:48:19.927 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:19.927 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 9, 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:48:19.928 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f77f006e340 msgr2=0x7f77f00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f77f006e340 0x7f77f00707f0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77f8005950 tx=0x7f77f800b410 comp rx=0 tx=0).stop 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 msgr2=0x7f78081963c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f78081963c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f77fc0092e0 tx=0x7f77fc00bd70 comp rx=0 tx=0).stop 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 shutdown_connections 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f77f006e340 0x7f77f00707f0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78080ff3c0 0x7f7808195e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 --2- 172.21.15.90:0/518284469 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78080ffd60 0x7f78081963c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:19.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.927+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 >> 172.21.15.90:0/518284469 conn(0x7f78080fac80 msgr2=0x7f7808068ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:19.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.928+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 shutdown_connections 2024-04-26T04:48:19.930 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:19.928+0000 7f780e108700 1 -- 172.21.15.90:0/518284469 wait complete. 2024-04-26T04:48:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 -- 172.21.15.90:0/508658681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 msgr2=0x7f676c1066f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 --2- 172.21.15.90:0/508658681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1066f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f675c009910 tx=0x7f675c009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 -- 172.21.15.90:0/508658681 shutdown_connections 2024-04-26T04:48:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 --2- 172.21.15.90:0/508658681 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f676c068540 0x7f676c0689b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 --2- 172.21.15.90:0/508658681 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1066f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 -- 172.21.15.90:0/508658681 >> 172.21.15.90:0/508658681 conn(0x7f676c075130 msgr2=0x7f676c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 -- 172.21.15.90:0/508658681 shutdown_connections 2024-04-26T04:48:20.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.037+0000 7f67724a0700 1 -- 172.21.15.90:0/508658681 wait complete. 2024-04-26T04:48:20.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f67724a0700 1 Processor -- start 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f67724a0700 1 -- start start 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f67724a0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f676c068540 0x7f676c195e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f67724a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f67724a0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c196a00 con 0x7f676c106320 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f67724a0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f676c196b40 con 0x7f676c068540 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f676b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f676b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45896/0 (socket says 172.21.15.90:45896) 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f676b7fe700 1 -- 172.21.15.90:0/1693721008 learned_addr learned my addr 172.21.15.90:0/1693721008 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.038+0000 7f676bfff700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f676c068540 0x7f676c195e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.039+0000 7f676b7fe700 1 -- 172.21.15.90:0/1693721008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f676c068540 msgr2=0x7f676c195e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.039+0000 7f676b7fe700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f676c068540 0x7f676c195e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.039+0000 7f676b7fe700 1 -- 172.21.15.90:0/1693721008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f675c0095f0 con 0x7f676c106320 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.039+0000 7f676b7fe700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1963b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f676000c930 tx=0x7f676000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:20.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.039+0000 7f67697fa700 1 -- 172.21.15.90:0/1693721008 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6760007ad0 con 0x7f676c106320 2024-04-26T04:48:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.039+0000 7f67697fa700 1 -- 172.21.15.90:0/1693721008 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6760007c30 con 0x7f676c106320 2024-04-26T04:48:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.039+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f676c19a990 con 0x7f676c106320 2024-04-26T04:48:20.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.040+0000 7f67697fa700 1 -- 172.21.15.90:0/1693721008 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6760018610 con 0x7f676c106320 2024-04-26T04:48:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.040+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f676c19ae80 con 0x7f676c106320 2024-04-26T04:48:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.041+0000 7f67697fa700 1 -- 172.21.15.90:0/1693721008 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f676000f450 con 0x7f676c106320 2024-04-26T04:48:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.041+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f676c108770 con 0x7f676c106320 2024-04-26T04:48:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.041+0000 7f67697fa700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f675406e340 0x7f67540707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.041+0000 7f67697fa700 1 -- 172.21.15.90:0/1693721008 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f676008dc50 con 0x7f676c106320 2024-04-26T04:48:20.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.041+0000 7f676bfff700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f675406e340 0x7f67540707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.042+0000 7f676bfff700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f675406e340 0x7f67540707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f675c00b5c0 tx=0x7f675c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:20.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.046+0000 7f67697fa700 1 -- 172.21.15.90:0/1693721008 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6760056d30 con 0x7f676c106320 2024-04-26T04:48:20.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:19 smithi090 ceph-mon[86550]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:20.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:19 smithi104 ceph-mon[64778]: from='client.34190 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.310+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f676c04f2a0 con 0x7f676c106320 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.311+0000 7f67697fa700 1 -- 172.21.15.90:0/1693721008 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f676005a3f0 con 0x7f676c106320 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:20.312 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:48:20.313 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:48:20.314 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:48:20.315 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:20.316 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:20.316 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:20.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f675406e340 msgr2=0x7f67540707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f675406e340 0x7f67540707f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f675c00b5c0 tx=0x7f675c01a040 comp rx=0 tx=0).stop 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 msgr2=0x7f676c1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1963b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f676000c930 tx=0x7f676000ccf0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 shutdown_connections 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f675406e340 0x7f67540707f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f676c068540 0x7f676c195e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 --2- 172.21.15.90:0/1693721008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f676c106320 0x7f676c1963b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 >> 172.21.15.90:0/1693721008 conn(0x7f676c075130 msgr2=0x7f676c0fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 shutdown_connections 2024-04-26T04:48:20.317 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.313+0000 7f67724a0700 1 -- 172.21.15.90:0/1693721008 wait complete. 2024-04-26T04:48:20.318 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:48:20.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 -- 172.21.15.90:0/3087866310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 msgr2=0x7f706c100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 --2- 172.21.15.90:0/3087866310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c100890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7068009910 tx=0x7f7068009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:20.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 -- 172.21.15.90:0/3087866310 shutdown_connections 2024-04-26T04:48:20.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 --2- 172.21.15.90:0/3087866310 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f706c100e60 0x7f706c104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.427 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 --2- 172.21.15.90:0/3087866310 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c100890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 -- 172.21.15.90:0/3087866310 >> 172.21.15.90:0/3087866310 conn(0x7f706c0fbd80 msgr2=0x7f706c0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 -- 172.21.15.90:0/3087866310 shutdown_connections 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.424+0000 7f70742d9700 1 -- 172.21.15.90:0/3087866310 wait complete. 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f70742d9700 1 Processor -- start 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f70742d9700 1 -- start start 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f70742d9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f70742d9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f706c100e60 0x7f706c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f70742d9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c073ad0 con 0x7f706c1004c0 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f70742d9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f706c073c10 con 0x7f706c100e60 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f7072075700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.428 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f7072075700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45908/0 (socket says 172.21.15.90:45908) 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f7072075700 1 -- 172.21.15.90:0/3856713956 learned_addr learned my addr 172.21.15.90:0/3856713956 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.425+0000 7f7071874700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f706c100e60 0x7f706c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f7072075700 1 -- 172.21.15.90:0/3856713956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f706c100e60 msgr2=0x7f706c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f7072075700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f706c100e60 0x7f706c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f7072075700 1 -- 172.21.15.90:0/3856713956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70680095f0 con 0x7f706c1004c0 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f7072075700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c073050 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f706800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f70637fe700 1 -- 172.21.15.90:0/3856713956 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f706801d070 con 0x7f706c1004c0 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f70637fe700 1 -- 172.21.15.90:0/3856713956 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f706800f460 con 0x7f706c1004c0 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f70637fe700 1 -- 172.21.15.90:0/3856713956 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7068017610 con 0x7f706c1004c0 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706c073e90 con 0x7f706c1004c0 2024-04-26T04:48:20.429 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.426+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706c1a2430 con 0x7f706c1004c0 2024-04-26T04:48:20.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.427+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f706c1087d0 con 0x7f706c1004c0 2024-04-26T04:48:20.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.428+0000 7f70637fe700 1 -- 172.21.15.90:0/3856713956 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f706800f5d0 con 0x7f706c1004c0 2024-04-26T04:48:20.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.428+0000 7f70637fe700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f705806e330 0x7f70580707e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.428+0000 7f7071874700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f705806e330 0x7f70580707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.428+0000 7f70637fe700 1 -- 172.21.15.90:0/3856713956 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f706808f7f0 con 0x7f706c1004c0 2024-04-26T04:48:20.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.429+0000 7f7071874700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f705806e330 0x7f70580707e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f706c074840 tx=0x7f705c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:20.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.433+0000 7f70637fe700 1 -- 172.21.15.90:0/3856713956 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f70680589e0 con 0x7f706c1004c0 2024-04-26T04:48:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.662+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f706c061190 con 0x7f705806e330 2024-04-26T04:48:20.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.663+0000 7f70637fe700 1 -- 172.21.15.90:0/3856713956 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f706c061190 con 0x7f705806e330 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:48:20.665 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "6/23 daemons upgraded", 2024-04-26T04:48:20.666 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:48:20.666 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:48:20.666 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:48:20.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f705806e330 msgr2=0x7f70580707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f705806e330 0x7f70580707e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f706c074840 tx=0x7f705c00a300 comp rx=0 tx=0).stop 2024-04-26T04:48:20.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 msgr2=0x7f706c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c073050 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7068006010 tx=0x7f706800bd70 comp rx=0 tx=0).stop 2024-04-26T04:48:20.667 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 shutdown_connections 2024-04-26T04:48:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f705806e330 0x7f70580707e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f706c1004c0 0x7f706c073050 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 --2- 172.21.15.90:0/3856713956 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f706c100e60 0x7f706c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 >> 172.21.15.90:0/3856713956 conn(0x7f706c0fbd80 msgr2=0x7f706c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 shutdown_connections 2024-04-26T04:48:20.668 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.666+0000 7f70742d9700 1 -- 172.21.15.90:0/3856713956 wait complete. 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3193977434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e4104130 msgr2=0x7f79e4104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3193977434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e4104130 0x7f79e4104500 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f79d4009910 tx=0x7f79d4009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3193977434 shutdown_connections 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3193977434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79e40ffbd0 0x7f79e4100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3193977434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e4104130 0x7f79e4104500 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3193977434 >> 172.21.15.90:0/3193977434 conn(0x7f79e4075130 msgr2=0x7f79e4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3193977434 shutdown_connections 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.777+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3193977434 wait complete. 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e9fe6700 1 Processor -- start 2024-04-26T04:48:20.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e9fe6700 1 -- start start 2024-04-26T04:48:20.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e9fe6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e40ffbd0 0x7f79e4196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e9fe6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79e4104130 0x7f79e41971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e9fe6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4197810 con 0x7f79e40ffbd0 2024-04-26T04:48:20.781 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e9fe6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79e4197950 con 0x7f79e4104130 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e40ffbd0 0x7f79e4196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e40ffbd0 0x7f79e4196cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45920/0 (socket says 172.21.15.90:45920) 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e37fe700 1 -- 172.21.15.90:0/3843087204 learned_addr learned my addr 172.21.15.90:0/3843087204 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79daffd700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79e4104130 0x7f79e41971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e37fe700 1 -- 172.21.15.90:0/3843087204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79e4104130 msgr2=0x7f79e41971f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e37fe700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79e4104130 0x7f79e41971f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.778+0000 7f79e37fe700 1 -- 172.21.15.90:0/3843087204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79d40095f0 con 0x7f79e40ffbd0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.779+0000 7f79e37fe700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e40ffbd0 0x7f79e4196cb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b7d0 tx=0x7f79d400f460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.779+0000 7f79e17fa700 1 -- 172.21.15.90:0/3843087204 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d401d070 con 0x7f79e40ffbd0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.779+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79e419fd20 con 0x7f79e40ffbd0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.779+0000 7f79e17fa700 1 -- 172.21.15.90:0/3843087204 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f79d40052a0 con 0x7f79e40ffbd0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.779+0000 7f79e17fa700 1 -- 172.21.15.90:0/3843087204 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79d4022470 con 0x7f79e40ffbd0 2024-04-26T04:48:20.782 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.779+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79e41a0130 con 0x7f79e40ffbd0 2024-04-26T04:48:20.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.780+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79e41076a0 con 0x7f79e40ffbd0 2024-04-26T04:48:20.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.780+0000 7f79e17fa700 1 -- 172.21.15.90:0/3843087204 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 37) v1 ==== 92305+0+0 (secure 0 0 0) 0x7f79d400f8c0 con 0x7f79e40ffbd0 2024-04-26T04:48:20.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.781+0000 7f79e17fa700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f79d006e340 0x7f79d00707f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:20.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.781+0000 7f79daffd700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f79d006e340 0x7f79d00707f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:20.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.781+0000 7f79e17fa700 1 -- 172.21.15.90:0/3843087204 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f79d408f870 con 0x7f79e40ffbd0 2024-04-26T04:48:20.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.781+0000 7f79daffd700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f79d006e340 0x7f79d00707f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79e4068df0 tx=0x7f79cc009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:20.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:20.786+0000 7f79e17fa700 1 -- 172.21.15.90:0/3843087204 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f79d40589e0 con 0x7f79e40ffbd0 2024-04-26T04:48:20.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:20 smithi090 ceph-mon[86550]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:20.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:20 smithi090 ceph-mon[86550]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:20.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:20 smithi090 ceph-mon[86550]: pgmap v47: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:20.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:20 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/518284469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:20.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:20 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1693721008' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:48:21.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.080+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f79e402cc30 con 0x7f79e40ffbd0 2024-04-26T04:48:21.081 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.080+0000 7f79e17fa700 1 -- 172.21.15.90:0/3843087204 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+1383 (secure 0 0 0) 0x7f79d405c0a0 con 0x7f79e40ffbd0 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 osds down; Degraded data redundancy: 68/348 objects degraded (19.540%), 20 pgs degraded 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: osd.0 (root=default,host=smithi090) is down 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] PG_DEGRADED: Degraded data redundancy: 68/348 objects degraded (19.540%), 20 pgs degraded 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: pg 1.0 is active+undersized+degraded, acting [3,1] 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.0 is active+undersized+degraded, acting [3,1] 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1 is active+undersized+degraded, acting [2,1] 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.2 is active+undersized+degraded, acting [5,1] 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.4 is active+undersized+degraded, acting [1,4] 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.5 is active+undersized+degraded, acting [3,4] 2024-04-26T04:48:21.082 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.8 is active+undersized+degraded, acting [3,5] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.9 is active+undersized+degraded, acting [1,4] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.c is active+undersized+degraded, acting [2,3] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.e is active+undersized+degraded, acting [2,3] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.f is active+undersized+degraded, acting [4,5] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.10 is active+undersized+degraded, acting [2,1] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.12 is active+undersized+degraded, acting [3,1] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.13 is active+undersized+degraded, acting [4,2] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.15 is active+undersized+degraded, acting [1,3] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.19 is active+undersized+degraded, acting [4,2] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1b is active+undersized+degraded, acting [1,5] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1d is active+undersized+degraded, acting [3,5] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1e is active+undersized+degraded, acting [2,5] 2024-04-26T04:48:21.083 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1f is active+undersized+degraded, acting [3,4] 2024-04-26T04:48:21.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f79d006e340 msgr2=0x7f79d00707f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:21.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f79d006e340 0x7f79d00707f0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f79e4068df0 tx=0x7f79cc009260 comp rx=0 tx=0).stop 2024-04-26T04:48:21.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e40ffbd0 msgr2=0x7f79e4196cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:21.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e40ffbd0 0x7f79e4196cb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f79d400b7d0 tx=0x7f79d400f460 comp rx=0 tx=0).stop 2024-04-26T04:48:21.084 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 shutdown_connections 2024-04-26T04:48:21.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f79d006e340 0x7f79d00707f0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:21.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79e40ffbd0 0x7f79e4196cb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:21.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 --2- 172.21.15.90:0/3843087204 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79e4104130 0x7f79e41971f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:21.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 >> 172.21.15.90:0/3843087204 conn(0x7f79e4075130 msgr2=0x7f79e40fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:21.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 shutdown_connections 2024-04-26T04:48:21.085 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:21.083+0000 7f79e9fe6700 1 -- 172.21.15.90:0/3843087204 wait complete. 2024-04-26T04:48:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:20 smithi104 ceph-mon[64778]: from='client.34194 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:20 smithi104 ceph-mon[64778]: from='client.34198 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:20 smithi104 ceph-mon[64778]: pgmap v47: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:20 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/518284469' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:20 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1693721008' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:48:22.101 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:21 smithi090 ceph-mon[86550]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:22.101 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:21 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3843087204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:48:22.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:21 smithi104 ceph-mon[64778]: from='client.34210 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:22.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:21 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3843087204' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:48:23.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:22 smithi090 ceph-mon[86550]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:22 smithi104 ceph-mon[64778]: pgmap v48: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:23.838 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:23 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[98097]: 2024-04-26T04:48:23.519+0000 7fcd1dd517c0 -1 osd.0 0 read_superblock omap replica is missing. 2024-04-26T04:48:24.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:23 smithi090 ceph-mon[86550]: pgmap v49: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:24.221 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:23 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[98097]: 2024-04-26T04:48:23.951+0000 7fcd1dd517c0 -1 osd.0 50 log_to_monitors true 2024-04-26T04:48:24.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:23 smithi104 ceph-mon[64778]: pgmap v49: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:25.221 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:48:24 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[98097]: 2024-04-26T04:48:24.856+0000 7fcd0f1a9700 -1 osd.0 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:48:25.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:24 smithi090 ceph-mon[86550]: from='osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:48:25.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:25.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:25.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:24 smithi104 ceph-mon[64778]: from='osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-26T04:48:25.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:25.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: from='osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: osdmap e53: 6 total, 5 up, 6 in 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: from='osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:26.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:26.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: from='osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-26T04:48:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: osdmap e53: 6 total, 5 up, 6 in 2024-04-26T04:48:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: from='osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:48:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: pgmap v51: 65 pgs: 20 active+undersized+degraded, 14 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 68/348 objects degraded (19.540%) 2024-04-26T04:48:26.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:26.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:27.145 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:26 smithi090 ceph-mon[86550]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:48:27.146 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:26 smithi090 ceph-mon[86550]: osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208] boot 2024-04-26T04:48:27.146 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:26 smithi090 ceph-mon[86550]: osdmap e54: 6 total, 6 up, 6 in 2024-04-26T04:48:27.146 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:48:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:26 smithi104 ceph-mon[64778]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:48:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:26 smithi104 ceph-mon[64778]: osd.0 [v2:172.21.15.90:6802/2303726208,v1:172.21.15.90:6803/2303726208] boot 2024-04-26T04:48:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:26 smithi104 ceph-mon[64778]: osdmap e54: 6 total, 6 up, 6 in 2024-04-26T04:48:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-26T04:48:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:27 smithi090 ceph-mon[86550]: osdmap e55: 6 total, 6 up, 6 in 2024-04-26T04:48:28.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:27 smithi090 ceph-mon[86550]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s; 12/348 objects degraded (3.448%) 2024-04-26T04:48:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:27 smithi104 ceph-mon[64778]: osdmap e55: 6 total, 6 up, 6 in 2024-04-26T04:48:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:27 smithi104 ceph-mon[64778]: pgmap v54: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s; 12/348 objects degraded (3.448%) 2024-04-26T04:48:28.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: Health check update: Degraded data redundancy: 12/348 objects degraded (3.448%), 3 pgs degraded (PG_DEGRADED) 2024-04-26T04:48:28.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: mgrmap e38: smithi090.iwrvwa(active, since 92s), standbys: smithi104.wwoewi 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:28.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:29.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: Health check update: Degraded data redundancy: 12/348 objects degraded (3.448%), 3 pgs degraded (PG_DEGRADED) 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: mgrmap e38: smithi090.iwrvwa(active, since 92s), standbys: smithi104.wwoewi 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:29.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:29.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:30.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:29 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:30.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:29 smithi090 ceph-mon[86550]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-26T04:48:30.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:29 smithi090 ceph-mon[86550]: pgmap v55: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s; 12/348 objects degraded (3.448%) 2024-04-26T04:48:30.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:29 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:30.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:29 smithi104 ceph-mon[64778]: Upgrade: unsafe to stop osd(s) at this time (15 PGs are or would become offline) 2024-04-26T04:48:30.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:29 smithi104 ceph-mon[64778]: pgmap v55: 65 pgs: 25 peering, 3 active+undersized+degraded, 6 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 853 B/s wr, 0 op/s; 12/348 objects degraded (3.448%) 2024-04-26T04:48:32.122 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:31 smithi090 ceph-mon[86550]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/348 objects degraded (3.448%), 3 pgs degraded) 2024-04-26T04:48:32.123 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:31 smithi090 ceph-mon[86550]: Cluster is now healthy 2024-04-26T04:48:32.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:31 smithi104 ceph-mon[64778]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 12/348 objects degraded (3.448%), 3 pgs degraded) 2024-04-26T04:48:32.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:31 smithi104 ceph-mon[64778]: Cluster is now healthy 2024-04-26T04:48:33.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:32 smithi090 ceph-mon[86550]: pgmap v56: 65 pgs: 25 peering, 40 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 750 B/s wr, 0 op/s 2024-04-26T04:48:33.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:32 smithi104 ceph-mon[64778]: pgmap v56: 65 pgs: 25 peering, 40 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 750 B/s wr, 0 op/s 2024-04-26T04:48:35.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:34 smithi090 ceph-mon[86550]: pgmap v57: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s wr, 0 op/s 2024-04-26T04:48:35.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:34 smithi104 ceph-mon[64778]: pgmap v57: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 639 B/s wr, 0 op/s 2024-04-26T04:48:37.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:36 smithi090 ceph-mon[86550]: pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 521 B/s wr, 0 op/s 2024-04-26T04:48:37.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:36 smithi104 ceph-mon[64778]: pgmap v58: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 521 B/s wr, 0 op/s 2024-04-26T04:48:39.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:38 smithi090 ceph-mon[86550]: pgmap v59: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:39.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:38 smithi104 ceph-mon[64778]: pgmap v59: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:40 smithi104 ceph-mon[64778]: pgmap v60: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:41.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:40 smithi090 ceph-mon[86550]: pgmap v60: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:42 smithi090 ceph-mon[86550]: pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:42 smithi104 ceph-mon[64778]: pgmap v61: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:44 smithi090 ceph-mon[86550]: pgmap v62: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:44 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:44 smithi104 ceph-mon[64778]: pgmap v62: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:44 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:45 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:45 smithi104 ceph-mon[64778]: Upgrade: osd.1 is safe to restart 2024-04-26T04:48:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:45 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:45 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:48:46.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:45 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:46.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:45 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["1"], "max": 16}]: dispatch 2024-04-26T04:48:46.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:45 smithi090 ceph-mon[86550]: Upgrade: osd.1 is safe to restart 2024-04-26T04:48:46.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:45 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:46.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:45 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-26T04:48:46.048 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:45 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:48:47.108 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:46 smithi090 ceph-mon[86550]: Upgrade: Updating osd.1 2024-04-26T04:48:47.108 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:46 smithi090 ceph-mon[86550]: Deploying daemon osd.1 on smithi090 2024-04-26T04:48:47.108 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:46 smithi090 ceph-mon[86550]: pgmap v63: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:46 smithi104 ceph-mon[64778]: Upgrade: Updating osd.1 2024-04-26T04:48:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:46 smithi104 ceph-mon[64778]: Deploying daemon osd.1 on smithi090 2024-04-26T04:48:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:46 smithi104 ceph-mon[64778]: pgmap v63: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:49.066 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:48 smithi090 ceph-mon[86550]: pgmap v64: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:49.066 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:48 smithi090 systemd[1]: Stopping Ceph osd.1 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:48:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:48 smithi104 ceph-mon[64778]: pgmap v64: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:49.471 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:49 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[47139]: 2024-04-26T04:48:49.064+0000 7f4c2392e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:48:49.471 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:49 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[47139]: 2024-04-26T04:48:49.064+0000 7f4c2392e700 -1 osd.1 55 *** Got signal Terminated *** 2024-04-26T04:48:49.471 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:49 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[47139]: 2024-04-26T04:48:49.064+0000 7f4c2392e700 -1 osd.1 55 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:48:50.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:49 smithi090 ceph-mon[86550]: osd.1 marked itself down and dead 2024-04-26T04:48:50.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:49 smithi104 ceph-mon[64778]: osd.1 marked itself down and dead 2024-04-26T04:48:50.693 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:50 smithi090 podman[103368]: 2024-04-26 04:48:50.383062988 +0000 UTC m=+1.410018845 container died 1674cbd259474b9331b1780b02960ab1ebb20d9f44ffd4e3045db846406cc6ed (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, RELEASE=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T04:48:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:50 smithi090 ceph-mon[86550]: pgmap v65: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:50 smithi090 ceph-mon[86550]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:48:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:50 smithi090 ceph-mon[86550]: osdmap e56: 6 total, 5 up, 6 in 2024-04-26T04:48:50.971 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:50 smithi090 podman[103368]: 2024-04-26 04:48:50.791695411 +0000 UTC m=+1.818651266 container cleanup 1674cbd259474b9331b1780b02960ab1ebb20d9f44ffd4e3045db846406cc6ed (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=HEAD, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:48:50.971 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:50 smithi090 bash[103368]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1 2024-04-26T04:48:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:50 smithi104 ceph-mon[64778]: pgmap v65: 65 pgs: 65 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:50 smithi104 ceph-mon[64778]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:48:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:50 smithi104 ceph-mon[64778]: osdmap e56: 6 total, 5 up, 6 in 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 -- 172.21.15.90:0/2132972476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100460 msgr2=0x7fc718100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/2132972476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100460 0x7fc718100830 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc708009940 tx=0x7fc708009c50 comp rx=0 tx=0).stop 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 -- 172.21.15.90:0/2132972476 shutdown_connections 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/2132972476 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc718100d70 0x7fc7181094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/2132972476 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100460 0x7fc718100830 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 -- 172.21.15.90:0/2132972476 >> 172.21.15.90:0/2132972476 conn(0x7fc7180748b0 msgr2=0x7fc718074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 -- 172.21.15.90:0/2132972476 shutdown_connections 2024-04-26T04:48:51.207 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 -- 172.21.15.90:0/2132972476 wait complete. 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 Processor -- start 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.204+0000 7fc71e4b1700 1 -- start start 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc71e4b1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc718100460 0x7fc718195dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc71e4b1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100d70 0x7fc718196300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc71e4b1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc718196950 con 0x7fc718100d70 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc71e4b1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc718196a90 con 0x7fc718100460 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7177fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100d70 0x7fc718196300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7177fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100d70 0x7fc718196300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51644/0 (socket says 172.21.15.90:51644) 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7177fe700 1 -- 172.21.15.90:0/3112090237 learned_addr learned my addr 172.21.15.90:0/3112090237 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc717fff700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc718100460 0x7fc718195dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7177fe700 1 -- 172.21.15.90:0/3112090237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc718100460 msgr2=0x7fc718195dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7177fe700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc718100460 0x7fc718195dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7177fe700 1 -- 172.21.15.90:0/3112090237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7080095f0 con 0x7fc718100d70 2024-04-26T04:48:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7177fe700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100d70 0x7fc718196300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c930 tx=0x7fc70c00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7157fa700 1 -- 172.21.15.90:0/3112090237 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc70c007ad0 con 0x7fc718100d70 2024-04-26T04:48:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.205+0000 7fc7157fa700 1 -- 172.21.15.90:0/3112090237 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc70c00ce80 con 0x7fc718100d70 2024-04-26T04:48:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.206+0000 7fc7157fa700 1 -- 172.21.15.90:0/3112090237 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc70c018630 con 0x7fc718100d70 2024-04-26T04:48:51.209 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.206+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc71819a890 con 0x7fc718100d70 2024-04-26T04:48:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.206+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc71819ade0 con 0x7fc718100d70 2024-04-26T04:48:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.207+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc71804f2a0 con 0x7fc718100d70 2024-04-26T04:48:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.208+0000 7fc7157fa700 1 -- 172.21.15.90:0/3112090237 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc70c007c30 con 0x7fc718100d70 2024-04-26T04:48:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.208+0000 7fc7157fa700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc70006e200 0x7fc7000706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.208+0000 7fc7157fa700 1 -- 172.21.15.90:0/3112090237 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc70c08e190 con 0x7fc718100d70 2024-04-26T04:48:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.208+0000 7fc717fff700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc70006e200 0x7fc7000706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.210 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.209+0000 7fc717fff700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc70006e200 0x7fc7000706b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc708006010 tx=0x7fc70800b9d0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.214 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.213+0000 7fc7157fa700 1 -- 172.21.15.90:0/3112090237 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc70c0574c0 con 0x7fc718100d70 2024-04-26T04:48:51.439 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:51 smithi090 podman[103385]: 2024-04-26 04:48:51.06018466 +0000 UTC m=+0.669130354 container remove 1674cbd259474b9331b1780b02960ab1ebb20d9f44ffd4e3045db846406cc6ed (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, io.buildah.version=1.29.1) 2024-04-26T04:48:51.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.456+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc71819b1d0 con 0x7fc70006e200 2024-04-26T04:48:51.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.457+0000 7fc7157fa700 1 -- 172.21.15.90:0/3112090237 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7fc71819b1d0 con 0x7fc70006e200 2024-04-26T04:48:51.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc70006e200 msgr2=0x7fc7000706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc70006e200 0x7fc7000706b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc708006010 tx=0x7fc70800b9d0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100d70 msgr2=0x7fc718196300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100d70 0x7fc718196300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc70c00c930 tx=0x7fc70c00cc40 comp rx=0 tx=0).stop 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 shutdown_connections 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc70006e200 0x7fc7000706b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc718100460 0x7fc718195dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 --2- 172.21.15.90:0/3112090237 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc718100d70 0x7fc718196300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 >> 172.21.15.90:0/3112090237 conn(0x7fc7180748b0 msgr2=0x7fc7180ff0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 shutdown_connections 2024-04-26T04:48:51.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.459+0000 7fc71e4b1700 1 -- 172.21.15.90:0/3112090237 wait complete. 2024-04-26T04:48:51.472 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:48:51.569 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 -- 172.21.15.90:0/913505201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 msgr2=0x7f1be4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 --2- 172.21.15.90:0/913505201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be4103d30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f1be0009910 tx=0x7f1be0009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 -- 172.21.15.90:0/913505201 shutdown_connections 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 --2- 172.21.15.90:0/913505201 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be4103d30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 --2- 172.21.15.90:0/913505201 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be40ff3c0 0x7f1be40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 -- 172.21.15.90:0/913505201 >> 172.21.15.90:0/913505201 conn(0x7f1be40fac60 msgr2=0x7f1be40fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 -- 172.21.15.90:0/913505201 shutdown_connections 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 -- 172.21.15.90:0/913505201 wait complete. 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 Processor -- start 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.567+0000 7f1bec720700 1 -- start start 2024-04-26T04:48:51.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bec720700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be40ff3c0 0x7f1be4198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bec720700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be41985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bec720700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be4198cb0 con 0x7f1be40ffd60 2024-04-26T04:48:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bec720700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419ca40 con 0x7f1be40ff3c0 2024-04-26T04:48:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1be9cbb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be41985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1be9cbb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be41985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51662/0 (socket says 172.21.15.90:51662) 2024-04-26T04:48:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1be9cbb700 1 -- 172.21.15.90:0/1944706220 learned_addr learned my addr 172.21.15.90:0/1944706220 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:51.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bea4bc700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be40ff3c0 0x7f1be4198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1be9cbb700 1 -- 172.21.15.90:0/1944706220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be40ff3c0 msgr2=0x7f1be4198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1be9cbb700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be40ff3c0 0x7f1be4198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1be9cbb700 1 -- 172.21.15.90:0/1944706220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1be00095f0 con 0x7f1be40ffd60 2024-04-26T04:48:51.572 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1be9cbb700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be41985d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1be000bd70 tx=0x7f1be000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bdb7fe700 1 -- 172.21.15.90:0/1944706220 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be001d070 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bdb7fe700 1 -- 172.21.15.90:0/1944706220 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1be0005dc0 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be419ccc0 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bdb7fe700 1 -- 172.21.15.90:0/1944706220 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1be0017400 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.568+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be419d1b0 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.569+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be404f2a0 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.570+0000 7f1bdb7fe700 1 -- 172.21.15.90:0/1944706220 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1be000f7f0 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.570+0000 7f1bdb7fe700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1bd006e200 0x7f1bd00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.570+0000 7f1bdb7fe700 1 -- 172.21.15.90:0/1944706220 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f1be008f600 con 0x7f1be40ffd60 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.571+0000 7f1bea4bc700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1bd006e200 0x7f1bd00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.571+0000 7f1bea4bc700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1bd006e200 0x7f1bd00706b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005950 tx=0x7f1bd40058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.581 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.575+0000 7f1bdb7fe700 1 -- 172.21.15.90:0/1944706220 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1be0058990 con 0x7f1be40ffd60 2024-04-26T04:48:51.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:51 smithi090 podman[103499]: 2024-04-26 04:48:51.341983199 +0000 UTC m=+0.017996392 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:48:51.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:51 smithi090 podman[103499]: 2024-04-26 04:48:51.444622354 +0000 UTC m=+0.120635547 container create 1984ffbc16788c51fbfba24e77bf20bfa92c9ea3e52b50d657b2eb2285f24ab1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:48:51.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:51 smithi090 podman[103499]: 2024-04-26 04:48:51.656585677 +0000 UTC m=+0.332598881 container init 1984ffbc16788c51fbfba24e77bf20bfa92c9ea3e52b50d657b2eb2285f24ab1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-26T04:48:51.722 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:51 smithi090 podman[103499]: 2024-04-26 04:48:51.660196464 +0000 UTC m=+0.336209667 container start 1984ffbc16788c51fbfba24e77bf20bfa92c9ea3e52b50d657b2eb2285f24ab1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-04-26T04:48:51.810 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.809+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f1be419d480 con 0x7f1bd006e200 2024-04-26T04:48:51.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.810+0000 7f1bdb7fe700 1 -- 172.21.15.90:0/1944706220 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f1be419d480 con 0x7f1bd006e200 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.812+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1bd006e200 msgr2=0x7f1bd00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.812+0000 7f1bec720700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1bd006e200 0x7f1bd00706b0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1bd4005950 tx=0x7f1bd40058e0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.812+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 msgr2=0x7f1be41985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.812+0000 7f1bec720700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be41985d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f1be000bd70 tx=0x7f1be000be50 comp rx=0 tx=0).stop 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.812+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 shutdown_connections 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.812+0000 7f1bec720700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1bd006e200 0x7f1bd00706b0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.812+0000 7f1bec720700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1be40ff3c0 0x7f1be4198090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.813+0000 7f1bec720700 1 --2- 172.21.15.90:0/1944706220 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1be40ffd60 0x7f1be41985d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.813+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 >> 172.21.15.90:0/1944706220 conn(0x7f1be40fac60 msgr2=0x7f1be4102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:51.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.813+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 shutdown_connections 2024-04-26T04:48:51.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.813+0000 7f1bec720700 1 -- 172.21.15.90:0/1944706220 wait complete. 2024-04-26T04:48:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 -- 172.21.15.90:0/1259803740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ff350 msgr2=0x7fed940ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 --2- 172.21.15.90:0/1259803740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ff350 0x7fed940ff720 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fed84009910 tx=0x7fed84009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 -- 172.21.15.90:0/1259803740 shutdown_connections 2024-04-26T04:48:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 --2- 172.21.15.90:0/1259803740 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed940ffc60 0x7fed9410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 --2- 172.21.15.90:0/1259803740 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ff350 0x7fed940ff720 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 -- 172.21.15.90:0/1259803740 >> 172.21.15.90:0/1259803740 conn(0x7fed940748b0 msgr2=0x7fed94074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 -- 172.21.15.90:0/1259803740 shutdown_connections 2024-04-26T04:48:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.921+0000 7fed9aa29700 1 -- 172.21.15.90:0/1259803740 wait complete. 2024-04-26T04:48:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.922+0000 7fed9aa29700 1 Processor -- start 2024-04-26T04:48:51.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.922+0000 7fed9aa29700 1 -- start start 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.922+0000 7fed9aa29700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed940ff350 0x7fed94195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.922+0000 7fed9aa29700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ffc60 0x7fed94196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.922+0000 7fed9aa29700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed94196be0 con 0x7fed940ffc60 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.922+0000 7fed9aa29700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed9419a970 con 0x7fed940ff350 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed937fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ffc60 0x7fed94196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed937fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ffc60 0x7fed94196500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51670/0 (socket says 172.21.15.90:51670) 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed937fe700 1 -- 172.21.15.90:0/2907686901 learned_addr learned my addr 172.21.15.90:0/2907686901 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed93fff700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed940ff350 0x7fed94195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed937fe700 1 -- 172.21.15.90:0/2907686901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed940ff350 msgr2=0x7fed94195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed937fe700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed940ff350 0x7fed94195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:51.926 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed937fe700 1 -- 172.21.15.90:0/2907686901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed840095f0 con 0x7fed940ffc60 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed937fe700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ffc60 0x7fed94196500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed8800b4b0 tx=0x7fed8800b7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed917fa700 1 -- 172.21.15.90:0/2907686901 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed88011800 con 0x7fed940ffc60 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed9419ac50 con 0x7fed940ffc60 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed917fa700 1 -- 172.21.15.90:0/2907686901 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fed88011e20 con 0x7fed940ffc60 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed917fa700 1 -- 172.21.15.90:0/2907686901 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fed8800f2e0 con 0x7fed940ffc60 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.923+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed9419b1a0 con 0x7fed940ffc60 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.924+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed94107dd0 con 0x7fed940ffc60 2024-04-26T04:48:51.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.925+0000 7fed917fa700 1 -- 172.21.15.90:0/2907686901 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fed88010380 con 0x7fed940ffc60 2024-04-26T04:48:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.925+0000 7fed917fa700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fed7c06e200 0x7fed7c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.925+0000 7fed917fa700 1 -- 172.21.15.90:0/2907686901 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fed8808dc30 con 0x7fed940ffc60 2024-04-26T04:48:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.925+0000 7fed93fff700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fed7c06e200 0x7fed7c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:51.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.926+0000 7fed93fff700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fed7c06e200 0x7fed7c0706b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed84006010 tx=0x7fed8401a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:51.931 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:51.930+0000 7fed917fa700 1 -- 172.21.15.90:0/2907686901 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fed88056fc0 con 0x7fed940ffc60 2024-04-26T04:48:52.143 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.142+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fed9419b420 con 0x7fed7c06e200 2024-04-26T04:48:52.144 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:51 smithi090 ceph-mon[86550]: osdmap e57: 6 total, 5 up, 6 in 2024-04-26T04:48:52.144 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:51 smithi090 podman[103499]: 2024-04-26 04:48:51.799660613 +0000 UTC m=+0.475673825 container attach 1984ffbc16788c51fbfba24e77bf20bfa92c9ea3e52b50d657b2eb2285f24ab1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-04-26T04:48:52.144 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:51 smithi090 podman[103499]: 2024-04-26 04:48:51.960972766 +0000 UTC m=+0.636985960 container died 1984ffbc16788c51fbfba24e77bf20bfa92c9ea3e52b50d657b2eb2285f24ab1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-26T04:48:52.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.153+0000 7fed917fa700 1 -- 172.21.15.90:0/2907686901 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fed9419b420 con 0x7fed7c06e200 2024-04-26T04:48:52.154 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (3m) 27s ago 14m 25.6M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (14m) 27s ago 14m 8124k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (13m) 50s ago 13m 8032k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (59s) 27s ago 14m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (53s) 50s ago 13m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (3m) 27s ago 14m 73.8M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (8m) 27s ago 8m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (9m) 27s ago 9m 16.4M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (8m) 50s ago 8m 16.6M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (9m) 50s ago 8m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (5m) 27s ago 15m 498M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (4m) 50s ago 13m 448M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (2m) 27s ago 15m 44.4M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:48:52.155 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (100s) 50s ago 13m 38.1M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (4m) 27s ago 14m 16.9M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (4m) 50s ago 13m 14.8M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (34s) 27s ago 12m 11.9M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (12m) 27s ago 12m 247M 1216M 18.2.0 dc2bc1663786 1674cbd25947 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (11m) 27s ago 11m 203M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (11m) 50s ago 11m 271M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (10m) 50s ago 10m 236M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (10m) 50s ago 10m 241M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:48:52.156 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (3m) 27s ago 13m 69.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:48:52.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fed7c06e200 msgr2=0x7fed7c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fed7c06e200 0x7fed7c0706b0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fed84006010 tx=0x7fed8401a040 comp rx=0 tx=0).stop 2024-04-26T04:48:52.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ffc60 msgr2=0x7fed94196500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.157 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ffc60 0x7fed94196500 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fed8800b4b0 tx=0x7fed8800b7c0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 shutdown_connections 2024-04-26T04:48:52.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fed7c06e200 0x7fed7c0706b0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fed940ff350 0x7fed94195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 --2- 172.21.15.90:0/2907686901 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fed940ffc60 0x7fed94196500 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 >> 172.21.15.90:0/2907686901 conn(0x7fed940748b0 msgr2=0x7fed94104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:52.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 shutdown_connections 2024-04-26T04:48:52.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.156+0000 7fed9aa29700 1 -- 172.21.15.90:0/2907686901 wait complete. 2024-04-26T04:48:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:51 smithi104 ceph-mon[64778]: osdmap e57: 6 total, 5 up, 6 in 2024-04-26T04:48:52.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 -- 172.21.15.90:0/3163218858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc104130 msgr2=0x7fa4fc104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 --2- 172.21.15.90:0/3163218858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc104130 0x7fa4fc104500 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fa4f400b0a0 tx=0x7fa4f400b3b0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 -- 172.21.15.90:0/3163218858 shutdown_connections 2024-04-26T04:48:52.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 --2- 172.21.15.90:0/3163218858 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4fc0ffbd0 0x7fa4fc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 --2- 172.21.15.90:0/3163218858 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc104130 0x7fa4fc104500 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 -- 172.21.15.90:0/3163218858 >> 172.21.15.90:0/3163218858 conn(0x7fa4fc075130 msgr2=0x7fa4fc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:52.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 -- 172.21.15.90:0/3163218858 shutdown_connections 2024-04-26T04:48:52.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.262+0000 7fa503439700 1 -- 172.21.15.90:0/3163218858 wait complete. 2024-04-26T04:48:52.265 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa503439700 1 Processor -- start 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa503439700 1 -- start start 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa503439700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc0ffbd0 0x7fa4fc195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa503439700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4fc104130 0x7fa4fc196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa503439700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc196a70 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa503439700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4fc19a800 con 0x7fa4fc104130 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5011d5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc0ffbd0 0x7fa4fc195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5011d5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc0ffbd0 0x7fa4fc195e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51690/0 (socket says 172.21.15.90:51690) 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5011d5700 1 -- 172.21.15.90:0/655478998 learned_addr learned my addr 172.21.15.90:0/655478998 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5009d4700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4fc104130 0x7fa4fc196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5011d5700 1 -- 172.21.15.90:0/655478998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4fc104130 msgr2=0x7fa4fc196390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5011d5700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4fc104130 0x7fa4fc196390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5011d5700 1 -- 172.21.15.90:0/655478998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f4009d00 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5009d4700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4fc104130 0x7fa4fc196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:48:52.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.263+0000 7fa5011d5700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc0ffbd0 0x7fa4fc195e50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4000f80 tx=0x7fa4f400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:52.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.264+0000 7fa4f27fc700 1 -- 172.21.15.90:0/655478998 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4f400e040 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.264+0000 7fa4f27fc700 1 -- 172.21.15.90:0/655478998 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa4f4003c10 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.264+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4fc19aa80 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.264+0000 7fa4f27fc700 1 -- 172.21.15.90:0/655478998 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa4f401d730 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.264+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4fc19af70 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.265+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4fc1086a0 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.265+0000 7fa4f27fc700 1 -- 172.21.15.90:0/655478998 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa4f4019070 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.265+0000 7fa4f27fc700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa4e806e200 0x7fa4e80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:52.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.266+0000 7fa4f27fc700 1 -- 172.21.15.90:0/655478998 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fa4f408f580 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.266+0000 7fa5009d4700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa4e806e200 0x7fa4e80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:52.268 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.266+0000 7fa5009d4700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa4e806e200 0x7fa4e80706b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc197470 tx=0x7fa4ec009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:52.271 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.270+0000 7fa4f27fc700 1 -- 172.21.15.90:0/655478998 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa4f4058910 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.561+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fa4fc1972b0 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.563 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.562+0000 7fa4f27fc700 1 -- 172.21.15.90:0/655478998 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fa4f405bfd0 con 0x7fa4fc0ffbd0 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:48:52.564 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:48:52.565 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:48:52.565 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:48:52.565 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-04-26T04:48:52.565 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 5 2024-04-26T04:48:52.565 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:48:52.565 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa4e806e200 msgr2=0x7fa4e80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa4e806e200 0x7fa4e80706b0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc197470 tx=0x7fa4ec009260 comp rx=0 tx=0).stop 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc0ffbd0 msgr2=0x7fa4fc195e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc0ffbd0 0x7fa4fc195e50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fa4f4000f80 tx=0x7fa4f400bd10 comp rx=0 tx=0).stop 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 shutdown_connections 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa4e806e200 0x7fa4e80706b0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa4fc0ffbd0 0x7fa4fc195e50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 --2- 172.21.15.90:0/655478998 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa4fc104130 0x7fa4fc196390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 >> 172.21.15.90:0/655478998 conn(0x7fa4fc075130 msgr2=0x7fa4fc0fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 shutdown_connections 2024-04-26T04:48:52.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.564+0000 7fa503439700 1 -- 172.21.15.90:0/655478998 wait complete. 2024-04-26T04:48:52.679 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 -- 172.21.15.90:0/4203265561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 msgr2=0x7f8d98104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 --2- 172.21.15.90:0/4203265561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d98104e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c009910 tx=0x7f8d8c009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:52.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 -- 172.21.15.90:0/4203265561 shutdown_connections 2024-04-26T04:48:52.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 --2- 172.21.15.90:0/4203265561 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d98104e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 --2- 172.21.15.90:0/4203265561 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d981004a0 0x7f8d98100870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 -- 172.21.15.90:0/4203265561 >> 172.21.15.90:0/4203265561 conn(0x7f8d980fbd40 msgr2=0x7f8d980fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:52.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 -- 172.21.15.90:0/4203265561 shutdown_connections 2024-04-26T04:48:52.680 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.678+0000 7f8d9d562700 1 -- 172.21.15.90:0/4203265561 wait complete. 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d9d562700 1 Processor -- start 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d9d562700 1 -- start start 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d9d562700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d981004a0 0x7f8d98195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d9d562700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d981963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d9d562700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d98196ad0 con 0x7f8d98100e40 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d9d562700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d9819a860 con 0x7f8d981004a0 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d981963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d967fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d981963f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51704/0 (socket says 172.21.15.90:51704) 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.679+0000 7f8d967fc700 1 -- 172.21.15.90:0/1959820859 learned_addr learned my addr 172.21.15.90:0/1959820859 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d96ffd700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d981004a0 0x7f8d98195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d967fc700 1 -- 172.21.15.90:0/1959820859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d981004a0 msgr2=0x7f8d98195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.682 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d967fc700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d981004a0 0x7f8d98195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d967fc700 1 -- 172.21.15.90:0/1959820859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c0095f0 con 0x7f8d98100e40 2024-04-26T04:48:52.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d967fc700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d981963f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0092e0 tx=0x7f8d8c00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:52.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d7ffff700 1 -- 172.21.15.90:0/1959820859 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d8c01d070 con 0x7f8d98100e40 2024-04-26T04:48:52.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d9819aae0 con 0x7f8d98100e40 2024-04-26T04:48:52.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d7ffff700 1 -- 172.21.15.90:0/1959820859 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8d8c00f460 con 0x7f8d98100e40 2024-04-26T04:48:52.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d7ffff700 1 -- 172.21.15.90:0/1959820859 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8d8c0175f0 con 0x7f8d98100e40 2024-04-26T04:48:52.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.680+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d9819afd0 con 0x7f8d98100e40 2024-04-26T04:48:52.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.681+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d98108810 con 0x7f8d98100e40 2024-04-26T04:48:52.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.682+0000 7f8d7ffff700 1 -- 172.21.15.90:0/1959820859 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8d8c00f5d0 con 0x7f8d98100e40 2024-04-26T04:48:52.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.682+0000 7f8d7ffff700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8d80072610 0x7f8d80074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:52.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.682+0000 7f8d7ffff700 1 -- 172.21.15.90:0/1959820859 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f8d8c08f600 con 0x7f8d98100e40 2024-04-26T04:48:52.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.682+0000 7f8d96ffd700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8d80072610 0x7f8d80074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:52.684 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.683+0000 7f8d96ffd700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8d80072610 0x7f8d80074ac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d8800a560 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:52.689 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.687+0000 7f8d7ffff700 1 -- 172.21.15.90:0/1959820859 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8d8c058990 con 0x7f8d98100e40 2024-04-26T04:48:52.808 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:52 smithi090 podman[103572]: 2024-04-26 04:48:52.527462033 +0000 UTC m=+0.561212189 container remove 1984ffbc16788c51fbfba24e77bf20bfa92c9ea3e52b50d657b2eb2285f24ab1 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T04:48:52.808 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:52 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.1.service: Deactivated successfully. 2024-04-26T04:48:52.808 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:52 smithi090 systemd[1]: Stopped Ceph osd.1 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:48:52.808 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:52 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.1.service: Consumed 1min 3.079s CPU time. 2024-04-26T04:48:52.808 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:52 smithi090 systemd[1]: Starting Ceph osd.1 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:48:52.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.956+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f8d9804f2a0 con 0x7f8d98100e40 2024-04-26T04:48:52.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.956+0000 7f8d7ffff700 1 -- 172.21.15.90:0/1959820859 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f8d8c05c050 con 0x7f8d98100e40 2024-04-26T04:48:52.958 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:48:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:48:52.960 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:52.961 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8d80072610 msgr2=0x7f8d80074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8d80072610 0x7f8d80074ac0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f8d88005fd0 tx=0x7f8d8800a560 comp rx=0 tx=0).stop 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 msgr2=0x7f8d981963f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d981963f0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f8d8c0092e0 tx=0x7f8d8c00be20 comp rx=0 tx=0).stop 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 shutdown_connections 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8d80072610 0x7f8d80074ac0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8d981004a0 0x7f8d98195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 --2- 172.21.15.90:0/1959820859 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8d98100e40 0x7f8d981963f0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 >> 172.21.15.90:0/1959820859 conn(0x7f8d980fbd40 msgr2=0x7f8d980fe110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 shutdown_connections 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:52.959+0000 7f8d9d562700 1 -- 172.21.15.90:0/1959820859 wait complete. 2024-04-26T04:48:52.963 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:48:53.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 -- 172.21.15.90:0/3341508652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 msgr2=0x7f36901008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 --2- 172.21.15.90:0/3341508652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f36901008b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3680009910 tx=0x7f3680009c20 comp rx=0 tx=0).stop 2024-04-26T04:48:53.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 -- 172.21.15.90:0/3341508652 shutdown_connections 2024-04-26T04:48:53.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 --2- 172.21.15.90:0/3341508652 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3690100e80 0x7f3690104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 --2- 172.21.15.90:0/3341508652 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f36901008b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 -- 172.21.15.90:0/3341508652 >> 172.21.15.90:0/3341508652 conn(0x7f36900fbda0 msgr2=0x7f36900fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:53.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 -- 172.21.15.90:0/3341508652 shutdown_connections 2024-04-26T04:48:53.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 -- 172.21.15.90:0/3341508652 wait complete. 2024-04-26T04:48:53.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:52 smithi090 ceph-mon[86550]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:53.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:52 smithi090 ceph-mon[86550]: pgmap v68: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:53.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:52 smithi090 ceph-mon[86550]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-26T04:48:53.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:52 smithi090 ceph-mon[86550]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:53.069 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:52 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/655478998' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:53.070 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:52 smithi090 podman[103724]: 2024-04-26 04:48:52.901510922 +0000 UTC m=+0.018306415 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:48:53.070 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 podman[103724]: 2024-04-26 04:48:53.022356228 +0000 UTC m=+0.139151695 container create 49a5f877cb806b565a0241326a74327658bd725c6ae515290f319fd0bc9b74e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:48:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.066+0000 7f3695eaa700 1 Processor -- start 2024-04-26T04:48:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f3695eaa700 1 -- start start 2024-04-26T04:48:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f3695eaa700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f3690195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f3695eaa700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3690100e80 0x7f3690196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f3695eaa700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3690196b30 con 0x7f36901004e0 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f3695eaa700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f369019a8c0 con 0x7f3690100e80 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f3690195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f3690195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51730/0 (socket says 172.21.15.90:51730) 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368f7fe700 1 -- 172.21.15.90:0/2260792719 learned_addr learned my addr 172.21.15.90:0/2260792719 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368effd700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3690100e80 0x7f3690196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368f7fe700 1 -- 172.21.15.90:0/2260792719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3690100e80 msgr2=0x7f3690196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368f7fe700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3690100e80 0x7f3690196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368f7fe700 1 -- 172.21.15.90:0/2260792719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36800095f0 con 0x7f36901004e0 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.067+0000 7f368f7fe700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f3690195f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f36800092e0 tx=0x7f368000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:53.074 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.068+0000 7f368cff9700 1 -- 172.21.15.90:0/2260792719 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f368001d070 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.068+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f369019ab40 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.068+0000 7f368cff9700 1 -- 172.21.15.90:0/2260792719 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3680005e00 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.068+0000 7f368cff9700 1 -- 172.21.15.90:0/2260792719 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3680017400 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.068+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f369019b030 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.069+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36901087f0 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.069+0000 7f368cff9700 1 -- 172.21.15.90:0/2260792719 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f368000f6f0 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.070+0000 7f368cff9700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f367806e200 0x7f36780706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.070+0000 7f368cff9700 1 -- 172.21.15.90:0/2260792719 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f368008f5c0 con 0x7f36901004e0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.070+0000 7f368effd700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f367806e200 0x7f36780706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.070+0000 7f368effd700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f367806e200 0x7f36780706b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3690197530 tx=0x7f368400b3c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.074+0000 7f368cff9700 1 -- 172.21.15.90:0/2260792719 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3680058950 con 0x7f36901004e0 2024-04-26T04:48:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:52 smithi104 ceph-mon[64778]: from='client.34220 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:52 smithi104 ceph-mon[64778]: pgmap v68: 65 pgs: 5 peering, 9 stale+active+clean, 51 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:48:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:52 smithi104 ceph-mon[64778]: Health check failed: Reduced data availability: 3 pgs peering (PG_AVAILABILITY) 2024-04-26T04:48:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:52 smithi104 ceph-mon[64778]: from='client.34224 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:52 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/655478998' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:48:53.302 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.301+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f369019b250 con 0x7f367806e200 2024-04-26T04:48:53.303 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.302+0000 7f368cff9700 1 -- 172.21.15.90:0/2260792719 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f369019b250 con 0x7f367806e200 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "7/23 daemons upgraded", 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:48:53.304 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.304+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f367806e200 msgr2=0x7f36780706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.304+0000 7f3695eaa700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f367806e200 0x7f36780706b0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3690197530 tx=0x7f368400b3c0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.304+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 msgr2=0x7f3690195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f3690195f10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f36800092e0 tx=0x7f368000bd70 comp rx=0 tx=0).stop 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 shutdown_connections 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f367806e200 0x7f36780706b0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36901004e0 0x7f3690195f10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.306 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 --2- 172.21.15.90:0/2260792719 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3690100e80 0x7f3690196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 >> 172.21.15.90:0/2260792719 conn(0x7f36900fbda0 msgr2=0x7f36900fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:53.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 shutdown_connections 2024-04-26T04:48:53.307 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.305+0000 7f3695eaa700 1 -- 172.21.15.90:0/2260792719 wait complete. 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 -- 172.21.15.90:0/690252420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 msgr2=0x7f556c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 --2- 172.21.15.90:0/690252420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c100830 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f5568009940 tx=0x7f5568009c50 comp rx=0 tx=0).stop 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 -- 172.21.15.90:0/690252420 shutdown_connections 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 --2- 172.21.15.90:0/690252420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c100d70 0x7f556c1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 --2- 172.21.15.90:0/690252420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c100830 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 -- 172.21.15.90:0/690252420 >> 172.21.15.90:0/690252420 conn(0x7f556c0748b0 msgr2=0x7f556c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 -- 172.21.15.90:0/690252420 shutdown_connections 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.413+0000 7f557399b700 1 -- 172.21.15.90:0/690252420 wait complete. 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f557399b700 1 Processor -- start 2024-04-26T04:48:53.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f557399b700 1 -- start start 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f557399b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c101540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f557399b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c100d70 0x7f556c101a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f557399b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c101fc0 con 0x7f556c100460 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f557399b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f556c102100 con 0x7f556c100d70 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f5571737700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c101540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f5571737700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c101540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:51740/0 (socket says 172.21.15.90:51740) 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f5571737700 1 -- 172.21.15.90:0/975808150 learned_addr learned my addr 172.21.15.90:0/975808150 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:48:53.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f5570f36700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c100d70 0x7f556c101a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f5571737700 1 -- 172.21.15.90:0/975808150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c100d70 msgr2=0x7f556c101a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f5571737700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c100d70 0x7f556c101a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.414+0000 7f5571737700 1 -- 172.21.15.90:0/975808150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55680095f0 con 0x7f556c100460 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.415+0000 7f5571737700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c101540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55680055f0 tx=0x7f55680056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.415+0000 7f55627fc700 1 -- 172.21.15.90:0/975808150 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f556801d070 con 0x7f556c100460 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.415+0000 7f55627fc700 1 -- 172.21.15.90:0/975808150 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f556800f460 con 0x7f556c100460 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.415+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f556c102380 con 0x7f556c100460 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.415+0000 7f55627fc700 1 -- 172.21.15.90:0/975808150 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5568021620 con 0x7f556c100460 2024-04-26T04:48:53.418 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.415+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f556c1a2460 con 0x7f556c100460 2024-04-26T04:48:53.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.416+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f556c107e40 con 0x7f556c100460 2024-04-26T04:48:53.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.416+0000 7f55627fc700 1 -- 172.21.15.90:0/975808150 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f556800fad0 con 0x7f556c100460 2024-04-26T04:48:53.419 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.417+0000 7f55627fc700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f555806e200 0x7f55580706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:48:53.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.417+0000 7f55627fc700 1 -- 172.21.15.90:0/975808150 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f556808f3e0 con 0x7f556c100460 2024-04-26T04:48:53.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.417+0000 7f5570f36700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f555806e200 0x7f55580706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:48:53.420 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.417+0000 7f5570f36700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f555806e200 0x7f55580706b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f556c102d30 tx=0x7f555c005cf0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:48:53.422 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.422+0000 7f55627fc700 1 -- 172.21.15.90:0/975808150 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f55680588b0 con 0x7f556c100460 2024-04-26T04:48:53.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 podman[103724]: 2024-04-26 04:48:53.344197376 +0000 UTC m=+0.460992858 container init 49a5f877cb806b565a0241326a74327658bd725c6ae515290f319fd0bc9b74e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:48:53.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 podman[103724]: 2024-04-26 04:48:53.348767947 +0000 UTC m=+0.465563425 container start 49a5f877cb806b565a0241326a74327658bd725c6ae515290f319fd0bc9b74e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:48:53.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 podman[103724]: 2024-04-26 04:48:53.418897891 +0000 UTC m=+0.535693361 container attach 49a5f877cb806b565a0241326a74327658bd725c6ae515290f319fd0bc9b74e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.build-date=20240424) 2024-04-26T04:48:53.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.722+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f556c04f2a0 con 0x7f556c100460 2024-04-26T04:48:53.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.722+0000 7f55627fc700 1 -- 172.21.15.90:0/975808150 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+438 (secure 0 0 0) 0x7f556805bf70 con 0x7f556c100460 2024-04-26T04:48:53.724 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 osds down; Reduced data availability: 3 pgs peering 2024-04-26T04:48:53.724 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:48:53.724 INFO:teuthology.orchestra.run.smithi090.stdout: osd.1 (root=default,host=smithi090) is down 2024-04-26T04:48:53.724 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] PG_AVAILABILITY: Reduced data availability: 3 pgs peering 2024-04-26T04:48:53.724 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.4 is stuck peering for 5m, current state peering, last acting [0,4] 2024-04-26T04:48:53.724 INFO:teuthology.orchestra.run.smithi090.stdout: pg 2.1b is stuck peering for 6m, current state peering, last acting [0,5] 2024-04-26T04:48:53.724 INFO:teuthology.orchestra.run.smithi090.stdout: pg 3.b is stuck peering for 5m, current state peering, last acting [0,4] 2024-04-26T04:48:53.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f555806e200 msgr2=0x7f55580706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f555806e200 0x7f55580706b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f556c102d30 tx=0x7f555c005cf0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 msgr2=0x7f556c101540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:48:53.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c101540 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f55680055f0 tx=0x7f55680056d0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 shutdown_connections 2024-04-26T04:48:53.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f555806e200 0x7f55580706b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f556c100460 0x7f556c101540 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 --2- 172.21.15.90:0/975808150 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f556c100d70 0x7f556c101a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:48:53.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 >> 172.21.15.90:0/975808150 conn(0x7f556c0748b0 msgr2=0x7f556c0fef40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:48:53.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 shutdown_connections 2024-04-26T04:48:53.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:48:53.725+0000 7f557399b700 1 -- 172.21.15.90:0/975808150 wait complete. 2024-04-26T04:48:54.114 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-mon[86550]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:54.114 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1959820859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:48:54.115 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/975808150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate[103762]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate[103762]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 bash[103724]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 bash[103724]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-1 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 bash[103724]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 bash[103724]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 bash[103724]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 bash[103724]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 bash[103724]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate[103762]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:48:54.115 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate[103762]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:48:54.116 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate[103762]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-1/block 2024-04-26T04:48:54.116 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate[103762]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-26T04:48:54.116 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate[103762]: --> ceph-volume raw activate successful for osd ID: 1 2024-04-26T04:48:54.116 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:53 smithi090 podman[103724]: 2024-04-26 04:48:53.829154745 +0000 UTC m=+0.945950242 container died 49a5f877cb806b565a0241326a74327658bd725c6ae515290f319fd0bc9b74e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:48:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:53 smithi104 ceph-mon[64778]: from='client.34228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:53 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1959820859' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:48:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:53 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/975808150' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:48:54.712 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 podman[103928]: 2024-04-26 04:48:54.390273105 +0000 UTC m=+0.555255223 container remove 49a5f877cb806b565a0241326a74327658bd725c6ae515290f319fd0bc9b74e7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True) 2024-04-26T04:48:54.712 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 podman[103992]: 2024-04-26 04:48:54.509060778 +0000 UTC m=+0.018371880 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:48:54.713 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 podman[103992]: 2024-04-26 04:48:54.608066142 +0000 UTC m=+0.117377232 container create 264b38fc9f3109120ebb265eee301143780e02a8f868d06beca0de5e711e361b (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424) 2024-04-26T04:48:54.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:54 smithi090 ceph-mon[86550]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:54.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:54 smithi090 ceph-mon[86550]: pgmap v69: 65 pgs: 14 active+undersized+degraded, 5 peering, 15 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/348 objects degraded (13.506%) 2024-04-26T04:48:54.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:54 smithi090 ceph-mon[86550]: Health check failed: Degraded data redundancy: 47/348 objects degraded (13.506%), 14 pgs degraded (PG_DEGRADED) 2024-04-26T04:48:54.972 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 podman[103992]: 2024-04-26 04:48:54.837412885 +0000 UTC m=+0.346723975 container init 264b38fc9f3109120ebb265eee301143780e02a8f868d06beca0de5e711e361b (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:48:54.972 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 podman[103992]: 2024-04-26 04:48:54.840477292 +0000 UTC m=+0.349788383 container start 264b38fc9f3109120ebb265eee301143780e02a8f868d06beca0de5e711e361b (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T04:48:54.972 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 ceph-osd[104011]: -- 172.21.15.90:0/1261303245 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5591bd23d680 con 0x5591bd207000 2024-04-26T04:48:54.972 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 bash[103992]: 264b38fc9f3109120ebb265eee301143780e02a8f868d06beca0de5e711e361b 2024-04-26T04:48:54.972 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:54 smithi090 systemd[1]: Started Ceph osd.1 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:48:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:54 smithi104 ceph-mon[64778]: from='client.34240 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:48:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:54 smithi104 ceph-mon[64778]: pgmap v69: 65 pgs: 14 active+undersized+degraded, 5 peering, 15 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 47/348 objects degraded (13.506%) 2024-04-26T04:48:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:54 smithi104 ceph-mon[64778]: Health check failed: Degraded data redundancy: 47/348 objects degraded (13.506%), 14 pgs degraded (PG_DEGRADED) 2024-04-26T04:48:55.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:55 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[104007]: 2024-04-26T04:48:55.383+0000 7f6afec9f7c0 -1 Falling back to public interface 2024-04-26T04:48:56.082 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:56.082 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:56.082 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:56.082 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:55 smithi090 ceph-mon[86550]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:48:56.082 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:56.082 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:48:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:55 smithi104 ceph-mon[64778]: pgmap v70: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:48:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:48:56.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:48:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:56 smithi104 ceph-mon[64778]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-26T04:48:57.284 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:56 smithi090 ceph-mon[86550]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 3 pgs peering) 2024-04-26T04:48:58.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:48:57 smithi090 ceph-mon[86550]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:48:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:48:57 smithi104 ceph-mon[64778]: pgmap v71: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:00.221 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:48:59 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[104007]: 2024-04-26T04:48:59.968+0000 7f6afec9f7c0 -1 osd.1 0 read_superblock omap replica is missing. 2024-04-26T04:49:00.971 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:49:00 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[104007]: 2024-04-26T04:49:00.484+0000 7f6afec9f7c0 -1 osd.1 55 log_to_monitors true 2024-04-26T04:49:00.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:00 smithi090 ceph-mon[86550]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:00.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:00.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:00.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:00 smithi090 ceph-mon[86550]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:49:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:00 smithi104 ceph-mon[64778]: pgmap v72: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.5 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:00 smithi104 ceph-mon[64778]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-26T04:49:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:01 smithi104 ceph-mon[64778]: Health check update: Degraded data redundancy: 54/348 objects degraded (15.517%), 16 pgs degraded (PG_DEGRADED) 2024-04-26T04:49:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:01 smithi104 ceph-mon[64778]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:49:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:01 smithi104 ceph-mon[64778]: osdmap e58: 6 total, 5 up, 6 in 2024-04-26T04:49:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:01 smithi104 ceph-mon[64778]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:49:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:01 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:02.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:01 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:02.077 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:49:01 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[104007]: 2024-04-26T04:49:01.941+0000 7f6aec0ef700 -1 osd.1 55 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:49:02.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:01 smithi090 ceph-mon[86550]: Health check update: Degraded data redundancy: 54/348 objects degraded (15.517%), 16 pgs degraded (PG_DEGRADED) 2024-04-26T04:49:02.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:01 smithi090 ceph-mon[86550]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-26T04:49:02.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:01 smithi090 ceph-mon[86550]: osdmap e58: 6 total, 5 up, 6 in 2024-04-26T04:49:02.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:01 smithi090 ceph-mon[86550]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:49:02.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:01 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:02.078 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:01 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:03.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:02 smithi104 ceph-mon[64778]: pgmap v74: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:03.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:02 smithi104 ceph-mon[64778]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' 2024-04-26T04:49:03.089 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:02 smithi090 ceph-mon[86550]: pgmap v74: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:03.090 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:02 smithi090 ceph-mon[86550]: from='osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421]' entity='osd.1' 2024-04-26T04:49:04.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:03 smithi104 ceph-mon[64778]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:49:04.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:03 smithi104 ceph-mon[64778]: osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421] boot 2024-04-26T04:49:04.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:03 smithi104 ceph-mon[64778]: osdmap e59: 6 total, 6 up, 6 in 2024-04-26T04:49:04.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:03 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:49:04.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:03 smithi090 ceph-mon[86550]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:49:04.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:03 smithi090 ceph-mon[86550]: osd.1 [v2:172.21.15.90:6810/2770756421,v1:172.21.15.90:6811/2770756421] boot 2024-04-26T04:49:04.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:03 smithi090 ceph-mon[86550]: osdmap e59: 6 total, 6 up, 6 in 2024-04-26T04:49:04.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-26T04:49:04.907 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:04 smithi090 ceph-mon[86550]: pgmap v76: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:04.907 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:04 smithi090 ceph-mon[86550]: osdmap e60: 6 total, 6 up, 6 in 2024-04-26T04:49:05.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:04 smithi104 ceph-mon[64778]: pgmap v76: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:05.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:04 smithi104 ceph-mon[64778]: osdmap e60: 6 total, 6 up, 6 in 2024-04-26T04:49:06.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:06.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:06.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:06.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:06.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:06.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:06.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:06.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:06.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:06.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:06.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:06.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-26T04:49:06.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:05 smithi090 ceph-mon[86550]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 16 active+undersized, 33 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:06.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:06.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:06.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:06.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:06.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: Upgrade: unsafe to stop osd(s) at this time (12 PGs are or would become offline) 2024-04-26T04:49:06.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:05 smithi104 ceph-mon[64778]: pgmap v78: 65 pgs: 16 active+undersized+degraded, 16 active+undersized, 33 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 54/348 objects degraded (15.517%) 2024-04-26T04:49:08.064 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:07 smithi104 ceph-mon[64778]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/348 objects degraded (15.517%), 16 pgs degraded) 2024-04-26T04:49:08.064 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:07 smithi104 ceph-mon[64778]: Cluster is now healthy 2024-04-26T04:49:08.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:07 smithi090 ceph-mon[86550]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 54/348 objects degraded (15.517%), 16 pgs degraded) 2024-04-26T04:49:08.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:07 smithi090 ceph-mon[86550]: Cluster is now healthy 2024-04-26T04:49:09.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:08 smithi090 ceph-mon[86550]: pgmap v79: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:09.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:08 smithi104 ceph-mon[64778]: pgmap v79: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:10 smithi090 ceph-mon[86550]: pgmap v80: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:10 smithi104 ceph-mon[64778]: pgmap v80: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:13.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:12 smithi090 ceph-mon[86550]: pgmap v81: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:12 smithi104 ceph-mon[64778]: pgmap v81: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:15.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:14 smithi090 ceph-mon[86550]: pgmap v82: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:14 smithi104 ceph-mon[64778]: pgmap v82: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:17.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:16 smithi090 ceph-mon[86550]: pgmap v83: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:17.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:17.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:16 smithi104 ceph-mon[64778]: pgmap v83: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:17.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:18 smithi090 ceph-mon[86550]: pgmap v84: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:18 smithi104 ceph-mon[64778]: pgmap v84: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:20.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:20 smithi090 ceph-mon[86550]: pgmap v85: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:20.981 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:20 smithi104 ceph-mon[64778]: pgmap v85: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:22.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:21 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:22.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:21 smithi090 ceph-mon[86550]: Upgrade: osd.2 is safe to restart 2024-04-26T04:49:22.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:21 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["2"], "max": 16}]: dispatch 2024-04-26T04:49:22.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:21 smithi104 ceph-mon[64778]: Upgrade: osd.2 is safe to restart 2024-04-26T04:49:22.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:22 smithi090 ceph-mon[86550]: pgmap v86: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:22.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:22 smithi090 ceph-mon[86550]: Upgrade: Updating osd.2 2024-04-26T04:49:22.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:22.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:49:22.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:22.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:22 smithi090 ceph-mon[86550]: Deploying daemon osd.2 on smithi090 2024-04-26T04:49:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:22 smithi104 ceph-mon[64778]: pgmap v86: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:22 smithi104 ceph-mon[64778]: Upgrade: Updating osd.2 2024-04-26T04:49:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-26T04:49:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:22 smithi104 ceph-mon[64778]: Deploying daemon osd.2 on smithi090 2024-04-26T04:49:23.842 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 -- 172.21.15.90:0/1577290250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ffc60 msgr2=0x7f060810a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 --2- 172.21.15.90:0/1577290250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ffc60 0x7f060810a6d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f05f8009910 tx=0x7f05f8009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 -- 172.21.15.90:0/1577290250 shutdown_connections 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 --2- 172.21.15.90:0/1577290250 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ffc60 0x7f060810a6d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 --2- 172.21.15.90:0/1577290250 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06080ff350 0x7f06080ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 -- 172.21.15.90:0/1577290250 >> 172.21.15.90:0/1577290250 conn(0x7f06080748b0 msgr2=0x7f0608074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 -- 172.21.15.90:0/1577290250 shutdown_connections 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.835+0000 7f060da36700 1 -- 172.21.15.90:0/1577290250 wait complete. 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f060da36700 1 Processor -- start 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f060da36700 1 -- start start 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f060da36700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ff350 0x7f0608195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f060da36700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06080ffc60 0x7f06081964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:23.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f060da36700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0608196ba0 con 0x7f06080ff350 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f060da36700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f060819a930 con 0x7f06080ffc60 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f0606ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ff350 0x7f0608195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f0606ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ff350 0x7f0608195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48182/0 (socket says 172.21.15.90:48182) 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f0606ffd700 1 -- 172.21.15.90:0/773693444 learned_addr learned my addr 172.21.15.90:0/773693444 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.836+0000 7f06067fc700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06080ffc60 0x7f06081964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f0606ffd700 1 -- 172.21.15.90:0/773693444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06080ffc60 msgr2=0x7f06081964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f0606ffd700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06080ffc60 0x7f06081964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f0606ffd700 1 -- 172.21.15.90:0/773693444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f80095f0 con 0x7f06080ff350 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f0606ffd700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ff350 0x7f0608195f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00b480 tx=0x7f05fc00b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f060ca34700 1 -- 172.21.15.90:0/773693444 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05fc011800 con 0x7f06080ff350 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f060ca34700 1 -- 172.21.15.90:0/773693444 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f05fc011e20 con 0x7f06080ff350 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f060819ac10 con 0x7f06080ff350 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f060819b160 con 0x7f06080ff350 2024-04-26T04:49:23.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.837+0000 7f060ca34700 1 -- 172.21.15.90:0/773693444 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f05fc010ea0 con 0x7f06080ff350 2024-04-26T04:49:23.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.838+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0608107dd0 con 0x7f06080ff350 2024-04-26T04:49:23.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.839+0000 7f060ca34700 1 -- 172.21.15.90:0/773693444 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f05fc011960 con 0x7f06080ff350 2024-04-26T04:49:23.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.839+0000 7f060ca34700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f05f006e200 0x7f05f00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:23.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.839+0000 7f060ca34700 1 -- 172.21.15.90:0/773693444 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f05fc08db50 con 0x7f06080ff350 2024-04-26T04:49:23.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.839+0000 7f06067fc700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f05f006e200 0x7f05f00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:23.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.840+0000 7f06067fc700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f05f006e200 0x7f05f00706b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06081975a0 tx=0x7f05f801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:23.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:23.844+0000 7f060ca34700 1 -- 172.21.15.90:0/773693444 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f05fc056f60 con 0x7f06080ff350 2024-04-26T04:49:24.090 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.089+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f060819b430 con 0x7f05f006e200 2024-04-26T04:49:24.091 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.090+0000 7f060ca34700 1 -- 172.21.15.90:0/773693444 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f060819b430 con 0x7f05f006e200 2024-04-26T04:49:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.092+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f05f006e200 msgr2=0x7f05f00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.092+0000 7f060da36700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f05f006e200 0x7f05f00706b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f06081975a0 tx=0x7f05f801a040 comp rx=0 tx=0).stop 2024-04-26T04:49:24.094 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.092+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ff350 msgr2=0x7f0608195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.092+0000 7f060da36700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ff350 0x7f0608195f80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f05fc00b480 tx=0x7f05fc00b790 comp rx=0 tx=0).stop 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.093+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 shutdown_connections 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.093+0000 7f060da36700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f05f006e200 0x7f05f00706b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.093+0000 7f060da36700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f06080ff350 0x7f0608195f80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.093+0000 7f060da36700 1 --2- 172.21.15.90:0/773693444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f06080ffc60 0x7f06081964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.093+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 >> 172.21.15.90:0/773693444 conn(0x7f06080748b0 msgr2=0x7f0608104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.093+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 shutdown_connections 2024-04-26T04:49:24.095 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.093+0000 7f060da36700 1 -- 172.21.15.90:0/773693444 wait complete. 2024-04-26T04:49:24.104 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:49:24.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.198+0000 7ff907a4f700 1 -- 172.21.15.90:0/3978815169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 msgr2=0x7ff900103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.198+0000 7ff907a4f700 1 --2- 172.21.15.90:0/3978815169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900103d30 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc009910 tx=0x7ff8fc009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:24.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.198+0000 7ff907a4f700 1 -- 172.21.15.90:0/3978815169 shutdown_connections 2024-04-26T04:49:24.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.198+0000 7ff907a4f700 1 --2- 172.21.15.90:0/3978815169 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900103d30 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.198+0000 7ff907a4f700 1 --2- 172.21.15.90:0/3978815169 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9000ff3c0 0x7ff9000ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 -- 172.21.15.90:0/3978815169 >> 172.21.15.90:0/3978815169 conn(0x7ff9000fac80 msgr2=0x7ff9000fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 -- 172.21.15.90:0/3978815169 shutdown_connections 2024-04-26T04:49:24.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 -- 172.21.15.90:0/3978815169 wait complete. 2024-04-26T04:49:24.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 Processor -- start 2024-04-26T04:49:24.201 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 -- start start 2024-04-26T04:49:24.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9000ff3c0 0x7ff900196df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900197330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff900197a10 con 0x7ff9000ffd60 2024-04-26T04:49:24.202 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.199+0000 7ff907a4f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff900197b50 con 0x7ff9000ff3c0 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff904fea700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900197330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff904fea700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900197330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48204/0 (socket says 172.21.15.90:48204) 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff904fea700 1 -- 172.21.15.90:0/2990694721 learned_addr learned my addr 172.21.15.90:0/2990694721 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff9057eb700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9000ff3c0 0x7ff900196df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff904fea700 1 -- 172.21.15.90:0/2990694721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9000ff3c0 msgr2=0x7ff900196df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff904fea700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9000ff3c0 0x7ff900196df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff904fea700 1 -- 172.21.15.90:0/2990694721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8fc0095f0 con 0x7ff9000ffd60 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff904fea700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900197330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00bda0 tx=0x7ff8fc00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff8f67fc700 1 -- 172.21.15.90:0/2990694721 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8fc01d070 con 0x7ff9000ffd60 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff90019fe60 con 0x7ff9000ffd60 2024-04-26T04:49:24.203 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff8f67fc700 1 -- 172.21.15.90:0/2990694721 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff8fc00f460 con 0x7ff9000ffd60 2024-04-26T04:49:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff8f67fc700 1 -- 172.21.15.90:0/2990694721 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff8fc017630 con 0x7ff9000ffd60 2024-04-26T04:49:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.200+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9001a02d0 con 0x7ff9000ffd60 2024-04-26T04:49:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.201+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff900107680 con 0x7ff9000ffd60 2024-04-26T04:49:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.202+0000 7ff8f67fc700 1 -- 172.21.15.90:0/2990694721 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff8fc00fab0 con 0x7ff9000ffd60 2024-04-26T04:49:24.204 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.202+0000 7ff8f67fc700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff8ec06e200 0x7ff8ec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.202+0000 7ff8f67fc700 1 -- 172.21.15.90:0/2990694721 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff8fc08f5b0 con 0x7ff9000ffd60 2024-04-26T04:49:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.202+0000 7ff9057eb700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff8ec06e200 0x7ff8ec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.205 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.203+0000 7ff9057eb700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff8ec06e200 0x7ff8ec0706b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009a10 tx=0x7ff8f0009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.208 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.207+0000 7ff8f67fc700 1 -- 172.21.15.90:0/2990694721 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff8fc058940 con 0x7ff9000ffd60 2024-04-26T04:49:24.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.429+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff900061190 con 0x7ff8ec06e200 2024-04-26T04:49:24.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.430+0000 7ff8f67fc700 1 -- 172.21.15.90:0/2990694721 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff900061190 con 0x7ff8ec06e200 2024-04-26T04:49:24.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff8ec06e200 msgr2=0x7ff8ec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff8ec06e200 0x7ff8ec0706b0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff8f0009a10 tx=0x7ff8f0009190 comp rx=0 tx=0).stop 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 msgr2=0x7ff900197330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900197330 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7ff8fc00bda0 tx=0x7ff8fc00bdd0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 shutdown_connections 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff8ec06e200 0x7ff8ec0706b0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff9000ff3c0 0x7ff900196df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 --2- 172.21.15.90:0/2990694721 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff9000ffd60 0x7ff900197330 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 >> 172.21.15.90:0/2990694721 conn(0x7ff9000fac80 msgr2=0x7ff9000fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 shutdown_connections 2024-04-26T04:49:24.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.432+0000 7ff907a4f700 1 -- 172.21.15.90:0/2990694721 wait complete. 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 -- 172.21.15.90:0/1801655275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c0ffbd0 msgr2=0x7feb0c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 --2- 172.21.15.90:0/1801655275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c0ffbd0 0x7feb0c100040 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7feb08009910 tx=0x7feb08009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 -- 172.21.15.90:0/1801655275 shutdown_connections 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 --2- 172.21.15.90:0/1801655275 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c0ffbd0 0x7feb0c100040 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 --2- 172.21.15.90:0/1801655275 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0c104130 0x7feb0c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 -- 172.21.15.90:0/1801655275 >> 172.21.15.90:0/1801655275 conn(0x7feb0c075130 msgr2=0x7feb0c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 -- 172.21.15.90:0/1801655275 shutdown_connections 2024-04-26T04:49:24.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.539+0000 7feb143df700 1 -- 172.21.15.90:0/1801655275 wait complete. 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb143df700 1 Processor -- start 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb143df700 1 -- start start 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb143df700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0c0ffbd0 0x7feb0c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb143df700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c104130 0x7feb0c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb143df700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1018b0 con 0x7feb0c104130 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb143df700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1019f0 con 0x7feb0c0ffbd0 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1197a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c104130 0x7feb0c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1197a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c104130 0x7feb0c101370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48228/0 (socket says 172.21.15.90:48228) 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1197a700 1 -- 172.21.15.90:0/2709091298 learned_addr learned my addr 172.21.15.90:0/2709091298 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1217b700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0c0ffbd0 0x7feb0c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1197a700 1 -- 172.21.15.90:0/2709091298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0c0ffbd0 msgr2=0x7feb0c100e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1197a700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0c0ffbd0 0x7feb0c100e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1197a700 1 -- 172.21.15.90:0/2709091298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080095f0 con 0x7feb0c104130 2024-04-26T04:49:24.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1217b700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0c0ffbd0 0x7feb0c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:49:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.540+0000 7feb1197a700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c104130 0x7feb0c101370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb0800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.541+0000 7feb037fe700 1 -- 172.21.15.90:0/2709091298 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0801d070 con 0x7feb0c104130 2024-04-26T04:49:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.541+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c101c70 con 0x7feb0c104130 2024-04-26T04:49:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.541+0000 7feb037fe700 1 -- 172.21.15.90:0/2709091298 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb08004fb0 con 0x7feb0c104130 2024-04-26T04:49:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.541+0000 7feb037fe700 1 -- 172.21.15.90:0/2709091298 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb0800f690 con 0x7feb0c104130 2024-04-26T04:49:24.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.541+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1a2300 con 0x7feb0c104130 2024-04-26T04:49:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.541+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c1086a0 con 0x7feb0c104130 2024-04-26T04:49:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.542+0000 7feb037fe700 1 -- 172.21.15.90:0/2709091298 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feb08022470 con 0x7feb0c104130 2024-04-26T04:49:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.542+0000 7feb037fe700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feaf806e200 0x7feaf80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.543+0000 7feb1217b700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feaf806e200 0x7feaf80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.543+0000 7feb037fe700 1 -- 172.21.15.90:0/2709091298 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7feb0808f8a0 con 0x7feb0c104130 2024-04-26T04:49:24.545 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.543+0000 7feb1217b700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feaf806e200 0x7feaf80706b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feafc005fd0 tx=0x7feafc005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.549 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.548+0000 7feb037fe700 1 -- 172.21.15.90:0/2709091298 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feb08058c30 con 0x7feb0c104130 2024-04-26T04:49:24.770 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.768+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7feb0c102440 con 0x7feaf806e200 2024-04-26T04:49:24.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.779+0000 7feb037fe700 1 -- 172.21.15.90:0/2709091298 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7feb0c102440 con 0x7feaf806e200 2024-04-26T04:49:24.780 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (4m) 24s ago 14m 25.6M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (15m) 24s ago 15m 8388k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (14m) 83s ago 14m 8032k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (91s) 24s ago 15m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (86s) 83s ago 14m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (3m) 24s ago 14m 76.5M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (9m) 24s ago 9m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (9m) 24s ago 9m 16.6M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (9m) 83s ago 9m 16.6M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (9m) 83s ago 9m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (5m) 24s ago 16m 504M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (5m) 83s ago 13m 448M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:49:24.781 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (2m) 24s ago 16m 47.4M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (2m) 83s ago 13m 38.1M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (4m) 24s ago 15m 16.4M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (4m) 83s ago 14m 14.8M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (66s) 24s ago 13m 87.9M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (30s) 24s ago 12m 12.6M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (12m) 24s ago 12m 201M 1216M 18.2.0 dc2bc1663786 3fb7d8d5fee4 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (11m) 83s ago 11m 271M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (11m) 83s ago 11m 236M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (10m) 83s ago 10m 241M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:49:24.782 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (4m) 24s ago 14m 69.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feaf806e200 msgr2=0x7feaf80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feaf806e200 0x7feaf80706b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7feafc005fd0 tx=0x7feafc005e20 comp rx=0 tx=0).stop 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c104130 msgr2=0x7feb0c101370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c104130 0x7feb0c101370 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feb08006010 tx=0x7feb0800be20 comp rx=0 tx=0).stop 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 shutdown_connections 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feaf806e200 0x7feaf80706b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7feb0c0ffbd0 0x7feb0c100e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 --2- 172.21.15.90:0/2709091298 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7feb0c104130 0x7feb0c101370 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 >> 172.21.15.90:0/2709091298 conn(0x7feb0c075130 msgr2=0x7feb0c0febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 shutdown_connections 2024-04-26T04:49:24.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.781+0000 7feb143df700 1 -- 172.21.15.90:0/2709091298 wait complete. 2024-04-26T04:49:24.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1262128931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 msgr2=0x7fd6d0104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1262128931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d0104500 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd6c0009910 tx=0x7fd6c0009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:24.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1262128931 shutdown_connections 2024-04-26T04:49:24.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1262128931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d00ffbd0 0x7fd6d0100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1262128931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d0104500 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.898 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1262128931 >> 172.21.15.90:0/1262128931 conn(0x7fd6d0075130 msgr2=0x7fd6d0075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1262128931 shutdown_connections 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1262128931 wait complete. 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.895+0000 7fd6d5c8f700 1 Processor -- start 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6d5c8f700 1 -- start start 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6d5c8f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d00ffbd0 0x7fd6d0196cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6d5c8f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d01971f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6d5c8f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0197810 con 0x7fd6d0104130 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6d5c8f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d0197950 con 0x7fd6d00ffbd0 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6ceffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d01971f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6ceffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d01971f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48254/0 (socket says 172.21.15.90:48254) 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6ceffd700 1 -- 172.21.15.90:0/1630970807 learned_addr learned my addr 172.21.15.90:0/1630970807 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6cf7fe700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d00ffbd0 0x7fd6d0196cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6ceffd700 1 -- 172.21.15.90:0/1630970807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d00ffbd0 msgr2=0x7fd6d0196cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6ceffd700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d00ffbd0 0x7fd6d0196cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.896+0000 7fd6ceffd700 1 -- 172.21.15.90:0/1630970807 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c00095f0 con 0x7fd6d0104130 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.897+0000 7fd6ceffd700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d01971f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400c8a0 tx=0x7fd6c400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.897+0000 7fd6ccff9700 1 -- 172.21.15.90:0/1630970807 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c400cea0 con 0x7fd6d0104130 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.897+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6d019fd20 con 0x7fd6d0104130 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.897+0000 7fd6ccff9700 1 -- 172.21.15.90:0/1630970807 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd6c4004830 con 0x7fd6d0104130 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.897+0000 7fd6ccff9700 1 -- 172.21.15.90:0/1630970807 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd6c40055b0 con 0x7fd6d0104130 2024-04-26T04:49:24.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.897+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6d01a01e0 con 0x7fd6d0104130 2024-04-26T04:49:24.901 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:24 smithi090 ceph-mon[86550]: pgmap v87: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:24.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.898+0000 7fd6ccff9700 1 -- 172.21.15.90:0/1630970807 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd6c40049a0 con 0x7fd6d0104130 2024-04-26T04:49:24.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.899+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6d01076a0 con 0x7fd6d0104130 2024-04-26T04:49:24.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.899+0000 7fd6ccff9700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd6b806e200 0x7fd6b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:24.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.899+0000 7fd6ccff9700 1 -- 172.21.15.90:0/1630970807 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd6c408dc90 con 0x7fd6d0104130 2024-04-26T04:49:24.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.899+0000 7fd6cf7fe700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd6b806e200 0x7fd6b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:24.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.899+0000 7fd6cf7fe700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd6b806e200 0x7fd6b80706b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b5c0 tx=0x7fd6c001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:24.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:24.908+0000 7fd6ccff9700 1 -- 172.21.15.90:0/1630970807 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd6c40570a0 con 0x7fd6d0104130 2024-04-26T04:49:25.221 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:25 smithi090 systemd[1]: Stopping Ceph osd.2 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:49:25.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.249+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd6d004f2a0 con 0x7fd6d0104130 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.249+0000 7fd6ccff9700 1 -- 172.21.15.90:0/1630970807 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fd6c401d070 con 0x7fd6d0104130 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:49:25.251 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4, 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 8, 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:49:25.252 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd6b806e200 msgr2=0x7fd6b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd6b806e200 0x7fd6b80706b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd6c000b5c0 tx=0x7fd6c001a040 comp rx=0 tx=0).stop 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 msgr2=0x7fd6d01971f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d01971f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c400c8a0 tx=0x7fd6c400cc60 comp rx=0 tx=0).stop 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 shutdown_connections 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd6b806e200 0x7fd6b80706b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd6d00ffbd0 0x7fd6d0196cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 --2- 172.21.15.90:0/1630970807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd6d0104130 0x7fd6d01971f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.251+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 >> 172.21.15.90:0/1630970807 conn(0x7fd6d0075130 msgr2=0x7fd6d00fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:25.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.252+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 shutdown_connections 2024-04-26T04:49:25.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.252+0000 7fd6d5c8f700 1 -- 172.21.15.90:0/1630970807 wait complete. 2024-04-26T04:49:25.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:24 smithi104 ceph-mon[64778]: pgmap v87: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:25.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 -- 172.21.15.90:0/2170590698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 msgr2=0x7f313c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 --2- 172.21.15.90:0/2170590698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c104e50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f3130009910 tx=0x7f3130009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 -- 172.21.15.90:0/2170590698 shutdown_connections 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 --2- 172.21.15.90:0/2170590698 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c104e50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 --2- 172.21.15.90:0/2170590698 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f313c1004e0 0x7f313c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 -- 172.21.15.90:0/2170590698 >> 172.21.15.90:0/2170590698 conn(0x7f313c0fbda0 msgr2=0x7f313c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 -- 172.21.15.90:0/2170590698 shutdown_connections 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.356+0000 7f3141860700 1 -- 172.21.15.90:0/2170590698 wait complete. 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f3141860700 1 Processor -- start 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f3141860700 1 -- start start 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f3141860700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f313c1004e0 0x7f313c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f3141860700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f3141860700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c196b30 con 0x7f313c100e80 2024-04-26T04:49:25.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f3141860700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c19a8c0 con 0x7f313c1004e0 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f313a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f313a7fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48274/0 (socket says 172.21.15.90:48274) 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.357+0000 7f313a7fc700 1 -- 172.21.15.90:0/343122458 learned_addr learned my addr 172.21.15.90:0/343122458 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f313affd700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f313c1004e0 0x7f313c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f313a7fc700 1 -- 172.21.15.90:0/343122458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f313c1004e0 msgr2=0x7f313c195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f313a7fc700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f313c1004e0 0x7f313c195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f313a7fc700 1 -- 172.21.15.90:0/343122458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31300095f0 con 0x7f313c100e80 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f313a7fc700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c196450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3130005f50 tx=0x7f313000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f314085e700 1 -- 172.21.15.90:0/343122458 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f313001d070 con 0x7f313c100e80 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c19ab40 con 0x7f313c100e80 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f314085e700 1 -- 172.21.15.90:0/343122458 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f313000f460 con 0x7f313c100e80 2024-04-26T04:49:25.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f314085e700 1 -- 172.21.15.90:0/343122458 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3130017630 con 0x7f313c100e80 2024-04-26T04:49:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.358+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c19b030 con 0x7f313c100e80 2024-04-26T04:49:25.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.359+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c108850 con 0x7f313c100e80 2024-04-26T04:49:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.360+0000 7f314085e700 1 -- 172.21.15.90:0/343122458 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f313000f5d0 con 0x7f313c100e80 2024-04-26T04:49:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.360+0000 7f314085e700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f312406e200 0x7f31240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.360+0000 7f314085e700 1 -- 172.21.15.90:0/343122458 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f313008f660 con 0x7f313c100e80 2024-04-26T04:49:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.360+0000 7f313affd700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f312406e200 0x7f31240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:25.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.360+0000 7f313affd700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f312406e200 0x7f31240706b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f312c005950 tx=0x7f312c00a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:25.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.365+0000 7f314085e700 1 -- 172.21.15.90:0/343122458 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f31300589f0 con 0x7f313c100e80 2024-04-26T04:49:25.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.610+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f313c197270 con 0x7f313c100e80 2024-04-26T04:49:25.612 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[52647]: 2024-04-26T04:49:25.318+0000 7f369905a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:49:25.612 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[52647]: 2024-04-26T04:49:25.318+0000 7f369905a700 -1 osd.2 60 *** Got signal Terminated *** 2024-04-26T04:49:25.613 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[52647]: 2024-04-26T04:49:25.318+0000 7f369905a700 -1 osd.2 60 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:49:25.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.610+0000 7f314085e700 1 -- 172.21.15.90:0/343122458 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f313005c0b0 con 0x7f313c100e80 2024-04-26T04:49:25.613 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:49:25.613 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:49:25.613 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:49:25.613 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:49:25.613 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:49:25.614 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:49:25.615 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:25.616 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:25.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f312406e200 msgr2=0x7f31240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f312406e200 0x7f31240706b0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f312c005950 tx=0x7f312c00a300 comp rx=0 tx=0).stop 2024-04-26T04:49:25.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 msgr2=0x7f313c196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c196450 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f3130005f50 tx=0x7f313000bd70 comp rx=0 tx=0).stop 2024-04-26T04:49:25.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 shutdown_connections 2024-04-26T04:49:25.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f312406e200 0x7f31240706b0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f313c1004e0 0x7f313c195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 --2- 172.21.15.90:0/343122458 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f313c100e80 0x7f313c196450 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 >> 172.21.15.90:0/343122458 conn(0x7f313c0fbda0 msgr2=0x7f313c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:25.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 shutdown_connections 2024-04-26T04:49:25.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.613+0000 7f3141860700 1 -- 172.21.15.90:0/343122458 wait complete. 2024-04-26T04:49:25.618 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 -- 172.21.15.90:0/409807661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 msgr2=0x7ff46c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 --2- 172.21.15.90:0/409807661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c103c80 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff46400b0a0 tx=0x7ff46400b3b0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 -- 172.21.15.90:0/409807661 shutdown_connections 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 --2- 172.21.15.90:0/409807661 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff46c0ff350 0x7ff46c0ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 --2- 172.21.15.90:0/409807661 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c103c80 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 -- 172.21.15.90:0/409807661 >> 172.21.15.90:0/409807661 conn(0x7ff46c0748b0 msgr2=0x7ff46c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 -- 172.21.15.90:0/409807661 shutdown_connections 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 -- 172.21.15.90:0/409807661 wait complete. 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 Processor -- start 2024-04-26T04:49:25.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.720+0000 7ff474503700 1 -- start start 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff474503700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff46c0ff350 0x7ff46c199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff474503700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff474503700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c193a60 con 0x7ff46c1038b0 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff474503700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff46c193bd0 con 0x7ff46c0ff350 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff471a9e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff471a9e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c199fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48298/0 (socket says 172.21.15.90:48298) 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff471a9e700 1 -- 172.21.15.90:0/2564666753 learned_addr learned my addr 172.21.15.90:0/2564666753 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff47229f700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff46c0ff350 0x7ff46c199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff471a9e700 1 -- 172.21.15.90:0/2564666753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff46c0ff350 msgr2=0x7ff46c199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff471a9e700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff46c0ff350 0x7ff46c199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff471a9e700 1 -- 172.21.15.90:0/2564666753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff464009d00 con 0x7ff46c1038b0 2024-04-26T04:49:25.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff471a9e700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c199fb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff46000ed80 tx=0x7ff46000c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff46b7fe700 1 -- 172.21.15.90:0/2564666753 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff46000ce70 con 0x7ff46c1038b0 2024-04-26T04:49:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff46b7fe700 1 -- 172.21.15.90:0/2564666753 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff4600044e0 con 0x7ff46c1038b0 2024-04-26T04:49:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff46c193eb0 con 0x7ff46c1038b0 2024-04-26T04:49:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff46b7fe700 1 -- 172.21.15.90:0/2564666753 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff460005410 con 0x7ff46c1038b0 2024-04-26T04:49:25.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.721+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff46c194400 con 0x7ff46c1038b0 2024-04-26T04:49:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.722+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff46c106e20 con 0x7ff46c1038b0 2024-04-26T04:49:25.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.723+0000 7ff46b7fe700 1 -- 172.21.15.90:0/2564666753 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff460005570 con 0x7ff46c1038b0 2024-04-26T04:49:25.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.723+0000 7ff46b7fe700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff454070410 0x7ff4540728c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:25.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.723+0000 7ff46b7fe700 1 -- 172.21.15.90:0/2564666753 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff46008dfe0 con 0x7ff46c1038b0 2024-04-26T04:49:25.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.723+0000 7ff47229f700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff454070410 0x7ff4540728c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:25.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.724+0000 7ff47229f700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff454070410 0x7ff4540728c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff46400b830 tx=0x7ff4640061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:25.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.728+0000 7ff46b7fe700 1 -- 172.21.15.90:0/2564666753 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff4600573f0 con 0x7ff46c1038b0 2024-04-26T04:49:25.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-mon[86550]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:25.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-mon[86550]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:25.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-mon[86550]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:25.960 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1630970807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:25.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-mon[86550]: osd.2 marked itself down and dead 2024-04-26T04:49:25.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/343122458' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:49:25.961 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.958+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff46c10b230 con 0x7ff454070410 2024-04-26T04:49:25.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.959+0000 7ff46b7fe700 1 -- 172.21.15.90:0/2564666753 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff46c10b230 con 0x7ff454070410 2024-04-26T04:49:25.962 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:49:25.962 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:49:25.963 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:49:25.963 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:49:25.963 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:49:25.963 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:49:25.963 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:49:25.963 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:49:25.964 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:49:25.964 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "8/23 daemons upgraded", 2024-04-26T04:49:25.964 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:49:25.964 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:49:25.964 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:49:25.966 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff454070410 msgr2=0x7ff4540728c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff454070410 0x7ff4540728c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff46400b830 tx=0x7ff4640061f0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 msgr2=0x7ff46c199fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:25.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c199fb0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff46000ed80 tx=0x7ff46000c650 comp rx=0 tx=0).stop 2024-04-26T04:49:25.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 shutdown_connections 2024-04-26T04:49:25.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff454070410 0x7ff4540728c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.967 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff46c0ff350 0x7ff46c199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 --2- 172.21.15.90:0/2564666753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff46c1038b0 0x7ff46c199fb0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:25.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 >> 172.21.15.90:0/2564666753 conn(0x7ff46c0748b0 msgr2=0x7ff46c0fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:25.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 shutdown_connections 2024-04-26T04:49:25.968 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:25.963+0000 7ff474503700 1 -- 172.21.15.90:0/2564666753 wait complete. 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 -- 172.21.15.90:0/1813971931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101d30 msgr2=0x7f44b010a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 --2- 172.21.15.90:0/1813971931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101d30 0x7f44b010a230 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009910 tx=0x7f44a0009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 -- 172.21.15.90:0/1813971931 shutdown_connections 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 --2- 172.21.15.90:0/1813971931 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101d30 0x7f44b010a230 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 --2- 172.21.15.90:0/1813971931 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44b0101420 0x7f44b01017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 -- 172.21.15.90:0/1813971931 >> 172.21.15.90:0/1813971931 conn(0x7f44b00fac80 msgr2=0x7f44b00fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 -- 172.21.15.90:0/1813971931 shutdown_connections 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 -- 172.21.15.90:0/1813971931 wait complete. 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 Processor -- start 2024-04-26T04:49:26.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.067+0000 7f44b7494700 1 -- start start 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b7494700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101420 0x7f44b0195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b7494700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44b0101d30 0x7f44b0196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b7494700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b0196c10 con 0x7f44b0101420 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b7494700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44b019a9a0 con 0x7f44b0101d30 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b5230700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101420 0x7f44b0195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b5230700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101420 0x7f44b0195ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:48326/0 (socket says 172.21.15.90:48326) 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b5230700 1 -- 172.21.15.90:0/1760825708 learned_addr learned my addr 172.21.15.90:0/1760825708 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b4a2f700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44b0101d30 0x7f44b0196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b5230700 1 -- 172.21.15.90:0/1760825708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44b0101d30 msgr2=0x7f44b0196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b5230700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44b0101d30 0x7f44b0196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b5230700 1 -- 172.21.15.90:0/1760825708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a00095f0 con 0x7f44b0101420 2024-04-26T04:49:26.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b5230700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101420 0x7f44b0195ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00b4b0 tx=0x7f44ac00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:26.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44a67fc700 1 -- 172.21.15.90:0/1760825708 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44ac011800 con 0x7f44b0101420 2024-04-26T04:49:26.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44b019ac80 con 0x7f44b0101420 2024-04-26T04:49:26.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44a67fc700 1 -- 172.21.15.90:0/1760825708 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f44ac011e20 con 0x7f44b0101420 2024-04-26T04:49:26.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44a67fc700 1 -- 172.21.15.90:0/1760825708 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f44ac00f2e0 con 0x7f44b0101420 2024-04-26T04:49:26.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.068+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44b019b1d0 con 0x7f44b0101420 2024-04-26T04:49:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.069+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44b01078a0 con 0x7f44b0101420 2024-04-26T04:49:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.070+0000 7f44a67fc700 1 -- 172.21.15.90:0/1760825708 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f44ac010380 con 0x7f44b0101420 2024-04-26T04:49:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.070+0000 7f44a67fc700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f449c06e200 0x7f449c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.071+0000 7f44b4a2f700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f449c06e200 0x7f449c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.071+0000 7f44a67fc700 1 -- 172.21.15.90:0/1760825708 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f44ac08dbf0 con 0x7f44b0101420 2024-04-26T04:49:26.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.071+0000 7f44b4a2f700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f449c06e200 0x7f449c0706b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44a00059d0 tx=0x7f44a001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:26.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.075+0000 7f44a67fc700 1 -- 172.21.15.90:0/1760825708 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f44ac057000 con 0x7f44b0101420 2024-04-26T04:49:26.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:25 smithi104 ceph-mon[64778]: from='client.34252 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:26.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:25 smithi104 ceph-mon[64778]: from='client.34256 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:26.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:25 smithi104 ceph-mon[64778]: from='client.34260 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:26.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:25 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1630970807' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:25 smithi104 ceph-mon[64778]: osd.2 marked itself down and dead 2024-04-26T04:49:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:25 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/343122458' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:49:26.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:26.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.377+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f44b004f2a0 con 0x7f44b0101420 2024-04-26T04:49:26.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.378+0000 7f44a67fc700 1 -- 172.21.15.90:0/1760825708 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+100 (secure 0 0 0) 0x7f44ac015080 con 0x7f44b0101420 2024-04-26T04:49:26.379 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 osds down 2024-04-26T04:49:26.379 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] OSD_DOWN: 1 osds down 2024-04-26T04:49:26.379 INFO:teuthology.orchestra.run.smithi090.stdout: osd.2 (root=default,host=smithi090) is down 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f449c06e200 msgr2=0x7f449c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f449c06e200 0x7f449c0706b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f44a00059d0 tx=0x7f44a001a040 comp rx=0 tx=0).stop 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101420 msgr2=0x7f44b0195ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101420 0x7f44b0195ff0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f44ac00b4b0 tx=0x7f44ac00b870 comp rx=0 tx=0).stop 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 shutdown_connections 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f449c06e200 0x7f449c0706b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f44b0101420 0x7f44b0195ff0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 --2- 172.21.15.90:0/1760825708 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f44b0101d30 0x7f44b0196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:26.382 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 >> 172.21.15.90:0/1760825708 conn(0x7f44b00fac80 msgr2=0x7f44b01049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 shutdown_connections 2024-04-26T04:49:26.383 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:26.380+0000 7f44b7494700 1 -- 172.21.15.90:0/1760825708 wait complete. 2024-04-26T04:49:26.599 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:26 smithi090 podman[108895]: 2024-04-26 04:49:26.325279074 +0000 UTC m=+1.132733024 container died 3fb7d8d5fee410eff60ddcc82402bdaefe0657b9fd877ce9065ac9a06779f789 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, io.buildah.version=1.29.1, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, GIT_REPO=https://github.com/ceph/ceph-container.git, CEPH_POINT_RELEASE=-18.2.0, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD) 2024-04-26T04:49:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:26 smithi090 ceph-mon[86550]: pgmap v88: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:26 smithi090 ceph-mon[86550]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:49:26.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:26 smithi090 ceph-mon[86550]: osdmap e61: 6 total, 5 up, 6 in 2024-04-26T04:49:26.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:26 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1760825708' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:49:26.972 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:26 smithi090 podman[108895]: 2024-04-26 04:49:26.714061138 +0000 UTC m=+1.521515078 container cleanup 3fb7d8d5fee410eff60ddcc82402bdaefe0657b9fd877ce9065ac9a06779f789 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=HEAD, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, io.buildah.version=1.29.1) 2024-04-26T04:49:26.972 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:26 smithi090 bash[108895]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2 2024-04-26T04:49:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:26 smithi104 ceph-mon[64778]: pgmap v88: 65 pgs: 65 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:26 smithi104 ceph-mon[64778]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:49:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:26 smithi104 ceph-mon[64778]: osdmap e61: 6 total, 5 up, 6 in 2024-04-26T04:49:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:26 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1760825708' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:49:27.287 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:26 smithi090 podman[108981]: 2024-04-26 04:49:26.974221152 +0000 UTC m=+0.649075482 container remove 3fb7d8d5fee410eff60ddcc82402bdaefe0657b9fd877ce9065ac9a06779f789 (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.29.1) 2024-04-26T04:49:27.604 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:27 smithi090 podman[109079]: 2024-04-26 04:49:27.189983271 +0000 UTC m=+0.018245326 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:49:27.604 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:27 smithi090 podman[109079]: 2024-04-26 04:49:27.300346119 +0000 UTC m=+0.128608159 container create 203cd7cc7cd7920402e8d7a5da2fd257a00a2ea52a516e69c97af47914faf8f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS) 2024-04-26T04:49:27.604 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:27 smithi090 podman[109079]: 2024-04-26 04:49:27.511876925 +0000 UTC m=+0.340138969 container init 203cd7cc7cd7920402e8d7a5da2fd257a00a2ea52a516e69c97af47914faf8f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:49:27.604 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:27 smithi090 podman[109079]: 2024-04-26 04:49:27.515103751 +0000 UTC m=+0.343365805 container start 203cd7cc7cd7920402e8d7a5da2fd257a00a2ea52a516e69c97af47914faf8f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:49:27.604 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:27 smithi090 podman[109079]: 2024-04-26 04:49:27.603693548 +0000 UTC m=+0.431955597 container attach 203cd7cc7cd7920402e8d7a5da2fd257a00a2ea52a516e69c97af47914faf8f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T04:49:27.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:27 smithi090 ceph-mon[86550]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:27.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:27 smithi090 ceph-mon[86550]: osdmap e62: 6 total, 5 up, 6 in 2024-04-26T04:49:27.972 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:27 smithi090 podman[109079]: 2024-04-26 04:49:27.808030991 +0000 UTC m=+0.636293038 container died 203cd7cc7cd7920402e8d7a5da2fd257a00a2ea52a516e69c97af47914faf8f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T04:49:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:27 smithi104 ceph-mon[64778]: from='client.34272 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:28.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:27 smithi104 ceph-mon[64778]: osdmap e62: 6 total, 5 up, 6 in 2024-04-26T04:49:28.721 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:28 smithi090 podman[109105]: 2024-04-26 04:49:28.359613824 +0000 UTC m=+0.546098618 container remove 203cd7cc7cd7920402e8d7a5da2fd257a00a2ea52a516e69c97af47914faf8f5 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424) 2024-04-26T04:49:28.722 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:28 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.2.service: Deactivated successfully. 2024-04-26T04:49:28.722 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:28 smithi090 systemd[1]: Stopped Ceph osd.2 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:49:28.722 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:28 smithi090 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.2.service: Consumed 50.436s CPU time. 2024-04-26T04:49:28.722 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:28 smithi090 systemd[1]: Starting Ceph osd.2 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:49:29.094 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:28 smithi090 ceph-mon[86550]: pgmap v91: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:29.094 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:28 smithi090 ceph-mon[86550]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-04-26T04:49:29.094 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:28 smithi090 podman[109211]: 2024-04-26 04:49:28.708102102 +0000 UTC m=+0.018136968 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:49:29.095 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:28 smithi090 podman[109211]: 2024-04-26 04:49:28.819296112 +0000 UTC m=+0.129330960 container create 8dae1f90aa827dc6c1eedb523c8c1c2e9d6bdbb2078ad9bcfcc38c1d0a4e4040 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:49:29.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:28 smithi104 ceph-mon[64778]: pgmap v91: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:29.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:28 smithi104 ceph-mon[64778]: Health check failed: Reduced data availability: 6 pgs peering (PG_AVAILABILITY) 2024-04-26T04:49:29.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 podman[109211]: 2024-04-26 04:49:29.094032265 +0000 UTC m=+0.404067115 container init 8dae1f90aa827dc6c1eedb523c8c1c2e9d6bdbb2078ad9bcfcc38c1d0a4e4040 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate, RELEASE=squid-b22e2eb, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-26T04:49:29.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 podman[109211]: 2024-04-26 04:49:29.097747847 +0000 UTC m=+0.407782703 container start 8dae1f90aa827dc6c1eedb523c8c1c2e9d6bdbb2078ad9bcfcc38c1d0a4e4040 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T04:49:29.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 podman[109211]: 2024-04-26 04:49:29.199111608 +0000 UTC m=+0.509146473 container attach 8dae1f90aa827dc6c1eedb523c8c1c2e9d6bdbb2078ad9bcfcc38c1d0a4e4040 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-26T04:49:29.845 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate[109226]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:49:29.845 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate[109226]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:49:29.845 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 bash[109211]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 bash[109211]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-2 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 bash[109211]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 bash[109211]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 bash[109211]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 bash[109211]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 bash[109211]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate[109226]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate[109226]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate[109226]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-2/block 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate[109226]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate[109226]: --> ceph-volume raw activate successful for osd ID: 2 2024-04-26T04:49:29.846 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:29 smithi090 podman[109211]: 2024-04-26 04:49:29.560590467 +0000 UTC m=+0.870625325 container died 8dae1f90aa827dc6c1eedb523c8c1c2e9d6bdbb2078ad9bcfcc38c1d0a4e4040 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T04:49:30.441 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 podman[109350]: 2024-04-26 04:49:30.112008243 +0000 UTC m=+0.546183764 container remove 8dae1f90aa827dc6c1eedb523c8c1c2e9d6bdbb2078ad9bcfcc38c1d0a4e4040 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T04:49:30.441 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 podman[109415]: 2024-04-26 04:49:30.228899658 +0000 UTC m=+0.018009636 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:49:30.441 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 podman[109415]: 2024-04-26 04:49:30.329737141 +0000 UTC m=+0.118847149 container create a6f8b02d7e7d2e7ba9e618ee9bbbb3180e71e330531de8e3eb0356a2ada74883 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424) 2024-04-26T04:49:30.836 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 podman[109415]: 2024-04-26 04:49:30.583799628 +0000 UTC m=+0.372909592 container init a6f8b02d7e7d2e7ba9e618ee9bbbb3180e71e330531de8e3eb0356a2ada74883 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T04:49:30.836 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 podman[109415]: 2024-04-26 04:49:30.587439999 +0000 UTC m=+0.376549971 container start a6f8b02d7e7d2e7ba9e618ee9bbbb3180e71e330531de8e3eb0356a2ada74883 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:49:30.836 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 ceph-osd[109434]: -- 172.21.15.90:0/939725942 <== mon.0 v2:172.21.15.90:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a533187680 con 0x55a533150c00 2024-04-26T04:49:30.836 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 bash[109415]: a6f8b02d7e7d2e7ba9e618ee9bbbb3180e71e330531de8e3eb0356a2ada74883 2024-04-26T04:49:30.837 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:30 smithi090 systemd[1]: Started Ceph osd.2 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:49:31.132 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:30 smithi090 ceph-mon[86550]: pgmap v92: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:31.132 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:31.132 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:31.132 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:31.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:30 smithi104 ceph-mon[64778]: pgmap v92: 65 pgs: 19 peering, 3 stale+active+clean, 43 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:31.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:31.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:31.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:31.450 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:31 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[109430]: 2024-04-26T04:49:31.130+0000 7fb0024bf7c0 -1 Falling back to public interface 2024-04-26T04:49:32.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:31 smithi090 ceph-mon[86550]: pgmap v93: 65 pgs: 7 active+undersized+degraded, 12 peering, 8 active+undersized, 38 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/348 objects degraded (6.897%) 2024-04-26T04:49:32.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:31 smithi090 ceph-mon[86550]: Health check failed: Degraded data redundancy: 24/348 objects degraded (6.897%), 7 pgs degraded (PG_DEGRADED) 2024-04-26T04:49:32.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:31 smithi104 ceph-mon[64778]: pgmap v93: 65 pgs: 7 active+undersized+degraded, 12 peering, 8 active+undersized, 38 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 24/348 objects degraded (6.897%) 2024-04-26T04:49:32.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:31 smithi104 ceph-mon[64778]: Health check failed: Degraded data redundancy: 24/348 objects degraded (6.897%), 7 pgs degraded (PG_DEGRADED) 2024-04-26T04:49:34.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:33 smithi090 ceph-mon[86550]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:49:34.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:33 smithi104 ceph-mon[64778]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:49:35.060 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:34 smithi090 ceph-mon[86550]: pgmap v94: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:35.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:34 smithi104 ceph-mon[64778]: pgmap v94: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:35.453 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:35 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[109430]: 2024-04-26T04:49:35.057+0000 7fb0024bf7c0 -1 osd.2 0 read_superblock omap replica is missing. 2024-04-26T04:49:35.453 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:35 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[109430]: 2024-04-26T04:49:35.451+0000 7fb0024bf7c0 -1 osd.2 60 log_to_monitors true 2024-04-26T04:49:36.082 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:35 smithi104 ceph-mon[64778]: from='osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:49:36.082 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:35 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:36.082 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:35 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:36.221 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:49:35 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[109430]: 2024-04-26T04:49:35.803+0000 7faff3917700 -1 osd.2 60 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:49:36.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:35 smithi090 ceph-mon[86550]: from='osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-26T04:49:36.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:35 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:36.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:35 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:37.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:36 smithi090 ceph-mon[86550]: pgmap v95: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:37.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:36 smithi090 ceph-mon[86550]: from='osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:49:37.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:36 smithi090 ceph-mon[86550]: osdmap e63: 6 total, 5 up, 6 in 2024-04-26T04:49:37.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:36 smithi090 ceph-mon[86550]: from='osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:49:37.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:36 smithi104 ceph-mon[64778]: pgmap v95: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.4 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:37.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:36 smithi104 ceph-mon[64778]: from='osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-26T04:49:37.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:36 smithi104 ceph-mon[64778]: osdmap e63: 6 total, 5 up, 6 in 2024-04-26T04:49:37.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:36 smithi104 ceph-mon[64778]: from='osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi090", "root=default"]}]: dispatch 2024-04-26T04:49:38.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:37 smithi104 ceph-mon[64778]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:49:38.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:37 smithi104 ceph-mon[64778]: osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709] boot 2024-04-26T04:49:38.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:37 smithi104 ceph-mon[64778]: osdmap e64: 6 total, 6 up, 6 in 2024-04-26T04:49:38.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:37 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:49:38.163 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:37 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:38.164 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:37 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:38.164 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:37 smithi104 ceph-mon[64778]: osdmap e65: 6 total, 6 up, 6 in 2024-04-26T04:49:38.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:37 smithi090 ceph-mon[86550]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:49:38.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:37 smithi090 ceph-mon[86550]: osd.2 [v2:172.21.15.90:6818/1256593709,v1:172.21.15.90:6819/1256593709] boot 2024-04-26T04:49:38.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:37 smithi090 ceph-mon[86550]: osdmap e64: 6 total, 6 up, 6 in 2024-04-26T04:49:38.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:37 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-26T04:49:38.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:37 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:38.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:37 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:38.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:37 smithi090 ceph-mon[86550]: osdmap e65: 6 total, 6 up, 6 in 2024-04-26T04:49:39.098 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:38 smithi090 ceph-mon[86550]: pgmap v98: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:39.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:38 smithi104 ceph-mon[64778]: pgmap v98: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:41.131 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: pgmap v100: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:41.131 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.131 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.131 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:41.131 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:41.131 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:41.132 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:40 smithi104 ceph-mon[64778]: Health check update: Degraded data redundancy: 41/348 objects degraded (11.782%), 13 pgs degraded (PG_DEGRADED) 2024-04-26T04:49:41.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: pgmap v100: 65 pgs: 13 active+undersized+degraded, 14 active+undersized, 38 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 41/348 objects degraded (11.782%) 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:41.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:41.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:40 smithi090 ceph-mon[86550]: Health check update: Degraded data redundancy: 41/348 objects degraded (11.782%), 13 pgs degraded (PG_DEGRADED) 2024-04-26T04:49:41.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:41 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:41.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:41 smithi090 ceph-mon[86550]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-26T04:49:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:41 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:41 smithi104 ceph-mon[64778]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-26T04:49:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:42 smithi090 ceph-mon[86550]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 16/348 objects degraded (4.598%) 2024-04-26T04:49:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:42 smithi104 ceph-mon[64778]: pgmap v101: 65 pgs: 5 active+undersized+degraded, 5 active+undersized, 55 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 16/348 objects degraded (4.598%) 2024-04-26T04:49:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:43 smithi090 ceph-mon[86550]: pgmap v102: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:43 smithi090 ceph-mon[86550]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/348 objects degraded (4.598%), 5 pgs degraded) 2024-04-26T04:49:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:43 smithi090 ceph-mon[86550]: Cluster is now healthy 2024-04-26T04:49:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:43 smithi104 ceph-mon[64778]: pgmap v102: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:43 smithi104 ceph-mon[64778]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 16/348 objects degraded (4.598%), 5 pgs degraded) 2024-04-26T04:49:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:43 smithi104 ceph-mon[64778]: Cluster is now healthy 2024-04-26T04:49:46.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:46 smithi090 ceph-mon[86550]: pgmap v103: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:46 smithi104 ceph-mon[64778]: pgmap v103: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:49.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:48 smithi104 ceph-mon[64778]: pgmap v104: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:49.150 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:48 smithi090 ceph-mon[86550]: pgmap v104: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:50 smithi104 ceph-mon[64778]: pgmap v105: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:50 smithi090 ceph-mon[86550]: pgmap v105: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:51 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:51 smithi104 ceph-mon[64778]: pgmap v106: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:52.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:51 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:52.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:51 smithi090 ceph-mon[86550]: pgmap v106: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:55.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:54 smithi104 ceph-mon[64778]: pgmap v107: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:54 smithi090 ceph-mon[86550]: pgmap v107: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:56.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:56.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:56.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:56.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:49:56.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 -- 172.21.15.90:0/3706567441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ffc60 msgr2=0x7f483010a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:56.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 --2- 172.21.15.90:0/3706567441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ffc60 0x7f483010a6d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f4820009910 tx=0x7f4820009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:56.500 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 -- 172.21.15.90:0/3706567441 shutdown_connections 2024-04-26T04:49:56.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 --2- 172.21.15.90:0/3706567441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ffc60 0x7f483010a6d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 --2- 172.21.15.90:0/3706567441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48300ff350 0x7f48300ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 -- 172.21.15.90:0/3706567441 >> 172.21.15.90:0/3706567441 conn(0x7f48300748b0 msgr2=0x7f4830074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:56.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 -- 172.21.15.90:0/3706567441 shutdown_connections 2024-04-26T04:49:56.501 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.499+0000 7f4834fdc700 1 -- 172.21.15.90:0/3706567441 wait complete. 2024-04-26T04:49:56.502 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f4834fdc700 1 Processor -- start 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f4834fdc700 1 -- start start 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f4834fdc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ff350 0x7f4830197f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f4834fdc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48300ffc60 0x7f48301984d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f4834fdc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4830198b20 con 0x7f48300ff350 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f4834fdc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4830198c60 con 0x7f48300ffc60 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f482e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ff350 0x7f4830197f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f482e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ff350 0x7f4830197f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47322/0 (socket says 172.21.15.90:47322) 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f482e59c700 1 -- 172.21.15.90:0/2349546557 learned_addr learned my addr 172.21.15.90:0/2349546557 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f482dd9b700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48300ffc60 0x7f48301984d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f482e59c700 1 -- 172.21.15.90:0/2349546557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48300ffc60 msgr2=0x7f48301984d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f482e59c700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48300ffc60 0x7f48301984d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.500+0000 7f482e59c700 1 -- 172.21.15.90:0/2349546557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48200095f0 con 0x7f48300ff350 2024-04-26T04:49:56.503 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.501+0000 7f482e59c700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ff350 0x7f4830197f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f482400c930 tx=0x7f482400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:56.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.501+0000 7f481f7fe700 1 -- 172.21.15.90:0/2349546557 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4824007ad0 con 0x7f48300ff350 2024-04-26T04:49:56.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.501+0000 7f481f7fe700 1 -- 172.21.15.90:0/2349546557 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f482400ce80 con 0x7f48300ff350 2024-04-26T04:49:56.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.501+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483019cab0 con 0x7f48300ff350 2024-04-26T04:49:56.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.501+0000 7f481f7fe700 1 -- 172.21.15.90:0/2349546557 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4824018630 con 0x7f48300ff350 2024-04-26T04:49:56.504 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.501+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483019d000 con 0x7f48300ff350 2024-04-26T04:49:56.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.502+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483010a230 con 0x7f48300ff350 2024-04-26T04:49:56.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.502+0000 7f481f7fe700 1 -- 172.21.15.90:0/2349546557 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4824007c30 con 0x7f48300ff350 2024-04-26T04:49:56.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.503+0000 7f481f7fe700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f481806e200 0x7f48180706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:56.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.503+0000 7f481f7fe700 1 -- 172.21.15.90:0/2349546557 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f482408e060 con 0x7f48300ff350 2024-04-26T04:49:56.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.503+0000 7f482dd9b700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f481806e200 0x7f48180706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:56.505 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.503+0000 7f482dd9b700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f481806e200 0x7f48180706b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48301995b0 tx=0x7f482001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:56.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.508+0000 7f481f7fe700 1 -- 172.21.15.90:0/2349546557 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f48240573f0 con 0x7f48300ff350 2024-04-26T04:49:56.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.727+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f483019d280 con 0x7f481806e200 2024-04-26T04:49:56.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.728+0000 7f481f7fe700 1 -- 172.21.15.90:0/2349546557 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f483019d280 con 0x7f481806e200 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f481806e200 msgr2=0x7f48180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f481806e200 0x7f48180706b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f48301995b0 tx=0x7f482001a040 comp rx=0 tx=0).stop 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ff350 msgr2=0x7f4830197f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ff350 0x7f4830197f90 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f482400c930 tx=0x7f482400cc40 comp rx=0 tx=0).stop 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 shutdown_connections 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f481806e200 0x7f48180706b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48300ff350 0x7f4830197f90 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 --2- 172.21.15.90:0/2349546557 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48300ffc60 0x7f48301984d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 >> 172.21.15.90:0/2349546557 conn(0x7f48300748b0 msgr2=0x7f4830100cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 shutdown_connections 2024-04-26T04:49:56.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.730+0000 7f4834fdc700 1 -- 172.21.15.90:0/2349546557 wait complete. 2024-04-26T04:49:56.744 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:49:56.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 -- 172.21.15.90:0/2848579753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 msgr2=0x7ff0e40ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:56.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/2848579753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e40ff720 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff0d8009910 tx=0x7ff0d8009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:56.841 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:56 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:56.841 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:56 smithi090 ceph-mon[86550]: Upgrade: osd.3 is safe to restart 2024-04-26T04:49:56.841 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:56 smithi090 ceph-mon[86550]: pgmap v108: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 -- 172.21.15.90:0/2848579753 shutdown_connections 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/2848579753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e40ffc60 0x7ff0e410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/2848579753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e40ff720 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 -- 172.21.15.90:0/2848579753 >> 172.21.15.90:0/2848579753 conn(0x7ff0e40748b0 msgr2=0x7ff0e4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 -- 172.21.15.90:0/2848579753 shutdown_connections 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.839+0000 7ff0ea44b700 1 -- 172.21.15.90:0/2848579753 wait complete. 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.840+0000 7ff0ea44b700 1 Processor -- start 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.840+0000 7ff0ea44b700 1 -- start start 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.840+0000 7ff0ea44b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e4195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.840+0000 7ff0ea44b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e40ffc60 0x7ff0e41964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:56.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.840+0000 7ff0ea44b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e4196b80 con 0x7ff0e40ff350 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.840+0000 7ff0ea44b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0e419a910 con 0x7ff0e40ffc60 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e4195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e3fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e4195f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47342/0 (socket says 172.21.15.90:47342) 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e3fff700 1 -- 172.21.15.90:0/1049077737 learned_addr learned my addr 172.21.15.90:0/1049077737 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e37fe700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e40ffc60 0x7ff0e41964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e3fff700 1 -- 172.21.15.90:0/1049077737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e40ffc60 msgr2=0x7ff0e41964a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e3fff700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e40ffc60 0x7ff0e41964a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e3fff700 1 -- 172.21.15.90:0/1049077737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0d80095f0 con 0x7ff0e40ff350 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e3fff700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e4195f60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80092e0 tx=0x7ff0d800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e17fa700 1 -- 172.21.15.90:0/1049077737 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d801d070 con 0x7ff0e40ff350 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e17fa700 1 -- 172.21.15.90:0/1049077737 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff0d800f460 con 0x7ff0e40ff350 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0e419ab90 con 0x7ff0e40ff350 2024-04-26T04:49:56.844 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0e17fa700 1 -- 172.21.15.90:0/1049077737 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff0d80175d0 con 0x7ff0e40ff350 2024-04-26T04:49:56.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.841+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0e419b030 con 0x7ff0e40ff350 2024-04-26T04:49:56.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.842+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0e4107dd0 con 0x7ff0e40ff350 2024-04-26T04:49:56.845 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.843+0000 7ff0e17fa700 1 -- 172.21.15.90:0/1049077737 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff0d800f5d0 con 0x7ff0e40ff350 2024-04-26T04:49:56.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.844+0000 7ff0e17fa700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff0cc06e200 0x7ff0cc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:56.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.844+0000 7ff0e17fa700 1 -- 172.21.15.90:0/1049077737 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ff0d808f590 con 0x7ff0e40ff350 2024-04-26T04:49:56.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.844+0000 7ff0e37fe700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff0cc06e200 0x7ff0cc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:56.846 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.844+0000 7ff0e37fe700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff0cc06e200 0x7ff0cc0706b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4197580 tx=0x7ff0d400a300 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:56.849 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:56.848+0000 7ff0e17fa700 1 -- 172.21.15.90:0/1049077737 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff0d8058920 con 0x7ff0e40ff350 2024-04-26T04:49:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:56 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["3"], "max": 16}]: dispatch 2024-04-26T04:49:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:56 smithi104 ceph-mon[64778]: Upgrade: osd.3 is safe to restart 2024-04-26T04:49:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:56 smithi104 ceph-mon[64778]: pgmap v108: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:57.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.070+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0e419b360 con 0x7ff0cc06e200 2024-04-26T04:49:57.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.071+0000 7ff0e17fa700 1 -- 172.21.15.90:0/1049077737 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7ff0e419b360 con 0x7ff0cc06e200 2024-04-26T04:49:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff0cc06e200 msgr2=0x7ff0cc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff0cc06e200 0x7ff0cc0706b0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4197580 tx=0x7ff0d400a300 comp rx=0 tx=0).stop 2024-04-26T04:49:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 msgr2=0x7ff0e4195f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e4195f60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7ff0d80092e0 tx=0x7ff0d800bd70 comp rx=0 tx=0).stop 2024-04-26T04:49:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 shutdown_connections 2024-04-26T04:49:57.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff0cc06e200 0x7ff0cc0706b0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff0e40ff350 0x7ff0e4195f60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 --2- 172.21.15.90:0/1049077737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff0e40ffc60 0x7ff0e41964a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 >> 172.21.15.90:0/1049077737 conn(0x7ff0e40748b0 msgr2=0x7ff0e4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:57.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 shutdown_connections 2024-04-26T04:49:57.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.074+0000 7ff0ea44b700 1 -- 172.21.15.90:0/1049077737 wait complete. 2024-04-26T04:49:57.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 -- 172.21.15.90:0/3908594217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 msgr2=0x7fd55c10c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 --2- 172.21.15.90:0/3908594217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c10c620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fd544009910 tx=0x7fd544009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:57.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 -- 172.21.15.90:0/3908594217 shutdown_connections 2024-04-26T04:49:57.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 --2- 172.21.15.90:0/3908594217 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c10c620 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 --2- 172.21.15.90:0/3908594217 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd55c0fefb0 0x7fd55c0ff380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 -- 172.21.15.90:0/3908594217 >> 172.21.15.90:0/3908594217 conn(0x7fd55c0fac80 msgr2=0x7fd55c0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:57.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 -- 172.21.15.90:0/3908594217 shutdown_connections 2024-04-26T04:49:57.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 -- 172.21.15.90:0/3908594217 wait complete. 2024-04-26T04:49:57.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 Processor -- start 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.180+0000 7fd561fd0700 1 -- start start 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd561fd0700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd55c0fefb0 0x7fd55c198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd561fd0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd561fd0700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c198c60 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd561fd0700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd55c19c9f0 con 0x7fd55c0fefb0 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd553fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd553fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:47362/0 (socket says 172.21.15.90:47362) 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd553fff700 1 -- 172.21.15.90:0/2383439266 learned_addr learned my addr 172.21.15.90:0/2383439266 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd55b7fe700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd55c0fefb0 0x7fd55c198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd553fff700 1 -- 172.21.15.90:0/2383439266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd55c0fefb0 msgr2=0x7fd55c198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd553fff700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd55c0fefb0 0x7fd55c198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd553fff700 1 -- 172.21.15.90:0/2383439266 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5440095f0 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.181+0000 7fd553fff700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c198580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd544005f50 tx=0x7fd54400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:57.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.182+0000 7fd5597fa700 1 -- 172.21.15.90:0/2383439266 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd54401d070 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.182+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd55c19cc70 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.182+0000 7fd5597fa700 1 -- 172.21.15.90:0/2383439266 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd54400b610 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.182+0000 7fd5597fa700 1 -- 172.21.15.90:0/2383439266 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd544022470 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.182+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd55c19d1c0 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.183+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd55c109d90 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.183+0000 7fd5597fa700 1 -- 172.21.15.90:0/2383439266 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd544003780 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.184+0000 7fd5597fa700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd54806e200 0x7fd5480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.184+0000 7fd5597fa700 1 -- 172.21.15.90:0/2383439266 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fd54408f530 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.184+0000 7fd55b7fe700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd54806e200 0x7fd5480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.186 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.184+0000 7fd55b7fe700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd54806e200 0x7fd5480706b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009c60 tx=0x7fd54c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:57.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.188+0000 7fd5597fa700 1 -- 172.21.15.90:0/2383439266 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd5440588c0 con 0x7fd55c0ff8c0 2024-04-26T04:49:57.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.396+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fd55c19d3e0 con 0x7fd54806e200 2024-04-26T04:49:57.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.407+0000 7fd5597fa700 1 -- 172.21.15.90:0/2383439266 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fd55c19d3e0 con 0x7fd54806e200 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (4m) 21s ago 15m 24.8M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (15m) 21s ago 15m 8627k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (14m) 115s ago 14m 8032k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (2m) 21s ago 15m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (119s) 115s ago 14m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (4m) 21s ago 15m 80.6M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (10m) 21s ago 10m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (10m) 21s ago 10m 16.9M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (9m) 115s ago 9m 16.6M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (10m) 115s ago 10m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:49:57.409 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (6m) 21s ago 16m 505M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (5m) 115s ago 14m 448M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (3m) 21s ago 16m 48.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (2m) 115s ago 14m 38.1M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (5m) 21s ago 15m 16.5M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (5m) 115s ago 14m 14.8M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (99s) 21s ago 13m 88.4M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (62s) 21s ago 13m 100M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (27s) 21s ago 12m 12.7M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (12m) 115s ago 12m 271M 2240M 18.2.0 dc2bc1663786 4107eb5ec98e 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (11m) 115s ago 11m 236M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (11m) 115s ago 11m 241M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:49:57.410 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (5m) 21s ago 15m 69.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:49:57.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.409+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd54806e200 msgr2=0x7fd5480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.409+0000 7fd561fd0700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd54806e200 0x7fd5480706b0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fd54c009c60 tx=0x7fd54c009260 comp rx=0 tx=0).stop 2024-04-26T04:49:57.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.409+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 msgr2=0x7fd55c198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.409+0000 7fd561fd0700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c198580 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd544005f50 tx=0x7fd54400f690 comp rx=0 tx=0).stop 2024-04-26T04:49:57.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.410+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 shutdown_connections 2024-04-26T04:49:57.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.410+0000 7fd561fd0700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd54806e200 0x7fd5480706b0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.410+0000 7fd561fd0700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd55c0fefb0 0x7fd55c198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.410+0000 7fd561fd0700 1 --2- 172.21.15.90:0/2383439266 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd55c0ff8c0 0x7fd55c198580 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.410+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 >> 172.21.15.90:0/2383439266 conn(0x7fd55c0fac80 msgr2=0x7fd55c0fc2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:57.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.410+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 shutdown_connections 2024-04-26T04:49:57.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.410+0000 7fd561fd0700 1 -- 172.21.15.90:0/2383439266 wait complete. 2024-04-26T04:49:57.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 -- 172.21.15.90:0/21501451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ffd50 msgr2=0x7f1160103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 --2- 172.21.15.90:0/21501451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ffd50 0x7f1160103d20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f1150009910 tx=0x7f1150009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:57.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 -- 172.21.15.90:0/21501451 shutdown_connections 2024-04-26T04:49:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 --2- 172.21.15.90:0/21501451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ffd50 0x7f1160103d20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 --2- 172.21.15.90:0/21501451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11600ff3b0 0x7f11600ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 -- 172.21.15.90:0/21501451 >> 172.21.15.90:0/21501451 conn(0x7f11600fac90 msgr2=0x7f11600fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 -- 172.21.15.90:0/21501451 shutdown_connections 2024-04-26T04:49:57.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.517+0000 7f1167876700 1 -- 172.21.15.90:0/21501451 wait complete. 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.518+0000 7f1167876700 1 Processor -- start 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.518+0000 7f1167876700 1 -- start start 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.518+0000 7f1167876700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ff3b0 0x7f1160194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.518+0000 7f1167876700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11600ffd50 0x7f1160195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.518+0000 7f1167876700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160195a20 con 0x7f11600ff3b0 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.518+0000 7f1167876700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1160199740 con 0x7f11600ffd50 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1165612700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ff3b0 0x7f1160194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1165612700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ff3b0 0x7f1160194e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54220/0 (socket says 172.21.15.90:54220) 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1165612700 1 -- 172.21.15.90:0/594603413 learned_addr learned my addr 172.21.15.90:0/594603413 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1164e11700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11600ffd50 0x7f1160195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1164e11700 1 -- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ff3b0 msgr2=0x7f1160194e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1164e11700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ff3b0 0x7f1160194e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1164e11700 1 -- 172.21.15.90:0/594603413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11500095f0 con 0x7f11600ffd50 2024-04-26T04:49:57.521 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1165612700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ff3b0 0x7f1160194e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:49:57.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1164e11700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11600ffd50 0x7f1160195340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1150006010 tx=0x7f115000f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:57.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f11567fc700 1 -- 172.21.15.90:0/594603413 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f115001d070 con 0x7f11600ffd50 2024-04-26T04:49:57.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f11567fc700 1 -- 172.21.15.90:0/594603413 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f115000b610 con 0x7f11600ffd50 2024-04-26T04:49:57.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11601999c0 con 0x7f11600ffd50 2024-04-26T04:49:57.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f11567fc700 1 -- 172.21.15.90:0/594603413 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1150022470 con 0x7f11600ffd50 2024-04-26T04:49:57.522 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.519+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1160199e80 con 0x7f11600ffd50 2024-04-26T04:49:57.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.520+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1160107720 con 0x7f11600ffd50 2024-04-26T04:49:57.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.521+0000 7f11567fc700 1 -- 172.21.15.90:0/594603413 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f11500050d0 con 0x7f11600ffd50 2024-04-26T04:49:57.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.521+0000 7f11567fc700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f114c06e200 0x7f114c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.521+0000 7f11567fc700 1 -- 172.21.15.90:0/594603413 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f115008f580 con 0x7f11600ffd50 2024-04-26T04:49:57.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.521+0000 7f1165612700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f114c06e200 0x7f114c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.523 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.522+0000 7f1165612700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f114c06e200 0x7f114c0706b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f115c011550 tx=0x7f115c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:57.527 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.526+0000 7f11567fc700 1 -- 172.21.15.90:0/594603413 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1150058990 con 0x7f11600ffd50 2024-04-26T04:49:57.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.815+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f1160196190 con 0x7f11600ffd50 2024-04-26T04:49:57.816 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:57 smithi090 ceph-mon[86550]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:57.817 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:57.817 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:49:57.817 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:57.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.815+0000 7f11567fc700 1 -- 172.21.15.90:0/594603413 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7f115005c050 con 0x7f11600ffd50 2024-04-26T04:49:57.817 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:49:57.817 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:49:57.817 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 3 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:49:57.818 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:49:57.819 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 7, 2024-04-26T04:49:57.819 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 7 2024-04-26T04:49:57.819 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:49:57.819 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.818+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f114c06e200 msgr2=0x7f114c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.818+0000 7f1167876700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f114c06e200 0x7f114c0706b0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f115c011550 tx=0x7f115c009260 comp rx=0 tx=0).stop 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.818+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11600ffd50 msgr2=0x7f1160195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.818+0000 7f1167876700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11600ffd50 0x7f1160195340 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1150006010 tx=0x7f115000f690 comp rx=0 tx=0).stop 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.819+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 shutdown_connections 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.819+0000 7f1167876700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f114c06e200 0x7f114c0706b0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.819+0000 7f1167876700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f11600ff3b0 0x7f1160194e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.819+0000 7f1167876700 1 --2- 172.21.15.90:0/594603413 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f11600ffd50 0x7f1160195340 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.819+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 >> 172.21.15.90:0/594603413 conn(0x7f11600fac90 msgr2=0x7f11600fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:57.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.819+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 shutdown_connections 2024-04-26T04:49:57.821 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.819+0000 7f1167876700 1 -- 172.21.15.90:0/594603413 wait complete. 2024-04-26T04:49:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.922+0000 7f36dff7f700 1 -- 172.21.15.90:0/1589023702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 msgr2=0x7f36d8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.922+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1589023702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d8103d30 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f36c8009910 tx=0x7f36c8009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 -- 172.21.15.90:0/1589023702 shutdown_connections 2024-04-26T04:49:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1589023702 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d8103d30 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1589023702 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36d80ff3c0 0x7f36d80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 -- 172.21.15.90:0/1589023702 >> 172.21.15.90:0/1589023702 conn(0x7f36d80fac80 msgr2=0x7f36d80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:57.924 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 -- 172.21.15.90:0/1589023702 shutdown_connections 2024-04-26T04:49:57.925 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 -- 172.21.15.90:0/1589023702 wait complete. 2024-04-26T04:49:57.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 Processor -- start 2024-04-26T04:49:57.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 -- start start 2024-04-26T04:49:57.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36d80ff3c0 0x7f36d8196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d81972c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d81978e0 con 0x7f36d80ffd60 2024-04-26T04:49:57.927 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.923+0000 7f36dff7f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d8197a20 con 0x7f36d80ff3c0 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dd51a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d81972c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dd51a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d81972c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54244/0 (socket says 172.21.15.90:54244) 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dd51a700 1 -- 172.21.15.90:0/1603697753 learned_addr learned my addr 172.21.15.90:0/1603697753 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36ddd1b700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36d80ff3c0 0x7f36d8196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dd51a700 1 -- 172.21.15.90:0/1603697753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36d80ff3c0 msgr2=0x7f36d8196d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dd51a700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36d80ff3c0 0x7f36d8196d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dd51a700 1 -- 172.21.15.90:0/1603697753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36c80095f0 con 0x7f36d80ffd60 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dd51a700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d81972c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f36c80092e0 tx=0x7f36c800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36ceffd700 1 -- 172.21.15.90:0/1603697753 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c801d070 con 0x7f36d80ffd60 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d819fdf0 con 0x7f36d80ffd60 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36ceffd700 1 -- 172.21.15.90:0/1603697753 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f36c8005dc0 con 0x7f36d80ffd60 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36ceffd700 1 -- 172.21.15.90:0/1603697753 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f36c8017400 con 0x7f36d80ffd60 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.924+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d81a0200 con 0x7f36d80ffd60 2024-04-26T04:49:57.928 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.925+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d8107730 con 0x7f36d80ffd60 2024-04-26T04:49:57.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.926+0000 7f36ceffd700 1 -- 172.21.15.90:0/1603697753 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f36c800f6f0 con 0x7f36d80ffd60 2024-04-26T04:49:57.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.926+0000 7f36ceffd700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f36c406e200 0x7f36c40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:57.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.926+0000 7f36ceffd700 1 -- 172.21.15.90:0/1603697753 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f36c808f4c0 con 0x7f36d80ffd60 2024-04-26T04:49:57.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.926+0000 7f36ddd1b700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f36c406e200 0x7f36c40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:57.929 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.927+0000 7f36ddd1b700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f36c406e200 0x7f36c40706b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005950 tx=0x7f36d400b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:57.932 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:57.931+0000 7f36ceffd700 1 -- 172.21.15.90:0/1603697753 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f36c8058850 con 0x7f36d80ffd60 2024-04-26T04:49:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:57 smithi104 ceph-mon[64778]: from='client.34284 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:49:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-26T04:49:58.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:49:58.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.175+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f36d804f2a0 con 0x7f36d80ffd60 2024-04-26T04:49:58.176 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.175+0000 7f36ceffd700 1 -- 172.21.15.90:0/1603697753 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f36c805bf10 con 0x7f36d80ffd60 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:49:58.177 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:49:58.178 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:49:58.179 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:58.180 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:49:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f36c406e200 msgr2=0x7f36c40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f36c406e200 0x7f36c40706b0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f36d4005950 tx=0x7f36d400b410 comp rx=0 tx=0).stop 2024-04-26T04:49:58.180 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 msgr2=0x7f36d81972c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d81972c0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f36c80092e0 tx=0x7f36c800bd70 comp rx=0 tx=0).stop 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 shutdown_connections 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f36c406e200 0x7f36c40706b0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f36d80ff3c0 0x7f36d8196d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 --2- 172.21.15.90:0/1603697753 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f36d80ffd60 0x7f36d81972c0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 >> 172.21.15.90:0/1603697753 conn(0x7f36d80fac80 msgr2=0x7f36d80fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.178+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 shutdown_connections 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.179+0000 7f36dff7f700 1 -- 172.21.15.90:0/1603697753 wait complete. 2024-04-26T04:49:58.181 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:49:58.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 -- 172.21.15.90:0/505746288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ffc60 msgr2=0x7f7c3c10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 --2- 172.21.15.90:0/505746288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ffc60 0x7f7c3c10a6d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c30009910 tx=0x7f7c30009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:58.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 -- 172.21.15.90:0/505746288 shutdown_connections 2024-04-26T04:49:58.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 --2- 172.21.15.90:0/505746288 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ffc60 0x7f7c3c10a6d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 --2- 172.21.15.90:0/505746288 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c3c0ff350 0x7f7c3c0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 -- 172.21.15.90:0/505746288 >> 172.21.15.90:0/505746288 conn(0x7f7c3c0748b0 msgr2=0x7f7c3c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:58.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 -- 172.21.15.90:0/505746288 shutdown_connections 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.283+0000 7f7c42005700 1 -- 172.21.15.90:0/505746288 wait complete. 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c42005700 1 Processor -- start 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c42005700 1 -- start start 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c42005700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ff350 0x7f7c3c195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c42005700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c3c0ffc60 0x7f7c3c196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c42005700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c196b00 con 0x7f7c3c0ff350 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c42005700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c3c19a890 con 0x7f7c3c0ffc60 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c3b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ff350 0x7f7c3c195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c3b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ff350 0x7f7c3c195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54264/0 (socket says 172.21.15.90:54264) 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.284+0000 7f7c3b7fe700 1 -- 172.21.15.90:0/4068008714 learned_addr learned my addr 172.21.15.90:0/4068008714 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c3affd700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c3c0ffc60 0x7f7c3c196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c3b7fe700 1 -- 172.21.15.90:0/4068008714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c3c0ffc60 msgr2=0x7f7c3c196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c3b7fe700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c3c0ffc60 0x7f7c3c196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c3b7fe700 1 -- 172.21.15.90:0/4068008714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c300095f0 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c3affd700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c3c0ffc60 0x7f7c3c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c3b7fe700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ff350 0x7f7c3c195ee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d900 tx=0x7f7c2c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c38ff9700 1 -- 172.21.15.90:0/4068008714 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c2c0041f0 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c38ff9700 1 -- 172.21.15.90:0/4068008714 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7c2c004350 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c3c19ab70 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c38ff9700 1 -- 172.21.15.90:0/4068008714 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7c2c003dc0 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.285+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c3c19b0c0 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.286+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c3c107dd0 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.287+0000 7f7c38ff9700 1 -- 172.21.15.90:0/4068008714 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7c2c009730 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.287+0000 7f7c38ff9700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7c2406e200 0x7f7c240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.287+0000 7f7c38ff9700 1 -- 172.21.15.90:0/4068008714 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f7c2c08e2b0 con 0x7f7c3c0ff350 2024-04-26T04:49:58.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.287+0000 7f7c3affd700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7c2406e200 0x7f7c240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.287+0000 7f7c3affd700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7c2406e200 0x7f7c240706b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b5c0 tx=0x7f7c300053a0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:58.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.292+0000 7f7c38ff9700 1 -- 172.21.15.90:0/4068008714 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7c2c057640 con 0x7f7c3c0ff350 2024-04-26T04:49:58.509 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.508+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f7c3c19b340 con 0x7f7c2406e200 2024-04-26T04:49:58.510 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.509+0000 7f7c38ff9700 1 -- 172.21.15.90:0/4068008714 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+400 (secure 0 0 0) 0x7f7c3c19b340 con 0x7f7c2406e200 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:49:58.511 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "9/23 daemons upgraded", 2024-04-26T04:49:58.512 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:49:58.512 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:49:58.512 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:49:58.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7c2406e200 msgr2=0x7f7c240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7c2406e200 0x7f7c240706b0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7c3000b5c0 tx=0x7f7c300053a0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ff350 msgr2=0x7f7c3c195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ff350 0x7f7c3c195ee0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7c2c00d900 tx=0x7f7c2c00dcc0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 shutdown_connections 2024-04-26T04:49:58.513 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7c2406e200 0x7f7c240706b0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7c3c0ff350 0x7f7c3c195ee0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 --2- 172.21.15.90:0/4068008714 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7c3c0ffc60 0x7f7c3c196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 >> 172.21.15.90:0/4068008714 conn(0x7f7c3c0748b0 msgr2=0x7f7c3c104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:58.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 shutdown_connections 2024-04-26T04:49:58.514 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.512+0000 7f7c42005700 1 -- 172.21.15.90:0/4068008714 wait complete. 2024-04-26T04:49:58.618 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.615+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3236924023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 msgr2=0x7fc928103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.615+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3236924023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928103d30 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fc918009910 tx=0x7fc918009c20 comp rx=0 tx=0).stop 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.615+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3236924023 shutdown_connections 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.615+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3236924023 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928103d30 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.615+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3236924023 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280ff3c0 0x7fc9280ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.615+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3236924023 >> 172.21.15.90:0/3236924023 conn(0x7fc9280fac80 msgr2=0x7fc9280fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3236924023 shutdown_connections 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3236924023 wait complete. 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 Processor -- start 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 -- start start 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280ff3c0 0x7fc928196d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928197260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc928197880 con 0x7fc9280ffd60 2024-04-26T04:49:58.619 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92fcf4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9281979f0 con 0x7fc9280ff3c0 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92d28f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928197260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92d28f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928197260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:54286/0 (socket says 172.21.15.90:54286) 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.616+0000 7fc92d28f700 1 -- 172.21.15.90:0/3406672053 learned_addr learned my addr 172.21.15.90:0/3406672053 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc92da90700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280ff3c0 0x7fc928196d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc92d28f700 1 -- 172.21.15.90:0/3406672053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280ff3c0 msgr2=0x7fc928196d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc92d28f700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280ff3c0 0x7fc928196d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc92d28f700 1 -- 172.21.15.90:0/3406672053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9180095f0 con 0x7fc9280ffd60 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc92d28f700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928197260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc9180092e0 tx=0x7fc91800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc91effd700 1 -- 172.21.15.90:0/3406672053 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc91801d070 con 0x7fc9280ffd60 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc91effd700 1 -- 172.21.15.90:0/3406672053 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc918005dc0 con 0x7fc9280ffd60 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc91effd700 1 -- 172.21.15.90:0/3406672053 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc918017400 con 0x7fc9280ffd60 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc92819fd90 con 0x7fc9280ffd60 2024-04-26T04:49:58.620 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.617+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9281a0200 con 0x7fc9280ffd60 2024-04-26T04:49:58.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.618+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc928107730 con 0x7fc9280ffd60 2024-04-26T04:49:58.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.618+0000 7fc91effd700 1 -- 172.21.15.90:0/3406672053 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc91800f460 con 0x7fc9280ffd60 2024-04-26T04:49:58.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.619+0000 7fc91effd700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc91406e200 0x7fc9140706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:49:58.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.619+0000 7fc91effd700 1 -- 172.21.15.90:0/3406672053 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fc91808f5a0 con 0x7fc9280ffd60 2024-04-26T04:49:58.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.619+0000 7fc92da90700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc91406e200 0x7fc9140706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:49:58.622 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.619+0000 7fc92da90700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc91406e200 0x7fc9140706b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc924005950 tx=0x7fc9240058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:49:58.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.624+0000 7fc91effd700 1 -- 172.21.15.90:0/3406672053 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc918058930 con 0x7fc9280ffd60 2024-04-26T04:49:58.914 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.913+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fc92804f2a0 con 0x7fc9280ffd60 2024-04-26T04:49:58.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:58 smithi090 ceph-mon[86550]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:58.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:58 smithi090 ceph-mon[86550]: Upgrade: Updating osd.3 2024-04-26T04:49:58.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:58 smithi090 ceph-mon[86550]: Deploying daemon osd.3 on smithi104 2024-04-26T04:49:58.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:58 smithi090 ceph-mon[86550]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:58.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:58 smithi090 ceph-mon[86550]: pgmap v109: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:58.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:58 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/594603413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:58.915 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:58 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1603697753' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:49:58.915 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.913+0000 7fc91effd700 1 -- 172.21.15.90:0/3406672053 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fc91805bff0 con 0x7fc9280ffd60 2024-04-26T04:49:58.915 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_OK 2024-04-26T04:49:58.916 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc91406e200 msgr2=0x7fc9140706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc91406e200 0x7fc9140706b0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fc924005950 tx=0x7fc9240058e0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 msgr2=0x7fc928197260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928197260 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc9180092e0 tx=0x7fc91800be20 comp rx=0 tx=0).stop 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 shutdown_connections 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc91406e200 0x7fc9140706b0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc9280ff3c0 0x7fc928196d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 --2- 172.21.15.90:0/3406672053 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc9280ffd60 0x7fc928197260 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.915+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 >> 172.21.15.90:0/3406672053 conn(0x7fc9280fac80 msgr2=0x7fc9280fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.916+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 shutdown_connections 2024-04-26T04:49:58.917 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:49:58.916+0000 7fc92fcf4700 1 -- 172.21.15.90:0/3406672053 wait complete. 2024-04-26T04:49:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:58 smithi104 ceph-mon[64778]: from='client.34288 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:58 smithi104 ceph-mon[64778]: Upgrade: Updating osd.3 2024-04-26T04:49:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:58 smithi104 ceph-mon[64778]: Deploying daemon osd.3 on smithi104 2024-04-26T04:49:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:58 smithi104 ceph-mon[64778]: from='client.34292 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:58 smithi104 ceph-mon[64778]: pgmap v109: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:49:59.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:58 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/594603413' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:49:59.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:58 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1603697753' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:49:59.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:59 smithi104 ceph-mon[64778]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:49:59.969 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:49:59 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3406672053' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:50:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:59 smithi090 ceph-mon[86550]: from='client.34304 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:49:59 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3406672053' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:50:00.633 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:00 smithi104 systemd[1]: Stopping Ceph osd.3 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:50:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:00 smithi104 ceph-mon[64778]: pgmap v110: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:00 smithi104 ceph-mon[64778]: overall HEALTH_OK 2024-04-26T04:50:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:00 smithi104 ceph-mon[64778]: osd.3 marked itself down and dead 2024-04-26T04:50:01.012 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:00 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[38332]: 2024-04-26T04:50:00.631+0000 7ff7ddf89700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:50:01.012 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:00 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[38332]: 2024-04-26T04:50:00.631+0000 7ff7ddf89700 -1 osd.3 65 *** Got signal Terminated *** 2024-04-26T04:50:01.012 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:00 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[38332]: 2024-04-26T04:50:00.631+0000 7ff7ddf89700 -1 osd.3 65 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:50:01.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:00 smithi090 ceph-mon[86550]: pgmap v110: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:01.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:00 smithi090 ceph-mon[86550]: overall HEALTH_OK 2024-04-26T04:50:01.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:00 smithi090 ceph-mon[86550]: osd.3 marked itself down and dead 2024-04-26T04:50:01.391 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:01 smithi104 podman[74358]: 2024-04-26 04:50:01.112283257 +0000 UTC m=+0.596737763 container died 4107eb5ec98e8c2b4bb09b6fd500bddb6dfc4a60f0e1f7eaf5d978b26fed930a (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.29.1, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20231212, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-26T04:50:01.741 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:01 smithi104 podman[74358]: 2024-04-26 04:50:01.481526746 +0000 UTC m=+0.965981239 container cleanup 4107eb5ec98e8c2b4bb09b6fd500bddb6dfc4a60f0e1f7eaf5d978b26fed930a (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, io.buildah.version=1.29.1, org.label-schema.schema-version=1.0, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T04:50:01.741 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:01 smithi104 bash[74358]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3 2024-04-26T04:50:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:01 smithi104 ceph-mon[64778]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:50:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:01 smithi104 ceph-mon[64778]: osdmap e66: 6 total, 5 up, 6 in 2024-04-26T04:50:02.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:01 smithi104 ceph-mon[64778]: osdmap e67: 6 total, 5 up, 6 in 2024-04-26T04:50:02.011 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:01 smithi104 podman[74376]: 2024-04-26 04:50:01.741220244 +0000 UTC m=+0.626880309 container remove 4107eb5ec98e8c2b4bb09b6fd500bddb6dfc4a60f0e1f7eaf5d978b26fed930a (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=HEAD, ceph=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20231212, io.buildah.version=1.29.1) 2024-04-26T04:50:02.122 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:01 smithi090 ceph-mon[86550]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:50:02.122 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:01 smithi090 ceph-mon[86550]: osdmap e66: 6 total, 5 up, 6 in 2024-04-26T04:50:02.122 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:01 smithi090 ceph-mon[86550]: osdmap e67: 6 total, 5 up, 6 in 2024-04-26T04:50:02.436 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:02 smithi104 podman[74470]: 2024-04-26 04:50:02.004651758 +0000 UTC m=+0.017626778 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:50:02.437 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:02 smithi104 podman[74470]: 2024-04-26 04:50:02.108969878 +0000 UTC m=+0.121944890 container create 8c2bf893f86d92318a866fc182b0e727d7be7d2eb9c08e0a3a39431997c37527 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T04:50:02.437 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:02 smithi104 podman[74470]: 2024-04-26 04:50:02.344719542 +0000 UTC m=+0.357694554 container init 8c2bf893f86d92318a866fc182b0e727d7be7d2eb9c08e0a3a39431997c37527 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-b22e2eb, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-26T04:50:02.437 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:02 smithi104 podman[74470]: 2024-04-26 04:50:02.347868657 +0000 UTC m=+0.360843677 container start 8c2bf893f86d92318a866fc182b0e727d7be7d2eb9c08e0a3a39431997c37527 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-26T04:50:02.437 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:02 smithi104 podman[74470]: 2024-04-26 04:50:02.436389894 +0000 UTC m=+0.449364929 container attach 8c2bf893f86d92318a866fc182b0e727d7be7d2eb9c08e0a3a39431997c37527 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-26T04:50:02.761 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:02 smithi104 podman[74470]: 2024-04-26 04:50:02.638512951 +0000 UTC m=+0.651487954 container died 8c2bf893f86d92318a866fc182b0e727d7be7d2eb9c08e0a3a39431997c37527 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, RELEASE=squid-b22e2eb, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T04:50:03.140 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:02 smithi104 ceph-mon[64778]: pgmap v112: 65 pgs: 22 peering, 6 stale+active+clean, 37 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:03.140 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:02 smithi104 ceph-mon[64778]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-04-26T04:50:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:02 smithi090 ceph-mon[86550]: pgmap v112: 65 pgs: 22 peering, 6 stale+active+clean, 37 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:02 smithi090 ceph-mon[86550]: Health check failed: Reduced data availability: 10 pgs peering (PG_AVAILABILITY) 2024-04-26T04:50:03.761 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:03 smithi104 podman[74496]: 2024-04-26 04:50:03.456976346 +0000 UTC m=+0.808754610 container remove 8c2bf893f86d92318a866fc182b0e727d7be7d2eb9c08e0a3a39431997c37527 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-26T04:50:03.761 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:03 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.3.service: Deactivated successfully. 2024-04-26T04:50:03.762 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:03 smithi104 systemd[1]: Stopped Ceph osd.3 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:50:03.762 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:03 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.3.service: Consumed 1min 9.254s CPU time. 2024-04-26T04:50:03.762 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:03 smithi104 systemd[1]: Starting Ceph osd.3 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:50:04.169 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:03 smithi104 podman[74600]: 2024-04-26 04:50:03.800694219 +0000 UTC m=+0.017875951 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:50:04.169 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:03 smithi104 podman[74600]: 2024-04-26 04:50:03.933403895 +0000 UTC m=+0.150585650 container create fd938aefb20edf21af8d7957a833285e3c21a4aee5a9057ee7187f30a76027e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb) 2024-04-26T04:50:04.169 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 podman[74600]: 2024-04-26 04:50:04.169272805 +0000 UTC m=+0.386454515 container init fd938aefb20edf21af8d7957a833285e3c21a4aee5a9057ee7187f30a76027e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-04-26T04:50:04.511 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 podman[74600]: 2024-04-26 04:50:04.172718225 +0000 UTC m=+0.389899935 container start fd938aefb20edf21af8d7957a833285e3c21a4aee5a9057ee7187f30a76027e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:50:04.511 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 podman[74600]: 2024-04-26 04:50:04.243967671 +0000 UTC m=+0.461149389 container attach fd938aefb20edf21af8d7957a833285e3c21a4aee5a9057ee7187f30a76027e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T04:50:04.980 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-mon[64778]: pgmap v114: 65 pgs: 31 peering, 3 stale+active+clean, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate[74615]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate[74615]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 bash[74600]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 bash[74600]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-3 --no-mon-config --dev /dev/mapper/vg_nvme-lv_4 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 bash[74600]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 bash[74600]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 bash[74600]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 bash[74600]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:50:04.981 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 bash[74600]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-26T04:50:04.982 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate[74615]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_4 2024-04-26T04:50:04.982 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate[74615]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-3 2024-04-26T04:50:04.982 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate[74615]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3/block 2024-04-26T04:50:04.982 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate[74615]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-26T04:50:04.982 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate[74615]: --> ceph-volume raw activate successful for osd ID: 3 2024-04-26T04:50:04.982 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:04 smithi104 podman[74600]: 2024-04-26 04:50:04.641022896 +0000 UTC m=+0.858204616 container died fd938aefb20edf21af8d7957a833285e3c21a4aee5a9057ee7187f30a76027e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.build-date=20240424, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5) 2024-04-26T04:50:05.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:04 smithi090 ceph-mon[86550]: pgmap v114: 65 pgs: 31 peering, 3 stale+active+clean, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:05.511 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 podman[74746]: 2024-04-26 04:50:05.239566127 +0000 UTC m=+0.593141020 container remove fd938aefb20edf21af8d7957a833285e3c21a4aee5a9057ee7187f30a76027e9 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-activate, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-04-26T04:50:05.512 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 podman[74812]: 2024-04-26 04:50:05.354373528 +0000 UTC m=+0.018024933 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:50:05.512 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 podman[74812]: 2024-04-26 04:50:05.482224552 +0000 UTC m=+0.145875941 container create d860fd1e9069b1e6738a4317122f80ef199f3569b3cb29e8a00e7174fb5f7e5a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T04:50:05.835 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:05 smithi104 ceph-mon[64778]: Health check failed: Degraded data redundancy: 9/348 objects degraded (2.586%), 3 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:05.836 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 podman[74812]: 2024-04-26 04:50:05.735096145 +0000 UTC m=+0.398747544 container init d860fd1e9069b1e6738a4317122f80ef199f3569b3cb29e8a00e7174fb5f7e5a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T04:50:05.836 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 podman[74812]: 2024-04-26 04:50:05.738341967 +0000 UTC m=+0.401993358 container start d860fd1e9069b1e6738a4317122f80ef199f3569b3cb29e8a00e7174fb5f7e5a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=) 2024-04-26T04:50:05.836 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 ceph-osd[74831]: -- 172.21.15.104:0/1466704819 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cece5fb680 con 0x55cece5c5000 2024-04-26T04:50:06.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:05 smithi090 ceph-mon[86550]: Health check failed: Degraded data redundancy: 9/348 objects degraded (2.586%), 3 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:06.261 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 bash[74812]: d860fd1e9069b1e6738a4317122f80ef199f3569b3cb29e8a00e7174fb5f7e5a 2024-04-26T04:50:06.279 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:05 smithi104 systemd[1]: Started Ceph osd.3 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:50:06.723 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:06 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[74827]: 2024-04-26T04:50:06.285+0000 7f1fe54ec7c0 -1 Falling back to public interface 2024-04-26T04:50:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:06 smithi104 ceph-mon[64778]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 31 peering, 4 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 9/348 objects degraded (2.586%) 2024-04-26T04:50:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:07.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:07.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:06 smithi090 ceph-mon[86550]: pgmap v115: 65 pgs: 3 active+undersized+degraded, 31 peering, 4 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 9/348 objects degraded (2.586%) 2024-04-26T04:50:07.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:07.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:07.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:08.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:07 smithi090 ceph-mon[86550]: pgmap v116: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:08.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:07 smithi090 ceph-mon[86550]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 13 pgs peering) 2024-04-26T04:50:08.227 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:07 smithi104 ceph-mon[64778]: pgmap v116: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:08.228 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:07 smithi104 ceph-mon[64778]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 13 pgs peering) 2024-04-26T04:50:10.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:10.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:10.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:10 smithi090 ceph-mon[86550]: pgmap v117: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:10.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:10.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:10.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:10 smithi104 ceph-mon[64778]: pgmap v117: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:11.117 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:10 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[74827]: 2024-04-26T04:50:10.926+0000 7f1fe54ec7c0 -1 osd.3 0 read_superblock omap replica is missing. 2024-04-26T04:50:11.411 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:11 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:11.411 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:11 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:11.411 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:11 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:11.411 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:11 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:11 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:11 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:11 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:11.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:11 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:11.745 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[74827]: 2024-04-26T04:50:11.445+0000 7f1fe54ec7c0 -1 osd.3 65 log_to_monitors true 2024-04-26T04:50:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:12 smithi090 ceph-mon[86550]: from='osd.3 [v2:172.21.15.104:6800/2129064810,v1:172.21.15.104:6801/2129064810]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:50:12.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:12 smithi090 ceph-mon[86550]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:50:12.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:12 smithi090 ceph-mon[86550]: pgmap v118: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:12 smithi104 ceph-mon[64778]: from='osd.3 [v2:172.21.15.104:6800/2129064810,v1:172.21.15.104:6801/2129064810]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:50:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:12 smithi104 ceph-mon[64778]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-26T04:50:12.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:12 smithi104 ceph-mon[64778]: pgmap v118: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:13 smithi090 ceph-mon[86550]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:50:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:13 smithi090 ceph-mon[86550]: osdmap e68: 6 total, 5 up, 6 in 2024-04-26T04:50:13.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:13 smithi090 ceph-mon[86550]: from='osd.3 [v2:172.21.15.104:6800/2129064810,v1:172.21.15.104:6801/2129064810]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:13.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:13 smithi090 ceph-mon[86550]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:13.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:13 smithi090 ceph-mon[86550]: from='osd.3 ' entity='osd.3' 2024-04-26T04:50:13.511 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:50:13 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[74827]: 2024-04-26T04:50:13.059+0000 7f1fd293c700 -1 osd.3 65 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:50:13.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:13 smithi104 ceph-mon[64778]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-26T04:50:13.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:13 smithi104 ceph-mon[64778]: osdmap e68: 6 total, 5 up, 6 in 2024-04-26T04:50:13.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:13 smithi104 ceph-mon[64778]: from='osd.3 [v2:172.21.15.104:6800/2129064810,v1:172.21.15.104:6801/2129064810]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:13.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:13 smithi104 ceph-mon[64778]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:13.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:13 smithi104 ceph-mon[64778]: from='osd.3 ' entity='osd.3' 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: pgmap v120: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:15.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:15.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:15.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:50:15.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: osd.3 [v2:172.21.15.104:6800/2129064810,v1:172.21.15.104:6801/2129064810] boot 2024-04-26T04:50:15.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: osdmap e69: 6 total, 6 up, 6 in 2024-04-26T04:50:15.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:50:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: pgmap v120: 65 pgs: 20 active+undersized+degraded, 18 active+undersized, 27 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 73/348 objects degraded (20.977%) 2024-04-26T04:50:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: osd.3 [v2:172.21.15.104:6800/2129064810,v1:172.21.15.104:6801/2129064810] boot 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: osdmap e69: 6 total, 6 up, 6 in 2024-04-26T04:50:15.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-26T04:50:16.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:15 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:16.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:15 smithi090 ceph-mon[86550]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-26T04:50:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:15 smithi090 ceph-mon[86550]: osdmap e70: 6 total, 6 up, 6 in 2024-04-26T04:50:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:15 smithi090 ceph-mon[86550]: pgmap v123: 65 pgs: 4 peering, 19 active+undersized+degraded, 15 active+undersized, 27 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 69/348 objects degraded (19.828%) 2024-04-26T04:50:16.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:15 smithi090 ceph-mon[86550]: Health check update: Degraded data redundancy: 73/348 objects degraded (20.977%), 20 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:16.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:15 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:16.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:15 smithi104 ceph-mon[64778]: Upgrade: unsafe to stop osd(s) at this time (18 PGs are or would become offline) 2024-04-26T04:50:16.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:15 smithi104 ceph-mon[64778]: osdmap e70: 6 total, 6 up, 6 in 2024-04-26T04:50:16.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:15 smithi104 ceph-mon[64778]: pgmap v123: 65 pgs: 4 peering, 19 active+undersized+degraded, 15 active+undersized, 27 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 69/348 objects degraded (19.828%) 2024-04-26T04:50:16.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:15 smithi104 ceph-mon[64778]: Health check update: Degraded data redundancy: 73/348 objects degraded (20.977%), 20 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:18.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:17 smithi090 ceph-mon[86550]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 69/348 objects degraded (19.828%), 19 pgs degraded) 2024-04-26T04:50:18.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:17 smithi090 ceph-mon[86550]: Cluster is now healthy 2024-04-26T04:50:18.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:17 smithi104 ceph-mon[64778]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 69/348 objects degraded (19.828%), 19 pgs degraded) 2024-04-26T04:50:18.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:17 smithi104 ceph-mon[64778]: Cluster is now healthy 2024-04-26T04:50:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:18 smithi090 ceph-mon[86550]: pgmap v124: 65 pgs: 4 peering, 61 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:18 smithi104 ceph-mon[64778]: pgmap v124: 65 pgs: 4 peering, 61 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:21.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:20 smithi090 ceph-mon[86550]: pgmap v125: 65 pgs: 4 peering, 61 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:20 smithi104 ceph-mon[64778]: pgmap v125: 65 pgs: 4 peering, 61 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:23.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:22 smithi090 ceph-mon[86550]: pgmap v126: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:22 smithi104 ceph-mon[64778]: pgmap v126: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:25.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:24 smithi090 ceph-mon[86550]: pgmap v127: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:25.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:24 smithi104 ceph-mon[64778]: pgmap v127: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:26 smithi090 ceph-mon[86550]: pgmap v128: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:26.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:26 smithi104 ceph-mon[64778]: pgmap v128: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:28.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:28 smithi090 ceph-mon[86550]: pgmap v129: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:29.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:28 smithi104 ceph-mon[64778]: pgmap v129: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:29.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 -- 172.21.15.90:0/3086590761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ff3c0 msgr2=0x7ffa4c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/3086590761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ff3c0 0x7ffa4c0ff790 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7ffa40009910 tx=0x7ffa40009c20 comp rx=0 tx=0).stop 2024-04-26T04:50:29.038 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 -- 172.21.15.90:0/3086590761 shutdown_connections 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/3086590761 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa4c0ffd60 0x7ffa4c103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/3086590761 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ff3c0 0x7ffa4c0ff790 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 -- 172.21.15.90:0/3086590761 >> 172.21.15.90:0/3086590761 conn(0x7ffa4c0fac60 msgr2=0x7ffa4c0fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 -- 172.21.15.90:0/3086590761 shutdown_connections 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.035+0000 7ffa52d8b700 1 -- 172.21.15.90:0/3086590761 wait complete. 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa52d8b700 1 Processor -- start 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa52d8b700 1 -- start start 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa52d8b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa4c0ff3c0 0x7ffa4c196d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa52d8b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ffd60 0x7ffa4c197250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa52d8b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c197930 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa52d8b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa4c197a70 con 0x7ffa4c0ff3c0 2024-04-26T04:50:29.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa4bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ffd60 0x7ffa4c197250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa4bfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ffd60 0x7ffa4c197250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45320/0 (socket says 172.21.15.90:45320) 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa4bfff700 1 -- 172.21.15.90:0/1650097453 learned_addr learned my addr 172.21.15.90:0/1650097453 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa50b27700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa4c0ff3c0 0x7ffa4c196d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa4bfff700 1 -- 172.21.15.90:0/1650097453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa4c0ff3c0 msgr2=0x7ffa4c196d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa4bfff700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa4c0ff3c0 0x7ffa4c196d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.036+0000 7ffa4bfff700 1 -- 172.21.15.90:0/1650097453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa400095f0 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.037+0000 7ffa4bfff700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ffd60 0x7ffa4c197250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d900 tx=0x7ffa3800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.037+0000 7ffa49ffb700 1 -- 172.21.15.90:0/1650097453 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa38009900 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.037+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa4c19fd80 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.037+0000 7ffa49ffb700 1 -- 172.21.15.90:0/1650097453 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ffa38010460 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.037+0000 7ffa49ffb700 1 -- 172.21.15.90:0/1650097453 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ffa3800f5f0 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.037+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa4c1a02a0 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.038+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa4c1076d0 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.038+0000 7ffa49ffb700 1 -- 172.21.15.90:0/1650097453 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ffa380105d0 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.039+0000 7ffa49ffb700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ffa3c06e200 0x7ffa3c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.041 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.039+0000 7ffa49ffb700 1 -- 172.21.15.90:0/1650097453 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7ffa3808dca0 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.039+0000 7ffa50b27700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ffa3c06e200 0x7ffa3c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.040+0000 7ffa50b27700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ffa3c06e200 0x7ffa3c0706b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006010 tx=0x7ffa4001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:29.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.044+0000 7ffa49ffb700 1 -- 172.21.15.90:0/1650097453 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ffa38092070 con 0x7ffa4c0ffd60 2024-04-26T04:50:29.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.261+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa4c061190 con 0x7ffa3c06e200 2024-04-26T04:50:29.264 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.262+0000 7ffa49ffb700 1 -- 172.21.15.90:0/1650097453 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7ffa4c061190 con 0x7ffa3c06e200 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.264+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ffa3c06e200 msgr2=0x7ffa3c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.264+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ffa3c06e200 0x7ffa3c0706b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa40006010 tx=0x7ffa4001a040 comp rx=0 tx=0).stop 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.264+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ffd60 msgr2=0x7ffa4c197250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.264+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ffd60 0x7ffa4c197250 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7ffa3800d900 tx=0x7ffa3800dc10 comp rx=0 tx=0).stop 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.265+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 shutdown_connections 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.265+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ffa3c06e200 0x7ffa3c0706b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.265+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ffa4c0ff3c0 0x7ffa4c196d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.265+0000 7ffa52d8b700 1 --2- 172.21.15.90:0/1650097453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ffa4c0ffd60 0x7ffa4c197250 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.265+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 >> 172.21.15.90:0/1650097453 conn(0x7ffa4c0fac60 msgr2=0x7ffa4c0fc270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.265+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 shutdown_connections 2024-04-26T04:50:29.267 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.265+0000 7ffa52d8b700 1 -- 172.21.15.90:0/1650097453 wait complete. 2024-04-26T04:50:29.277 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 -- 172.21.15.90:0/3265786912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc100e60 msgr2=0x7fccdc104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 --2- 172.21.15.90:0/3265786912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc100e60 0x7fccdc104e30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fccd0009910 tx=0x7fccd0009c20 comp rx=0 tx=0).stop 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 -- 172.21.15.90:0/3265786912 shutdown_connections 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 --2- 172.21.15.90:0/3265786912 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc100e60 0x7fccdc104e30 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 --2- 172.21.15.90:0/3265786912 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccdc1004c0 0x7fccdc100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 -- 172.21.15.90:0/3265786912 >> 172.21.15.90:0/3265786912 conn(0x7fccdc0fbd80 msgr2=0x7fccdc0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 -- 172.21.15.90:0/3265786912 shutdown_connections 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.369+0000 7fcce290a700 1 -- 172.21.15.90:0/3265786912 wait complete. 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fcce290a700 1 Processor -- start 2024-04-26T04:50:29.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fcce290a700 1 -- start start 2024-04-26T04:50:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fcce290a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc1004c0 0x7fccdc073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fcce290a700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccdc100e60 0x7fccdc073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fcce290a700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc073ad0 con 0x7fccdc1004c0 2024-04-26T04:50:29.374 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fcce290a700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc073c10 con 0x7fccdc100e60 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fccdbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc1004c0 0x7fccdc073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fccdbfff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc1004c0 0x7fccdc073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45344/0 (socket says 172.21.15.90:45344) 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fccdbfff700 1 -- 172.21.15.90:0/1182026717 learned_addr learned my addr 172.21.15.90:0/1182026717 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.370+0000 7fccdb7fe700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccdc100e60 0x7fccdc073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fccdbfff700 1 -- 172.21.15.90:0/1182026717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccdc100e60 msgr2=0x7fccdc073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fccdbfff700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccdc100e60 0x7fccdc073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fccdbfff700 1 -- 172.21.15.90:0/1182026717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd00095f0 con 0x7fccdc1004c0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fccdbfff700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc1004c0 0x7fccdc073050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d900 tx=0x7fcccc00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fccd97fa700 1 -- 172.21.15.90:0/1182026717 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc004a00 con 0x7fccdc1004c0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc1a2070 con 0x7fccdc1004c0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc1a2480 con 0x7fccdc1004c0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fccd97fa700 1 -- 172.21.15.90:0/1182026717 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcccc0054e0 con 0x7fccdc1004c0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.371+0000 7fccd97fa700 1 -- 172.21.15.90:0/1182026717 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcccc009d00 con 0x7fccdc1004c0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.372+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc108740 con 0x7fccdc1004c0 2024-04-26T04:50:29.375 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.372+0000 7fccd97fa700 1 -- 172.21.15.90:0/1182026717 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcccc005020 con 0x7fccdc1004c0 2024-04-26T04:50:29.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.373+0000 7fccd97fa700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fccc406e200 0x7fccc40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.373+0000 7fccd97fa700 1 -- 172.21.15.90:0/1182026717 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fcccc08dfc0 con 0x7fccdc1004c0 2024-04-26T04:50:29.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.373+0000 7fccdb7fe700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fccc406e200 0x7fccc40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.376 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.373+0000 7fccdb7fe700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fccc406e200 0x7fccc40706b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fccdc074840 tx=0x7fccd001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:29.379 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.378+0000 7fccd97fa700 1 -- 172.21.15.90:0/1182026717 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcccc0573d0 con 0x7fccdc1004c0 2024-04-26T04:50:29.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.595+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fccdc061190 con 0x7fccc406e200 2024-04-26T04:50:29.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.596+0000 7fccd97fa700 1 -- 172.21.15.90:0/1182026717 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fccdc061190 con 0x7fccc406e200 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.598+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fccc406e200 msgr2=0x7fccc40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.598+0000 7fcce290a700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fccc406e200 0x7fccc40706b0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fccdc074840 tx=0x7fccd001a040 comp rx=0 tx=0).stop 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.598+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc1004c0 msgr2=0x7fccdc073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.598+0000 7fcce290a700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc1004c0 0x7fccdc073050 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcccc00d900 tx=0x7fcccc00dc10 comp rx=0 tx=0).stop 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.599+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 shutdown_connections 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.599+0000 7fcce290a700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fccc406e200 0x7fccc40706b0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.599+0000 7fcce290a700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fccdc1004c0 0x7fccdc073050 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.599+0000 7fcce290a700 1 --2- 172.21.15.90:0/1182026717 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fccdc100e60 0x7fccdc073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.599+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 >> 172.21.15.90:0/1182026717 conn(0x7fccdc0fbd80 msgr2=0x7fccdc1046a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:29.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.599+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 shutdown_connections 2024-04-26T04:50:29.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.599+0000 7fcce290a700 1 -- 172.21.15.90:0/1182026717 wait complete. 2024-04-26T04:50:29.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 -- 172.21.15.90:0/3106438444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 msgr2=0x7f8680104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 --2- 172.21.15.90:0/3106438444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680104e50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8674009910 tx=0x7f8674009c20 comp rx=0 tx=0).stop 2024-04-26T04:50:29.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 -- 172.21.15.90:0/3106438444 shutdown_connections 2024-04-26T04:50:29.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 --2- 172.21.15.90:0/3106438444 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680104e50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.704 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 --2- 172.21.15.90:0/3106438444 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86801004e0 0x7f86801008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 -- 172.21.15.90:0/3106438444 >> 172.21.15.90:0/3106438444 conn(0x7f86800fbda0 msgr2=0x7f86800fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:29.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 -- 172.21.15.90:0/3106438444 shutdown_connections 2024-04-26T04:50:29.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.703+0000 7f8686a0e700 1 -- 172.21.15.90:0/3106438444 wait complete. 2024-04-26T04:50:29.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f8686a0e700 1 Processor -- start 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f8686a0e700 1 -- start start 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f8686a0e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86801004e0 0x7f8680195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f8686a0e700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f8686a0e700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680196aa0 con 0x7f8680100e80 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f8686a0e700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8680196be0 con 0x7f86801004e0 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f867f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f867f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680196450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45360/0 (socket says 172.21.15.90:45360) 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f867f7fe700 1 -- 172.21.15.90:0/2771099630 learned_addr learned my addr 172.21.15.90:0/2771099630 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.704+0000 7f867ffff700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86801004e0 0x7f8680195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f867f7fe700 1 -- 172.21.15.90:0/2771099630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86801004e0 msgr2=0x7f8680195f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f867f7fe700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86801004e0 0x7f8680195f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f867f7fe700 1 -- 172.21.15.90:0/2771099630 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86740095f0 con 0x7f8680100e80 2024-04-26T04:50:29.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f867f7fe700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680196450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f867400bda0 tx=0x7f867400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:29.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f867d7fa700 1 -- 172.21.15.90:0/2771099630 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f867401d070 con 0x7f8680100e80 2024-04-26T04:50:29.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f868019a9d0 con 0x7f8680100e80 2024-04-26T04:50:29.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f867d7fa700 1 -- 172.21.15.90:0/2771099630 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8674004fb0 con 0x7f8680100e80 2024-04-26T04:50:29.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f867d7fa700 1 -- 172.21.15.90:0/2771099630 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f867400f480 con 0x7f8680100e80 2024-04-26T04:50:29.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.705+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f868019aec0 con 0x7f8680100e80 2024-04-26T04:50:29.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.706+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8680108850 con 0x7f8680100e80 2024-04-26T04:50:29.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.707+0000 7f867d7fa700 1 -- 172.21.15.90:0/2771099630 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8674022870 con 0x7f8680100e80 2024-04-26T04:50:29.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.707+0000 7f867d7fa700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f866806e200 0x7f86680706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:29.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.707+0000 7f867d7fa700 1 -- 172.21.15.90:0/2771099630 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f867408f610 con 0x7f8680100e80 2024-04-26T04:50:29.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.707+0000 7f867ffff700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f866806e200 0x7f86680706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:29.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.708+0000 7f867ffff700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f866806e200 0x7f86680706b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8670005ea0 tx=0x7f8670005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:29.714 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.713+0000 7f867d7fa700 1 -- 172.21.15.90:0/2771099630 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8674058a50 con 0x7f8680100e80 2024-04-26T04:50:29.912 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:29 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:29.923 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.922+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f86801972e0 con 0x7f866806e200 2024-04-26T04:50:29.933 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.932+0000 7f867d7fa700 1 -- 172.21.15.90:0/2771099630 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f86801972e0 con 0x7f866806e200 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (5m) 54s ago 15m 24.8M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (16m) 54s ago 16m 8627k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (15m) 20s ago 15m 8208k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (2m) 54s ago 16m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (2m) 20s ago 15m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (4m) 54s ago 15m 80.6M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (10m) 54s ago 10m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (10m) 54s ago 10m 16.9M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (10m) 20s ago 10m 17.5M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:50:29.934 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (10m) 20s ago 10m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (6m) 54s ago 17m 505M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (6m) 20s ago 15m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (3m) 54s ago 17m 48.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (3m) 20s ago 14m 42.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (5m) 54s ago 16m 16.5M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (5m) 20s ago 15m 14.7M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (2m) 54s ago 14m 88.4M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (95s) 54s ago 13m 100M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (59s) 54s ago 13m 12.7M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (24s) 20s ago 12m 16.4M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d860fd1e9069 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (12m) 20s ago 12m 238M 2240M 18.2.0 dc2bc1663786 112060ed5503 2024-04-26T04:50:29.935 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (12m) 20s ago 12m 240M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:50:29.936 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (5m) 54s ago 15m 69.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:50:29.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f866806e200 msgr2=0x7f86680706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f866806e200 0x7f86680706b0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8670005ea0 tx=0x7f8670005e30 comp rx=0 tx=0).stop 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 msgr2=0x7f8680196450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680196450 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f867400bda0 tx=0x7f867400be80 comp rx=0 tx=0).stop 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 shutdown_connections 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f866806e200 0x7f86680706b0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f86801004e0 0x7f8680195f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 --2- 172.21.15.90:0/2771099630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8680100e80 0x7f8680196450 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 >> 172.21.15.90:0/2771099630 conn(0x7f86800fbda0 msgr2=0x7f86800fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 shutdown_connections 2024-04-26T04:50:29.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:29.935+0000 7f8686a0e700 1 -- 172.21.15.90:0/2771099630 wait complete. 2024-04-26T04:50:29.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:29 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:30.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 -- 172.21.15.90:0/3206081626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ff350 msgr2=0x7efedc0ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 --2- 172.21.15.90:0/3206081626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ff350 0x7efedc0ff720 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7efecc009910 tx=0x7efecc009c20 comp rx=0 tx=0).stop 2024-04-26T04:50:30.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 -- 172.21.15.90:0/3206081626 shutdown_connections 2024-04-26T04:50:30.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 --2- 172.21.15.90:0/3206081626 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ffc60 0x7efedc10a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 --2- 172.21.15.90:0/3206081626 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ff350 0x7efedc0ff720 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 -- 172.21.15.90:0/3206081626 >> 172.21.15.90:0/3206081626 conn(0x7efedc0748b0 msgr2=0x7efedc074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 -- 172.21.15.90:0/3206081626 shutdown_connections 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.038+0000 7efee24a4700 1 -- 172.21.15.90:0/3206081626 wait complete. 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.039+0000 7efee24a4700 1 Processor -- start 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.039+0000 7efee24a4700 1 -- start start 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.039+0000 7efee24a4700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ff350 0x7efedc195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.039+0000 7efee24a4700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ffc60 0x7efedc1963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.039+0000 7efee24a4700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc196ac0 con 0x7efedc0ffc60 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.039+0000 7efee24a4700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efedc19a850 con 0x7efedc0ff350 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedb7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ffc60 0x7efedc1963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedb7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ffc60 0x7efedc1963e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45380/0 (socket says 172.21.15.90:45380) 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedb7fe700 1 -- 172.21.15.90:0/2303315511 learned_addr learned my addr 172.21.15.90:0/2303315511 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:50:30.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedbfff700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ff350 0x7efedc195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedb7fe700 1 -- 172.21.15.90:0/2303315511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ff350 msgr2=0x7efedc195ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedb7fe700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ff350 0x7efedc195ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedb7fe700 1 -- 172.21.15.90:0/2303315511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efecc0095f0 con 0x7efedc0ffc60 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efedb7fe700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ffc60 0x7efedc1963e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efed000d8d0 tx=0x7efed000dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efed97fa700 1 -- 172.21.15.90:0/2303315511 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed00098a0 con 0x7efedc0ffc60 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efedc19ab30 con 0x7efedc0ffc60 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efed97fa700 1 -- 172.21.15.90:0/2303315511 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efed0010460 con 0x7efedc0ffc60 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efed97fa700 1 -- 172.21.15.90:0/2303315511 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efed000f5f0 con 0x7efedc0ffc60 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.040+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efedc19b080 con 0x7efedc0ffc60 2024-04-26T04:50:30.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.041+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efedc107dd0 con 0x7efedc0ffc60 2024-04-26T04:50:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.042+0000 7efed97fa700 1 -- 172.21.15.90:0/2303315511 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efed0009a00 con 0x7efedc0ffc60 2024-04-26T04:50:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.042+0000 7efed97fa700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efec406e200 0x7efec40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.042+0000 7efed97fa700 1 -- 172.21.15.90:0/2303315511 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7efed008dfc0 con 0x7efedc0ffc60 2024-04-26T04:50:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.042+0000 7efedbfff700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efec406e200 0x7efec40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.043+0000 7efedbfff700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efec406e200 0x7efec40706b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efecc006010 tx=0x7efecc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:30.048 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.047+0000 7efed97fa700 1 -- 172.21.15.90:0/2303315511 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efed0057350 con 0x7efedc0ffc60 2024-04-26T04:50:30.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.334+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7efedc04f2a0 con 0x7efedc0ffc60 2024-04-26T04:50:30.335 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.334+0000 7efed97fa700 1 -- 172.21.15.90:0/2303315511 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7efed005aa10 con 0x7efedc0ffc60 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 2, 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:50:30.336 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:50:30.337 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:50:30.337 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:50:30.337 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:50:30.337 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 6, 2024-04-26T04:50:30.337 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 8 2024-04-26T04:50:30.337 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:50:30.337 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:50:30.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.336+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efec406e200 msgr2=0x7efec40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.336+0000 7efee24a4700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efec406e200 0x7efec40706b0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7efecc006010 tx=0x7efecc01a040 comp rx=0 tx=0).stop 2024-04-26T04:50:30.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.336+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ffc60 msgr2=0x7efedc1963e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.336+0000 7efee24a4700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ffc60 0x7efedc1963e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7efed000d8d0 tx=0x7efed000dbe0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.337+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 shutdown_connections 2024-04-26T04:50:30.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.337+0000 7efee24a4700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efec406e200 0x7efec40706b0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.338 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.337+0000 7efee24a4700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efedc0ff350 0x7efedc195ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.337+0000 7efee24a4700 1 --2- 172.21.15.90:0/2303315511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efedc0ffc60 0x7efedc1963e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.337+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 >> 172.21.15.90:0/2303315511 conn(0x7efedc0748b0 msgr2=0x7efedc104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:30.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.337+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 shutdown_connections 2024-04-26T04:50:30.339 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.337+0000 7efee24a4700 1 -- 172.21.15.90:0/2303315511 wait complete. 2024-04-26T04:50:30.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.439+0000 7f0e67f68700 1 -- 172.21.15.90:0/2804644788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e600ffbd0 msgr2=0x7f0e60100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.439+0000 7f0e67f68700 1 --2- 172.21.15.90:0/2804644788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e600ffbd0 0x7f0e60100040 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009910 tx=0x7f0e50009c20 comp rx=0 tx=0).stop 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.439+0000 7f0e67f68700 1 -- 172.21.15.90:0/2804644788 shutdown_connections 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.439+0000 7f0e67f68700 1 --2- 172.21.15.90:0/2804644788 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e600ffbd0 0x7f0e60100040 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.439+0000 7f0e67f68700 1 --2- 172.21.15.90:0/2804644788 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e60104130 0x7f0e60104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.439+0000 7f0e67f68700 1 -- 172.21.15.90:0/2804644788 >> 172.21.15.90:0/2804644788 conn(0x7f0e60075130 msgr2=0x7f0e60075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 -- 172.21.15.90:0/2804644788 shutdown_connections 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 -- 172.21.15.90:0/2804644788 wait complete. 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 Processor -- start 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 -- start start 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e600ffbd0 0x7f0e60195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e60104130 0x7f0e60196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e60196b70 con 0x7f0e60104130 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.440+0000 7f0e67f68700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e6019a900 con 0x7f0e600ffbd0 2024-04-26T04:50:30.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65503700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e60104130 0x7f0e60196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65503700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e60104130 0x7f0e60196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45388/0 (socket says 172.21.15.90:45388) 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65503700 1 -- 172.21.15.90:0/3033416634 learned_addr learned my addr 172.21.15.90:0/3033416634 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65d04700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e600ffbd0 0x7f0e60195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65503700 1 -- 172.21.15.90:0/3033416634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e600ffbd0 msgr2=0x7f0e60195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65503700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e600ffbd0 0x7f0e60195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65503700 1 -- 172.21.15.90:0/3033416634 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e500095f0 con 0x7f0e60104130 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65d04700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e600ffbd0 0x7f0e60195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e65503700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e60104130 0x7f0e60196490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000bd70 tx=0x7f0e5000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e56ffd700 1 -- 172.21.15.90:0/3033416634 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e5001d070 con 0x7f0e60104130 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e6019ab80 con 0x7f0e60104130 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e56ffd700 1 -- 172.21.15.90:0/3033416634 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0e5000f460 con 0x7f0e60104130 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e56ffd700 1 -- 172.21.15.90:0/3033416634 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0e50017630 con 0x7f0e60104130 2024-04-26T04:50:30.444 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.441+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e6019b070 con 0x7f0e60104130 2024-04-26T04:50:30.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.442+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e601086a0 con 0x7f0e60104130 2024-04-26T04:50:30.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.443+0000 7f0e56ffd700 1 -- 172.21.15.90:0/3033416634 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0e5000f5d0 con 0x7f0e60104130 2024-04-26T04:50:30.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.443+0000 7f0e56ffd700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0e4c06e200 0x7f0e4c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.443+0000 7f0e56ffd700 1 -- 172.21.15.90:0/3033416634 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f0e5008f6f0 con 0x7f0e60104130 2024-04-26T04:50:30.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.443+0000 7f0e65d04700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0e4c06e200 0x7f0e4c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.445 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.444+0000 7f0e65d04700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0e4c06e200 0x7f0e4c0706b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0099d0 tx=0x7f0e5c008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:30.449 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.448+0000 7f0e56ffd700 1 -- 172.21.15.90:0/3033416634 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0e50058a80 con 0x7f0e60104130 2024-04-26T04:50:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.700+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f0e6002cc30 con 0x7f0e60104130 2024-04-26T04:50:30.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.700+0000 7f0e56ffd700 1 -- 172.21.15.90:0/3033416634 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f0e5005c140 con 0x7f0e60104130 2024-04-26T04:50:30.701 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:50:30.701 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:50:30.702 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:50:30.703 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:30.704 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:50:30.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0e4c06e200 msgr2=0x7f0e4c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0e4c06e200 0x7f0e4c0706b0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0e5c0099d0 tx=0x7f0e5c008040 comp rx=0 tx=0).stop 2024-04-26T04:50:30.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e60104130 msgr2=0x7f0e60196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e60104130 0x7f0e60196490 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f0e5000bd70 tx=0x7f0e5000be50 comp rx=0 tx=0).stop 2024-04-26T04:50:30.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 shutdown_connections 2024-04-26T04:50:30.705 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0e4c06e200 0x7f0e4c0706b0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0e600ffbd0 0x7f0e60195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 --2- 172.21.15.90:0/3033416634 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0e60104130 0x7f0e60196490 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.702+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 >> 172.21.15.90:0/3033416634 conn(0x7f0e60075130 msgr2=0x7f0e600fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:30.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.703+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 shutdown_connections 2024-04-26T04:50:30.706 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.703+0000 7f0e67f68700 1 -- 172.21.15.90:0/3033416634 wait complete. 2024-04-26T04:50:30.706 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:50:30.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.809+0000 7fe26646f700 1 -- 172.21.15.90:0/1217565008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 msgr2=0x7fe260103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.811 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.809+0000 7fe26646f700 1 --2- 172.21.15.90:0/1217565008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260103c80 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe250009910 tx=0x7fe250009c20 comp rx=0 tx=0).stop 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 -- 172.21.15.90:0/1217565008 shutdown_connections 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 --2- 172.21.15.90:0/1217565008 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2600ff350 0x7fe2600ff7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 --2- 172.21.15.90:0/1217565008 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260103c80 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 -- 172.21.15.90:0/1217565008 >> 172.21.15.90:0/1217565008 conn(0x7fe2600748b0 msgr2=0x7fe260074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 -- 172.21.15.90:0/1217565008 shutdown_connections 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 -- 172.21.15.90:0/1217565008 wait complete. 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 Processor -- start 2024-04-26T04:50:30.812 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.810+0000 7fe26646f700 1 -- start start 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe26646f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2600ff350 0x7fe260195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe26646f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe26646f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe260196a70 con 0x7fe2601038b0 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe26646f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe26019a800 con 0x7fe2600ff350 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45410/0 (socket says 172.21.15.90:45410) 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25f7fe700 1 -- 172.21.15.90:0/3547702191 learned_addr learned my addr 172.21.15.90:0/3547702191 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25ffff700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2600ff350 0x7fe260195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25f7fe700 1 -- 172.21.15.90:0/3547702191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2600ff350 msgr2=0x7fe260195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:30.814 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25f7fe700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2600ff350 0x7fe260195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25f7fe700 1 -- 172.21.15.90:0/3547702191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2500095f0 con 0x7fe2601038b0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25f7fe700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260196390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe25400dd40 tx=0x7fe25400a530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25d7fa700 1 -- 172.21.15.90:0/3547702191 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe25400ad10 con 0x7fe2601038b0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25d7fa700 1 -- 172.21.15.90:0/3547702191 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe25400ae70 con 0x7fe2601038b0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe26019aae0 con 0x7fe2601038b0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.811+0000 7fe25d7fa700 1 -- 172.21.15.90:0/3547702191 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe2540168a0 con 0x7fe2601038b0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.812+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe26019b030 con 0x7fe2601038b0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.813+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe260109030 con 0x7fe2601038b0 2024-04-26T04:50:30.815 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.813+0000 7fe25d7fa700 1 -- 172.21.15.90:0/3547702191 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe25400e450 con 0x7fe2601038b0 2024-04-26T04:50:30.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.813+0000 7fe25d7fa700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe24806e200 0x7fe2480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:30.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.814+0000 7fe25d7fa700 1 -- 172.21.15.90:0/3547702191 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe25408e030 con 0x7fe2601038b0 2024-04-26T04:50:30.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.814+0000 7fe25ffff700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe24806e200 0x7fe2480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:30.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.814+0000 7fe25ffff700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe24806e200 0x7fe2480706b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe250000c00 tx=0x7fe25001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:30.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:30.818+0000 7fe25d7fa700 1 -- 172.21.15.90:0/3547702191 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe254057470 con 0x7fe2601038b0 2024-04-26T04:50:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: Upgrade: osd.4 is safe to restart 2024-04-26T04:50:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: pgmap v130: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2303315511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:50:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:30 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3033416634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:50:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["4"], "max": 16}]: dispatch 2024-04-26T04:50:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: Upgrade: osd.4 is safe to restart 2024-04-26T04:50:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='client.34314 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='client.34318 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: pgmap v130: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2303315511' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:31.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-26T04:50:31.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:31.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:30 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3033416634' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:50:31.039 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.038+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe26019b320 con 0x7fe24806e200 2024-04-26T04:50:31.040 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.039+0000 7fe25d7fa700 1 -- 172.21.15.90:0/3547702191 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe26019b320 con 0x7fe24806e200 2024-04-26T04:50:31.040 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:50:31.040 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:50:31.040 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:50:31.040 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:50:31.040 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "10/23 daemons upgraded", 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:50:31.041 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe24806e200 msgr2=0x7fe2480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe24806e200 0x7fe2480706b0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe250000c00 tx=0x7fe25001a040 comp rx=0 tx=0).stop 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 msgr2=0x7fe260196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260196390 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fe25400dd40 tx=0x7fe25400a530 comp rx=0 tx=0).stop 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 shutdown_connections 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe24806e200 0x7fe2480706b0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2600ff350 0x7fe260195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 --2- 172.21.15.90:0/3547702191 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2601038b0 0x7fe260196390 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.041+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 >> 172.21.15.90:0/3547702191 conn(0x7fe2600748b0 msgr2=0x7fe2600fe350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.042+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 shutdown_connections 2024-04-26T04:50:31.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.042+0000 7fe26646f700 1 -- 172.21.15.90:0/3547702191 wait complete. 2024-04-26T04:50:31.150 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 -- 172.21.15.90:0/768893110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c100e80 msgr2=0x7f596c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 --2- 172.21.15.90:0/768893110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c100e80 0x7f596c104e50 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f5960009910 tx=0x7f5960009c20 comp rx=0 tx=0).stop 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 -- 172.21.15.90:0/768893110 shutdown_connections 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 --2- 172.21.15.90:0/768893110 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c100e80 0x7f596c104e50 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 --2- 172.21.15.90:0/768893110 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f596c1004e0 0x7f596c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 -- 172.21.15.90:0/768893110 >> 172.21.15.90:0/768893110 conn(0x7f596c0fbda0 msgr2=0x7f596c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 -- 172.21.15.90:0/768893110 shutdown_connections 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 -- 172.21.15.90:0/768893110 wait complete. 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.149+0000 7f5971e5f700 1 Processor -- start 2024-04-26T04:50:31.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f5971e5f700 1 -- start start 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f5971e5f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c1004e0 0x7f596c195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f5971e5f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f596c100e80 0x7f596c196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f5971e5f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c196b30 con 0x7f596c1004e0 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f5971e5f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f596c19a8c0 con 0x7f596c100e80 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c1004e0 0x7f596c195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c1004e0 0x7f596c195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:45428/0 (socket says 172.21.15.90:45428) 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596b7fe700 1 -- 172.21.15.90:0/771989895 learned_addr learned my addr 172.21.15.90:0/771989895 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596affd700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f596c100e80 0x7f596c196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596b7fe700 1 -- 172.21.15.90:0/771989895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f596c100e80 msgr2=0x7f596c196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596b7fe700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f596c100e80 0x7f596c196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596b7fe700 1 -- 172.21.15.90:0/771989895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59600095f0 con 0x7f596c1004e0 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.150+0000 7f596b7fe700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c1004e0 0x7f596c195f10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f595c0060b0 tx=0x7f595c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:31.153 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.151+0000 7f5968ff9700 1 -- 172.21.15.90:0/771989895 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f595c00dda0 con 0x7f596c1004e0 2024-04-26T04:50:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.151+0000 7f5968ff9700 1 -- 172.21.15.90:0/771989895 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f595c00f040 con 0x7f596c1004e0 2024-04-26T04:50:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.151+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f596c19aba0 con 0x7f596c1004e0 2024-04-26T04:50:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.151+0000 7f5968ff9700 1 -- 172.21.15.90:0/771989895 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f595c0146f0 con 0x7f596c1004e0 2024-04-26T04:50:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.151+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f596c19b0c0 con 0x7f596c1004e0 2024-04-26T04:50:31.154 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.152+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f596c108850 con 0x7f596c1004e0 2024-04-26T04:50:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.152+0000 7f5968ff9700 1 -- 172.21.15.90:0/771989895 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f595c015400 con 0x7f596c1004e0 2024-04-26T04:50:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.152+0000 7f5968ff9700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f595406e200 0x7f59540706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:50:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.153+0000 7f5968ff9700 1 -- 172.21.15.90:0/771989895 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f595c08da90 con 0x7f596c1004e0 2024-04-26T04:50:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.153+0000 7f596affd700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f595406e200 0x7f59540706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:50:31.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.153+0000 7f596affd700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f595406e200 0x7f59540706b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5960006010 tx=0x7f596001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:50:31.158 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.157+0000 7f5968ff9700 1 -- 172.21.15.90:0/771989895 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f595c056ed0 con 0x7f596c1004e0 2024-04-26T04:50:31.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.438+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f596c04f2a0 con 0x7f596c1004e0 2024-04-26T04:50:31.440 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.439+0000 7f5968ff9700 1 -- 172.21.15.90:0/771989895 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7f595c05a590 con 0x7f596c1004e0 2024-04-26T04:50:31.440 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_OK 2024-04-26T04:50:31.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f595406e200 msgr2=0x7f59540706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:31.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f595406e200 0x7f59540706b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5960006010 tx=0x7f596001a040 comp rx=0 tx=0).stop 2024-04-26T04:50:31.442 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c1004e0 msgr2=0x7f596c195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c1004e0 0x7f596c195f10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f595c0060b0 tx=0x7f595c00d3b0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 shutdown_connections 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f595406e200 0x7f59540706b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f596c1004e0 0x7f596c195f10 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 --2- 172.21.15.90:0/771989895 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f596c100e80 0x7f596c196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 >> 172.21.15.90:0/771989895 conn(0x7f596c0fbda0 msgr2=0x7f596c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 shutdown_connections 2024-04-26T04:50:31.443 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:50:31.441+0000 7f5971e5f700 1 -- 172.21.15.90:0/771989895 wait complete. 2024-04-26T04:50:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:31 smithi090 ceph-mon[86550]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:31 smithi090 ceph-mon[86550]: Upgrade: Updating osd.4 2024-04-26T04:50:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:31 smithi090 ceph-mon[86550]: Deploying daemon osd.4 on smithi104 2024-04-26T04:50:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:31 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:31 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/771989895' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:50:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:31 smithi104 ceph-mon[64778]: from='client.34322 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:31 smithi104 ceph-mon[64778]: Upgrade: Updating osd.4 2024-04-26T04:50:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:31 smithi104 ceph-mon[64778]: Deploying daemon osd.4 on smithi104 2024-04-26T04:50:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:31 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:31 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/771989895' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:50:32.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:32 smithi090 ceph-mon[86550]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:32.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:32 smithi090 ceph-mon[86550]: pgmap v131: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:33.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:32 smithi104 ceph-mon[64778]: from='client.34334 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:50:33.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:32 smithi104 ceph-mon[64778]: pgmap v131: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:34.003 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:33 smithi104 systemd[1]: Stopping Ceph osd.4 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:50:34.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:34 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[43046]: 2024-04-26T04:50:34.000+0000 7f9e3300c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:50:34.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:34 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[43046]: 2024-04-26T04:50:34.000+0000 7f9e3300c700 -1 osd.4 70 *** Got signal Terminated *** 2024-04-26T04:50:34.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:34 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[43046]: 2024-04-26T04:50:34.000+0000 7f9e3300c700 -1 osd.4 70 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:50:35.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:34 smithi104 ceph-mon[64778]: pgmap v132: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:35.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:34 smithi104 ceph-mon[64778]: osd.4 marked itself down and dead 2024-04-26T04:50:35.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:34 smithi090 ceph-mon[86550]: pgmap v132: 65 pgs: 65 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:35.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:34 smithi090 ceph-mon[86550]: osd.4 marked itself down and dead 2024-04-26T04:50:35.436 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:35 smithi104 podman[79556]: 2024-04-26 04:50:35.183400429 +0000 UTC m=+1.288758523 container died 112060ed55032bb5541be33000e058311a8e3da23eaac155c36301edddea796b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, ceph=True, CEPH_POINT_RELEASE=-18.2.0, org.label-schema.vendor=CentOS) 2024-04-26T04:50:35.703 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:35 smithi104 podman[79556]: 2024-04-26 04:50:35.526250791 +0000 UTC m=+1.631608882 container cleanup 112060ed55032bb5541be33000e058311a8e3da23eaac155c36301edddea796b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, org.label-schema.build-date=20231212, org.label-schema.vendor=CentOS, RELEASE=HEAD, GIT_REPO=https://github.com/ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, maintainer=Guillaume Abrioux ) 2024-04-26T04:50:35.703 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:35 smithi104 bash[79556]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4 2024-04-26T04:50:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:35 smithi104 ceph-mon[64778]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:50:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:35 smithi104 ceph-mon[64778]: osdmap e71: 6 total, 5 up, 6 in 2024-04-26T04:50:36.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:35 smithi104 ceph-mon[64778]: osdmap e72: 6 total, 5 up, 6 in 2024-04-26T04:50:36.012 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:35 smithi104 podman[79586]: 2024-04-26 04:50:35.702474483 +0000 UTC m=+0.515095006 container remove 112060ed55032bb5541be33000e058311a8e3da23eaac155c36301edddea796b (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=HEAD, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, io.buildah.version=1.29.1, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20231212, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-26T04:50:36.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:35 smithi090 ceph-mon[86550]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:50:36.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:35 smithi090 ceph-mon[86550]: osdmap e71: 6 total, 5 up, 6 in 2024-04-26T04:50:36.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:35 smithi090 ceph-mon[86550]: osdmap e72: 6 total, 5 up, 6 in 2024-04-26T04:50:36.448 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:36 smithi104 podman[79682]: 2024-04-26 04:50:36.041899877 +0000 UTC m=+0.017464862 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:50:36.449 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:36 smithi104 podman[79682]: 2024-04-26 04:50:36.219193327 +0000 UTC m=+0.194758284 container create bdcfdfdad2ebad82e994692eab74c7faa83ad6f5a5839d87b3a8fd05fec72ad7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T04:50:36.449 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:36 smithi104 podman[79682]: 2024-04-26 04:50:36.447997479 +0000 UTC m=+0.423562446 container init bdcfdfdad2ebad82e994692eab74c7faa83ad6f5a5839d87b3a8fd05fec72ad7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, ceph=True) 2024-04-26T04:50:36.740 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:36 smithi104 podman[79682]: 2024-04-26 04:50:36.451561417 +0000 UTC m=+0.427126384 container start bdcfdfdad2ebad82e994692eab74c7faa83ad6f5a5839d87b3a8fd05fec72ad7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:50:36.740 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:36 smithi104 podman[79682]: 2024-04-26 04:50:36.547681458 +0000 UTC m=+0.523246431 container attach bdcfdfdad2ebad82e994692eab74c7faa83ad6f5a5839d87b3a8fd05fec72ad7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_CLEAN=True) 2024-04-26T04:50:37.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:36 smithi104 podman[79682]: 2024-04-26 04:50:36.740079573 +0000 UTC m=+0.715644541 container died bdcfdfdad2ebad82e994692eab74c7faa83ad6f5a5839d87b3a8fd05fec72ad7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424) 2024-04-26T04:50:37.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:36 smithi104 ceph-mon[64778]: pgmap v134: 65 pgs: 8 peering, 4 stale+active+clean, 53 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:37.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:36 smithi104 ceph-mon[64778]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-26T04:50:37.187 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:36 smithi090 ceph-mon[86550]: pgmap v134: 65 pgs: 8 peering, 4 stale+active+clean, 53 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:37.187 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:36 smithi090 ceph-mon[86550]: Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2024-04-26T04:50:37.721 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 podman[79708]: 2024-04-26 04:50:37.285020465 +0000 UTC m=+0.539860979 container remove bdcfdfdad2ebad82e994692eab74c7faa83ad6f5a5839d87b3a8fd05fec72ad7 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T04:50:37.721 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.4.service: Deactivated successfully. 2024-04-26T04:50:37.721 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 systemd[1]: Stopped Ceph osd.4 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:50:37.721 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.4.service: Consumed 58.925s CPU time. 2024-04-26T04:50:37.722 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 systemd[1]: Starting Ceph osd.4 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:50:37.972 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:37 smithi104 ceph-mon[64778]: Health check failed: Degraded data redundancy: 45/348 objects degraded (12.931%), 12 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:37.973 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 podman[79811]: 2024-04-26 04:50:37.623614812 +0000 UTC m=+0.017502438 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:50:37.973 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 podman[79811]: 2024-04-26 04:50:37.736087704 +0000 UTC m=+0.129975328 container create e53948c6a8996419018ca5fa9bb96180a56dd5f776e5f23715d66284bdcc1c76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-26T04:50:37.973 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 podman[79811]: 2024-04-26 04:50:37.972156701 +0000 UTC m=+0.366044321 container init e53948c6a8996419018ca5fa9bb96180a56dd5f776e5f23715d66284bdcc1c76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-26T04:50:38.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:37 smithi090 ceph-mon[86550]: Health check failed: Degraded data redundancy: 45/348 objects degraded (12.931%), 12 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:38.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:37 smithi104 podman[79811]: 2024-04-26 04:50:37.975717254 +0000 UTC m=+0.369604880 container start e53948c6a8996419018ca5fa9bb96180a56dd5f776e5f23715d66284bdcc1c76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD) 2024-04-26T04:50:38.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 podman[79811]: 2024-04-26 04:50:38.05537044 +0000 UTC m=+0.449258075 container attach e53948c6a8996419018ca5fa9bb96180a56dd5f776e5f23715d66284bdcc1c76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate[79825]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate[79825]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 bash[79811]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 bash[79811]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-4 --no-mon-config --dev /dev/mapper/vg_nvme-lv_3 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 bash[79811]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 bash[79811]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 bash[79811]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-26T04:50:38.753 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 bash[79811]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:50:38.754 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 bash[79811]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-26T04:50:38.754 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate[79825]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_3 2024-04-26T04:50:38.754 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate[79825]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-2 2024-04-26T04:50:38.754 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate[79825]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-4/block 2024-04-26T04:50:38.754 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate[79825]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-26T04:50:38.754 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate[79825]: --> ceph-volume raw activate successful for osd ID: 4 2024-04-26T04:50:38.754 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:38 smithi104 podman[79811]: 2024-04-26 04:50:38.447949312 +0000 UTC m=+0.841836935 container died e53948c6a8996419018ca5fa9bb96180a56dd5f776e5f23715d66284bdcc1c76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, ceph=True) 2024-04-26T04:50:39.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:38 smithi104 ceph-mon[64778]: pgmap v136: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/348 objects degraded (12.931%) 2024-04-26T04:50:39.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:38 smithi090 ceph-mon[86550]: pgmap v136: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/348 objects degraded (12.931%) 2024-04-26T04:50:39.327 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 podman[79957]: 2024-04-26 04:50:39.011129964 +0000 UTC m=+0.558335969 container remove e53948c6a8996419018ca5fa9bb96180a56dd5f776e5f23715d66284bdcc1c76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-activate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb) 2024-04-26T04:50:39.327 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 podman[80018]: 2024-04-26 04:50:39.125045878 +0000 UTC m=+0.018085367 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:50:39.328 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 podman[80018]: 2024-04-26 04:50:39.22873663 +0000 UTC m=+0.121776128 container create fbfd5c1c44eb3edbdbb79985ee001e116d8ba5b05a63812a5a1575e528a86e14 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T04:50:39.761 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 podman[80018]: 2024-04-26 04:50:39.406106413 +0000 UTC m=+0.299145880 container init fbfd5c1c44eb3edbdbb79985ee001e116d8ba5b05a63812a5a1575e528a86e14 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T04:50:39.761 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 podman[80018]: 2024-04-26 04:50:39.409502108 +0000 UTC m=+0.302541563 container start fbfd5c1c44eb3edbdbb79985ee001e116d8ba5b05a63812a5a1575e528a86e14 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:50:39.762 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 ceph-osd[80037]: -- 172.21.15.104:0/149192135 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556e81b27680 con 0x556e81af0c00 2024-04-26T04:50:39.762 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 bash[80018]: fbfd5c1c44eb3edbdbb79985ee001e116d8ba5b05a63812a5a1575e528a86e14 2024-04-26T04:50:39.762 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 systemd[1]: Started Ceph osd.4 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:50:40.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:39 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:50:39.952+0000 7f43f9c307c0 -1 Falling back to public interface 2024-04-26T04:50:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:40 smithi090 ceph-mon[86550]: pgmap v137: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/348 objects degraded (12.931%) 2024-04-26T04:50:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:40 smithi104 ceph-mon[64778]: pgmap v137: 65 pgs: 12 active+undersized+degraded, 8 peering, 14 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 45/348 objects degraded (12.931%) 2024-04-26T04:50:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:41 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:41.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:41 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:41 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:42.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:41 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:42 smithi090 ceph-mon[86550]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/348 objects degraded (16.954%) 2024-04-26T04:50:42.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:42 smithi090 ceph-mon[86550]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:50:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:42 smithi104 ceph-mon[64778]: pgmap v138: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/348 objects degraded (16.954%) 2024-04-26T04:50:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:42 smithi104 ceph-mon[64778]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 2 pgs peering) 2024-04-26T04:50:44.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:43 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:44.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:43 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:43 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:43 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:44.784 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:44 smithi104 ceph-mon[64778]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/348 objects degraded (16.954%) 2024-04-26T04:50:44.785 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:44 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:44.785 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:44 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:44.785 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:44 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:50:44.531+0000 7f43f9c307c0 -1 osd.4 0 read_superblock omap replica is missing. 2024-04-26T04:50:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:44 smithi090 ceph-mon[86550]: pgmap v139: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/348 objects degraded (16.954%) 2024-04-26T04:50:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:44 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:44 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:45.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:45 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:50:45.022+0000 7f43f9c307c0 -1 osd.4 70 log_to_monitors true 2024-04-26T04:50:45.790 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:45 smithi104 ceph-mon[64778]: from='osd.4 [v2:172.21.15.104:6808/319614461,v1:172.21.15.104:6809/319614461]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:50:45.790 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:45 smithi104 ceph-mon[64778]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:50:46.052 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:50:45 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:50:45.789+0000 7f43eb088700 -1 osd.4 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:50:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:45 smithi090 ceph-mon[86550]: from='osd.4 [v2:172.21.15.104:6808/319614461,v1:172.21.15.104:6809/319614461]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:50:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:45 smithi090 ceph-mon[86550]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-26T04:50:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:46 smithi104 ceph-mon[64778]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/348 objects degraded (16.954%) 2024-04-26T04:50:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:46 smithi104 ceph-mon[64778]: Health check update: Degraded data redundancy: 59/348 objects degraded (16.954%), 16 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:46 smithi104 ceph-mon[64778]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:50:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:46 smithi104 ceph-mon[64778]: osdmap e73: 6 total, 5 up, 6 in 2024-04-26T04:50:47.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:46 smithi104 ceph-mon[64778]: from='osd.4 [v2:172.21.15.104:6808/319614461,v1:172.21.15.104:6809/319614461]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:47.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:46 smithi104 ceph-mon[64778]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:46 smithi090 ceph-mon[86550]: pgmap v140: 65 pgs: 16 active+undersized+degraded, 18 active+undersized, 31 active+clean; 293 MiB data, 1.3 GiB used, 535 GiB / 536 GiB avail; 59/348 objects degraded (16.954%) 2024-04-26T04:50:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:46 smithi090 ceph-mon[86550]: Health check update: Degraded data redundancy: 59/348 objects degraded (16.954%), 16 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:46 smithi090 ceph-mon[86550]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-26T04:50:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:46 smithi090 ceph-mon[86550]: osdmap e73: 6 total, 5 up, 6 in 2024-04-26T04:50:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:46 smithi090 ceph-mon[86550]: from='osd.4 [v2:172.21.15.104:6808/319614461,v1:172.21.15.104:6809/319614461]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:46 smithi090 ceph-mon[86550]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:50:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:50:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: osd.4 [v2:172.21.15.104:6808/319614461,v1:172.21.15.104:6809/319614461] boot 2024-04-26T04:50:47.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: osdmap e74: 6 total, 6 up, 6 in 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:47.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:47 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:50:48.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:50:48.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: osd.4 [v2:172.21.15.104:6808/319614461,v1:172.21.15.104:6809/319614461] boot 2024-04-26T04:50:48.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: osdmap e74: 6 total, 6 up, 6 in 2024-04-26T04:50:48.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:50:48.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:47 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:50:49.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:48 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:50:49.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:48 smithi090 ceph-mon[86550]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-26T04:50:49.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:48 smithi090 ceph-mon[86550]: pgmap v143: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/348 objects degraded (9.770%) 2024-04-26T04:50:49.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:48 smithi090 ceph-mon[86550]: osdmap e75: 6 total, 6 up, 6 in 2024-04-26T04:50:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:48 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:50:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:48 smithi104 ceph-mon[64778]: Upgrade: unsafe to stop osd(s) at this time (11 PGs are or would become offline) 2024-04-26T04:50:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:48 smithi104 ceph-mon[64778]: pgmap v143: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/348 objects degraded (9.770%) 2024-04-26T04:50:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:48 smithi104 ceph-mon[64778]: osdmap e75: 6 total, 6 up, 6 in 2024-04-26T04:50:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:50 smithi090 ceph-mon[86550]: pgmap v145: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/348 objects degraded (9.770%) 2024-04-26T04:50:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:50 smithi090 ceph-mon[86550]: Health check update: Degraded data redundancy: 34/348 objects degraded (9.770%), 11 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:50 smithi104 ceph-mon[64778]: pgmap v145: 65 pgs: 10 peering, 11 active+undersized+degraded, 13 active+undersized, 31 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 34/348 objects degraded (9.770%) 2024-04-26T04:50:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:50 smithi104 ceph-mon[64778]: Health check update: Degraded data redundancy: 34/348 objects degraded (9.770%), 11 pgs degraded (PG_DEGRADED) 2024-04-26T04:50:52.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:51 smithi090 ceph-mon[86550]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/348 objects degraded (9.770%), 11 pgs degraded) 2024-04-26T04:50:52.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:51 smithi090 ceph-mon[86550]: Cluster is now healthy 2024-04-26T04:50:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:51 smithi104 ceph-mon[64778]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 34/348 objects degraded (9.770%), 11 pgs degraded) 2024-04-26T04:50:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:51 smithi104 ceph-mon[64778]: Cluster is now healthy 2024-04-26T04:50:53.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:52 smithi090 ceph-mon[86550]: pgmap v146: 65 pgs: 10 peering, 55 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:52 smithi104 ceph-mon[64778]: pgmap v146: 65 pgs: 10 peering, 55 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:54 smithi090 ceph-mon[86550]: pgmap v147: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:54 smithi104 ceph-mon[64778]: pgmap v147: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:56 smithi090 ceph-mon[86550]: pgmap v148: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:56.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:56.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:56 smithi104 ceph-mon[64778]: pgmap v148: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:50:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:50:58.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:50:58 smithi090 ceph-mon[86550]: pgmap v149: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:50:59.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:50:58 smithi104 ceph-mon[64778]: pgmap v149: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:00.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:00 smithi090 ceph-mon[86550]: pgmap v150: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:01.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:00 smithi104 ceph-mon[64778]: pgmap v150: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 -- 172.21.15.90:0/742440522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 msgr2=0x7fe748103c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/742440522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe748103c20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe738009910 tx=0x7fe738009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 -- 172.21.15.90:0/742440522 shutdown_connections 2024-04-26T04:51:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/742440522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe748103c20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/742440522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7480ff2b0 0x7fe7480ff680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 -- 172.21.15.90:0/742440522 >> 172.21.15.90:0/742440522 conn(0x7fe7480fab50 msgr2=0x7fe7480fcf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:01.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 -- 172.21.15.90:0/742440522 shutdown_connections 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.560+0000 7fe74d1a5700 1 -- 172.21.15.90:0/742440522 wait complete. 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.561+0000 7fe74d1a5700 1 Processor -- start 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.561+0000 7fe74d1a5700 1 -- start start 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74d1a5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7480ff2b0 0x7fe748196bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74d1a5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe7481970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74d1a5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748197710 con 0x7fe7480ffc50 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74d1a5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe748197880 con 0x7fe7480ff2b0 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe7481970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe7481970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58316/0 (socket says 172.21.15.90:58316) 2024-04-26T04:51:01.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74659c700 1 -- 172.21.15.90:0/602166087 learned_addr learned my addr 172.21.15.90:0/602166087 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe746d9d700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7480ff2b0 0x7fe748196bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74659c700 1 -- 172.21.15.90:0/602166087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7480ff2b0 msgr2=0x7fe748196bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74659c700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7480ff2b0 0x7fe748196bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74659c700 1 -- 172.21.15.90:0/602166087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380095f0 con 0x7fe7480ffc50 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74659c700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe7481970f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7380092e0 tx=0x7fe73800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe73ffff700 1 -- 172.21.15.90:0/602166087 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe73801d070 con 0x7fe7480ffc50 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe74819fc20 con 0x7fe7480ffc50 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe73ffff700 1 -- 172.21.15.90:0/602166087 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe738005dc0 con 0x7fe7480ffc50 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.562+0000 7fe73ffff700 1 -- 172.21.15.90:0/602166087 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe738017400 con 0x7fe7480ffc50 2024-04-26T04:51:01.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.563+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7481a0090 con 0x7fe7480ffc50 2024-04-26T04:51:01.567 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.564+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe74804f2a0 con 0x7fe7480ffc50 2024-04-26T04:51:01.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.564+0000 7fe73ffff700 1 -- 172.21.15.90:0/602166087 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe73800f6f0 con 0x7fe7480ffc50 2024-04-26T04:51:01.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.565+0000 7fe73ffff700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe73406e200 0x7fe7340706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:01.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.565+0000 7fe73ffff700 1 -- 172.21.15.90:0/602166087 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fe73808f4c0 con 0x7fe7480ffc50 2024-04-26T04:51:01.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.565+0000 7fe746d9d700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe73406e200 0x7fe7340706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:01.568 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.565+0000 7fe746d9d700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe73406e200 0x7fe7340706b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe730005950 tx=0x7fe73000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:01.571 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.569+0000 7fe73ffff700 1 -- 172.21.15.90:0/602166087 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe738058850 con 0x7fe7480ffc50 2024-04-26T04:51:01.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.795+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7480fce60 con 0x7fe73406e200 2024-04-26T04:51:01.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.796+0000 7fe73ffff700 1 -- 172.21.15.90:0/602166087 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7fe7480fce60 con 0x7fe73406e200 2024-04-26T04:51:01.799 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe73406e200 msgr2=0x7fe7340706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe73406e200 0x7fe7340706b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe730005950 tx=0x7fe73000b410 comp rx=0 tx=0).stop 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 msgr2=0x7fe7481970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe7481970f0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fe7380092e0 tx=0x7fe73800bd70 comp rx=0 tx=0).stop 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 shutdown_connections 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe73406e200 0x7fe7340706b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe7480ff2b0 0x7fe748196bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 --2- 172.21.15.90:0/602166087 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7480ffc50 0x7fe7481970f0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 >> 172.21.15.90:0/602166087 conn(0x7fe7480fab50 msgr2=0x7fe7480fc0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.798+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 shutdown_connections 2024-04-26T04:51:01.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.799+0000 7fe74d1a5700 1 -- 172.21.15.90:0/602166087 wait complete. 2024-04-26T04:51:01.810 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.903+0000 7f787e35f700 1 -- 172.21.15.90:0/499900983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 msgr2=0x7f78781008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.903+0000 7f787e35f700 1 --2- 172.21.15.90:0/499900983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f78781008b0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f7868009910 tx=0x7f7868009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 -- 172.21.15.90:0/499900983 shutdown_connections 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 --2- 172.21.15.90:0/499900983 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7878100e80 0x7f7878104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 --2- 172.21.15.90:0/499900983 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f78781008b0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 -- 172.21.15.90:0/499900983 >> 172.21.15.90:0/499900983 conn(0x7f78780fbda0 msgr2=0x7f78780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 -- 172.21.15.90:0/499900983 shutdown_connections 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 -- 172.21.15.90:0/499900983 wait complete. 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 Processor -- start 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.904+0000 7f787e35f700 1 -- start start 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f787e35f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f7878195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:01.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f787e35f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7878100e80 0x7f7878196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f787e35f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878196a80 con 0x7f78781004e0 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f787e35f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7878196bc0 con 0x7f7878100e80 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f7877fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f7878195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f7877fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f7878195ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58324/0 (socket says 172.21.15.90:58324) 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f7877fff700 1 -- 172.21.15.90:0/3799490434 learned_addr learned my addr 172.21.15.90:0/3799490434 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f78777fe700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7878100e80 0x7f7878196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f7877fff700 1 -- 172.21.15.90:0/3799490434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7878100e80 msgr2=0x7f7878196430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f7877fff700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7878100e80 0x7f7878196430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f7877fff700 1 -- 172.21.15.90:0/3799490434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78680095f0 con 0x7f78781004e0 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f7877fff700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f7878195ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7868006010 tx=0x7f786800be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f78757fa700 1 -- 172.21.15.90:0/3799490434 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f786801d070 con 0x7f78781004e0 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787819a9b0 con 0x7f78781004e0 2024-04-26T04:51:01.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f78757fa700 1 -- 172.21.15.90:0/3799490434 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7868004fb0 con 0x7f78781004e0 2024-04-26T04:51:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f78757fa700 1 -- 172.21.15.90:0/3799490434 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7868022ea0 con 0x7f78781004e0 2024-04-26T04:51:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.905+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787819aea0 con 0x7f78781004e0 2024-04-26T04:51:01.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.906+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78781087f0 con 0x7f78781004e0 2024-04-26T04:51:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.907+0000 7f78757fa700 1 -- 172.21.15.90:0/3799490434 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7868022470 con 0x7f78781004e0 2024-04-26T04:51:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.907+0000 7f78757fa700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f786006e200 0x7f78600706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.907+0000 7f78757fa700 1 -- 172.21.15.90:0/3799490434 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f786808f590 con 0x7f78781004e0 2024-04-26T04:51:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.907+0000 7f78777fe700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f786006e200 0x7f78600706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:01.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.908+0000 7f78777fe700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f786006e200 0x7f78600706b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f786c005ea0 tx=0x7f786c005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:01.913 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:01.912+0000 7f78757fa700 1 -- 172.21.15.90:0/3799490434 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7868058a50 con 0x7f78781004e0 2024-04-26T04:51:02.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.136+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f787819b110 con 0x7f786006e200 2024-04-26T04:51:02.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.137+0000 7f78757fa700 1 -- 172.21.15.90:0/3799490434 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7f787819b110 con 0x7f786006e200 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.139+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f786006e200 msgr2=0x7f78600706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.139+0000 7f787e35f700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f786006e200 0x7f78600706b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f786c005ea0 tx=0x7f786c005e30 comp rx=0 tx=0).stop 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.139+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 msgr2=0x7f7878195ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f7878195ef0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f7868006010 tx=0x7f786800be20 comp rx=0 tx=0).stop 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 shutdown_connections 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f786006e200 0x7f78600706b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78781004e0 0x7f7878195ef0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 --2- 172.21.15.90:0/3799490434 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7878100e80 0x7f7878196430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.141 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 >> 172.21.15.90:0/3799490434 conn(0x7f78780fbda0 msgr2=0x7f78780fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:02.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 shutdown_connections 2024-04-26T04:51:02.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.140+0000 7f787e35f700 1 -- 172.21.15.90:0/3799490434 wait complete. 2024-04-26T04:51:02.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:02.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:02 smithi090 ceph-mon[86550]: pgmap v151: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:02.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:02 smithi090 ceph-mon[86550]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 -- 172.21.15.90:0/2214889520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 msgr2=0x7f45e0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 --2- 172.21.15.90:0/2214889520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0104e50 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f45d4009910 tx=0x7f45d4009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 -- 172.21.15.90:0/2214889520 shutdown_connections 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 --2- 172.21.15.90:0/2214889520 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0104e50 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 --2- 172.21.15.90:0/2214889520 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e01004e0 0x7f45e01008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 -- 172.21.15.90:0/2214889520 >> 172.21.15.90:0/2214889520 conn(0x7f45e00fbda0 msgr2=0x7f45e00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 -- 172.21.15.90:0/2214889520 shutdown_connections 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.245+0000 7f45e7f05700 1 -- 172.21.15.90:0/2214889520 wait complete. 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e7f05700 1 Processor -- start 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e7f05700 1 -- start start 2024-04-26T04:51:02.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e7f05700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e01004e0 0x7f45e0195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e7f05700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e7f05700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e0196b50 con 0x7f45e0100e80 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e7f05700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45e019a8e0 con 0x7f45e01004e0 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e54a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e54a0700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58334/0 (socket says 172.21.15.90:58334) 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e54a0700 1 -- 172.21.15.90:0/1782302494 learned_addr learned my addr 172.21.15.90:0/1782302494 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e5ca1700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e01004e0 0x7f45e0195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e54a0700 1 -- 172.21.15.90:0/1782302494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e01004e0 msgr2=0x7f45e0195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e54a0700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e01004e0 0x7f45e0195f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e54a0700 1 -- 172.21.15.90:0/1782302494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45d40095f0 con 0x7f45e0100e80 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.246+0000 7f45e54a0700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0196470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45d40092e0 tx=0x7f45d400bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:02.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.247+0000 7f45deffd700 1 -- 172.21.15.90:0/1782302494 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d401d070 con 0x7f45e0100e80 2024-04-26T04:51:02.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.247+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45e019ab60 con 0x7f45e0100e80 2024-04-26T04:51:02.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.247+0000 7f45deffd700 1 -- 172.21.15.90:0/1782302494 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f45d4005dc0 con 0x7f45e0100e80 2024-04-26T04:51:02.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.247+0000 7f45deffd700 1 -- 172.21.15.90:0/1782302494 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f45d4017400 con 0x7f45e0100e80 2024-04-26T04:51:02.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.247+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45e019b050 con 0x7f45e0100e80 2024-04-26T04:51:02.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.247+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45e0108850 con 0x7f45e0100e80 2024-04-26T04:51:02.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.248+0000 7f45deffd700 1 -- 172.21.15.90:0/1782302494 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f45d400f6f0 con 0x7f45e0100e80 2024-04-26T04:51:02.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.248+0000 7f45deffd700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f45c806e200 0x7f45c80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.249+0000 7f45deffd700 1 -- 172.21.15.90:0/1782302494 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f45d408f4c0 con 0x7f45e0100e80 2024-04-26T04:51:02.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.249+0000 7f45e5ca1700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f45c806e200 0x7f45c80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.251 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.249+0000 7f45e5ca1700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f45c806e200 0x7f45c80706b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005950 tx=0x7f45d000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:02.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.253+0000 7f45deffd700 1 -- 172.21.15.90:0/1782302494 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f45d4058850 con 0x7f45e0100e80 2024-04-26T04:51:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:02 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:02 smithi104 ceph-mon[64778]: pgmap v151: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:02 smithi104 ceph-mon[64778]: from='client.34344 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:02.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.462+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f45e019b270 con 0x7f45c806e200 2024-04-26T04:51:02.474 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.473+0000 7f45deffd700 1 -- 172.21.15.90:0/1782302494 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f45e019b270 con 0x7f45c806e200 2024-04-26T04:51:02.474 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (5m) 86s ago 16m 24.8M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (16m) 86s ago 16m 8627k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (15m) 19s ago 15m 8446k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (3m) 86s ago 16m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (3m) 19s ago 15m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (5m) 86s ago 16m 80.6M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (11m) 86s ago 11m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (11m) 86s ago 11m 16.9M - 18.2.0 dc2bc1663786 59d021254c1e 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (11m) 19s ago 11m 17.7M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (11m) 19s ago 11m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (7m) 86s ago 17m 505M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:51:02.475 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (6m) 19s ago 15m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (4m) 86s ago 18m 48.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (3m) 19s ago 15m 42.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (6m) 86s ago 16m 16.5M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (6m) 19s ago 15m 14.6M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (2m) 86s ago 14m 88.4M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (2m) 86s ago 14m 100M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (92s) 86s ago 13m 12.7M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (57s) 19s ago 13m 107M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d860fd1e9069 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (23s) 19s ago 13m 11.6M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 fbfd5c1c44eb 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (12m) 19s ago 12m 240M 2240M 18.2.0 dc2bc1663786 99079694ead0 2024-04-26T04:51:02.476 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (6m) 86s ago 16m 69.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f45c806e200 msgr2=0x7f45c80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f45c806e200 0x7f45c80706b0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f45d0005950 tx=0x7f45d000b410 comp rx=0 tx=0).stop 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 msgr2=0x7f45e0196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0196470 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f45d40092e0 tx=0x7f45d400bd70 comp rx=0 tx=0).stop 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 shutdown_connections 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f45c806e200 0x7f45c80706b0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f45e01004e0 0x7f45e0195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 --2- 172.21.15.90:0/1782302494 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f45e0100e80 0x7f45e0196470 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 >> 172.21.15.90:0/1782302494 conn(0x7f45e00fbda0 msgr2=0x7f45e00fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.476+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 shutdown_connections 2024-04-26T04:51:02.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.477+0000 7f45e7f05700 1 -- 172.21.15.90:0/1782302494 wait complete. 2024-04-26T04:51:02.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 -- 172.21.15.90:0/3768350503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c1004e0 msgr2=0x7fab5c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.593 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 --2- 172.21.15.90:0/3768350503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c1004e0 0x7fab5c1008b0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fab5400b0a0 tx=0x7fab5400b3b0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 -- 172.21.15.90:0/3768350503 shutdown_connections 2024-04-26T04:51:02.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 --2- 172.21.15.90:0/3768350503 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5c100e80 0x7fab5c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 --2- 172.21.15.90:0/3768350503 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c1004e0 0x7fab5c1008b0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 -- 172.21.15.90:0/3768350503 >> 172.21.15.90:0/3768350503 conn(0x7fab5c0fbda0 msgr2=0x7fab5c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:02.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 -- 172.21.15.90:0/3768350503 shutdown_connections 2024-04-26T04:51:02.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.586+0000 7fab64266700 1 -- 172.21.15.90:0/3768350503 wait complete. 2024-04-26T04:51:02.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab64266700 1 Processor -- start 2024-04-26T04:51:02.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab64266700 1 -- start start 2024-04-26T04:51:02.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab64266700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5c1004e0 0x7fab5c195f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab64266700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c100e80 0x7fab5c1964b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab64266700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c196b90 con 0x7fab5c100e80 2024-04-26T04:51:02.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab64266700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab5c19a920 con 0x7fab5c1004e0 2024-04-26T04:51:02.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab61801700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c100e80 0x7fab5c1964b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab61801700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c100e80 0x7fab5c1964b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58350/0 (socket says 172.21.15.90:58350) 2024-04-26T04:51:02.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab61801700 1 -- 172.21.15.90:0/3642807792 learned_addr learned my addr 172.21.15.90:0/3642807792 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:02.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab62002700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5c1004e0 0x7fab5c195f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab61801700 1 -- 172.21.15.90:0/3642807792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5c1004e0 msgr2=0x7fab5c195f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab61801700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5c1004e0 0x7fab5c195f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.587+0000 7fab61801700 1 -- 172.21.15.90:0/3642807792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab54009d00 con 0x7fab5c100e80 2024-04-26T04:51:02.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.588+0000 7fab61801700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c100e80 0x7fab5c1964b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00d900 tx=0x7fab4c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:02.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.588+0000 7fab52ffd700 1 -- 172.21.15.90:0/3642807792 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab4c009900 con 0x7fab5c100e80 2024-04-26T04:51:02.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.588+0000 7fab52ffd700 1 -- 172.21.15.90:0/3642807792 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fab4c010460 con 0x7fab5c100e80 2024-04-26T04:51:02.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.588+0000 7fab52ffd700 1 -- 172.21.15.90:0/3642807792 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fab4c00f5f0 con 0x7fab5c100e80 2024-04-26T04:51:02.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.588+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab5c19ac00 con 0x7fab5c100e80 2024-04-26T04:51:02.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.588+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab5c19b150 con 0x7fab5c100e80 2024-04-26T04:51:02.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.589+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab5c1087f0 con 0x7fab5c100e80 2024-04-26T04:51:02.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.589+0000 7fab52ffd700 1 -- 172.21.15.90:0/3642807792 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fab4c0105d0 con 0x7fab5c100e80 2024-04-26T04:51:02.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.590+0000 7fab52ffd700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fab4806e200 0x7fab480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.590+0000 7fab52ffd700 1 -- 172.21.15.90:0/3642807792 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7fab4c08dcb0 con 0x7fab5c100e80 2024-04-26T04:51:02.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.590+0000 7fab62002700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fab4806e200 0x7fab480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.590+0000 7fab62002700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fab4806e200 0x7fab480706b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab54000f80 tx=0x7fab540061f0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:02.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.594+0000 7fab52ffd700 1 -- 172.21.15.90:0/3642807792 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fab4c057040 con 0x7fab5c100e80 2024-04-26T04:51:02.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.887+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fab5c19b370 con 0x7fab5c100e80 2024-04-26T04:51:02.888 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.887+0000 7fab52ffd700 1 -- 172.21.15.90:0/3642807792 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+791 (secure 0 0 0) 0x7fab4c05a700 con 0x7fab5c100e80 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 5 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:51:02.889 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 4 2024-04-26T04:51:02.890 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:02.890 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:51:02.890 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 5, 2024-04-26T04:51:02.890 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 9 2024-04-26T04:51:02.890 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:51:02.890 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fab4806e200 msgr2=0x7fab480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fab4806e200 0x7fab480706b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fab54000f80 tx=0x7fab540061f0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c100e80 msgr2=0x7fab5c1964b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c100e80 0x7fab5c1964b0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fab4c00d900 tx=0x7fab4c00dc10 comp rx=0 tx=0).stop 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 shutdown_connections 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fab4806e200 0x7fab480706b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fab5c1004e0 0x7fab5c195f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 --2- 172.21.15.90:0/3642807792 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fab5c100e80 0x7fab5c1964b0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.890+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 >> 172.21.15.90:0/3642807792 conn(0x7fab5c0fbda0 msgr2=0x7fab5c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.891+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 shutdown_connections 2024-04-26T04:51:02.892 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.891+0000 7fab64266700 1 -- 172.21.15.90:0/3642807792 wait complete. 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.994+0000 7f5b3ad35700 1 -- 172.21.15.90:0/2170003309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ffd50 msgr2=0x7f5b34103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.994+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/2170003309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ffd50 0x7f5b34103d20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5b20009910 tx=0x7f5b20009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 -- 172.21.15.90:0/2170003309 shutdown_connections 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/2170003309 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ffd50 0x7f5b34103d20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/2170003309 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ff3b0 0x7f5b340ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 -- 172.21.15.90:0/2170003309 >> 172.21.15.90:0/2170003309 conn(0x7f5b340fac90 msgr2=0x7f5b340fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 -- 172.21.15.90:0/2170003309 shutdown_connections 2024-04-26T04:51:02.996 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 -- 172.21.15.90:0/2170003309 wait complete. 2024-04-26T04:51:02.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 Processor -- start 2024-04-26T04:51:02.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 -- start start 2024-04-26T04:51:02.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:02.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b34195a20 con 0x7f5b340ff3b0 2024-04-26T04:51:02.998 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.995+0000 7f5b3ad35700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b341997b0 con 0x7f5b340ffd50 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b38ad1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b38ad1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34194e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58378/0 (socket says 172.21.15.90:58378) 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b38ad1700 1 -- 172.21.15.90:0/417507930 learned_addr learned my addr 172.21.15.90:0/417507930 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b33fff700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b38ad1700 1 -- 172.21.15.90:0/417507930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 msgr2=0x7f5b34195340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b38ad1700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34195340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b38ad1700 1 -- 172.21.15.90:0/417507930 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b200095f0 con 0x7f5b340ff3b0 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b33fff700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b38ad1700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34194e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b280076b0 tx=0x7f5b280079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b31ffb700 1 -- 172.21.15.90:0/417507930 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b28004020 con 0x7f5b340ff3b0 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b31ffb700 1 -- 172.21.15.90:0/417507930 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5b28004830 con 0x7f5b340ff3b0 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b34199a30 con 0x7f5b340ff3b0 2024-04-26T04:51:02.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b31ffb700 1 -- 172.21.15.90:0/417507930 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5b280055d0 con 0x7f5b340ff3b0 2024-04-26T04:51:03.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.996+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b34199f80 con 0x7f5b340ff3b0 2024-04-26T04:51:03.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.997+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3404f2a0 con 0x7f5b340ff3b0 2024-04-26T04:51:03.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.998+0000 7f5b31ffb700 1 -- 172.21.15.90:0/417507930 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5b280049a0 con 0x7f5b340ff3b0 2024-04-26T04:51:03.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.998+0000 7f5b31ffb700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2406e200 0x7f5b240706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:03.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.998+0000 7f5b31ffb700 1 -- 172.21.15.90:0/417507930 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f5b2808d820 con 0x7f5b340ff3b0 2024-04-26T04:51:03.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.998+0000 7f5b33fff700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2406e200 0x7f5b240706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:03.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:02.999+0000 7f5b33fff700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2406e200 0x7f5b240706b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5b34196420 tx=0x7f5b2000b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:03.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.003+0000 7f5b31ffb700 1 -- 172.21.15.90:0/417507930 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5b28056bb0 con 0x7f5b340ff3b0 2024-04-26T04:51:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:03 smithi090 ceph-mon[86550]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:03 smithi090 ceph-mon[86550]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:03 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:51:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:03 smithi090 ceph-mon[86550]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:51:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:03 smithi090 ceph-mon[86550]: Upgrade: osd.5 is safe to restart 2024-04-26T04:51:03.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:03 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3642807792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.253+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f5b3419a240 con 0x7f5b340ff3b0 2024-04-26T04:51:03.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.253+0000 7f5b31ffb700 1 -- 172.21.15.90:0/417507930 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 21 v21) v1 ==== 76+0+1886 (secure 0 0 0) 0x7f5b2805a270 con 0x7f5b340ff3b0 2024-04-26T04:51:03.254 INFO:teuthology.orchestra.run.smithi090.stdout:e21 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 21 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:46:31.691772+0000 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:51:03.255 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 46 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=14486,1=24273} 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:51:03.256 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{0:14486} state up:active seq 6 join_fscid=1 addr [v2:172.21.15.90:6828/370183803,v1:172.21.15.90:6829/370183803] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{1:24273} state up:active seq 4 join_fscid=1 addr [v2:172.21.15.104:6824/1058172006,v1:172.21.15.104:6825/1058172006] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:03.257 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:14502} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/650878075,v1:172.21.15.90:6827/650878075] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:03.258 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2406e200 msgr2=0x7f5b240706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2406e200 0x7f5b240706b0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5b34196420 tx=0x7f5b2000b540 comp rx=0 tx=0).stop 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 msgr2=0x7f5b34194e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34194e00 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f5b280076b0 tx=0x7f5b280079c0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 shutdown_connections 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f5b2406e200 0x7f5b240706b0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f5b340ff3b0 0x7f5b34194e00 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 --2- 172.21.15.90:0/417507930 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f5b340ffd50 0x7f5b34195340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.255+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 >> 172.21.15.90:0/417507930 conn(0x7f5b340fac90 msgr2=0x7f5b340fd0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.256+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 shutdown_connections 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.256+0000 7f5b3ad35700 1 -- 172.21.15.90:0/417507930 wait complete. 2024-04-26T04:51:03.259 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:51:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:03 smithi104 ceph-mon[64778]: from='client.34348 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:03 smithi104 ceph-mon[64778]: from='client.34352 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:03 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:51:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:03 smithi104 ceph-mon[64778]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "osd ok-to-stop", "ids": ["5"], "max": 16}]: dispatch 2024-04-26T04:51:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:03 smithi104 ceph-mon[64778]: Upgrade: osd.5 is safe to restart 2024-04-26T04:51:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:03 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3642807792' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 -- 172.21.15.90:0/2529294390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 msgr2=0x7faf2c1094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 --2- 172.21.15.90:0/2529294390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c1094d0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7faf28009910 tx=0x7faf28009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 -- 172.21.15.90:0/2529294390 shutdown_connections 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 --2- 172.21.15.90:0/2529294390 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c1094d0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 --2- 172.21.15.90:0/2529294390 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2c100460 0x7faf2c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 -- 172.21.15.90:0/2529294390 >> 172.21.15.90:0/2529294390 conn(0x7faf2c0748b0 msgr2=0x7faf2c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 -- 172.21.15.90:0/2529294390 shutdown_connections 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 -- 172.21.15.90:0/2529294390 wait complete. 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.360+0000 7faf33541700 1 Processor -- start 2024-04-26T04:51:03.363 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf33541700 1 -- start start 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf33541700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2c100460 0x7faf2c101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf33541700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf33541700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c103af0 con 0x7faf2c100d70 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf33541700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c103c60 con 0x7faf2c100460 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf30adc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf30adc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c102310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58388/0 (socket says 172.21.15.90:58388) 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf30adc700 1 -- 172.21.15.90:0/4232323887 learned_addr learned my addr 172.21.15.90:0/4232323887 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf312dd700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2c100460 0x7faf2c101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf30adc700 1 -- 172.21.15.90:0/4232323887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2c100460 msgr2=0x7faf2c101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf30adc700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2c100460 0x7faf2c101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf30adc700 1 -- 172.21.15.90:0/4232323887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf280095f0 con 0x7faf2c100d70 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf30adc700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c102310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf28006010 tx=0x7faf2800bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:03.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf1e7fc700 1 -- 172.21.15.90:0/4232323887 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf2801d070 con 0x7faf2c100d70 2024-04-26T04:51:03.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf1e7fc700 1 -- 172.21.15.90:0/4232323887 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf2800f460 con 0x7faf2c100d70 2024-04-26T04:51:03.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c102850 con 0x7faf2c100d70 2024-04-26T04:51:03.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.361+0000 7faf1e7fc700 1 -- 172.21.15.90:0/4232323887 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf28017630 con 0x7faf2c100d70 2024-04-26T04:51:03.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.362+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c1a6810 con 0x7faf2c100d70 2024-04-26T04:51:03.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.362+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c04f2a0 con 0x7faf2c100d70 2024-04-26T04:51:03.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.363+0000 7faf1e7fc700 1 -- 172.21.15.90:0/4232323887 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faf2800f5d0 con 0x7faf2c100d70 2024-04-26T04:51:03.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.363+0000 7faf1e7fc700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf1806e200 0x7faf180706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:03.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.364+0000 7faf1e7fc700 1 -- 172.21.15.90:0/4232323887 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7faf2808f620 con 0x7faf2c100d70 2024-04-26T04:51:03.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.364+0000 7faf312dd700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf1806e200 0x7faf180706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:03.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.364+0000 7faf312dd700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf1806e200 0x7faf180706b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf2000ac50 tx=0x7faf2000a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:03.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.368+0000 7faf1e7fc700 1 -- 172.21.15.90:0/4232323887 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faf280589b0 con 0x7faf2c100d70 2024-04-26T04:51:03.590 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.589+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7faf2c0ff3d0 con 0x7faf1806e200 2024-04-26T04:51:03.591 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.590+0000 7faf1e7fc700 1 -- 172.21.15.90:0/4232323887 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+401 (secure 0 0 0) 0x7faf2c0ff3d0 con 0x7faf1806e200 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "mon" 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "11/23 daemons upgraded", 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading osd daemons", 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:51:03.592 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:51:03.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf1806e200 msgr2=0x7faf180706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf1806e200 0x7faf180706b0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7faf2000ac50 tx=0x7faf2000a380 comp rx=0 tx=0).stop 2024-04-26T04:51:03.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 msgr2=0x7faf2c102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.594 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c102310 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7faf28006010 tx=0x7faf2800bd70 comp rx=0 tx=0).stop 2024-04-26T04:51:03.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 shutdown_connections 2024-04-26T04:51:03.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf1806e200 0x7faf180706b0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf2c100460 0x7faf2c101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 --2- 172.21.15.90:0/4232323887 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf2c100d70 0x7faf2c102310 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 >> 172.21.15.90:0/4232323887 conn(0x7faf2c0748b0 msgr2=0x7faf2c0fecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:03.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 shutdown_connections 2024-04-26T04:51:03.595 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.593+0000 7faf33541700 1 -- 172.21.15.90:0/4232323887 wait complete. 2024-04-26T04:51:03.698 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 -- 172.21.15.90:0/2863518188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 msgr2=0x7febac103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2863518188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac103d30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feb9c009910 tx=0x7feb9c009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 -- 172.21.15.90:0/2863518188 shutdown_connections 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2863518188 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac103d30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2863518188 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac0ff3c0 0x7febac0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 -- 172.21.15.90:0/2863518188 >> 172.21.15.90:0/2863518188 conn(0x7febac0fac80 msgr2=0x7febac0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 -- 172.21.15.90:0/2863518188 shutdown_connections 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.697+0000 7febb1dcc700 1 -- 172.21.15.90:0/2863518188 wait complete. 2024-04-26T04:51:03.699 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febb1dcc700 1 Processor -- start 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febb1dcc700 1 -- start start 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febb1dcc700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac0ff3c0 0x7febac100dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febb1dcc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac101310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febb1dcc700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac101850 con 0x7febac0ffd60 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febb1dcc700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febac101990 con 0x7febac0ff3c0 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febaaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac101310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febaaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac101310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:58394/0 (socket says 172.21.15.90:58394) 2024-04-26T04:51:03.701 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febaaffd700 1 -- 172.21.15.90:0/2169463474 learned_addr learned my addr 172.21.15.90:0/2169463474 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.698+0000 7febab7fe700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac0ff3c0 0x7febac100dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7febaaffd700 1 -- 172.21.15.90:0/2169463474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac0ff3c0 msgr2=0x7febac100dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7febaaffd700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac0ff3c0 0x7febac100dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7febaaffd700 1 -- 172.21.15.90:0/2169463474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb9c0095f0 con 0x7febac0ffd60 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7febaaffd700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac101310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00bda0 tx=0x7feb9c00bdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7feba8ff9700 1 -- 172.21.15.90:0/2169463474 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9c01d070 con 0x7febac0ffd60 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7feba8ff9700 1 -- 172.21.15.90:0/2169463474 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7feb9c00f460 con 0x7febac0ffd60 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febac101c10 con 0x7febac0ffd60 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7feba8ff9700 1 -- 172.21.15.90:0/2169463474 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7feb9c017630 con 0x7febac0ffd60 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.699+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febac1a2340 con 0x7febac0ffd60 2024-04-26T04:51:03.702 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.700+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febac108740 con 0x7febac0ffd60 2024-04-26T04:51:03.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.701+0000 7feba8ff9700 1 -- 172.21.15.90:0/2169463474 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7feb9c00fab0 con 0x7febac0ffd60 2024-04-26T04:51:03.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.701+0000 7feba8ff9700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feb9806e200 0x7feb980706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:03.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.701+0000 7feba8ff9700 1 -- 172.21.15.90:0/2169463474 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 6414+0+0 (secure 0 0 0) 0x7feb9c08f4f0 con 0x7febac0ffd60 2024-04-26T04:51:03.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.701+0000 7febab7fe700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feb9806e200 0x7feb980706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:03.703 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.701+0000 7febab7fe700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feb9806e200 0x7feb980706b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb9400ac50 tx=0x7feb9400a380 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:03.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.706+0000 7feba8ff9700 1 -- 172.21.15.90:0/2169463474 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7feb9c058900 con 0x7febac0ffd60 2024-04-26T04:51:03.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.995+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7febac04f2a0 con 0x7febac0ffd60 2024-04-26T04:51:03.997 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.996+0000 7feba8ff9700 1 -- 172.21.15.90:0/2169463474 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7feb9c05bfc0 con 0x7febac0ffd60 2024-04-26T04:51:03.997 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_OK 2024-04-26T04:51:03.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feb9806e200 msgr2=0x7feb980706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feb9806e200 0x7feb980706b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7feb9400ac50 tx=0x7feb9400a380 comp rx=0 tx=0).stop 2024-04-26T04:51:03.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 msgr2=0x7febac101310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:03.999 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac101310 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7feb9c00bda0 tx=0x7feb9c00bdd0 comp rx=0 tx=0).stop 2024-04-26T04:51:04.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 shutdown_connections 2024-04-26T04:51:04.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7feb9806e200 0x7feb980706b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:04.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7febac0ff3c0 0x7febac100dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:04.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 --2- 172.21.15.90:0/2169463474 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7febac0ffd60 0x7febac101310 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:04.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 >> 172.21.15.90:0/2169463474 conn(0x7febac0fac80 msgr2=0x7febac0690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:04.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 shutdown_connections 2024-04-26T04:51:04.000 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:03.998+0000 7febb1dcc700 1 -- 172.21.15.90:0/2169463474 wait complete. 2024-04-26T04:51:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:04 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/417507930' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:51:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:04 smithi090 ceph-mon[86550]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:04 smithi090 ceph-mon[86550]: pgmap v152: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:04 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2169463474' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:51:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:04 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/417507930' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:51:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:04 smithi104 ceph-mon[64778]: from='client.34364 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:04 smithi104 ceph-mon[64778]: pgmap v152: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:04 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2169463474' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:51:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:05 smithi090 ceph-mon[86550]: Upgrade: Updating osd.5 2024-04-26T04:51:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:05.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:51:05.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:05 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:05.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:05 smithi090 ceph-mon[86550]: Deploying daemon osd.5 on smithi104 2024-04-26T04:51:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:05 smithi104 ceph-mon[64778]: Upgrade: Updating osd.5 2024-04-26T04:51:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-26T04:51:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:05 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:05.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:05 smithi104 ceph-mon[64778]: Deploying daemon osd.5 on smithi104 2024-04-26T04:51:06.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:06 smithi104 ceph-mon[64778]: pgmap v153: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:06.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:06 smithi090 ceph-mon[86550]: pgmap v153: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:07.738 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:07 smithi104 systemd[1]: Stopping Ceph osd.5 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:51:07.738 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:07 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[47742]: 2024-04-26T04:51:07.670+0000 7f232f318700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:51:07.738 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:07 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[47742]: 2024-04-26T04:51:07.670+0000 7f232f318700 -1 osd.5 75 *** Got signal Terminated *** 2024-04-26T04:51:07.738 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:07 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[47742]: 2024-04-26T04:51:07.670+0000 7f232f318700 -1 osd.5 75 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:51:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:07 smithi090 ceph-mon[86550]: osd.5 marked itself down and dead 2024-04-26T04:51:08.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:07 smithi104 ceph-mon[64778]: osd.5 marked itself down and dead 2024-04-26T04:51:08.355 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:08 smithi104 podman[84931]: 2024-04-26 04:51:08.090963037 +0000 UTC m=+0.517307547 container died 99079694ead0c678825f872d9814d3d0748977c7f1805b2c43d9969485c483ad (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, ceph=True, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=-18.2.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20231212, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-26T04:51:08.698 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:08 smithi104 podman[84931]: 2024-04-26 04:51:08.462762238 +0000 UTC m=+0.889106734 container cleanup 99079694ead0c678825f872d9814d3d0748977c7f1805b2c43d9969485c483ad (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, GIT_CLEAN=True, GIT_REPO=https://github.com/ceph/ceph-container.git, org.label-schema.build-date=20231212, org.label-schema.schema-version=1.0, io.buildah.version=1.29.1, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=-18.2.0, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, RELEASE=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:51:08.698 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:08 smithi104 bash[84931]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5 2024-04-26T04:51:09.001 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:08 smithi104 ceph-mon[64778]: pgmap v154: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:09.002 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:08 smithi104 ceph-mon[64778]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:51:09.002 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:08 smithi104 ceph-mon[64778]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-26T04:51:09.002 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:08 smithi104 ceph-mon[64778]: osdmap e76: 6 total, 5 up, 6 in 2024-04-26T04:51:09.002 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:08 smithi104 ceph-mon[64778]: osdmap e77: 6 total, 5 up, 6 in 2024-04-26T04:51:09.002 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:08 smithi104 podman[84949]: 2024-04-26 04:51:08.697167851 +0000 UTC m=+0.603614484 container remove 99079694ead0c678825f872d9814d3d0748977c7f1805b2c43d9969485c483ad (image=quay.io/ceph/ceph@sha256:1793ff3af6ae74527c86e1a0b22401e9c42dc08d0ebb8379653be07db17d0007, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, org.label-schema.build-date=20231212, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=-18.2.0, RELEASE=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=0396eef90bef641b676c164ec7a3876f45010308, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.29.1, GIT_REPO=https://github.com/ceph/ceph-container.git) 2024-04-26T04:51:09.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:08 smithi090 ceph-mon[86550]: pgmap v154: 65 pgs: 65 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:09.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:08 smithi090 ceph-mon[86550]: Health check failed: 1 osds down (OSD_DOWN) 2024-04-26T04:51:09.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:08 smithi090 ceph-mon[86550]: Health check failed: all OSDs are running squid or later but require_osd_release < squid (OSD_UPGRADE_FINISHED) 2024-04-26T04:51:09.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:08 smithi090 ceph-mon[86550]: osdmap e76: 6 total, 5 up, 6 in 2024-04-26T04:51:09.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:08 smithi090 ceph-mon[86550]: osdmap e77: 6 total, 5 up, 6 in 2024-04-26T04:51:09.259 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:09 smithi104 podman[85039]: 2024-04-26 04:51:08.904228262 +0000 UTC m=+0.017807721 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:51:09.259 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:09 smithi104 podman[85039]: 2024-04-26 04:51:09.035752462 +0000 UTC m=+0.149331979 container create 9f1a292093e92dc655f5184d59490716e7abf14cc77cb294daf2ec20b9cf5e76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb) 2024-04-26T04:51:09.259 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:09 smithi104 podman[85039]: 2024-04-26 04:51:09.258835227 +0000 UTC m=+0.372414672 container init 9f1a292093e92dc655f5184d59490716e7abf14cc77cb294daf2ec20b9cf5e76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-26T04:51:09.511 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:09 smithi104 podman[85039]: 2024-04-26 04:51:09.261888127 +0000 UTC m=+0.375467575 container start 9f1a292093e92dc655f5184d59490716e7abf14cc77cb294daf2ec20b9cf5e76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T04:51:09.511 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:09 smithi104 podman[85039]: 2024-04-26 04:51:09.333740008 +0000 UTC m=+0.447319454 container attach 9f1a292093e92dc655f5184d59490716e7abf14cc77cb294daf2ec20b9cf5e76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=squid-b22e2eb) 2024-04-26T04:51:10.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:09 smithi104 podman[85039]: 2024-04-26 04:51:09.544337874 +0000 UTC m=+0.657917320 container died 9f1a292093e92dc655f5184d59490716e7abf14cc77cb294daf2ec20b9cf5e76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:51:10.356 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 podman[85065]: 2024-04-26 04:51:10.137288496 +0000 UTC m=+0.588472327 container remove 9f1a292093e92dc655f5184d59490716e7abf14cc77cb294daf2ec20b9cf5e76 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-04-26T04:51:10.356 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.5.service: Deactivated successfully. 2024-04-26T04:51:10.356 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 systemd[1]: Stopped Ceph osd.5 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:51:10.356 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 systemd[1]: ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.5.service: Consumed 1min 1.781s CPU time. 2024-04-26T04:51:10.697 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 systemd[1]: Starting Ceph osd.5 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:51:10.698 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 podman[85166]: 2024-04-26 04:51:10.473953333 +0000 UTC m=+0.017762208 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:51:10.698 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 podman[85166]: 2024-04-26 04:51:10.588507747 +0000 UTC m=+0.132316607 container create 65b88415fa0c7ef77aeb1d353484884ae8f6504306288c160ed8a6e2cb1f4f6d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-26T04:51:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:10 smithi104 ceph-mon[64778]: pgmap v157: 65 pgs: 13 stale+active+clean, 52 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:11.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:11.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 podman[85166]: 2024-04-26 04:51:10.832594884 +0000 UTC m=+0.376403744 container init 65b88415fa0c7ef77aeb1d353484884ae8f6504306288c160ed8a6e2cb1f4f6d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424) 2024-04-26T04:51:11.012 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 podman[85166]: 2024-04-26 04:51:10.835919556 +0000 UTC m=+0.379728418 container start 65b88415fa0c7ef77aeb1d353484884ae8f6504306288c160ed8a6e2cb1f4f6d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-04-26T04:51:11.012 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:10 smithi104 podman[85166]: 2024-04-26 04:51:10.92419104 +0000 UTC m=+0.467999915 container attach 65b88415fa0c7ef77aeb1d353484884ae8f6504306288c160ed8a6e2cb1f4f6d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:51:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:10 smithi090 ceph-mon[86550]: pgmap v157: 65 pgs: 13 stale+active+clean, 52 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:11.594 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate[85182]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:51:11.594 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate[85182]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:51:11.594 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 bash[85166]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:51:11.594 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 bash[85166]: Running command: /usr/bin/ceph-bluestore-tool prime-osd-dir --path /var/lib/ceph/osd/ceph-5 --no-mon-config --dev /dev/mapper/vg_nvme-lv_2 2024-04-26T04:51:11.594 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 bash[85166]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 bash[85166]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 bash[85166]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 bash[85166]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 bash[85166]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate[85182]: Running command: /usr/bin/chown -h ceph:ceph /dev/mapper/vg_nvme-lv_2 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate[85182]: Running command: /usr/bin/chown -R ceph:ceph /dev/dm-1 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate[85182]: Running command: /usr/bin/ln -s /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5/block 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate[85182]: Running command: /usr/bin/chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate[85182]: --> ceph-volume raw activate successful for osd ID: 5 2024-04-26T04:51:11.595 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 podman[85166]: 2024-04-26 04:51:11.304628453 +0000 UTC m=+0.848437305 container died 65b88415fa0c7ef77aeb1d353484884ae8f6504306288c160ed8a6e2cb1f4f6d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-26T04:51:11.870 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:11 smithi104 ceph-mon[64778]: Health check failed: Degraded data redundancy: 22/348 objects degraded (6.322%), 8 pgs degraded (PG_DEGRADED) 2024-04-26T04:51:12.210 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:11 smithi104 podman[85318]: 2024-04-26 04:51:11.869549424 +0000 UTC m=+0.555546409 container remove 65b88415fa0c7ef77aeb1d353484884ae8f6504306288c160ed8a6e2cb1f4f6d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-activate, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD) 2024-04-26T04:51:12.210 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 podman[85381]: 2024-04-26 04:51:11.981950177 +0000 UTC m=+0.017998328 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:51:12.210 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 podman[85381]: 2024-04-26 04:51:12.087357691 +0000 UTC m=+0.123405882 container create 64a9be8458bb7982ef15220b317e53fb21ce8b72996ae8230db67a255b141854 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:51:12.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:11 smithi090 ceph-mon[86550]: Health check failed: Degraded data redundancy: 22/348 objects degraded (6.322%), 8 pgs degraded (PG_DEGRADED) 2024-04-26T04:51:12.761 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 podman[85381]: 2024-04-26 04:51:12.314768515 +0000 UTC m=+0.350816651 container init 64a9be8458bb7982ef15220b317e53fb21ce8b72996ae8230db67a255b141854 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:51:12.761 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 podman[85381]: 2024-04-26 04:51:12.317824984 +0000 UTC m=+0.353873120 container start 64a9be8458bb7982ef15220b317e53fb21ce8b72996ae8230db67a255b141854 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T04:51:12.761 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 ceph-osd[85400]: -- 172.21.15.104:0/2503561311 <== mon.1 v2:172.21.15.104:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563b01a85680 con 0x563b01a4ec00 2024-04-26T04:51:12.761 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 bash[85381]: 64a9be8458bb7982ef15220b317e53fb21ce8b72996ae8230db67a255b141854 2024-04-26T04:51:12.762 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 systemd[1]: Started Ceph osd.5 for f018e902-0385-11ef-bc93-c7b262605968. 2024-04-26T04:51:12.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:12 smithi090 ceph-mon[86550]: pgmap v158: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 44 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/348 objects degraded (6.322%) 2024-04-26T04:51:12.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:12 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:12.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:12 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:12.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:12 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:13.245 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:12 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:51:12.865+0000 7fdd74eec7c0 -1 Falling back to public interface 2024-04-26T04:51:13.245 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:12 smithi104 ceph-mon[64778]: pgmap v158: 65 pgs: 8 active+undersized+degraded, 3 stale+active+clean, 10 active+undersized, 44 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 22/348 objects degraded (6.322%) 2024-04-26T04:51:13.245 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:12 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:13.246 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:12 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:13.246 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:12 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:14 smithi104 ceph-mon[64778]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:15.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:14 smithi090 ceph-mon[86550]: pgmap v159: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:16.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:16.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:16.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:16 smithi090 ceph-mon[86550]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:16.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:16.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:16.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:16 smithi104 ceph-mon[64778]: pgmap v160: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:17.402 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:17 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:51:17.317+0000 7fdd74eec7c0 -1 osd.5 0 read_superblock omap replica is missing. 2024-04-26T04:51:17.718 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:17 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:17.719 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:17 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:17.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:17 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:17.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:17 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:18.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:17 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:51:17.717+0000 7fdd74eec7c0 -1 osd.5 75 log_to_monitors true 2024-04-26T04:51:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:18 smithi104 ceph-mon[64778]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:18.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:18 smithi104 ceph-mon[64778]: from='osd.5 [v2:172.21.15.104:6816/1501294989,v1:172.21.15.104:6817/1501294989]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:51:18.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:18 smithi104 ceph-mon[64778]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:51:18.762 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:51:18 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:51:18.568+0000 7fdd66344700 -1 osd.5 75 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-26T04:51:18.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:18 smithi090 ceph-mon[86550]: pgmap v161: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.2 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:18.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:18 smithi090 ceph-mon[86550]: from='osd.5 [v2:172.21.15.104:6816/1501294989,v1:172.21.15.104:6817/1501294989]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:51:18.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:18 smithi090 ceph-mon[86550]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-26T04:51:19.830 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:19 smithi104 ceph-mon[64778]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:51:19.830 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:19 smithi104 ceph-mon[64778]: from='osd.5 [v2:172.21.15.104:6816/1501294989,v1:172.21.15.104:6817/1501294989]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:51:19.830 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:19 smithi104 ceph-mon[64778]: osdmap e78: 6 total, 5 up, 6 in 2024-04-26T04:51:19.830 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:19 smithi104 ceph-mon[64778]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:51:19.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:19 smithi090 ceph-mon[86550]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-26T04:51:19.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:19 smithi090 ceph-mon[86550]: from='osd.5 [v2:172.21.15.104:6816/1501294989,v1:172.21.15.104:6817/1501294989]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:51:19.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:19 smithi090 ceph-mon[86550]: osdmap e78: 6 total, 5 up, 6 in 2024-04-26T04:51:19.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:19 smithi090 ceph-mon[86550]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi104", "root=default"]}]: dispatch 2024-04-26T04:51:20.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[86546]: 2024-04-26T04:51:20.563+0000 7fd65dcff700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:51:20.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:51:20.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: osd.5 [v2:172.21.15.104:6816/1501294989,v1:172.21.15.104:6817/1501294989] boot 2024-04-26T04:51:20.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: osdmap e79: 6 total, 6 up, 6 in 2024-04-26T04:51:20.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:20.973 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-26T04:51:20.974 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-26T04:51:20.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-26T04:51:20.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-26T04:51:20.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:20 smithi090 ceph-mon[86550]: osdmap e80: 6 total, 6 up, 6 in 2024-04-26T04:51:21.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: Health check cleared: OSD_DOWN (was: 1 osds down) 2024-04-26T04:51:21.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: osd.5 [v2:172.21.15.104:6816/1501294989,v1:172.21.15.104:6817/1501294989] boot 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: osdmap e79: 6 total, 6 up, 6 in 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: pgmap v164: 65 pgs: 14 active+undersized+degraded, 14 active+undersized, 37 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 53/348 objects degraded (15.230%) 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:21.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.0"}]': finished 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.1"}]': finished 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.2"}]': finished 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.3"}]': finished 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.4"}]': finished 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]: dispatch 2024-04-26T04:51:21.013 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd.5"}]': finished 2024-04-26T04:51:21.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd require-osd-release", "release": "squid"}]: dispatch 2024-04-26T04:51:21.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: Health check cleared: OSD_UPGRADE_FINISHED (was: all OSDs are running squid or later but require_osd_release < squid) 2024-04-26T04:51:21.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "osd require-osd-release", "release": "squid"}]': finished 2024-04-26T04:51:21.014 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:20 smithi104 ceph-mon[64778]: osdmap e80: 6 total, 6 up, 6 in 2024-04-26T04:51:21.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all osd 2024-04-26T04:51:21.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: Upgrade: Setting require_osd_release to 19 squid 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: osdmap e81: 6 total, 6 up, 6 in 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-26T04:51:21.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:21 smithi090 ceph-mon[86550]: Health check update: Degraded data redundancy: 53/348 objects degraded (15.230%), 14 pgs degraded (PG_DEGRADED) 2024-04-26T04:51:22.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all osd 2024-04-26T04:51:22.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: Upgrade: Setting require_osd_release to 19 squid 2024-04-26T04:51:22.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: Upgrade: failing fs cephfs for rapid multi-rank mds upgrade 2024-04-26T04:51:22.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-26T04:51:22.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-26T04:51:22.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-26T04:51:22.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-26T04:51:22.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: osdmap e81: 6 total, 6 up, 6 in 2024-04-26T04:51:22.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 2 up:standby, 2 failed 2024-04-26T04:51:22.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:21 smithi104 ceph-mon[64778]: Health check update: Degraded data redundancy: 53/348 objects degraded (15.230%), 14 pgs degraded (PG_DEGRADED) 2024-04-26T04:51:22.911 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6828/1614774470,v1:172.21.15.90:6829/1614774470] up:boot 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.104:6824/1188215025,v1:172.21.15.104:6825/1188215025] up:boot 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: pgmap v167: 65 pgs: 1 peering, 10 active+undersized+degraded, 10 active+undersized, 44 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/348 objects degraded (12.644%) 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:51:22.912 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:22 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:23.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6828/1614774470,v1:172.21.15.90:6829/1614774470] up:boot 2024-04-26T04:51:23.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.104:6824/1188215025,v1:172.21.15.104:6825/1188215025] up:boot 2024-04-26T04:51:23.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:51:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:51:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:51:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: pgmap v167: 65 pgs: 1 peering, 10 active+undersized+degraded, 10 active+undersized, 44 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 44/348 objects degraded (12.644%) 2024-04-26T04:51:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.ehputv", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:51:23.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:22 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:23.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:23 smithi090 ceph-mon[86550]: Upgrade: Updating mds.cephfs.smithi090.ehputv 2024-04-26T04:51:23.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:23 smithi090 ceph-mon[86550]: Deploying daemon mds.cephfs.smithi090.ehputv on smithi090 2024-04-26T04:51:24.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:23 smithi104 ceph-mon[64778]: Upgrade: Updating mds.cephfs.smithi090.ehputv 2024-04-26T04:51:24.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:23 smithi104 ceph-mon[64778]: Deploying daemon mds.cephfs.smithi090.ehputv on smithi090 2024-04-26T04:51:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:24 smithi090 ceph-mon[86550]: pgmap v168: 65 pgs: 1 peering, 64 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:25.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:24 smithi104 ceph-mon[64778]: pgmap v168: 65 pgs: 1 peering, 64 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:25.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:25 smithi090 ceph-mon[86550]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/348 objects degraded (12.644%), 10 pgs degraded) 2024-04-26T04:51:25.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:25 smithi090 ceph-mon[86550]: osdmap e82: 6 total, 6 up, 6 in 2024-04-26T04:51:25.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:25 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:51:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:25 smithi104 ceph-mon[64778]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 44/348 objects degraded (12.644%), 10 pgs degraded) 2024-04-26T04:51:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:25 smithi104 ceph-mon[64778]: osdmap e82: 6 total, 6 up, 6 in 2024-04-26T04:51:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:25 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:51:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:26 smithi090 ceph-mon[86550]: pgmap v170: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:26.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:51:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:26 smithi104 ceph-mon[64778]: pgmap v170: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:27.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:51:28.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:28 smithi090 ceph-mon[86550]: pgmap v171: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:29.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:28 smithi104 ceph-mon[64778]: pgmap v171: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:30 smithi090 ceph-mon[86550]: pgmap v172: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:30.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:30.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:30 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:30 smithi104 ceph-mon[64778]: pgmap v172: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:30 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:31 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:boot 2024-04-26T04:51:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:31 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:51:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:31 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:51:31.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:31 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:31 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:boot 2024-04-26T04:51:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:31 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:51:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:31 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:51:32.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:31 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:32.975 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:32 smithi090 ceph-mon[86550]: pgmap v173: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:33.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:32 smithi104 ceph-mon[64778]: pgmap v173: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 -- 172.21.15.90:0/3190183355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ffd60 msgr2=0x7fa1e8103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 --2- 172.21.15.90:0/3190183355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ffd60 0x7fa1e8103d30 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa1dc009910 tx=0x7fa1dc009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 -- 172.21.15.90:0/3190183355 shutdown_connections 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 --2- 172.21.15.90:0/3190183355 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ffd60 0x7fa1e8103d30 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 --2- 172.21.15.90:0/3190183355 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1e80ff3c0 0x7fa1e80ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 -- 172.21.15.90:0/3190183355 >> 172.21.15.90:0/3190183355 conn(0x7fa1e80fac80 msgr2=0x7fa1e80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 -- 172.21.15.90:0/3190183355 shutdown_connections 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.105+0000 7fa1edf98700 1 -- 172.21.15.90:0/3190183355 wait complete. 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1edf98700 1 Processor -- start 2024-04-26T04:51:34.107 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1edf98700 1 -- start start 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1edf98700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ff3c0 0x7fa1e8195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1edf98700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1e80ffd60 0x7fa1e81963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1edf98700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e8196ab0 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1edf98700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e819a840 con 0x7fa1e80ffd60 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e77fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ff3c0 0x7fa1e8195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e77fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ff3c0 0x7fa1e8195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40304/0 (socket says 172.21.15.90:40304) 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e77fe700 1 -- 172.21.15.90:0/1559764438 learned_addr learned my addr 172.21.15.90:0/1559764438 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:34.109 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e6ffd700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1e80ffd60 0x7fa1e81963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e77fe700 1 -- 172.21.15.90:0/1559764438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1e80ffd60 msgr2=0x7fa1e81963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e77fe700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1e80ffd60 0x7fa1e81963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e77fe700 1 -- 172.21.15.90:0/1559764438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1dc0095f0 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.106+0000 7fa1e6ffd700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1e80ffd60 0x7fa1e81963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.107+0000 7fa1e77fe700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ff3c0 0x7fa1e8195e90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d8d0 tx=0x7fa1d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.107+0000 7fa1e4ff9700 1 -- 172.21.15.90:0/1559764438 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d8009940 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.107+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e819ab20 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.107+0000 7fa1e4ff9700 1 -- 172.21.15.90:0/1559764438 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa1d8010460 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.107+0000 7fa1e4ff9700 1 -- 172.21.15.90:0/1559764438 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa1d800f5f0 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.107+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e819b070 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.108+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e8108740 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.110 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.108+0000 7fa1e4ff9700 1 -- 172.21.15.90:0/1559764438 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa1d80105d0 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.109+0000 7fa1e4ff9700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa1d006e200 0x7fa1d00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.109+0000 7fa1e4ff9700 1 -- 172.21.15.90:0/1559764438 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa1d808e170 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.109+0000 7fa1e6ffd700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa1d006e200 0x7fa1d00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.111 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.109+0000 7fa1e6ffd700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa1d006e200 0x7fa1d00706b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1e81974b0 tx=0x7fa1dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:34.114 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.114+0000 7fa1e4ff9700 1 -- 172.21.15.90:0/1559764438 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa1d8057450 con 0x7fa1e80ff3c0 2024-04-26T04:51:34.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.354+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1e819b290 con 0x7fa1d006e200 2024-04-26T04:51:34.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.355+0000 7fa1e4ff9700 1 -- 172.21.15.90:0/1559764438 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa1e819b290 con 0x7fa1d006e200 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa1d006e200 msgr2=0x7fa1d00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa1d006e200 0x7fa1d00706b0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa1e81974b0 tx=0x7fa1dc01a040 comp rx=0 tx=0).stop 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ff3c0 msgr2=0x7fa1e8195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ff3c0 0x7fa1e8195e90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa1d800d8d0 tx=0x7fa1d800dc90 comp rx=0 tx=0).stop 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 shutdown_connections 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa1d006e200 0x7fa1d00706b0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa1e80ff3c0 0x7fa1e8195e90 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 --2- 172.21.15.90:0/1559764438 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa1e80ffd60 0x7fa1e81963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 >> 172.21.15.90:0/1559764438 conn(0x7fa1e80fac80 msgr2=0x7fa1e80690f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 shutdown_connections 2024-04-26T04:51:34.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.357+0000 7fa1edf98700 1 -- 172.21.15.90:0/1559764438 wait complete. 2024-04-26T04:51:34.369 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:51:34.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 -- 172.21.15.90:0/1631197616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010afd0 msgr2=0x7fc36010b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 --2- 172.21.15.90:0/1631197616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010afd0 0x7fc36010b3a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fc350009910 tx=0x7fc350009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:34.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 -- 172.21.15.90:0/1631197616 shutdown_connections 2024-04-26T04:51:34.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 --2- 172.21.15.90:0/1631197616 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc36010b970 0x7fc36010f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 --2- 172.21.15.90:0/1631197616 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010afd0 0x7fc36010b3a0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 -- 172.21.15.90:0/1631197616 >> 172.21.15.90:0/1631197616 conn(0x7fc36006add0 msgr2=0x7fc36006b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:34.475 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 -- 172.21.15.90:0/1631197616 shutdown_connections 2024-04-26T04:51:34.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.473+0000 7fc366666700 1 -- 172.21.15.90:0/1631197616 wait complete. 2024-04-26T04:51:34.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.474+0000 7fc366666700 1 Processor -- start 2024-04-26T04:51:34.476 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.474+0000 7fc366666700 1 -- start start 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.474+0000 7fc366666700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc36010afd0 0x7fc3601a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.474+0000 7fc366666700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010b970 0x7fc3601a6bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.474+0000 7fc366666700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3601a71c0 con 0x7fc36010b970 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.474+0000 7fc366666700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3601a0770 con 0x7fc36010afd0 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010b970 0x7fc3601a6bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010b970 0x7fc3601a6bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40324/0 (socket says 172.21.15.90:40324) 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35f7fe700 1 -- 172.21.15.90:0/1434128575 learned_addr learned my addr 172.21.15.90:0/1434128575 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35ffff700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc36010afd0 0x7fc3601a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.477 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35f7fe700 1 -- 172.21.15.90:0/1434128575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc36010afd0 msgr2=0x7fc3601a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35f7fe700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc36010afd0 0x7fc3601a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35f7fe700 1 -- 172.21.15.90:0/1434128575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3500095f0 con 0x7fc36010b970 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35f7fe700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010b970 0x7fc3601a6bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c960 tx=0x7fc35400cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35d7fa700 1 -- 172.21.15.90:0/1434128575 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc354007a70 con 0x7fc36010b970 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35d7fa700 1 -- 172.21.15.90:0/1434128575 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc354007bd0 con 0x7fc36010b970 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3601a0a50 con 0x7fc36010b970 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc35d7fa700 1 -- 172.21.15.90:0/1434128575 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc3540186c0 con 0x7fc36010b970 2024-04-26T04:51:34.478 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.475+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3601a0fa0 con 0x7fc36010b970 2024-04-26T04:51:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.476+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3601131c0 con 0x7fc36010b970 2024-04-26T04:51:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.477+0000 7fc35d7fa700 1 -- 172.21.15.90:0/1434128575 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc354018820 con 0x7fc36010b970 2024-04-26T04:51:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.477+0000 7fc35d7fa700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc34806e200 0x7fc3480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.477+0000 7fc35d7fa700 1 -- 172.21.15.90:0/1434128575 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc35408e180 con 0x7fc36010b970 2024-04-26T04:51:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.477+0000 7fc35ffff700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc34806e200 0x7fc3480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.479 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.478+0000 7fc35ffff700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc34806e200 0x7fc3480706b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc350006010 tx=0x7fc35001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:34.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.482+0000 7fc35d7fa700 1 -- 172.21.15.90:0/1434128575 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc35405ad70 con 0x7fc36010b970 2024-04-26T04:51:34.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.707+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3601a1c80 con 0x7fc34806e200 2024-04-26T04:51:34.709 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.708+0000 7fc35d7fa700 1 -- 172.21.15.90:0/1434128575 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fc3601a1c80 con 0x7fc34806e200 2024-04-26T04:51:34.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc34806e200 msgr2=0x7fc3480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc34806e200 0x7fc3480706b0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc350006010 tx=0x7fc35001a040 comp rx=0 tx=0).stop 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010b970 msgr2=0x7fc3601a6bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010b970 0x7fc3601a6bc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fc35400c960 tx=0x7fc35400cd20 comp rx=0 tx=0).stop 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 shutdown_connections 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc34806e200 0x7fc3480706b0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc36010afd0 0x7fc3601a6680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 --2- 172.21.15.90:0/1434128575 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc36010b970 0x7fc3601a6bc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 >> 172.21.15.90:0/1434128575 conn(0x7fc36006add0 msgr2=0x7fc360109860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 shutdown_connections 2024-04-26T04:51:34.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.710+0000 7fc366666700 1 -- 172.21.15.90:0/1434128575 wait complete. 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 -- 172.21.15.90:0/3375309157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ffd60 msgr2=0x7f7bcc103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/3375309157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ffd60 0x7f7bcc103d30 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f7bbc009910 tx=0x7f7bbc009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 -- 172.21.15.90:0/3375309157 shutdown_connections 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/3375309157 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ffd60 0x7f7bcc103d30 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/3375309157 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bcc0ff3c0 0x7f7bcc0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 -- 172.21.15.90:0/3375309157 >> 172.21.15.90:0/3375309157 conn(0x7f7bcc0fac80 msgr2=0x7f7bcc0fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 -- 172.21.15.90:0/3375309157 shutdown_connections 2024-04-26T04:51:34.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 -- 172.21.15.90:0/3375309157 wait complete. 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 Processor -- start 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.815+0000 7f7bd39ee700 1 -- start start 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd39ee700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ff3c0 0x7f7bcc194ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd39ee700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bcc0ffd60 0x7f7bcc195420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd39ee700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc195b00 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd39ee700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bcc199890 con 0x7f7bcc0ffd60 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd178a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ff3c0 0x7f7bcc194ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd178a700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ff3c0 0x7f7bcc194ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40348/0 (socket says 172.21.15.90:40348) 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd178a700 1 -- 172.21.15.90:0/2132764449 learned_addr learned my addr 172.21.15.90:0/2132764449 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd0f89700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bcc0ffd60 0x7f7bcc195420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd178a700 1 -- 172.21.15.90:0/2132764449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bcc0ffd60 msgr2=0x7f7bcc195420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd178a700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bcc0ffd60 0x7f7bcc195420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd178a700 1 -- 172.21.15.90:0/2132764449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bbc0095f0 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd178a700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ff3c0 0x7f7bcc194ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b4b0 tx=0x7f7bc800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bc27fc700 1 -- 172.21.15.90:0/2132764449 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc8011800 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bc27fc700 1 -- 172.21.15.90:0/2132764449 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7bc8011e20 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.816+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bcc199b70 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.817+0000 7f7bc27fc700 1 -- 172.21.15.90:0/2132764449 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7bc800f2e0 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.817+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bcc19a0c0 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.817+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bcc107660 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.818+0000 7f7bc27fc700 1 -- 172.21.15.90:0/2132764449 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f7bc8010380 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.818+0000 7f7bc27fc700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7bb806e200 0x7f7bb80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:34.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.818+0000 7f7bc27fc700 1 -- 172.21.15.90:0/2132764449 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f7bc808dca0 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.818+0000 7f7bd0f89700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7bb806e200 0x7f7bb80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:34.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.819+0000 7f7bd0f89700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7bb806e200 0x7f7bb80706b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc196500 tx=0x7f7bbc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:34.824 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:34.823+0000 7f7bc27fc700 1 -- 172.21.15.90:0/2132764449 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7bc8057000 con 0x7f7bcc0ff3c0 2024-04-26T04:51:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:34 smithi090 ceph-mon[86550]: pgmap v174: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:35.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:34 smithi104 ceph-mon[64778]: pgmap v174: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:35.050 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.049+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f7bcc19a390 con 0x7f7bb806e200 2024-04-26T04:51:35.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.060+0000 7f7bc27fc700 1 -- 172.21.15.90:0/2132764449 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f7bcc19a390 con 0x7f7bb806e200 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (6m) 119s ago 17m 24.8M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (17m) 119s ago 17m 8627k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (16m) 19s ago 16m 8711k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (3m) 119s ago 17m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (3m) 19s ago 16m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (5m) 119s ago 16m 80.6M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (11m) 119s ago 11m 129M - 18.2.0 dc2bc1663786 2add7aae65b8 2024-04-26T04:51:35.062 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 starting - - - - 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (11m) 19s ago 11m 17.9M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (11m) 19s ago 11m 130M - 18.2.0 dc2bc1663786 c423f48b535c 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (7m) 119s ago 18m 505M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (7m) 19s ago 16m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (4m) 119s ago 18m 48.2M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (4m) 19s ago 16m 43.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (7m) 119s ago 17m 16.5M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (6m) 19s ago 16m 14.7M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (3m) 119s ago 15m 88.4M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (2m) 119s ago 14m 100M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (2m) 119s ago 14m 12.7M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (89s) 19s ago 13m 108M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d860fd1e9069 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (55s) 19s ago 13m 92.1M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 fbfd5c1c44eb 2024-04-26T04:51:35.063 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (23s) 19s ago 13m 12.3M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 64a9be8458bb 2024-04-26T04:51:35.064 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (6m) 119s ago 16m 69.5M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:51:35.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7bb806e200 msgr2=0x7f7bb80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7bb806e200 0x7f7bb80706b0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f7bcc196500 tx=0x7f7bbc01a040 comp rx=0 tx=0).stop 2024-04-26T04:51:35.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ff3c0 msgr2=0x7f7bcc194ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ff3c0 0x7f7bcc194ee0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f7bc800b4b0 tx=0x7f7bc800b870 comp rx=0 tx=0).stop 2024-04-26T04:51:35.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 shutdown_connections 2024-04-26T04:51:35.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f7bb806e200 0x7f7bb80706b0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7bcc0ff3c0 0x7f7bcc194ee0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 --2- 172.21.15.90:0/2132764449 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7bcc0ffd60 0x7f7bcc195420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 >> 172.21.15.90:0/2132764449 conn(0x7f7bcc0fac80 msgr2=0x7f7bcc0fc440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:35.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 shutdown_connections 2024-04-26T04:51:35.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.064+0000 7f7bd39ee700 1 -- 172.21.15.90:0/2132764449 wait complete. 2024-04-26T04:51:35.178 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 -- 172.21.15.90:0/4009859688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec1004c0 msgr2=0x7fd7ec100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 --2- 172.21.15.90:0/4009859688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec1004c0 0x7fd7ec100890 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc009910 tx=0x7fd7dc009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:35.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 -- 172.21.15.90:0/4009859688 shutdown_connections 2024-04-26T04:51:35.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 --2- 172.21.15.90:0/4009859688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7ec100e60 0x7fd7ec104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 --2- 172.21.15.90:0/4009859688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec1004c0 0x7fd7ec100890 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 -- 172.21.15.90:0/4009859688 >> 172.21.15.90:0/4009859688 conn(0x7fd7ec0fbd80 msgr2=0x7fd7ec0fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:35.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 -- 172.21.15.90:0/4009859688 shutdown_connections 2024-04-26T04:51:35.179 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.177+0000 7fd7f2920700 1 -- 172.21.15.90:0/4009859688 wait complete. 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7f2920700 1 Processor -- start 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7f2920700 1 -- start start 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7f2920700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7ec1004c0 0x7fd7ec073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7f2920700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec100e60 0x7fd7ec073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7f2920700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec073ad0 con 0x7fd7ec100e60 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7f2920700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7ec073c10 con 0x7fd7ec1004c0 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7eb7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec100e60 0x7fd7ec073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7eb7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec100e60 0x7fd7ec073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40362/0 (socket says 172.21.15.90:40362) 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.178+0000 7fd7eb7fe700 1 -- 172.21.15.90:0/518186627 learned_addr learned my addr 172.21.15.90:0/518186627 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7ebfff700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7ec1004c0 0x7fd7ec073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.181 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7eb7fe700 1 -- 172.21.15.90:0/518186627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7ec1004c0 msgr2=0x7fd7ec073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7eb7fe700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7ec1004c0 0x7fd7ec073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7eb7fe700 1 -- 172.21.15.90:0/518186627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7dc0095f0 con 0x7fd7ec100e60 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7eb7fe700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec100e60 0x7fd7ec073590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c8f0 tx=0x7fd7e000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7e97fa700 1 -- 172.21.15.90:0/518186627 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e0007a50 con 0x7fd7ec100e60 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7ec1a2100 con 0x7fd7ec100e60 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7ec1a2510 con 0x7fd7ec100e60 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7e97fa700 1 -- 172.21.15.90:0/518186627 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd7e0007bb0 con 0x7fd7ec100e60 2024-04-26T04:51:35.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.179+0000 7fd7e97fa700 1 -- 172.21.15.90:0/518186627 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd7e0018630 con 0x7fd7ec100e60 2024-04-26T04:51:35.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.180+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d8005320 con 0x7fd7ec100e60 2024-04-26T04:51:35.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.181+0000 7fd7e97fa700 1 -- 172.21.15.90:0/518186627 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd7e000f450 con 0x7fd7ec100e60 2024-04-26T04:51:35.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.181+0000 7fd7e97fa700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd7d406e200 0x7fd7d40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.181+0000 7fd7e97fa700 1 -- 172.21.15.90:0/518186627 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd7e008e010 con 0x7fd7ec100e60 2024-04-26T04:51:35.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.181+0000 7fd7ebfff700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd7d406e200 0x7fd7d40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.182+0000 7fd7ebfff700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd7d406e200 0x7fd7d40706b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00b5c0 tx=0x7fd7dc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:35.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.186+0000 7fd7e97fa700 1 -- 172.21.15.90:0/518186627 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd7e0057370 con 0x7fd7ec100e60 2024-04-26T04:51:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.481+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fd7d8006200 con 0x7fd7ec100e60 2024-04-26T04:51:35.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.482+0000 7fd7e97fa700 1 -- 172.21.15.90:0/518186627 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fd7e005aa30 con 0x7fd7ec100e60 2024-04-26T04:51:35.483 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:51:35.483 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:51:35.483 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 1 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 3, 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 11 2024-04-26T04:51:35.484 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:51:35.485 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:51:35.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd7d406e200 msgr2=0x7fd7d40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd7d406e200 0x7fd7d40706b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd7dc00b5c0 tx=0x7fd7dc01a040 comp rx=0 tx=0).stop 2024-04-26T04:51:35.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec100e60 msgr2=0x7fd7ec073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec100e60 0x7fd7ec073590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd7e000c8f0 tx=0x7fd7e000ccb0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 shutdown_connections 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd7d406e200 0x7fd7d40706b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd7ec1004c0 0x7fd7ec073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 --2- 172.21.15.90:0/518186627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd7ec100e60 0x7fd7ec073590 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 >> 172.21.15.90:0/518186627 conn(0x7fd7ec0fbd80 msgr2=0x7fd7ec0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 shutdown_connections 2024-04-26T04:51:35.486 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.484+0000 7fd7f2920700 1 -- 172.21.15.90:0/518186627 wait complete. 2024-04-26T04:51:35.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.594+0000 7f61da882700 1 -- 172.21.15.90:0/1562777828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d40687a0 msgr2=0x7f61d4068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 --2- 172.21.15.90:0/1562777828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d40687a0 0x7f61d4068b70 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f61c4009910 tx=0x7f61c4009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:35.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 -- 172.21.15.90:0/1562777828 shutdown_connections 2024-04-26T04:51:35.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 --2- 172.21.15.90:0/1562777828 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61d4069140 0x7f61d4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 --2- 172.21.15.90:0/1562777828 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d40687a0 0x7f61d4068b70 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 -- 172.21.15.90:0/1562777828 >> 172.21.15.90:0/1562777828 conn(0x7f61d4075130 msgr2=0x7f61d4075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:35.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 -- 172.21.15.90:0/1562777828 shutdown_connections 2024-04-26T04:51:35.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 -- 172.21.15.90:0/1562777828 wait complete. 2024-04-26T04:51:35.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 Processor -- start 2024-04-26T04:51:35.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.595+0000 7f61da882700 1 -- start start 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61da882700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61d40687a0 0x7f61d4195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61da882700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d4069140 0x7f61d4196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61da882700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d4196b00 con 0x7f61d4069140 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61da882700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d419a890 con 0x7f61d40687a0 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d4069140 0x7f61d4196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d4069140 0x7f61d4196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40386/0 (socket says 172.21.15.90:40386) 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d37fe700 1 -- 172.21.15.90:0/2101051301 learned_addr learned my addr 172.21.15.90:0/2101051301 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d37fe700 1 -- 172.21.15.90:0/2101051301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61d40687a0 msgr2=0x7f61d4195ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d37fe700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61d40687a0 0x7f61d4195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d37fe700 1 -- 172.21.15.90:0/2101051301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c40095f0 con 0x7f61d4069140 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d37fe700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d4069140 0x7f61d4196420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61c800d900 tx=0x7f61c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:35.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d17fa700 1 -- 172.21.15.90:0/2101051301 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c8009900 con 0x7f61d4069140 2024-04-26T04:51:35.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61d17fa700 1 -- 172.21.15.90:0/2101051301 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f61c8010460 con 0x7f61d4069140 2024-04-26T04:51:35.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.596+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d419ab70 con 0x7f61d4069140 2024-04-26T04:51:35.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.597+0000 7f61d17fa700 1 -- 172.21.15.90:0/2101051301 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f61c800f5f0 con 0x7f61d4069140 2024-04-26T04:51:35.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.597+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d419b0c0 con 0x7f61d4069140 2024-04-26T04:51:35.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.598+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d4108770 con 0x7f61d4069140 2024-04-26T04:51:35.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.598+0000 7f61d17fa700 1 -- 172.21.15.90:0/2101051301 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f61c80105d0 con 0x7f61d4069140 2024-04-26T04:51:35.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.599+0000 7f61d17fa700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f61bc06e200 0x7f61bc0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.599+0000 7f61d17fa700 1 -- 172.21.15.90:0/2101051301 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f61c808dcf0 con 0x7f61d4069140 2024-04-26T04:51:35.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.599+0000 7f61d3fff700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f61bc06e200 0x7f61bc0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.599+0000 7f61d3fff700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f61bc06e200 0x7f61bc0706b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61c400b5c0 tx=0x7f61c4005960 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:35.604 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.603+0000 7f61d17fa700 1 -- 172.21.15.90:0/2101051301 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f61c8057050 con 0x7f61d4069140 2024-04-26T04:51:35.863 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:35 smithi090 ceph-mon[86550]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:35.863 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:35 smithi090 ceph-mon[86550]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:35.863 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:35 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:35.863 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:35 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:35.863 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:35 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/518186627' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.861+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7f61d419ad00 con 0x7f61d4069140 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.861+0000 7f61d17fa700 1 -- 172.21.15.90:0/2101051301 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 25 v25) v1 ==== 76+0+1872 (secure 0 0 0) 0x7f61c8020760 con 0x7f61d4069140 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stdout:e25 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:35.864 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 22 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:51:20.563438+0000 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:51:35.865 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 81 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:up {} 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:failed 0,1 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:51:35.866 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:35.867 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:35.867 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:51:35.867 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:51:35.867 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:35.867 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{-1:34374} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6828/1614774470,v1:172.21.15.90:6829/1614774470] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:35.867 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:51:35.867 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{-1:44269} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.104:6824/1188215025,v1:172.21.15.104:6825/1188215025] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f61bc06e200 msgr2=0x7f61bc0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f61bc06e200 0x7f61bc0706b0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f61c400b5c0 tx=0x7f61c4005960 comp rx=0 tx=0).stop 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d4069140 msgr2=0x7f61d4196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d4069140 0x7f61d4196420 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f61c800d900 tx=0x7f61c800dcc0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 shutdown_connections 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f61bc06e200 0x7f61bc0706b0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f61d40687a0 0x7f61d4195ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 --2- 172.21.15.90:0/2101051301 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f61d4069140 0x7f61d4196420 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 >> 172.21.15.90:0/2101051301 conn(0x7f61d4075130 msgr2=0x7f61d40fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 shutdown_connections 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.865+0000 7f61da882700 1 -- 172.21.15.90:0/2101051301 wait complete. 2024-04-26T04:51:35.868 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T04:51:35.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 -- 172.21.15.90:0/783762974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 msgr2=0x7f3a2c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 --2- 172.21.15.90:0/783762974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c100830 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f3a20009910 tx=0x7f3a20009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:35.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 -- 172.21.15.90:0/783762974 shutdown_connections 2024-04-26T04:51:35.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 --2- 172.21.15.90:0/783762974 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a2c100d70 0x7f3a2c1094d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 --2- 172.21.15.90:0/783762974 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c100830 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 -- 172.21.15.90:0/783762974 >> 172.21.15.90:0/783762974 conn(0x7f3a2c0748b0 msgr2=0x7f3a2c074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:35.976 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 -- 172.21.15.90:0/783762974 shutdown_connections 2024-04-26T04:51:35.977 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 -- 172.21.15.90:0/783762974 wait complete. 2024-04-26T04:51:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 Processor -- start 2024-04-26T04:51:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.975+0000 7f3a331a7700 1 -- start start 2024-04-26T04:51:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a331a7700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c194de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a331a7700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a2c100d70 0x7f3a2c195320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.978 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a331a7700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c195a00 con 0x7f3a2c100460 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a331a7700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a2c199790 con 0x7f3a2c100d70 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a30f43700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c194de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a30f43700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c194de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40402/0 (socket says 172.21.15.90:40402) 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a30f43700 1 -- 172.21.15.90:0/3684611305 learned_addr learned my addr 172.21.15.90:0/3684611305 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a2bfff700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a2c100d70 0x7f3a2c195320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a30f43700 1 -- 172.21.15.90:0/3684611305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a2c100d70 msgr2=0x7f3a2c195320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a30f43700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a2c100d70 0x7f3a2c195320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a30f43700 1 -- 172.21.15.90:0/3684611305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a200095f0 con 0x7f3a2c100460 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a30f43700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c194de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a20006010 tx=0x7f3a2000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a29ffb700 1 -- 172.21.15.90:0/3684611305 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a2001f070 con 0x7f3a2c100460 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a29ffb700 1 -- 172.21.15.90:0/3684611305 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3a20003680 con 0x7f3a2c100460 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a2c199a10 con 0x7f3a2c100460 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.976+0000 7f3a29ffb700 1 -- 172.21.15.90:0/3684611305 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3a2000f7f0 con 0x7f3a2c100460 2024-04-26T04:51:35.979 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.977+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a2c199f00 con 0x7f3a2c100460 2024-04-26T04:51:35.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.978+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a2c04f2a0 con 0x7f3a2c100460 2024-04-26T04:51:35.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.978+0000 7f3a29ffb700 1 -- 172.21.15.90:0/3684611305 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3a200037f0 con 0x7f3a2c100460 2024-04-26T04:51:35.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.979+0000 7f3a29ffb700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3a1c06e200 0x7f3a1c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:35.980 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.979+0000 7f3a29ffb700 1 -- 172.21.15.90:0/3684611305 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3a200915e0 con 0x7f3a2c100460 2024-04-26T04:51:35.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.979+0000 7f3a2bfff700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3a1c06e200 0x7f3a1c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:35.981 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.979+0000 7f3a2bfff700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3a1c06e200 0x7f3a1c0706b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006fd0 tx=0x7f3a18008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:35.985 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:35.984+0000 7f3a29ffb700 1 -- 172.21.15.90:0/3684611305 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3a2005a940 con 0x7f3a2c100460 2024-04-26T04:51:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:35 smithi104 ceph-mon[64778]: from='client.34384 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:35 smithi104 ceph-mon[64778]: from='client.34388 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:35 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:36.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:35 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:36.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:35 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/518186627' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:36.244 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.242+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a2c19a2a0 con 0x7f3a1c06e200 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.244+0000 7f3a29ffb700 1 -- 172.21.15.90:0/3684611305 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3a2c19a2a0 con 0x7f3a1c06e200 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:51:36.245 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-26T04:51:36.246 INFO:teuthology.orchestra.run.smithi090.stdout: "osd" 2024-04-26T04:51:36.246 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:51:36.246 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "13/23 daemons upgraded", 2024-04-26T04:51:36.246 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mds daemons", 2024-04-26T04:51:36.246 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:51:36.246 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3a1c06e200 msgr2=0x7f3a1c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3a1c06e200 0x7f3a1c0706b0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3a18006fd0 tx=0x7f3a18008040 comp rx=0 tx=0).stop 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 msgr2=0x7f3a2c194de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c194de0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f3a20006010 tx=0x7f3a2000be20 comp rx=0 tx=0).stop 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 shutdown_connections 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3a1c06e200 0x7f3a1c0706b0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3a2c100460 0x7f3a2c194de0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 --2- 172.21.15.90:0/3684611305 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3a2c100d70 0x7f3a2c195320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.246+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 >> 172.21.15.90:0/3684611305 conn(0x7f3a2c0748b0 msgr2=0x7f3a2c103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.247+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 shutdown_connections 2024-04-26T04:51:36.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.247+0000 7f3a331a7700 1 -- 172.21.15.90:0/3684611305 wait complete. 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.354+0000 7f6e999ce700 1 -- 172.21.15.90:0/3711537827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 msgr2=0x7f6e940ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.354+0000 7f6e999ce700 1 --2- 172.21.15.90:0/3711537827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e940ff720 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f6e84009910 tx=0x7f6e84009c20 comp rx=0 tx=0).stop 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.354+0000 7f6e999ce700 1 -- 172.21.15.90:0/3711537827 shutdown_connections 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.354+0000 7f6e999ce700 1 --2- 172.21.15.90:0/3711537827 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e940ffc60 0x7f6e9410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.354+0000 7f6e999ce700 1 --2- 172.21.15.90:0/3711537827 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e940ff720 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.354+0000 7f6e999ce700 1 -- 172.21.15.90:0/3711537827 >> 172.21.15.90:0/3711537827 conn(0x7f6e940748b0 msgr2=0x7f6e94074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.354+0000 7f6e999ce700 1 -- 172.21.15.90:0/3711537827 shutdown_connections 2024-04-26T04:51:36.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.355+0000 7f6e999ce700 1 -- 172.21.15.90:0/3711537827 wait complete. 2024-04-26T04:51:36.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.355+0000 7f6e999ce700 1 Processor -- start 2024-04-26T04:51:36.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e999ce700 1 -- start start 2024-04-26T04:51:36.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e999ce700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e94195ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:36.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e999ce700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e940ffc60 0x7f6e941963e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:36.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e999ce700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e94196a30 con 0x7f6e940ff350 2024-04-26T04:51:36.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e999ce700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e94196b70 con 0x7f6e940ffc60 2024-04-26T04:51:36.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e94195ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e92ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e94195ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40408/0 (socket says 172.21.15.90:40408) 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e92ffd700 1 -- 172.21.15.90:0/2358318984 learned_addr learned my addr 172.21.15.90:0/2358318984 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e927fc700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e940ffc60 0x7f6e941963e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e92ffd700 1 -- 172.21.15.90:0/2358318984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e940ffc60 msgr2=0x7f6e941963e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e92ffd700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e940ffc60 0x7f6e941963e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e92ffd700 1 -- 172.21.15.90:0/2358318984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e840095f0 con 0x7f6e940ff350 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.356+0000 7f6e92ffd700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e94195ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400bda0 tx=0x7f6e8400be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.357+0000 7f6e989cc700 1 -- 172.21.15.90:0/2358318984 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e8401d070 con 0x7f6e940ff350 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.357+0000 7f6e989cc700 1 -- 172.21.15.90:0/2358318984 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6e84004fb0 con 0x7f6e940ff350 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.357+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9419a960 con 0x7f6e940ff350 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.357+0000 7f6e989cc700 1 -- 172.21.15.90:0/2358318984 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6e8400f480 con 0x7f6e940ff350 2024-04-26T04:51:36.359 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.357+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9419ae50 con 0x7f6e940ff350 2024-04-26T04:51:36.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.358+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e94107dd0 con 0x7f6e940ff350 2024-04-26T04:51:36.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.358+0000 7f6e989cc700 1 -- 172.21.15.90:0/2358318984 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6e84022870 con 0x7f6e940ff350 2024-04-26T04:51:36.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.359+0000 7f6e989cc700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e7c06e200 0x7f6e7c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:51:36.360 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.359+0000 7f6e989cc700 1 -- 172.21.15.90:0/2358318984 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6e8408f6c0 con 0x7f6e940ff350 2024-04-26T04:51:36.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.359+0000 7f6e927fc700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e7c06e200 0x7f6e7c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:51:36.361 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.359+0000 7f6e927fc700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e7c06e200 0x7f6e7c0706b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e88005ea0 tx=0x7f6e88005e30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:51:36.364 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.364+0000 7f6e989cc700 1 -- 172.21.15.90:0/2358318984 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6e84058a50 con 0x7f6e940ff350 2024-04-26T04:51:36.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.659+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f6e9404f2a0 con 0x7f6e940ff350 2024-04-26T04:51:36.660 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.659+0000 7f6e989cc700 1 -- 172.21.15.90:0/2358318984 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7f6e8405c110 con 0x7f6e940ff350 2024-04-26T04:51:36.661 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-26T04:51:36.661 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T04:51:36.661 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs is degraded 2024-04-26T04:51:36.661 INFO:teuthology.orchestra.run.smithi090.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-26T04:51:36.661 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e7c06e200 msgr2=0x7f6e7c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e7c06e200 0x7f6e7c0706b0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e88005ea0 tx=0x7f6e88005e30 comp rx=0 tx=0).stop 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 msgr2=0x7f6e94195ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e94195ea0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400bda0 tx=0x7f6e8400be80 comp rx=0 tx=0).stop 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 shutdown_connections 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6e7c06e200 0x7f6e7c0706b0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6e940ff350 0x7f6e94195ea0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.663 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 --2- 172.21.15.90:0/2358318984 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6e940ffc60 0x7f6e941963e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:51:36.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 >> 172.21.15.90:0/2358318984 conn(0x7f6e940748b0 msgr2=0x7f6e94104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:51:36.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 shutdown_connections 2024-04-26T04:51:36.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:51:36.662+0000 7f6e999ce700 1 -- 172.21.15.90:0/2358318984 wait complete. 2024-04-26T04:51:36.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:36 smithi090 ceph-mon[86550]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:36.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:36 smithi090 ceph-mon[86550]: pgmap v175: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:36.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:36 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2101051301' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:51:36.924 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:36 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:36.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:36 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:36.925 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:36 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2358318984' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:51:37.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:36 smithi104 ceph-mon[64778]: from='client.34392 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:37.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:36 smithi104 ceph-mon[64778]: pgmap v175: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:37.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:36 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2101051301' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:51:37.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:36 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:37.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:36 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:37.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:36 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2358318984' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:51:38.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:37 smithi104 ceph-mon[64778]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:38.143 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:37 smithi090 ceph-mon[86550]: from='client.34404 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:51:39.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:38 smithi104 ceph-mon[64778]: pgmap v176: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:39.135 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:38 smithi090 ceph-mon[86550]: pgmap v176: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: pgmap v177: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:51:41.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:41.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:41.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:41.073 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: pgmap v177: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:51:41.074 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:42.905 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:42 smithi090 ceph-mon[86550]: pgmap v178: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:42.905 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:42 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:42.905 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:42 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:51:42.905 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:42 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:42 smithi104 ceph-mon[64778]: pgmap v178: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:42 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:42 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi090.dbkjuk", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:51:43.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:42 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:51:43.868 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:43 smithi090 ceph-mon[86550]: Upgrade: Updating mds.cephfs.smithi090.dbkjuk 2024-04-26T04:51:43.868 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:43 smithi090 ceph-mon[86550]: Deploying daemon mds.cephfs.smithi090.dbkjuk on smithi090 2024-04-26T04:51:44.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:43 smithi104 ceph-mon[64778]: Upgrade: Updating mds.cephfs.smithi090.dbkjuk 2024-04-26T04:51:44.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:43 smithi104 ceph-mon[64778]: Deploying daemon mds.cephfs.smithi090.dbkjuk on smithi090 2024-04-26T04:51:45.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:44 smithi104 ceph-mon[64778]: pgmap v179: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:44 smithi090 ceph-mon[86550]: pgmap v179: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:45 smithi104 ceph-mon[64778]: osdmap e83: 6 total, 6 up, 6 in 2024-04-26T04:51:46.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:45 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:51:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:45 smithi090 ceph-mon[86550]: osdmap e83: 6 total, 6 up, 6 in 2024-04-26T04:51:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:45 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:51:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:46 smithi090 ceph-mon[86550]: pgmap v181: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:46 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:51:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:46 smithi104 ceph-mon[64778]: pgmap v181: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:46 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:51:49.149 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:48 smithi090 ceph-mon[86550]: pgmap v182: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:48 smithi104 ceph-mon[64778]: pgmap v182: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:50 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:50 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:50 smithi090 ceph-mon[86550]: pgmap v183: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:50 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:50.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:50 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:boot 2024-04-26T04:51:50.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:50 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:51:50.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:50 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:51:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:50 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:50 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:50 smithi104 ceph-mon[64778]: pgmap v183: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:50 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:51:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:50 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:boot 2024-04-26T04:51:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:50 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:51:51.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:50 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:51:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:52 smithi090 ceph-mon[86550]: pgmap v184: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:52 smithi104 ceph-mon[64778]: pgmap v184: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:54 smithi090 ceph-mon[86550]: pgmap v185: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:54 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:54 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:54 smithi104 ceph-mon[64778]: pgmap v185: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:54 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:54 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:56.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:51:57.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:56 smithi090 ceph-mon[86550]: pgmap v186: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:57.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:57.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:56 smithi104 ceph-mon[64778]: pgmap v186: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:51:59.054 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:51:58 smithi090 ceph-mon[86550]: pgmap v187: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:51:59.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:51:58 smithi104 ceph-mon[64778]: pgmap v187: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:00.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:00.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:00 smithi090 ceph-mon[86550]: pgmap v188: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:00.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:00.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:00.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:00 smithi104 ceph-mon[64778]: pgmap v188: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:02 smithi104 ceph-mon[64778]: Upgrade: Updating mds.cephfs.smithi104.wpntdi 2024-04-26T04:52:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:02 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:02 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:52:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:02 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:02 smithi104 ceph-mon[64778]: Deploying daemon mds.cephfs.smithi104.wpntdi on smithi104 2024-04-26T04:52:02.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:02 smithi104 ceph-mon[64778]: pgmap v189: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:02 smithi090 ceph-mon[86550]: Upgrade: Updating mds.cephfs.smithi104.wpntdi 2024-04-26T04:52:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:02.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.wpntdi", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:52:02.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:02.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:02 smithi090 ceph-mon[86550]: Deploying daemon mds.cephfs.smithi104.wpntdi on smithi104 2024-04-26T04:52:02.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:02 smithi090 ceph-mon[86550]: pgmap v189: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:04 smithi090 ceph-mon[86550]: osdmap e84: 6 total, 6 up, 6 in 2024-04-26T04:52:04.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:04 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:52:04.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:04 smithi090 ceph-mon[86550]: pgmap v191: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:04 smithi104 ceph-mon[64778]: osdmap e84: 6 total, 6 up, 6 in 2024-04-26T04:52:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:04 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:52:04.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:04 smithi104 ceph-mon[64778]: pgmap v191: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:06.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/875047435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 msgr2=0x7f3d0410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:06.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/875047435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d0410a230 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4009910 tx=0x7f3cf4009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:06.784 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/875047435 shutdown_connections 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/875047435 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d0410a230 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/875047435 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d04101420 0x7f3d041017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/875047435 >> 172.21.15.90:0/875047435 conn(0x7f3d040fac80 msgr2=0x7f3d040fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/875047435 shutdown_connections 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.781+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/875047435 wait complete. 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0c0e1700 1 Processor -- start 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0c0e1700 1 -- start start 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0c0e1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d04101420 0x7f3d040ff080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0c0e1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d040ff5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0c0e1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040ffb00 con 0x7f3d04101d30 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0c0e1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d040ffc40 con 0x7f3d04101420 2024-04-26T04:52:06.785 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0967c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d040ff5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0967c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d040ff5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38124/0 (socket says 172.21.15.90:38124) 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d0967c700 1 -- 172.21.15.90:0/3734651139 learned_addr learned my addr 172.21.15.90:0/3734651139 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.782+0000 7f3d09e7d700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d04101420 0x7f3d040ff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3d0967c700 1 -- 172.21.15.90:0/3734651139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d04101420 msgr2=0x7f3d040ff080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3d0967c700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d04101420 0x7f3d040ff080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3d0967c700 1 -- 172.21.15.90:0/3734651139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cf40095f0 con 0x7f3d04101d30 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3d0967c700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d040ff5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4005f50 tx=0x7f3cf400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3cfaffd700 1 -- 172.21.15.90:0/3734651139 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf401d070 con 0x7f3d04101d30 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3cfaffd700 1 -- 172.21.15.90:0/3734651139 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3cf400b610 con 0x7f3d04101d30 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d040ffec0 con 0x7f3d04101d30 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3cfaffd700 1 -- 172.21.15.90:0/3734651139 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3cf4022470 con 0x7f3d04101d30 2024-04-26T04:52:06.786 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.783+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d041a2360 con 0x7f3d04101d30 2024-04-26T04:52:06.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.784+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d041078a0 con 0x7f3d04101d30 2024-04-26T04:52:06.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.784+0000 7f3cfaffd700 1 -- 172.21.15.90:0/3734651139 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3cf4003780 con 0x7f3d04101d30 2024-04-26T04:52:06.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.785+0000 7f3cfaffd700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3cf006e200 0x7f3cf00706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:06.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.785+0000 7f3cfaffd700 1 -- 172.21.15.90:0/3734651139 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f3cf408f580 con 0x7f3d04101d30 2024-04-26T04:52:06.787 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.785+0000 7f3d09e7d700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3cf006e200 0x7f3cf00706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:06.788 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.785+0000 7f3d09e7d700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3cf006e200 0x7f3cf00706b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009c60 tx=0x7f3d00009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:06.791 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:06.790+0000 7f3cfaffd700 1 -- 172.21.15.90:0/3734651139 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3cf4058860 con 0x7f3d04101d30 2024-04-26T04:52:07.011 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.010+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f3d041006c0 con 0x7f3cf006e200 2024-04-26T04:52:07.012 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.011+0000 7f3cfaffd700 1 -- 172.21.15.90:0/3734651139 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f3d041006c0 con 0x7f3cf006e200 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.013+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3cf006e200 msgr2=0x7f3cf00706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.013+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3cf006e200 0x7f3cf00706b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3d00009c60 tx=0x7f3d00009260 comp rx=0 tx=0).stop 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.013+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 msgr2=0x7f3d040ff5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.013+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d040ff5c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f3cf4005f50 tx=0x7f3cf400f690 comp rx=0 tx=0).stop 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.014+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 shutdown_connections 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.014+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f3cf006e200 0x7f3cf00706b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.014+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3d04101420 0x7f3d040ff080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.014+0000 7f3d0c0e1700 1 --2- 172.21.15.90:0/3734651139 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3d04101d30 0x7f3d040ff5c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.014+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 >> 172.21.15.90:0/3734651139 conn(0x7f3d040fac80 msgr2=0x7f3d041049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:07.015 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.014+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 shutdown_connections 2024-04-26T04:52:07.016 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.014+0000 7f3d0c0e1700 1 -- 172.21.15.90:0/3734651139 wait complete. 2024-04-26T04:52:07.025 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:52:07.025 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:06 smithi090 ceph-mon[86550]: pgmap v192: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:07.025 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:07.025 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:07.025 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:06 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 -- 172.21.15.90:0/4054657662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 msgr2=0x7f6974103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.120 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 --2- 172.21.15.90:0/4054657662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974103d30 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f6970009910 tx=0x7f6970009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:07.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 -- 172.21.15.90:0/4054657662 shutdown_connections 2024-04-26T04:52:07.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 --2- 172.21.15.90:0/4054657662 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974103d30 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 --2- 172.21.15.90:0/4054657662 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69740ff3c0 0x7f69740ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 -- 172.21.15.90:0/4054657662 >> 172.21.15.90:0/4054657662 conn(0x7f69740fac80 msgr2=0x7f69740fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:07.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 -- 172.21.15.90:0/4054657662 shutdown_connections 2024-04-26T04:52:07.121 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.119+0000 7f697b5c2700 1 -- 172.21.15.90:0/4054657662 wait complete. 2024-04-26T04:52:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f697b5c2700 1 Processor -- start 2024-04-26T04:52:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f697b5c2700 1 -- start start 2024-04-26T04:52:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f697b5c2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69740ff3c0 0x7f6974196000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f697b5c2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f697b5c2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6974196c20 con 0x7f69740ffd60 2024-04-26T04:52:07.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f697b5c2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697419a9b0 con 0x7f69740ff3c0 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f6978b5d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f6978b5d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974196540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38140/0 (socket says 172.21.15.90:38140) 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f6978b5d700 1 -- 172.21.15.90:0/3237536949 learned_addr learned my addr 172.21.15.90:0/3237536949 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f697935e700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69740ff3c0 0x7f6974196000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f6978b5d700 1 -- 172.21.15.90:0/3237536949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69740ff3c0 msgr2=0x7f6974196000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f6978b5d700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69740ff3c0 0x7f6974196000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f6978b5d700 1 -- 172.21.15.90:0/3237536949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700095f0 con 0x7f69740ffd60 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.120+0000 7f6978b5d700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974196540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f697000bda0 tx=0x7f697000be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.121+0000 7f696a7fc700 1 -- 172.21.15.90:0/3237536949 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f697001d070 con 0x7f69740ffd60 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.121+0000 7f696a7fc700 1 -- 172.21.15.90:0/3237536949 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6970004fb0 con 0x7f69740ffd60 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.121+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697419ac30 con 0x7f69740ffd60 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.121+0000 7f696a7fc700 1 -- 172.21.15.90:0/3237536949 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f697000f480 con 0x7f69740ffd60 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.121+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697419b120 con 0x7f69740ffd60 2024-04-26T04:52:07.124 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.122+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697404f2a0 con 0x7f69740ffd60 2024-04-26T04:52:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.122+0000 7f696a7fc700 1 -- 172.21.15.90:0/3237536949 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6970022870 con 0x7f69740ffd60 2024-04-26T04:52:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.123+0000 7f696a7fc700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f696006e200 0x7f69600706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.123+0000 7f697935e700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f696006e200 0x7f69600706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.123+0000 7f696a7fc700 1 -- 172.21.15.90:0/3237536949 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f697008f710 con 0x7f69740ffd60 2024-04-26T04:52:07.125 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.123+0000 7f697935e700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f696006e200 0x7f69600706b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6964006fd0 tx=0x7f6964008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:07.128 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.127+0000 7f696a7fc700 1 -- 172.21.15.90:0/3237536949 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6970058c40 con 0x7f69740ffd60 2024-04-26T04:52:07.218 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:06 smithi104 ceph-mon[64778]: pgmap v192: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:07.218 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:07.218 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:07.218 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:06 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:07.348 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.347+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f697419b400 con 0x7f696006e200 2024-04-26T04:52:07.349 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.348+0000 7f696a7fc700 1 -- 172.21.15.90:0/3237536949 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7f697419b400 con 0x7f696006e200 2024-04-26T04:52:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f696006e200 msgr2=0x7f69600706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f696006e200 0x7f69600706b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f6964006fd0 tx=0x7f6964008040 comp rx=0 tx=0).stop 2024-04-26T04:52:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 msgr2=0x7f6974196540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974196540 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f697000bda0 tx=0x7f697000be80 comp rx=0 tx=0).stop 2024-04-26T04:52:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 shutdown_connections 2024-04-26T04:52:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f696006e200 0x7f69600706b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.352 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f69740ff3c0 0x7f6974196000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 --2- 172.21.15.90:0/3237536949 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f69740ffd60 0x7f6974196540 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 >> 172.21.15.90:0/3237536949 conn(0x7f69740fac80 msgr2=0x7f69741034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:07.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 shutdown_connections 2024-04-26T04:52:07.353 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.351+0000 7f697b5c2700 1 -- 172.21.15.90:0/3237536949 wait complete. 2024-04-26T04:52:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 -- 172.21.15.90:0/2522848848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c0687a0 msgr2=0x7f479c068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.455 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 --2- 172.21.15.90:0/2522848848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c0687a0 0x7f479c068b70 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f479400b0a0 tx=0x7f479400b3b0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 -- 172.21.15.90:0/2522848848 shutdown_connections 2024-04-26T04:52:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 --2- 172.21.15.90:0/2522848848 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479c069140 0x7f479c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 --2- 172.21.15.90:0/2522848848 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c0687a0 0x7f479c068b70 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 -- 172.21.15.90:0/2522848848 >> 172.21.15.90:0/2522848848 conn(0x7f479c075130 msgr2=0x7f479c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 -- 172.21.15.90:0/2522848848 shutdown_connections 2024-04-26T04:52:07.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.454+0000 7f47a1011700 1 -- 172.21.15.90:0/2522848848 wait complete. 2024-04-26T04:52:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f47a1011700 1 Processor -- start 2024-04-26T04:52:07.457 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f47a1011700 1 -- start start 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f47a1011700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479c0687a0 0x7f479c0ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f47a1011700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c069140 0x7f479c1001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f47a1011700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c100720 con 0x7f479c069140 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f47a1011700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f479c100860 con 0x7f479c0687a0 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f479a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c069140 0x7f479c1001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f479a59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c069140 0x7f479c1001e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:38160/0 (socket says 172.21.15.90:38160) 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f479a59c700 1 -- 172.21.15.90:0/2284780653 learned_addr learned my addr 172.21.15.90:0/2284780653 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.455+0000 7f479ad9d700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479c0687a0 0x7f479c0ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f479a59c700 1 -- 172.21.15.90:0/2284780653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479c0687a0 msgr2=0x7f479c0ffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f479a59c700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479c0687a0 0x7f479c0ffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f479a59c700 1 -- 172.21.15.90:0/2284780653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4794009d00 con 0x7f479c069140 2024-04-26T04:52:07.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f479a59c700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c069140 0x7f479c1001e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f478400c8f0 tx=0x7f478400ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f4793fff700 1 -- 172.21.15.90:0/2284780653 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4784007a50 con 0x7f479c069140 2024-04-26T04:52:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f4793fff700 1 -- 172.21.15.90:0/2284780653 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4784007bb0 con 0x7f479c069140 2024-04-26T04:52:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f479c1a20b0 con 0x7f479c069140 2024-04-26T04:52:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f4793fff700 1 -- 172.21.15.90:0/2284780653 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4784018630 con 0x7f479c069140 2024-04-26T04:52:07.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.456+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f479c1a24c0 con 0x7f479c069140 2024-04-26T04:52:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.457+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f479c108770 con 0x7f479c069140 2024-04-26T04:52:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.458+0000 7f4793fff700 1 -- 172.21.15.90:0/2284780653 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4784007d20 con 0x7f479c069140 2024-04-26T04:52:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.458+0000 7f4793fff700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f478806e200 0x7f47880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.458+0000 7f4793fff700 1 -- 172.21.15.90:0/2284780653 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f478408e160 con 0x7f479c069140 2024-04-26T04:52:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.458+0000 7f479ad9d700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f478806e200 0x7f47880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.459+0000 7f479ad9d700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f478806e200 0x7f47880706b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f479400b830 tx=0x7f47940097c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:07.464 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.463+0000 7f4793fff700 1 -- 172.21.15.90:0/2284780653 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4784057440 con 0x7f479c069140 2024-04-26T04:52:07.672 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.671+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7f479c061190 con 0x7f478806e200 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.682+0000 7f4793fff700 1 -- 172.21.15.90:0/2284780653 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7f479c061190 con 0x7f478806e200 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (6m) 12s ago 17m 24.3M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (17m) 12s ago 17m 9017k - 18.2.0 dc2bc1663786 cb0a3313600c 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (16m) 52s ago 16m 8711k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (4m) 12s ago 17m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (4m) 52s ago 16m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:52:07.683 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (6m) 12s ago 17m 80.3M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (18s) 12s ago 12m 12.4M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a01df3d7e4d4 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (38s) 12s ago 12m 13.4M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 4bcc1e9fbb69 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (12m) 52s ago 12m 17.9M - 18.2.0 dc2bc1663786 8ec652b060fc 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 starting - - - - 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (8m) 12s ago 19m 508M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (7m) 52s ago 16m 450M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (5m) 12s ago 19m 55.9M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (4m) 52s ago 16m 43.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (7m) 12s ago 17m 16.2M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (7m) 52s ago 16m 14.7M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (3m) 12s ago 15m 93.0M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (3m) 12s ago 15m 104M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (2m) 12s ago 14m 83.7M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (2m) 52s ago 14m 108M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d860fd1e9069 2024-04-26T04:52:07.684 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (88s) 52s ago 14m 92.1M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 fbfd5c1c44eb 2024-04-26T04:52:07.685 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (55s) 52s ago 13m 12.3M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 64a9be8458bb 2024-04-26T04:52:07.685 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (7m) 12s ago 17m 69.2M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:52:07.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f478806e200 msgr2=0x7f47880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.685 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f478806e200 0x7f47880706b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f479400b830 tx=0x7f47940097c0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c069140 msgr2=0x7f479c1001e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c069140 0x7f479c1001e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f478400c8f0 tx=0x7f478400ccb0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 shutdown_connections 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f478806e200 0x7f47880706b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f479c0687a0 0x7f479c0ffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 --2- 172.21.15.90:0/2284780653 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f479c069140 0x7f479c1001e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 >> 172.21.15.90:0/2284780653 conn(0x7f479c075130 msgr2=0x7f479c0fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 shutdown_connections 2024-04-26T04:52:07.686 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.684+0000 7f47a1011700 1 -- 172.21.15.90:0/2284780653 wait complete. 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 -- 172.21.15.90:0/319249800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 msgr2=0x7fe720068b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 --2- 172.21.15.90:0/319249800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720068b70 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe71800b0a0 tx=0x7fe71800b3b0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 -- 172.21.15.90:0/319249800 shutdown_connections 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 --2- 172.21.15.90:0/319249800 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe720069140 0x7fe720104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 --2- 172.21.15.90:0/319249800 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720068b70 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 -- 172.21.15.90:0/319249800 >> 172.21.15.90:0/319249800 conn(0x7fe720075130 msgr2=0x7fe720075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 -- 172.21.15.90:0/319249800 shutdown_connections 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.788+0000 7fe726ab6700 1 -- 172.21.15.90:0/319249800 wait complete. 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe726ab6700 1 Processor -- start 2024-04-26T04:52:07.790 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe726ab6700 1 -- start start 2024-04-26T04:52:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe726ab6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720195ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe726ab6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe720069140 0x7fe720196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe726ab6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe720196b00 con 0x7fe7200687a0 2024-04-26T04:52:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe726ab6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72019a890 con 0x7fe720069140 2024-04-26T04:52:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe724852700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720195ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.792 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe724852700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720195ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35808/0 (socket says 172.21.15.90:35808) 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe724852700 1 -- 172.21.15.90:0/883099840 learned_addr learned my addr 172.21.15.90:0/883099840 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe71ffff700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe720069140 0x7fe720196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe724852700 1 -- 172.21.15.90:0/883099840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe720069140 msgr2=0x7fe720196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.789+0000 7fe724852700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe720069140 0x7fe720196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.790+0000 7fe724852700 1 -- 172.21.15.90:0/883099840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe718009d00 con 0x7fe7200687a0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.790+0000 7fe724852700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720195ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe718009bb0 tx=0x7fe71800bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.790+0000 7fe71dffb700 1 -- 172.21.15.90:0/883099840 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe71800e040 con 0x7fe7200687a0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.790+0000 7fe71dffb700 1 -- 172.21.15.90:0/883099840 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe718012b00 con 0x7fe7200687a0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.790+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72019ab10 con 0x7fe7200687a0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.790+0000 7fe71dffb700 1 -- 172.21.15.90:0/883099840 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe71801db80 con 0x7fe7200687a0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.790+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72019b000 con 0x7fe7200687a0 2024-04-26T04:52:07.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.791+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe720108790 con 0x7fe7200687a0 2024-04-26T04:52:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.792+0000 7fe71dffb700 1 -- 172.21.15.90:0/883099840 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe718019070 con 0x7fe7200687a0 2024-04-26T04:52:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.792+0000 7fe71dffb700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe70c06e200 0x7fe70c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.792+0000 7fe71ffff700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe70c06e200 0x7fe70c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.792+0000 7fe71dffb700 1 -- 172.21.15.90:0/883099840 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe71808f640 con 0x7fe7200687a0 2024-04-26T04:52:07.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.793+0000 7fe71ffff700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe70c06e200 0x7fe70c0706b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe720197500 tx=0x7fe71000b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:07.798 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:07.797+0000 7fe71dffb700 1 -- 172.21.15.90:0/883099840 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe7180589a0 con 0x7fe7200687a0 2024-04-26T04:52:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:07 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.104:6824/2998394587,v1:172.21.15.104:6825/2998394587] up:boot 2024-04-26T04:52:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:07 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:52:07.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:07 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:52:08.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:07 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.104:6824/2998394587,v1:172.21.15.104:6825/2998394587] up:boot 2024-04-26T04:52:08.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:07 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:52:08.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:07 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:52:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.082+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fe72004f2a0 con 0x7fe7200687a0 2024-04-26T04:52:08.083 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.082+0000 7fe71dffb700 1 -- 172.21.15.90:0/883099840 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+805 (secure 0 0 0) 0x7fe71805c060 con 0x7fe7200687a0 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:52:08.084 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-26T04:52:08.085 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 3 2024-04-26T04:52:08.085 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:08.085 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:52:08.085 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 18.2.0 (5dd24139a1eada541a3bc16b6941c5dde975e26d) reef (stable)": 1, 2024-04-26T04:52:08.085 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 13 2024-04-26T04:52:08.085 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:52:08.085 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:52:08.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe70c06e200 msgr2=0x7fe70c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe70c06e200 0x7fe70c0706b0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe720197500 tx=0x7fe71000b410 comp rx=0 tx=0).stop 2024-04-26T04:52:08.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 msgr2=0x7fe720195ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.086 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720195ee0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fe718009bb0 tx=0x7fe71800bd10 comp rx=0 tx=0).stop 2024-04-26T04:52:08.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 shutdown_connections 2024-04-26T04:52:08.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe70c06e200 0x7fe70c0706b0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe7200687a0 0x7fe720195ee0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 --2- 172.21.15.90:0/883099840 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe720069140 0x7fe720196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 >> 172.21.15.90:0/883099840 conn(0x7fe720075130 msgr2=0x7fe7200fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:08.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 shutdown_connections 2024-04-26T04:52:08.087 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.085+0000 7fe726ab6700 1 -- 172.21.15.90:0/883099840 wait complete. 2024-04-26T04:52:08.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 -- 172.21.15.90:0/3797477778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc558104130 msgr2=0x7fc558104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 --2- 172.21.15.90:0/3797477778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc558104130 0x7fc558104500 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fc548009910 tx=0x7fc548009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 -- 172.21.15.90:0/3797477778 shutdown_connections 2024-04-26T04:52:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 --2- 172.21.15.90:0/3797477778 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc5580ffbd0 0x7fc558100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 --2- 172.21.15.90:0/3797477778 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc558104130 0x7fc558104500 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 -- 172.21.15.90:0/3797477778 >> 172.21.15.90:0/3797477778 conn(0x7fc558075130 msgr2=0x7fc558075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 -- 172.21.15.90:0/3797477778 shutdown_connections 2024-04-26T04:52:08.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.190+0000 7fc55cb08700 1 -- 172.21.15.90:0/3797477778 wait complete. 2024-04-26T04:52:08.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55cb08700 1 Processor -- start 2024-04-26T04:52:08.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55cb08700 1 -- start start 2024-04-26T04:52:08.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55cb08700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5580ffbd0 0x7fc558194d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55cb08700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc558104130 0x7fc558195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55cb08700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc558195960 con 0x7fc5580ffbd0 2024-04-26T04:52:08.194 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55cb08700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5581996f0 con 0x7fc558104130 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5580ffbd0 0x7fc558194d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55659c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5580ffbd0 0x7fc558194d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35814/0 (socket says 172.21.15.90:35814) 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc55659c700 1 -- 172.21.15.90:0/1374580728 learned_addr learned my addr 172.21.15.90:0/1374580728 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.191+0000 7fc555d9b700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc558104130 0x7fc558195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc55659c700 1 -- 172.21.15.90:0/1374580728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc558104130 msgr2=0x7fc558195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc55659c700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc558104130 0x7fc558195280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc55659c700 1 -- 172.21.15.90:0/1374580728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5480095f0 con 0x7fc5580ffbd0 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc555d9b700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc558104130 0x7fc558195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc55659c700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5580ffbd0 0x7fc558194d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc54800b7d0 tx=0x7fc54800bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc5477fe700 1 -- 172.21.15.90:0/1374580728 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc54801d070 con 0x7fc5580ffbd0 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc5477fe700 1 -- 172.21.15.90:0/1374580728 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc5480052a0 con 0x7fc5580ffbd0 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc558199970 con 0x7fc5580ffbd0 2024-04-26T04:52:08.195 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc5477fe700 1 -- 172.21.15.90:0/1374580728 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc548022470 con 0x7fc5580ffbd0 2024-04-26T04:52:08.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.192+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc558199e60 con 0x7fc5580ffbd0 2024-04-26T04:52:08.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.193+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55804f2a0 con 0x7fc5580ffbd0 2024-04-26T04:52:08.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.194+0000 7fc5477fe700 1 -- 172.21.15.90:0/1374580728 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc54800f850 con 0x7fc5580ffbd0 2024-04-26T04:52:08.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.194+0000 7fc5477fe700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc54006e200 0x7fc5400706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.194+0000 7fc5477fe700 1 -- 172.21.15.90:0/1374580728 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc54808f6a0 con 0x7fc5580ffbd0 2024-04-26T04:52:08.196 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.194+0000 7fc555d9b700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc54006e200 0x7fc5400706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.197 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.195+0000 7fc555d9b700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc54006e200 0x7fc5400706b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc558196360 tx=0x7fc54c009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:08.200 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.199+0000 7fc5477fe700 1 -- 172.21.15.90:0/1374580728 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc548058980 con 0x7fc5580ffbd0 2024-04-26T04:52:08.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.452+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc55819a050 con 0x7fc5580ffbd0 2024-04-26T04:52:08.454 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.453+0000 7fc5477fe700 1 -- 172.21.15.90:0/1374580728 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) v1 ==== 76+0+1872 (secure 0 0 0) 0x7fc54805c040 con 0x7fc5580ffbd0 2024-04-26T04:52:08.454 INFO:teuthology.orchestra.run.smithi090.stdout:e29 2024-04-26T04:52:08.454 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:52:08.454 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:52:08.454 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 22 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:flags 13 allow_snaps allow_multimds_snaps 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:51:20.563438+0000 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:52:08.455 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 81 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:up {} 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:failed 0,1 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:52:08.456 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:24283} state up:standby seq 2 join_fscid=1 addr [v2:172.21.15.104:6826/2655656935,v1:172.21.15.104:6827/2655656935] compat {c=[1],r=[1],i=[7ff]}] 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{-1:34380} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{-1:34412} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:52:08.457 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.104:6824/2998394587,v1:172.21.15.104:6825/2998394587] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:52:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc54006e200 msgr2=0x7fc5400706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc54006e200 0x7fc5400706b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc558196360 tx=0x7fc54c009260 comp rx=0 tx=0).stop 2024-04-26T04:52:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5580ffbd0 msgr2=0x7fc558194d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.458 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5580ffbd0 0x7fc558194d40 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fc54800b7d0 tx=0x7fc54800bf90 comp rx=0 tx=0).stop 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 shutdown_connections 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc54006e200 0x7fc5400706b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc5580ffbd0 0x7fc558194d40 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 --2- 172.21.15.90:0/1374580728 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc558104130 0x7fc558195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 >> 172.21.15.90:0/1374580728 conn(0x7fc558075130 msgr2=0x7fc5580fec10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 shutdown_connections 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.455+0000 7fc55cb08700 1 -- 172.21.15.90:0/1374580728 wait complete. 2024-04-26T04:52:08.459 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-04-26T04:52:08.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 -- 172.21.15.90:0/1748553815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 msgr2=0x7fa1040689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/1748553815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040689b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8009910 tx=0x7fa0f8009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:08.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 -- 172.21.15.90:0/1748553815 shutdown_connections 2024-04-26T04:52:08.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/1748553815 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040689b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/1748553815 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa104105230 0x7fa104105600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 -- 172.21.15.90:0/1748553815 >> 172.21.15.90:0/1748553815 conn(0x7fa104075130 msgr2=0x7fa104075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:08.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 -- 172.21.15.90:0/1748553815 shutdown_connections 2024-04-26T04:52:08.562 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.560+0000 7fa10a5b5700 1 -- 172.21.15.90:0/1748553815 wait complete. 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa10a5b5700 1 Processor -- start 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa10a5b5700 1 -- start start 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa10a5b5700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040ffca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa10a5b5700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa104105230 0x7fa1041001e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa10a5b5700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104100720 con 0x7fa104068540 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa10a5b5700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa104100860 con 0x7fa104105230 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa103fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040ffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.564 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa103fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040ffca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35844/0 (socket says 172.21.15.90:35844) 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa103fff700 1 -- 172.21.15.90:0/2483321843 learned_addr learned my addr 172.21.15.90:0/2483321843 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.561+0000 7fa1037fe700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa104105230 0x7fa1041001e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa103fff700 1 -- 172.21.15.90:0/2483321843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa104105230 msgr2=0x7fa1041001e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa103fff700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa104105230 0x7fa1041001e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa103fff700 1 -- 172.21.15.90:0/2483321843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0f80095f0 con 0x7fa104068540 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa103fff700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040ffca0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400d900 tx=0x7fa0f400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa1017fa700 1 -- 172.21.15.90:0/2483321843 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f40041f0 con 0x7fa104068540 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa1017fa700 1 -- 172.21.15.90:0/2483321843 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa0f4004cf0 con 0x7fa104068540 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1041a1f90 con 0x7fa104068540 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa1017fa700 1 -- 172.21.15.90:0/2483321843 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa0f400b750 con 0x7fa104068540 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.562+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1041a23a0 con 0x7fa104068540 2024-04-26T04:52:08.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.563+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa104108750 con 0x7fa104068540 2024-04-26T04:52:08.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.563+0000 7fa1017fa700 1 -- 172.21.15.90:0/2483321843 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa0f4004750 con 0x7fa104068540 2024-04-26T04:52:08.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.564+0000 7fa1017fa700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa0ec06e200 0x7fa0ec0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.564+0000 7fa1017fa700 1 -- 172.21.15.90:0/2483321843 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa0f408e070 con 0x7fa104068540 2024-04-26T04:52:08.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.564+0000 7fa1037fe700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa0ec06e200 0x7fa0ec0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.564+0000 7fa1037fe700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa0ec06e200 0x7fa0ec0706b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa104101490 tx=0x7fa0f801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:08.570 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.568+0000 7fa1017fa700 1 -- 172.21.15.90:0/2483321843 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa0f4057350 con 0x7fa104068540 2024-04-26T04:52:08.793 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.791+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fa10402cca0 con 0x7fa0ec06e200 2024-04-26T04:52:08.794 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.793+0000 7fa1017fa700 1 -- 172.21.15.90:0/2483321843 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+416 (secure 0 0 0) 0x7fa10402cca0 con 0x7fa0ec06e200 2024-04-26T04:52:08.794 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:52:08.794 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:52:08.794 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:52:08.794 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:52:08.794 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:52:08.794 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout: "osd" 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "14/23 daemons upgraded", 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading mds daemons", 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:52:08.795 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa0ec06e200 msgr2=0x7fa0ec0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa0ec06e200 0x7fa0ec0706b0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fa104101490 tx=0x7fa0f801a040 comp rx=0 tx=0).stop 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 msgr2=0x7fa1040ffca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040ffca0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fa0f400d900 tx=0x7fa0f400dc10 comp rx=0 tx=0).stop 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 shutdown_connections 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa0ec06e200 0x7fa0ec0706b0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa104068540 0x7fa1040ffca0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 --2- 172.21.15.90:0/2483321843 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa104105230 0x7fa1041001e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.796 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 >> 172.21.15.90:0/2483321843 conn(0x7fa104075130 msgr2=0x7fa1040feb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:08.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 shutdown_connections 2024-04-26T04:52:08.797 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.795+0000 7fa10a5b5700 1 -- 172.21.15.90:0/2483321843 wait complete. 2024-04-26T04:52:08.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 -- 172.21.15.90:0/3222024178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2741004e0 msgr2=0x7fe2741008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.900 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 --2- 172.21.15.90:0/3222024178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2741004e0 0x7fe2741008b0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fe268009910 tx=0x7fe268009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 -- 172.21.15.90:0/3222024178 shutdown_connections 2024-04-26T04:52:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 --2- 172.21.15.90:0/3222024178 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe274100e80 0x7fe274104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 --2- 172.21.15.90:0/3222024178 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe2741004e0 0x7fe2741008b0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 -- 172.21.15.90:0/3222024178 >> 172.21.15.90:0/3222024178 conn(0x7fe2740fbda0 msgr2=0x7fe2740fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 -- 172.21.15.90:0/3222024178 shutdown_connections 2024-04-26T04:52:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.899+0000 7fe2798f9700 1 -- 172.21.15.90:0/3222024178 wait complete. 2024-04-26T04:52:08.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2798f9700 1 Processor -- start 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2798f9700 1 -- start start 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2798f9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2741004e0 0x7fe274073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2798f9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe274100e80 0x7fe274073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2798f9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe274073ad0 con 0x7fe274100e80 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2798f9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe274073c10 con 0x7fe2741004e0 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2727fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe274100e80 0x7fe274073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2727fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe274100e80 0x7fe274073590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:35848/0 (socket says 172.21.15.90:35848) 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2727fc700 1 -- 172.21.15.90:0/1870589532 learned_addr learned my addr 172.21.15.90:0/1870589532 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe272ffd700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2741004e0 0x7fe274073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2727fc700 1 -- 172.21.15.90:0/1870589532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2741004e0 msgr2=0x7fe274073050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2727fc700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2741004e0 0x7fe274073050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:08.903 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2727fc700 1 -- 172.21.15.90:0/1870589532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680095f0 con 0x7fe274100e80 2024-04-26T04:52:08.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.900+0000 7fe2727fc700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe274100e80 0x7fe274073590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe26400d900 tx=0x7fe26400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:08.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.901+0000 7fe2788f7700 1 -- 172.21.15.90:0/1870589532 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe264009900 con 0x7fe274100e80 2024-04-26T04:52:08.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.901+0000 7fe2788f7700 1 -- 172.21.15.90:0/1870589532 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe264010460 con 0x7fe274100e80 2024-04-26T04:52:08.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.901+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2741a20c0 con 0x7fe274100e80 2024-04-26T04:52:08.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.901+0000 7fe2788f7700 1 -- 172.21.15.90:0/1870589532 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe26400f5f0 con 0x7fe274100e80 2024-04-26T04:52:08.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.901+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2741a24d0 con 0x7fe274100e80 2024-04-26T04:52:08.904 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.902+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2741087f0 con 0x7fe274100e80 2024-04-26T04:52:08.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.902+0000 7fe2788f7700 1 -- 172.21.15.90:0/1870589532 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe2640105d0 con 0x7fe274100e80 2024-04-26T04:52:08.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.902+0000 7fe2788f7700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe25c06e200 0x7fe25c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:08.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.903+0000 7fe2788f7700 1 -- 172.21.15.90:0/1870589532 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe26408dcf0 con 0x7fe274100e80 2024-04-26T04:52:08.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.903+0000 7fe272ffd700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe25c06e200 0x7fe25c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:08.905 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.903+0000 7fe272ffd700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe25c06e200 0x7fe25c0706b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe26800b5c0 tx=0x7fe26801a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:08.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:08.907+0000 7fe2788f7700 1 -- 172.21.15.90:0/1870589532 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe264057050 con 0x7fe274100e80 2024-04-26T04:52:09.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:08 smithi104 ceph-mon[64778]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:09.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:08 smithi104 ceph-mon[64778]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:09.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:08 smithi104 ceph-mon[64778]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:09.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:08 smithi104 ceph-mon[64778]: pgmap v193: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:09.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:08 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/883099840' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:09.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:08 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1374580728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:09.187 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.186+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fe274073dd0 con 0x7fe274100e80 2024-04-26T04:52:09.187 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:08 smithi090 ceph-mon[86550]: from='client.34418 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:09.188 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:08 smithi090 ceph-mon[86550]: from='client.34422 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:09.188 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:08 smithi090 ceph-mon[86550]: from='client.34426 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:09.188 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:08 smithi090 ceph-mon[86550]: pgmap v193: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:09.188 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:08 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/883099840' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:09.188 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:08 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1374580728' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:09.189 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.186+0000 7fe2788f7700 1 -- 172.21.15.90:0/1870589532 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+233 (secure 0 0 0) 0x7fe264020100 con 0x7fe274100e80 2024-04-26T04:52:09.189 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_ERR 1 filesystem is degraded; 1 filesystem is offline 2024-04-26T04:52:09.189 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T04:52:09.190 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs is degraded 2024-04-26T04:52:09.190 INFO:teuthology.orchestra.run.smithi090.stdout:[ERR] MDS_ALL_DOWN: 1 filesystem is offline 2024-04-26T04:52:09.190 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs is offline because no MDS is active for it. 2024-04-26T04:52:09.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe25c06e200 msgr2=0x7fe25c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:09.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe25c06e200 0x7fe25c0706b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe26800b5c0 tx=0x7fe26801a040 comp rx=0 tx=0).stop 2024-04-26T04:52:09.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe274100e80 msgr2=0x7fe274073590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:09.191 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe274100e80 0x7fe274073590 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe26400d900 tx=0x7fe26400dcc0 comp rx=0 tx=0).stop 2024-04-26T04:52:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 shutdown_connections 2024-04-26T04:52:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe25c06e200 0x7fe25c0706b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe2741004e0 0x7fe274073050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 --2- 172.21.15.90:0/1870589532 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe274100e80 0x7fe274073590 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 >> 172.21.15.90:0/1870589532 conn(0x7fe2740fbda0 msgr2=0x7fe2740fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:09.192 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 shutdown_connections 2024-04-26T04:52:09.193 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:09.189+0000 7fe2798f9700 1 -- 172.21.15.90:0/1870589532 wait complete. 2024-04-26T04:52:10.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:09 smithi090 ceph-mon[86550]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:10.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:09 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1870589532' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:10.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:09 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:10.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:09 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:10.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:09 smithi104 ceph-mon[64778]: from='client.34438 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:10.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:09 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1870589532' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:10.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:09 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:10.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:09 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:11.053 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:10 smithi104 ceph-mon[64778]: pgmap v194: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:11.053 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:11.053 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:11.053 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:10 smithi090 ceph-mon[86550]: pgmap v194: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:11.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:11.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:11.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:13.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:12 smithi104 ceph-mon[64778]: pgmap v195: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:13.160 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:12 smithi090 ceph-mon[86550]: pgmap v195: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:15.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: pgmap v196: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:15.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:14 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:15.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: pgmap v196: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:15.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:15.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:14 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:16.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:16 smithi090 ceph-mon[86550]: Upgrade: Updating mds.cephfs.smithi104.ndiusl 2024-04-26T04:52:16.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:16.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:52:16.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:16 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:16.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:16 smithi090 ceph-mon[86550]: Deploying daemon mds.cephfs.smithi104.ndiusl on smithi104 2024-04-26T04:52:16.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:16 smithi090 ceph-mon[86550]: pgmap v197: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:16 smithi104 ceph-mon[64778]: Upgrade: Updating mds.cephfs.smithi104.ndiusl 2024-04-26T04:52:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "mds.cephfs.smithi104.ndiusl", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-26T04:52:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:16 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:16 smithi104 ceph-mon[64778]: Deploying daemon mds.cephfs.smithi104.ndiusl on smithi104 2024-04-26T04:52:17.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:16 smithi104 ceph-mon[64778]: pgmap v197: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:19.062 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:18 smithi104 ceph-mon[64778]: pgmap v198: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:18 smithi090 ceph-mon[86550]: pgmap v198: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:20.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:19 smithi090 ceph-mon[86550]: osdmap e85: 6 total, 6 up, 6 in 2024-04-26T04:52:20.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:19 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:52:20.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:19 smithi104 ceph-mon[64778]: osdmap e85: 6 total, 6 up, 6 in 2024-04-26T04:52:20.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:19 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 3 up:standby, 2 failed 2024-04-26T04:52:21.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:20 smithi090 ceph-mon[86550]: pgmap v200: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:20 smithi104 ceph-mon[64778]: pgmap v200: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:22.214 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:21 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:52:22.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:21 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:52:22.807 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:22 smithi104 ceph-mon[64778]: pgmap v201: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:23.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:22 smithi090 ceph-mon[86550]: pgmap v201: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:24.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:24.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:24.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:24.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:24 smithi104 ceph-mon[64778]: pgmap v202: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:24.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:24 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.104:6826/1280197644,v1:172.21.15.104:6827/1280197644] up:boot 2024-04-26T04:52:24.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:24 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:52:24.762 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:24 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:52:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:24 smithi090 ceph-mon[86550]: pgmap v202: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:24.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:24 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.104:6826/1280197644,v1:172.21.15.104:6827/1280197644] up:boot 2024-04-26T04:52:24.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:24 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:52:24.972 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:24 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "mds metadata", "who": "cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:52:26.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:26.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:27.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:26 smithi090 ceph-mon[86550]: pgmap v203: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:27.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:27.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:26 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:26 smithi104 ceph-mon[64778]: pgmap v203: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:26 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:28.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:28.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:28 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:28.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:28 smithi090 ceph-mon[86550]: pgmap v204: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:28 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:28.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:28 smithi104 ceph-mon[64778]: pgmap v204: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:30.974 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:30 smithi104 ceph-mon[64778]: pgmap v205: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:31.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:30 smithi090 ceph-mon[86550]: pgmap v205: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all mds 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.dbkjuk"}]': finished 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.ehputv"}]': finished 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.ndiusl"}]': finished 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.wpntdi"}]': finished 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: Upgrade: Setting filesystem cephfs Joinable 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-26T04:52:32.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:32 smithi090 ceph-mon[86550]: pgmap v206: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:32.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:32.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:32.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all mds 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.dbkjuk"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.dbkjuk"}]': finished 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.ehputv"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi090.ehputv"}]': finished 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.ndiusl"}]: dispatch 2024-04-26T04:52:32.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.ndiusl"}]': finished 2024-04-26T04:52:32.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.wpntdi"}]: dispatch 2024-04-26T04:52:32.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds.cephfs.smithi104.wpntdi"}]': finished 2024-04-26T04:52:32.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: Upgrade: Setting filesystem cephfs Joinable 2024-04-26T04:52:32.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-26T04:52:32.513 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:32 smithi104 ceph-mon[64778]: pgmap v206: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: Standby daemon mds.cephfs.smithi090.ehputv assigned to filesystem cephfs as rank 0 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: Standby daemon mds.cephfs.smithi090.dbkjuk assigned to filesystem cephfs as rank 1 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all rgw 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:replay,1=cephfs.smithi090.dbkjuk=up:replay} 2 up:standby 2024-04-26T04:52:33.331 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:33.332 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:33.332 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all rbd-mirror 2024-04-26T04:52:33.332 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:33.332 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:33 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:33.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-26T04:52:33.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: Standby daemon mds.cephfs.smithi090.ehputv assigned to filesystem cephfs as rank 0 2024-04-26T04:52:33.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: Standby daemon mds.cephfs.smithi090.dbkjuk assigned to filesystem cephfs as rank 1 2024-04-26T04:52:33.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-26T04:52:33.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: fsmap cephfs:0/2 4 up:standby, 2 failed 2024-04-26T04:52:33.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:33.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all rgw 2024-04-26T04:52:33.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:replay,1=cephfs.smithi090.dbkjuk=up:replay} 2 up:standby 2024-04-26T04:52:33.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:33.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:33.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all rbd-mirror 2024-04-26T04:52:33.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:33.512 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:33 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:34 smithi090 ceph-mon[86550]: pgmap v207: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 0 op/s 2024-04-26T04:52:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:34.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:52:34.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:34 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:34 smithi104 ceph-mon[64778]: pgmap v207: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.0 MiB/s rd, 0 op/s 2024-04-26T04:52:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi090", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:52:34.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:34 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:35.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:35 smithi104 ceph-mon[64778]: Upgrade: Updating ceph-exporter.smithi090 (1/2) 2024-04-26T04:52:35.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:35 smithi104 ceph-mon[64778]: Deploying daemon ceph-exporter.smithi090 on smithi090 2024-04-26T04:52:35.707 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:35 smithi090 ceph-mon[86550]: Upgrade: Updating ceph-exporter.smithi090 (1/2) 2024-04-26T04:52:35.707 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:35 smithi090 ceph-mon[86550]: Deploying daemon ceph-exporter.smithi090 on smithi090 2024-04-26T04:52:36.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:36 smithi090 ceph-mon[86550]: pgmap v208: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 1 op/s 2024-04-26T04:52:36.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:36 smithi104 ceph-mon[64778]: pgmap v208: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 3.3 MiB/s rd, 1 op/s 2024-04-26T04:52:38.865 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:38 smithi090 ceph-mon[86550]: pgmap v209: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-04-26T04:52:39.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:38 smithi104 ceph-mon[64778]: pgmap v209: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 18 MiB/s rd, 6 op/s 2024-04-26T04:52:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.351+0000 7fb68969b700 1 -- 172.21.15.90:0/2238583372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 msgr2=0x7fb68410a230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.351+0000 7fb68969b700 1 --2- 172.21.15.90:0/2238583372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb68410a230 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb678009910 tx=0x7fb678009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 -- 172.21.15.90:0/2238583372 shutdown_connections 2024-04-26T04:52:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 --2- 172.21.15.90:0/2238583372 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb68410a230 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.355 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 --2- 172.21.15.90:0/2238583372 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb684101420 0x7fb6841017f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 -- 172.21.15.90:0/2238583372 >> 172.21.15.90:0/2238583372 conn(0x7fb6840fac60 msgr2=0x7fb6840fd070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 -- 172.21.15.90:0/2238583372 shutdown_connections 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 -- 172.21.15.90:0/2238583372 wait complete. 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 Processor -- start 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.352+0000 7fb68969b700 1 -- start start 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb68969b700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb684101420 0x7fb684195e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb68969b700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb684196390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb68969b700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb684196a70 con 0x7fb684101d30 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb68969b700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb68419a800 con 0x7fb684101420 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb6827fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb684196390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb6827fc700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb684196390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60504/0 (socket says 172.21.15.90:60504) 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb6827fc700 1 -- 172.21.15.90:0/3182633944 learned_addr learned my addr 172.21.15.90:0/3182633944 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:39.356 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb682ffd700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb684101420 0x7fb684195e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb6827fc700 1 -- 172.21.15.90:0/3182633944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb684101420 msgr2=0x7fb684195e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb6827fc700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb684101420 0x7fb684195e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb6827fc700 1 -- 172.21.15.90:0/3182633944 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6780095f0 con 0x7fb684101d30 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb6827fc700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb684196390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb67800f690 tx=0x7fb67800f6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.353+0000 7fb66bfff700 1 -- 172.21.15.90:0/3182633944 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb67801d070 con 0x7fb684101d30 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.354+0000 7fb66bfff700 1 -- 172.21.15.90:0/3182633944 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb6780050d0 con 0x7fb684101d30 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.354+0000 7fb66bfff700 1 -- 172.21.15.90:0/3182633944 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb678022470 con 0x7fb684101d30 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.354+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb68419aa80 con 0x7fb684101d30 2024-04-26T04:52:39.357 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.354+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb68419ae60 con 0x7fb684101d30 2024-04-26T04:52:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.355+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6841078a0 con 0x7fb684101d30 2024-04-26T04:52:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.355+0000 7fb66bfff700 1 -- 172.21.15.90:0/3182633944 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb678003780 con 0x7fb684101d30 2024-04-26T04:52:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.356+0000 7fb66bfff700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb66c06e200 0x7fb66c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.356+0000 7fb66bfff700 1 -- 172.21.15.90:0/3182633944 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb67808f610 con 0x7fb684101d30 2024-04-26T04:52:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.356+0000 7fb682ffd700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb66c06e200 0x7fb66c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:39.358 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.356+0000 7fb682ffd700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb66c06e200 0x7fb66c0706b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb674009c60 tx=0x7fb674009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:39.362 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.361+0000 7fb66bfff700 1 -- 172.21.15.90:0/3182633944 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb6780588f0 con 0x7fb684101d30 2024-04-26T04:52:39.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.590+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb68402cca0 con 0x7fb66c06e200 2024-04-26T04:52:39.592 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.592+0000 7fb66bfff700 1 -- 172.21.15.90:0/3182633944 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fb68402cca0 con 0x7fb66c06e200 2024-04-26T04:52:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb66c06e200 msgr2=0x7fb66c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.596 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb66c06e200 0x7fb66c0706b0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fb674009c60 tx=0x7fb674009260 comp rx=0 tx=0).stop 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 msgr2=0x7fb684196390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb684196390 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb67800f690 tx=0x7fb67800f6c0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 shutdown_connections 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb66c06e200 0x7fb66c0706b0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb684101420 0x7fb684195e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 --2- 172.21.15.90:0/3182633944 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb684101d30 0x7fb684196390 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 >> 172.21.15.90:0/3182633944 conn(0x7fb6840fac60 msgr2=0x7fb6841049e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 shutdown_connections 2024-04-26T04:52:39.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.595+0000 7fb68969b700 1 -- 172.21.15.90:0/3182633944 wait complete. 2024-04-26T04:52:39.611 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:52:39.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 -- 172.21.15.90:0/1391483160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100d70 msgr2=0x7f84981094d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.707 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 --2- 172.21.15.90:0/1391483160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100d70 0x7f84981094d0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f848c009910 tx=0x7f848c009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 -- 172.21.15.90:0/1391483160 shutdown_connections 2024-04-26T04:52:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 --2- 172.21.15.90:0/1391483160 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100d70 0x7f84981094d0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 --2- 172.21.15.90:0/1391483160 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8498100460 0x7f8498100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 -- 172.21.15.90:0/1391483160 >> 172.21.15.90:0/1391483160 conn(0x7f84980748b0 msgr2=0x7f8498074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 -- 172.21.15.90:0/1391483160 shutdown_connections 2024-04-26T04:52:39.708 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.706+0000 7f849d6a9700 1 -- 172.21.15.90:0/1391483160 wait complete. 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.707+0000 7f849d6a9700 1 Processor -- start 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.707+0000 7f849d6a9700 1 -- start start 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.707+0000 7f849d6a9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100460 0x7f8498196c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.707+0000 7f849d6a9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8498100d70 0x7f8498197180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.707+0000 7f849d6a9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84981977a0 con 0x7f8498100460 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.707+0000 7f849d6a9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84981978e0 con 0x7f8498100d70 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f8496ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100460 0x7f8498196c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f8496ffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100460 0x7f8498196c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60530/0 (socket says 172.21.15.90:60530) 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f8496ffd700 1 -- 172.21.15.90:0/3998941030 learned_addr learned my addr 172.21.15.90:0/3998941030 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f84967fc700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8498100d70 0x7f8498197180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f8496ffd700 1 -- 172.21.15.90:0/3998941030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8498100d70 msgr2=0x7f8498197180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.710 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f8496ffd700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8498100d70 0x7f8498197180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f8496ffd700 1 -- 172.21.15.90:0/3998941030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848c0095f0 con 0x7f8498100460 2024-04-26T04:52:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f8496ffd700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100460 0x7f8498196c40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f848800e920 tx=0x7f848800ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f847ffff700 1 -- 172.21.15.90:0/3998941030 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f848800c9d0 con 0x7f8498100460 2024-04-26T04:52:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f849819fcb0 con 0x7f8498100460 2024-04-26T04:52:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f847ffff700 1 -- 172.21.15.90:0/3998941030 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8488010430 con 0x7f8498100460 2024-04-26T04:52:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f847ffff700 1 -- 172.21.15.90:0/3998941030 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8488018630 con 0x7f8498100460 2024-04-26T04:52:39.711 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.708+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84981a0170 con 0x7f8498100460 2024-04-26T04:52:39.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.710+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8498106bd0 con 0x7f8498100460 2024-04-26T04:52:39.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.710+0000 7f847ffff700 1 -- 172.21.15.90:0/3998941030 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f84880105a0 con 0x7f8498100460 2024-04-26T04:52:39.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.710+0000 7f847ffff700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f848006e200 0x7f84800706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:39.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.711+0000 7f847ffff700 1 -- 172.21.15.90:0/3998941030 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8488014070 con 0x7f8498100460 2024-04-26T04:52:39.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.711+0000 7f84967fc700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f848006e200 0x7f84800706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:39.712 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.711+0000 7f84967fc700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f848006e200 0x7f84800706b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84980fe140 tx=0x7f848c005a90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:39.716 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.715+0000 7f847ffff700 1 -- 172.21.15.90:0/3998941030 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8488053740 con 0x7f8498100460 2024-04-26T04:52:39.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.947+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f84980ffa80 con 0x7f848006e200 2024-04-26T04:52:39.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:39 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:resolve 2024-04-26T04:52:39.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:39 smithi090 ceph-mon[86550]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:replay,1=cephfs.smithi090.dbkjuk=up:resolve} 2 up:standby 2024-04-26T04:52:39.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:39 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:39.948 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:39 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:39.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.948+0000 7f847ffff700 1 -- 172.21.15.90:0/3998941030 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7f84980ffa80 con 0x7f848006e200 2024-04-26T04:52:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f848006e200 msgr2=0x7f84800706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f848006e200 0x7f84800706b0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f84980fe140 tx=0x7f848c005a90 comp rx=0 tx=0).stop 2024-04-26T04:52:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100460 msgr2=0x7f8498196c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100460 0x7f8498196c40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f848800e920 tx=0x7f848800ec30 comp rx=0 tx=0).stop 2024-04-26T04:52:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 shutdown_connections 2024-04-26T04:52:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f848006e200 0x7f84800706b0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.952 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8498100460 0x7f8498196c40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 --2- 172.21.15.90:0/3998941030 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8498100d70 0x7f8498197180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.950+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 >> 172.21.15.90:0/3998941030 conn(0x7f84980748b0 msgr2=0x7f8498103d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.951+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 shutdown_connections 2024-04-26T04:52:39.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:39.951+0000 7f849d6a9700 1 -- 172.21.15.90:0/3998941030 wait complete. 2024-04-26T04:52:40.045 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:39 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:resolve 2024-04-26T04:52:40.045 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:39 smithi104 ceph-mon[64778]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:replay,1=cephfs.smithi090.dbkjuk=up:resolve} 2 up:standby 2024-04-26T04:52:40.045 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:39 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:40.045 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:39 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 -- 172.21.15.90:0/2230616497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408069140 msgr2=0x7fc408104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 --2- 172.21.15.90:0/2230616497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408069140 0x7fc408104eb0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009910 tx=0x7fc3f8009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 -- 172.21.15.90:0/2230616497 shutdown_connections 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 --2- 172.21.15.90:0/2230616497 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc408069140 0x7fc408104eb0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 --2- 172.21.15.90:0/2230616497 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc4080687a0 0x7fc408068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 -- 172.21.15.90:0/2230616497 >> 172.21.15.90:0/2230616497 conn(0x7fc408075130 msgr2=0x7fc408075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 -- 172.21.15.90:0/2230616497 shutdown_connections 2024-04-26T04:52:40.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.056+0000 7fc40f645700 1 -- 172.21.15.90:0/2230616497 wait complete. 2024-04-26T04:52:40.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40f645700 1 Processor -- start 2024-04-26T04:52:40.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40f645700 1 -- start start 2024-04-26T04:52:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40f645700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4080687a0 0x7fc408195e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40f645700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc408069140 0x7fc4081963a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40f645700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc408196a80 con 0x7fc4080687a0 2024-04-26T04:52:40.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40f645700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc40819a810 con 0x7fc408069140 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40d3e1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4080687a0 0x7fc408195e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40d3e1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4080687a0 0x7fc408195e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60552/0 (socket says 172.21.15.90:60552) 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.057+0000 7fc40d3e1700 1 -- 172.21.15.90:0/3953929062 learned_addr learned my addr 172.21.15.90:0/3953929062 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc40cbe0700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc408069140 0x7fc4081963a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc40d3e1700 1 -- 172.21.15.90:0/3953929062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc408069140 msgr2=0x7fc4081963a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc40d3e1700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc408069140 0x7fc4081963a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc40d3e1700 1 -- 172.21.15.90:0/3953929062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f80095f0 con 0x7fc4080687a0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc40d3e1700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4080687a0 0x7fc408195e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e920 tx=0x7fc40400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc3fe7fc700 1 -- 172.21.15.90:0/3953929062 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc404009790 con 0x7fc4080687a0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc3fe7fc700 1 -- 172.21.15.90:0/3953929062 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc40400cba0 con 0x7fc4080687a0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc40819aaf0 con 0x7fc4080687a0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc3fe7fc700 1 -- 172.21.15.90:0/3953929062 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc404018650 con 0x7fc4080687a0 2024-04-26T04:52:40.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.058+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc40819b040 con 0x7fc4080687a0 2024-04-26T04:52:40.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.059+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc408108750 con 0x7fc4080687a0 2024-04-26T04:52:40.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.060+0000 7fc3fe7fc700 1 -- 172.21.15.90:0/3953929062 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc40400cd10 con 0x7fc4080687a0 2024-04-26T04:52:40.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.060+0000 7fc3fe7fc700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc3f406e200 0x7fc3f40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.061+0000 7fc3fe7fc700 1 -- 172.21.15.90:0/3953929062 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc404014070 con 0x7fc4080687a0 2024-04-26T04:52:40.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.061+0000 7fc40cbe0700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc3f406e200 0x7fc3f40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.061+0000 7fc40cbe0700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc3f406e200 0x7fc3f40706b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009fd0 tx=0x7fc3f8005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:40.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.065+0000 7fc3fe7fc700 1 -- 172.21.15.90:0/3953929062 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc404053740 con 0x7fc4080687a0 2024-04-26T04:52:40.329 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.328+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fc40819b260 con 0x7fc3f406e200 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.339+0000 7fc3fe7fc700 1 -- 172.21.15.90:0/3953929062 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3816 (secure 0 0 0) 0x7fc40819b260 con 0x7fc3f406e200 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (7m) 45s ago 18m 24.3M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 starting - - - - 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (17m) 13s ago 17m 8870k - 18.2.0 dc2bc1663786 38e53de65be3 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (4m) 45s ago 18m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (4m) 13s ago 17m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 45s ago 17m 80.3M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (51s) 45s ago 12m 12.4M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a01df3d7e4d4 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (70s) 45s ago 12m 13.4M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 4bcc1e9fbb69 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (17s) 13s ago 12m 15.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 0dbd9167e2e1 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (34s) 13s ago 12m 14.8M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 7d741ad3ecf3 2024-04-26T04:52:40.341 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (8m) 45s ago 19m 508M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (8m) 13s ago 17m 451M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (5m) 45s ago 19m 55.9M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (5m) 13s ago 17m 45.5M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (8m) 45s ago 18m 16.2M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (8m) 13s ago 17m 14.9M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (4m) 45s ago 16m 93.0M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (3m) 45s ago 15m 104M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (3m) 45s ago 15m 83.7M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (2m) 13s ago 15m 109M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d860fd1e9069 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (2m) 13s ago 14m 92.0M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 fbfd5c1c44eb 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (88s) 13s ago 14m 86.8M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 64a9be8458bb 2024-04-26T04:52:40.342 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (7m) 45s ago 17m 69.2M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:52:40.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc3f406e200 msgr2=0x7fc3f40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc3f406e200 0x7fc3f40706b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009fd0 tx=0x7fc3f8005940 comp rx=0 tx=0).stop 2024-04-26T04:52:40.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4080687a0 msgr2=0x7fc408195e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4080687a0 0x7fc408195e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fc40400e920 tx=0x7fc40400ece0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 shutdown_connections 2024-04-26T04:52:40.343 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc3f406e200 0x7fc3f40706b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc4080687a0 0x7fc408195e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 --2- 172.21.15.90:0/3953929062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc408069140 0x7fc4081963a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 >> 172.21.15.90:0/3953929062 conn(0x7fc408075130 msgr2=0x7fc408104720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:40.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 shutdown_connections 2024-04-26T04:52:40.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.342+0000 7fc40f645700 1 -- 172.21.15.90:0/3953929062 wait complete. 2024-04-26T04:52:40.447 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 -- 172.21.15.90:0/3323097174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ff350 msgr2=0x7fcab40ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 --2- 172.21.15.90:0/3323097174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ff350 0x7fcab40ff720 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009910 tx=0x7fcaa4009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 -- 172.21.15.90:0/3323097174 shutdown_connections 2024-04-26T04:52:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 --2- 172.21.15.90:0/3323097174 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab40ffc60 0x7fcab410a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 --2- 172.21.15.90:0/3323097174 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ff350 0x7fcab40ff720 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 -- 172.21.15.90:0/3323097174 >> 172.21.15.90:0/3323097174 conn(0x7fcab40748b0 msgr2=0x7fcab4074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 -- 172.21.15.90:0/3323097174 shutdown_connections 2024-04-26T04:52:40.448 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.446+0000 7fcaba483700 1 -- 172.21.15.90:0/3323097174 wait complete. 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.447+0000 7fcaba483700 1 Processor -- start 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.447+0000 7fcaba483700 1 -- start start 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.447+0000 7fcaba483700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab40ff350 0x7fcab4100530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.447+0000 7fcaba483700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ffc60 0x7fcab4100a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.447+0000 7fcaba483700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4100fb0 con 0x7fcab40ffc60 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcaba483700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab41010f0 con 0x7fcab40ff350 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ffc60 0x7fcab4100a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab37fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ffc60 0x7fcab4100a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60574/0 (socket says 172.21.15.90:60574) 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab37fe700 1 -- 172.21.15.90:0/718255436 learned_addr learned my addr 172.21.15.90:0/718255436 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab3fff700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab40ff350 0x7fcab4100530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab3fff700 1 -- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ffc60 msgr2=0x7fcab4100a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab3fff700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ffc60 0x7fcab4100a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.450 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab3fff700 1 -- 172.21.15.90:0/718255436 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaa40095f0 con 0x7fcab40ff350 2024-04-26T04:52:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab37fe700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ffc60 0x7fcab4100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-26T04:52:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab3fff700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab40ff350 0x7fcab4100530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009fd0 tx=0x7fcaa4005ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab17fa700 1 -- 172.21.15.90:0/718255436 <== mon.1 v2:172.21.15.104:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa401d070 con 0x7fcab40ff350 2024-04-26T04:52:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab17fa700 1 -- 172.21.15.90:0/718255436 <== mon.1 v2:172.21.15.104:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fcaa40052a0 con 0x7fcab40ff350 2024-04-26T04:52:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab41a2070 con 0x7fcab40ff350 2024-04-26T04:52:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.448+0000 7fcab17fa700 1 -- 172.21.15.90:0/718255436 <== mon.1 v2:172.21.15.104:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fcaa4003e70 con 0x7fcab40ff350 2024-04-26T04:52:40.451 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.449+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab41a2480 con 0x7fcab40ff350 2024-04-26T04:52:40.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.450+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab4107dd0 con 0x7fcab40ff350 2024-04-26T04:52:40.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.450+0000 7fcab17fa700 1 -- 172.21.15.90:0/718255436 <== mon.1 v2:172.21.15.104:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fcaa4003710 con 0x7fcab40ff350 2024-04-26T04:52:40.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.450+0000 7fcab17fa700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fca9c06e200 0x7fca9c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.450+0000 7fcab17fa700 1 -- 172.21.15.90:0/718255436 <== mon.1 v2:172.21.15.104:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fcaa408f430 con 0x7fcab40ff350 2024-04-26T04:52:40.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.450+0000 7fcab37fe700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fca9c06e200 0x7fca9c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.452 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.451+0000 7fcab37fe700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fca9c06e200 0x7fca9c0706b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcab4101d20 tx=0x7fcaa8009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:40.456 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.455+0000 7fcab17fa700 1 -- 172.21.15.90:0/718255436 <== mon.1 v2:172.21.15.104:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fcaa40586f0 con 0x7fcab40ff350 2024-04-26T04:52:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.760+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fcab4101280 con 0x7fcab40ff350 2024-04-26T04:52:40.761 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.760+0000 7fcab17fa700 1 -- 172.21.15.90:0/718255436 <== mon.1 v2:172.21.15.104:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fcaa405bdb0 con 0x7fcab40ff350 2024-04-26T04:52:40.761 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:52:40.761 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 14 2024-04-26T04:52:40.762 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:52:40.763 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.763+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fca9c06e200 msgr2=0x7fca9c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fca9c06e200 0x7fca9c0706b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fcab4101d20 tx=0x7fcaa8009260 comp rx=0 tx=0).stop 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab40ff350 msgr2=0x7fcab4100530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab40ff350 0x7fcab4100530 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009fd0 tx=0x7fcaa4005ee0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 shutdown_connections 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fca9c06e200 0x7fca9c0706b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fcab40ff350 0x7fcab4100530 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.765 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 --2- 172.21.15.90:0/718255436 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fcab40ffc60 0x7fcab4100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 >> 172.21.15.90:0/718255436 conn(0x7fcab40748b0 msgr2=0x7fcab4104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:40.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 shutdown_connections 2024-04-26T04:52:40.766 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.764+0000 7fcaba483700 1 -- 172.21.15.90:0/718255436 wait complete. 2024-04-26T04:52:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.874+0000 7fafa454f700 1 -- 172.21.15.90:0/3161514420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 msgr2=0x7faf9c073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.874+0000 7fafa454f700 1 --2- 172.21.15.90:0/3161514420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c073460 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7faf98009940 tx=0x7faf98009c50 comp rx=0 tx=0).stop 2024-04-26T04:52:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.875+0000 7fafa454f700 1 -- 172.21.15.90:0/3161514420 shutdown_connections 2024-04-26T04:52:40.876 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.875+0000 7fafa454f700 1 --2- 172.21.15.90:0/3161514420 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf9c0739a0 0x7faf9c10c5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.875+0000 7fafa454f700 1 --2- 172.21.15.90:0/3161514420 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c073460 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.875+0000 7fafa454f700 1 -- 172.21.15.90:0/3161514420 >> 172.21.15.90:0/3161514420 conn(0x7faf9c0fbda0 msgr2=0x7faf9c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.875+0000 7fafa454f700 1 -- 172.21.15.90:0/3161514420 shutdown_connections 2024-04-26T04:52:40.877 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.875+0000 7fafa454f700 1 -- 172.21.15.90:0/3161514420 wait complete. 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.875+0000 7fafa454f700 1 Processor -- start 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa454f700 1 -- start start 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa454f700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa454f700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf9c0739a0 0x7faf9c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa454f700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c198da0 con 0x7faf9c073090 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa454f700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf9c19cb30 con 0x7faf9c0739a0 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa22eb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa22eb700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60588/0 (socket says 172.21.15.90:60588) 2024-04-26T04:52:40.879 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa22eb700 1 -- 172.21.15.90:0/3562903759 learned_addr learned my addr 172.21.15.90:0/3562903759 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa1aea700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf9c0739a0 0x7faf9c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa22eb700 1 -- 172.21.15.90:0/3562903759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf9c0739a0 msgr2=0x7faf9c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa22eb700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf9c0739a0 0x7faf9c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.876+0000 7fafa22eb700 1 -- 172.21.15.90:0/3562903759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf980095f0 con 0x7faf9c073090 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.877+0000 7fafa22eb700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c198180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faf9800b580 tx=0x7faf980055f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.877+0000 7faf937fe700 1 -- 172.21.15.90:0/3562903759 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf9801d070 con 0x7faf9c073090 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.877+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf9c19cdb0 con 0x7faf9c073090 2024-04-26T04:52:40.880 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.877+0000 7faf937fe700 1 -- 172.21.15.90:0/3562903759 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7faf98005dc0 con 0x7faf9c073090 2024-04-26T04:52:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.877+0000 7faf937fe700 1 -- 172.21.15.90:0/3562903759 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7faf98021410 con 0x7faf9c073090 2024-04-26T04:52:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.877+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf9c19d2a0 con 0x7faf9c073090 2024-04-26T04:52:40.881 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.878+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf9c109ce0 con 0x7faf9c073090 2024-04-26T04:52:40.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.879+0000 7faf937fe700 1 -- 172.21.15.90:0/3562903759 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7faf9800f7f0 con 0x7faf9c073090 2024-04-26T04:52:40.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.880+0000 7faf937fe700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf8806e200 0x7faf880706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:40.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.880+0000 7faf937fe700 1 -- 172.21.15.90:0/3562903759 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7faf9808f8c0 con 0x7faf9c073090 2024-04-26T04:52:40.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.880+0000 7fafa1aea700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf8806e200 0x7faf880706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:40.882 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.880+0000 7fafa1aea700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf8806e200 0x7faf880706b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf9c1997a0 tx=0x7faf8c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:40.885 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:40.884+0000 7faf937fe700 1 -- 172.21.15.90:0/3562903759 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7faf98058b80 con 0x7faf9c073090 2024-04-26T04:52:41.151 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.149+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7faf9c04f2a0 con 0x7faf9c073090 2024-04-26T04:52:41.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:40 smithi090 ceph-mon[86550]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:41.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:40 smithi090 ceph-mon[86550]: pgmap v210: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-04-26T04:52:41.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:41.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:41.151 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:40 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/718255436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.150+0000 7faf937fe700 1 -- 172.21.15.90:0/3562903759 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 34 v34) v1 ==== 76+0+1936 (secure 0 0 0) 0x7faf9805c240 con 0x7faf9c073090 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:e34 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 34 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:52:38.433787+0000 2024-04-26T04:52:41.152 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 81 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=34380,1=34412} 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:52:41.153 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 0 members: 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{0:34380} state up:replay seq 1 join_fscid=1 addr [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{1:34412} state up:resolve seq 14 join_fscid=1 addr [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.104:6824/2998394587,v1:172.21.15.104:6825/2998394587] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:52:41.154 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.104:6826/1280197644,v1:172.21.15.104:6827/1280197644] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:52:41.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf8806e200 msgr2=0x7faf880706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf8806e200 0x7faf880706b0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7faf9c1997a0 tx=0x7faf8c00b410 comp rx=0 tx=0).stop 2024-04-26T04:52:41.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 msgr2=0x7faf9c198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.155 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c198180 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7faf9800b580 tx=0x7faf980055f0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 shutdown_connections 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7faf8806e200 0x7faf880706b0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7faf9c073090 0x7faf9c198180 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 --2- 172.21.15.90:0/3562903759 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7faf9c0739a0 0x7faf9c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 >> 172.21.15.90:0/3562903759 conn(0x7faf9c0fbda0 msgr2=0x7faf9c106e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 shutdown_connections 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.152+0000 7fafa454f700 1 -- 172.21.15.90:0/3562903759 wait complete. 2024-04-26T04:52:41.156 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 34 2024-04-26T04:52:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 -- 172.21.15.90:0/1729259168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ffc60 msgr2=0x7fb8ec10a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/1729259168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ffc60 0x7fb8ec10a6d0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fb8e0009910 tx=0x7fb8e0009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 -- 172.21.15.90:0/1729259168 shutdown_connections 2024-04-26T04:52:41.258 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/1729259168 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ffc60 0x7fb8ec10a6d0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/1729259168 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec0ff350 0x7fb8ec0ff720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 -- 172.21.15.90:0/1729259168 >> 172.21.15.90:0/1729259168 conn(0x7fb8ec0748b0 msgr2=0x7fb8ec074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:41.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 -- 172.21.15.90:0/1729259168 shutdown_connections 2024-04-26T04:52:41.259 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.257+0000 7fb8f18f9700 1 -- 172.21.15.90:0/1729259168 wait complete. 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8f18f9700 1 Processor -- start 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8f18f9700 1 -- start start 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8f18f9700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ff350 0x7fb8ec195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8f18f9700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec0ffc60 0x7fb8ec1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8f18f9700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec196ad0 con 0x7fb8ec0ff350 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8f18f9700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec19a860 con 0x7fb8ec0ffc60 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ff350 0x7fb8ec195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8eaffd700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ff350 0x7fb8ec195eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60616/0 (socket says 172.21.15.90:60616) 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8eaffd700 1 -- 172.21.15.90:0/2661380511 learned_addr learned my addr 172.21.15.90:0/2661380511 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8ea7fc700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec0ffc60 0x7fb8ec1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8eaffd700 1 -- 172.21.15.90:0/2661380511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec0ffc60 msgr2=0x7fb8ec1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8eaffd700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec0ffc60 0x7fb8ec1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.258+0000 7fb8eaffd700 1 -- 172.21.15.90:0/2661380511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e00095f0 con 0x7fb8ec0ff350 2024-04-26T04:52:41.261 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.259+0000 7fb8eaffd700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ff350 0x7fb8ec195eb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e8f0 tx=0x7fb8dc00ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.259+0000 7fb8f08f7700 1 -- 172.21.15.90:0/2661380511 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8dc009790 con 0x7fb8ec0ff350 2024-04-26T04:52:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.259+0000 7fb8f08f7700 1 -- 172.21.15.90:0/2661380511 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb8dc00c810 con 0x7fb8ec0ff350 2024-04-26T04:52:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.259+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec19ab40 con 0x7fb8ec0ff350 2024-04-26T04:52:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.259+0000 7fb8f08f7700 1 -- 172.21.15.90:0/2661380511 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb8dc003d20 con 0x7fb8ec0ff350 2024-04-26T04:52:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.259+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec19b090 con 0x7fb8ec0ff350 2024-04-26T04:52:41.262 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.260+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec107dd0 con 0x7fb8ec0ff350 2024-04-26T04:52:41.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:40 smithi104 ceph-mon[64778]: from='client.34448 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:41.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:40 smithi104 ceph-mon[64778]: pgmap v210: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 20 MiB/s rd, 7 op/s 2024-04-26T04:52:41.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:41.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:41.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:40 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/718255436' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.260+0000 7fb8f08f7700 1 -- 172.21.15.90:0/2661380511 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb8dc0041f0 con 0x7fb8ec0ff350 2024-04-26T04:52:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.261+0000 7fb8f08f7700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb8d406e200 0x7fb8d40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.261+0000 7fb8f08f7700 1 -- 172.21.15.90:0/2661380511 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb8dc014070 con 0x7fb8ec0ff350 2024-04-26T04:52:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.261+0000 7fb8ea7fc700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb8d406e200 0x7fb8d40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:41.263 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.261+0000 7fb8ea7fc700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb8d406e200 0x7fb8d40706b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec1974d0 tx=0x7fb8e0005940 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:41.266 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.265+0000 7fb8f08f7700 1 -- 172.21.15.90:0/2661380511 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb8dc057780 con 0x7fb8ec0ff350 2024-04-26T04:52:41.489 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.488+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fb8ec19b310 con 0x7fb8d406e200 2024-04-26T04:52:41.490 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.489+0000 7fb8f08f7700 1 -- 172.21.15.90:0/2661380511 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+441 (secure 0 0 0) 0x7fb8ec19b310 con 0x7fb8d406e200 2024-04-26T04:52:41.490 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:52:41.490 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": "quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6", 2024-04-26T04:52:41.490 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": true, 2024-04-26T04:52:41.490 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "Upgrading all daemon types on all hosts", 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [ 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "crash", 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr", 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "mds", 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "mon", 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "osd" 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: ], 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": "16/23 daemons upgraded", 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "Currently upgrading ceph-exporter daemons", 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:52:41.491 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb8d406e200 msgr2=0x7fb8d40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb8d406e200 0x7fb8d40706b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb8ec1974d0 tx=0x7fb8e0005940 comp rx=0 tx=0).stop 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ff350 msgr2=0x7fb8ec195eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ff350 0x7fb8ec195eb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc00e8f0 tx=0x7fb8dc00ec00 comp rx=0 tx=0).stop 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 shutdown_connections 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb8d406e200 0x7fb8d40706b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb8ec0ff350 0x7fb8ec195eb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.493 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.491+0000 7fb8f18f9700 1 --2- 172.21.15.90:0/2661380511 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb8ec0ffc60 0x7fb8ec1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.492+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 >> 172.21.15.90:0/2661380511 conn(0x7fb8ec0748b0 msgr2=0x7fb8ec104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:41.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.492+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 shutdown_connections 2024-04-26T04:52:41.494 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.492+0000 7fb8f18f9700 1 -- 172.21.15.90:0/2661380511 wait complete. 2024-04-26T04:52:41.597 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.595+0000 7f1b57cd2700 1 -- 172.21.15.90:0/3631369182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 msgr2=0x7f1b50104000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.595+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/3631369182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b50104000 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f1b40009910 tx=0x7f1b40009c20 comp rx=0 tx=0).stop 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.595+0000 7f1b57cd2700 1 -- 172.21.15.90:0/3631369182 shutdown_connections 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.595+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/3631369182 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b50104000 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.595+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/3631369182 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b500ff210 0x7f1b500ff5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.595+0000 7f1b57cd2700 1 -- 172.21.15.90:0/3631369182 >> 172.21.15.90:0/3631369182 conn(0x7f1b500fac80 msgr2=0x7f1b500fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 -- 172.21.15.90:0/3631369182 shutdown_connections 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 -- 172.21.15.90:0/3631369182 wait complete. 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 Processor -- start 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 -- start start 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b500ff210 0x7f1b5019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b5019c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b5019d0a0 con 0x7f1b500ffb20 2024-04-26T04:52:41.598 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b57cd2700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b501a0e30 con 0x7f1b500ff210 2024-04-26T04:52:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b5526d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b5019c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b5526d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b5019c9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:60638/0 (socket says 172.21.15.90:60638) 2024-04-26T04:52:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b5526d700 1 -- 172.21.15.90:0/993352302 learned_addr learned my addr 172.21.15.90:0/993352302 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:52:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.596+0000 7f1b55a6e700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b500ff210 0x7f1b5019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b5526d700 1 -- 172.21.15.90:0/993352302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b500ff210 msgr2=0x7f1b5019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b5526d700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b500ff210 0x7f1b5019c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.599 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b5526d700 1 -- 172.21.15.90:0/993352302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b400095f0 con 0x7f1b500ffb20 2024-04-26T04:52:41.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b5526d700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b5019c9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1b40006010 tx=0x7f1b4000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:41.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b46ffd700 1 -- 172.21.15.90:0/993352302 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b4001d070 con 0x7f1b500ffb20 2024-04-26T04:52:41.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b501a10b0 con 0x7f1b500ffb20 2024-04-26T04:52:41.600 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b46ffd700 1 -- 172.21.15.90:0/993352302 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1b40005dc0 con 0x7f1b500ffb20 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b46ffd700 1 -- 172.21.15.90:0/993352302 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1b40017400 con 0x7f1b500ffb20 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.597+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b501a15a0 con 0x7f1b500ffb20 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.598+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b501016d0 con 0x7f1b500ffb20 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.598+0000 7f1b46ffd700 1 -- 172.21.15.90:0/993352302 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1b40003a60 con 0x7f1b500ffb20 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.599+0000 7f1b46ffd700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1b3c06e200 0x7f1b3c0706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.599+0000 7f1b46ffd700 1 -- 172.21.15.90:0/993352302 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f1b4008f5b0 con 0x7f1b500ffb20 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.599+0000 7f1b55a6e700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1b3c06e200 0x7f1b3c0706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:52:41.601 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.600+0000 7f1b55a6e700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1b3c06e200 0x7f1b3c0706b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005950 tx=0x7f1b4c00b410 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:52:41.605 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.604+0000 7f1b46ffd700 1 -- 172.21.15.90:0/993352302 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1b400588b0 con 0x7f1b500ffb20 2024-04-26T04:52:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.897+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7f1b5002cc30 con 0x7f1b500ffb20 2024-04-26T04:52:41.899 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.898+0000 7f1b46ffd700 1 -- 172.21.15.90:0/993352302 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+107 (secure 0 0 0) 0x7f1b4005bf70 con 0x7f1b500ffb20 2024-04-26T04:52:41.899 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_WARN 1 filesystem is degraded 2024-04-26T04:52:41.899 INFO:teuthology.orchestra.run.smithi090.stdout:[WRN] FS_DEGRADED: 1 filesystem is degraded 2024-04-26T04:52:41.899 INFO:teuthology.orchestra.run.smithi090.stdout: fs cephfs is degraded 2024-04-26T04:52:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1b3c06e200 msgr2=0x7f1b3c0706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1b3c06e200 0x7f1b3c0706b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c005950 tx=0x7f1b4c00b410 comp rx=0 tx=0).stop 2024-04-26T04:52:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 msgr2=0x7f1b5019c9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:52:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b5019c9c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f1b40006010 tx=0x7f1b4000bd70 comp rx=0 tx=0).stop 2024-04-26T04:52:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 shutdown_connections 2024-04-26T04:52:41.901 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f1b3c06e200 0x7f1b3c0706b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f1b500ff210 0x7f1b5019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 --2- 172.21.15.90:0/993352302 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f1b500ffb20 0x7f1b5019c9c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:52:41.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 >> 172.21.15.90:0/993352302 conn(0x7f1b500fac80 msgr2=0x7f1b50110680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:52:41.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 shutdown_connections 2024-04-26T04:52:41.902 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:52:41.900+0000 7f1b57cd2700 1 -- 172.21.15.90:0/993352302 wait complete. 2024-04-26T04:52:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:41 smithi090 ceph-mon[86550]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:41 smithi090 ceph-mon[86550]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:41 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:41 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:52:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:41 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:42.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:41 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3562903759' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:41 smithi104 ceph-mon[64778]: from='client.34452 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:41 smithi104 ceph-mon[64778]: from='client.34456 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:41 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:41 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get-or-create", "entity": "client.ceph-exporter.smithi104", "caps": ["mon", "profile ceph-exporter", "mon", "allow r", "mgr", "allow r", "osd", "allow r"]}]: dispatch 2024-04-26T04:52:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:41 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:42.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:41 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3562903759' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:52:42.923 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:42 smithi104 ceph-mon[64778]: Upgrade: Updating ceph-exporter.smithi104 (2/2) 2024-04-26T04:52:42.923 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:42 smithi104 ceph-mon[64778]: Deploying daemon ceph-exporter.smithi104 on smithi104 2024-04-26T04:52:42.923 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:42 smithi104 ceph-mon[64778]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:42.923 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:42 smithi104 ceph-mon[64778]: pgmap v211: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-26T04:52:42.923 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:42 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/993352302' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:42.923 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:42 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:resolve 2024-04-26T04:52:42.924 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:42 smithi104 ceph-mon[64778]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:resolve,1=cephfs.smithi090.dbkjuk=up:resolve} 2 up:standby 2024-04-26T04:52:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:42 smithi090 ceph-mon[86550]: Upgrade: Updating ceph-exporter.smithi104 (2/2) 2024-04-26T04:52:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:42 smithi090 ceph-mon[86550]: Deploying daemon ceph-exporter.smithi104 on smithi104 2024-04-26T04:52:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:42 smithi090 ceph-mon[86550]: from='client.34468 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:52:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:42 smithi090 ceph-mon[86550]: pgmap v211: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 23 MiB/s rd, 7 op/s 2024-04-26T04:52:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:42 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/993352302' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:52:43.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:42 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:resolve 2024-04-26T04:52:43.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:42 smithi090 ceph-mon[86550]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:resolve,1=cephfs.smithi090.dbkjuk=up:resolve} 2 up:standby 2024-04-26T04:52:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: reconnect by client.24315 v1:172.21.15.90:0/3181179058 after 0 2024-04-26T04:52:44.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: reconnect by client.14540 v1:172.21.15.104:0/3108279391 after 0 2024-04-26T04:52:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: reconnect by client.14540 v1:172.21.15.104:0/3108279391 after 0.000999985 2024-04-26T04:52:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: reconnect by client.24315 v1:172.21.15.90:0/3181179058 after 0.000999985 2024-04-26T04:52:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:reconnect 2024-04-26T04:52:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:reconnect 2024-04-26T04:52:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:reconnect,1=cephfs.smithi090.dbkjuk=up:reconnect} 2 up:standby 2024-04-26T04:52:44.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:43 smithi090 ceph-mon[86550]: pgmap v212: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-04-26T04:52:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: reconnect by client.24315 v1:172.21.15.90:0/3181179058 after 0 2024-04-26T04:52:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: reconnect by client.14540 v1:172.21.15.104:0/3108279391 after 0 2024-04-26T04:52:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: reconnect by client.14540 v1:172.21.15.104:0/3108279391 after 0.000999985 2024-04-26T04:52:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: reconnect by client.24315 v1:172.21.15.90:0/3181179058 after 0.000999985 2024-04-26T04:52:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:reconnect 2024-04-26T04:52:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:reconnect 2024-04-26T04:52:44.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:reconnect,1=cephfs.smithi090.dbkjuk=up:reconnect} 2 up:standby 2024-04-26T04:52:44.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:43 smithi104 ceph-mon[64778]: pgmap v212: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 24 MiB/s rd, 8 op/s 2024-04-26T04:52:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:rejoin 2024-04-26T04:52:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:rejoin 2024-04-26T04:52:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:rejoin,1=cephfs.smithi090.dbkjuk=up:rejoin} 2 up:standby 2024-04-26T04:52:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: daemon mds.cephfs.smithi090.ehputv is now active in filesystem cephfs as rank 0 2024-04-26T04:52:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: daemon mds.cephfs.smithi090.dbkjuk is now active in filesystem cephfs as rank 1 2024-04-26T04:52:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:45.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:45.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:44 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:rejoin 2024-04-26T04:52:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:rejoin 2024-04-26T04:52:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: fsmap cephfs:2/2 {0=cephfs.smithi090.ehputv=up:rejoin,1=cephfs.smithi090.dbkjuk=up:rejoin} 2 up:standby 2024-04-26T04:52:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: daemon mds.cephfs.smithi090.ehputv is now active in filesystem cephfs as rank 0 2024-04-26T04:52:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: daemon mds.cephfs.smithi090.dbkjuk is now active in filesystem cephfs as rank 1 2024-04-26T04:52:45.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:45.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:45.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:44 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:45 smithi090 ceph-mon[86550]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T04:52:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:45 smithi090 ceph-mon[86550]: Cluster is now healthy 2024-04-26T04:52:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:45 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:active 2024-04-26T04:52:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:45 smithi090 ceph-mon[86550]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:active 2024-04-26T04:52:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:45 smithi090 ceph-mon[86550]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi090.dbkjuk=up:active} 2 up:standby 2024-04-26T04:52:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:45 smithi090 ceph-mon[86550]: pgmap v213: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-04-26T04:52:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:45 smithi104 ceph-mon[64778]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-26T04:52:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:45 smithi104 ceph-mon[64778]: Cluster is now healthy 2024-04-26T04:52:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:45 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] up:active 2024-04-26T04:52:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:45 smithi104 ceph-mon[64778]: mds.? [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] up:active 2024-04-26T04:52:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:45 smithi104 ceph-mon[64778]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi090.dbkjuk=up:active} 2 up:standby 2024-04-26T04:52:46.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:45 smithi104 ceph-mon[64778]: pgmap v213: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 22 MiB/s rd, 7 op/s 2024-04-26T04:52:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:48 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:48.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:48 smithi090 ceph-mon[86550]: pgmap v214: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 682 B/s wr, 12 op/s 2024-04-26T04:52:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:48 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:48.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:48 smithi104 ceph-mon[64778]: pgmap v214: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 21 MiB/s rd, 682 B/s wr, 12 op/s 2024-04-26T04:52:49.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:49 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:49.473 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:49 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:49.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:49 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:49.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:49 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:50.251 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:50 smithi104 ceph-mon[64778]: pgmap v215: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.8 MiB/s rd, 853 B/s wr, 7 op/s 2024-04-26T04:52:50.318 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:50 smithi090 ceph-mon[86550]: pgmap v215: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 5.8 MiB/s rd, 853 B/s wr, 7 op/s 2024-04-26T04:52:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:52 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:52 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:52 smithi090 ceph-mon[86550]: pgmap v216: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T04:52:52.721 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:52 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:52.722 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:52 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:52 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:52 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:52 smithi104 ceph-mon[64778]: pgmap v216: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 4.0 MiB/s rd, 1023 B/s wr, 8 op/s 2024-04-26T04:52:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:52 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:52.761 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:52 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:54.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:54 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:54.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:54 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:54.472 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:54 smithi090 ceph-mon[86550]: pgmap v217: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T04:52:54.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:54 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:54.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:54 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:54.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:54 smithi104 ceph-mon[64778]: pgmap v217: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.3 MiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T04:52:56.763 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:56.763 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:56 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:56.763 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:56 smithi090 ceph-mon[86550]: pgmap v218: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T04:52:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:56 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:52:57.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:56 smithi104 ceph-mon[64778]: pgmap v218: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T04:52:58.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:58.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:58.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:58.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:58.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:58.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:58.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:57 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-26T04:52:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:58.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:57 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]: dispatch 2024-04-26T04:52:59.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: Upgrade: Setting filesystem cephfs Joinable 2024-04-26T04:52:59.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: pgmap v219: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T04:52:59.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-26T04:52:59.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi090.dbkjuk=up:active} 2 up:standby 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all ceph-exporter 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]': finished 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi104"}]: dispatch 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi104"}]': finished 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.223 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all iscsi 2024-04-26T04:52:59.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.224 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-26T04:52:59.225 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-26T04:52:59.226 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:59.227 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:59.228 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.228 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:59.228 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:59.228 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:59.228 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: Upgrade: Setting filesystem cephfs Joinable 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: pgmap v219: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 6.7 KiB/s rd, 10 KiB/s wr, 11 op/s 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "joinable", "val": "true"}]': finished 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: fsmap cephfs:2 {0=cephfs.smithi090.ehputv=up:active,1=cephfs.smithi090.dbkjuk=up:active} 2 up:standby 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all ceph-exporter 2024-04-26T04:52:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]: dispatch 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi090"}]': finished 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi104"}]: dispatch 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter.smithi104"}]': finished 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all iscsi 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.263 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mgr"}]': finished 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mon"}]': finished 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.crash"}]': finished 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "osd"}]: dispatch 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "osd"}]': finished 2024-04-26T04:52:59.264 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mds"}]: dispatch 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "mds"}]': finished 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]: dispatch 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rgw"}]': finished 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]: dispatch 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.rbd-mirror"}]': finished 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]: dispatch 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.ceph-exporter"}]': finished 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]: dispatch 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.iscsi"}]': finished 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]: dispatch 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nfs"}]': finished 2024-04-26T04:52:59.265 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix": "config rm", "name": "container_image", "who": "client.nvmeof"}]': finished 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config rm", "name": "container_image", "who": "mon"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd='[{"prefix":"config-key del","key":"mgr/cephadm/upgrade_state"}]': finished 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:59.266 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:52:59.267 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:52:59.267 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:52:59.267 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:52:59.267 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:53:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:59 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all nfs 2024-04-26T04:53:00.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:59 smithi104 ceph-mon[64778]: Upgrade: Setting container_image for all nvmeof 2024-04-26T04:53:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:59 smithi104 ceph-mon[64778]: Upgrade: Finalizing container_image settings 2024-04-26T04:53:00.012 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:52:59 smithi104 ceph-mon[64778]: Upgrade: Complete! 2024-04-26T04:53:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:59 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all nfs 2024-04-26T04:53:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:59 smithi090 ceph-mon[86550]: Upgrade: Setting container_image for all nvmeof 2024-04-26T04:53:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:59 smithi090 ceph-mon[86550]: Upgrade: Finalizing container_image settings 2024-04-26T04:53:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:52:59 smithi090 ceph-mon[86550]: Upgrade: Complete! 2024-04-26T04:53:01.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:00 smithi090 ceph-mon[86550]: pgmap v220: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-04-26T04:53:01.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:00 smithi104 ceph-mon[64778]: pgmap v220: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 2.3 KiB/s rd, 9.6 KiB/s wr, 5 op/s 2024-04-26T04:53:02.419 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:02 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:53:02.419 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:02 smithi090 ceph-mon[86550]: pgmap v221: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 9.4 KiB/s wr, 4 op/s 2024-04-26T04:53:02.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:02 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:53:02.511 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:02 smithi104 ceph-mon[64778]: pgmap v221: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 1.2 KiB/s rd, 9.4 KiB/s wr, 4 op/s 2024-04-26T04:53:05.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:04 smithi090 ceph-mon[86550]: pgmap v222: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.2 KiB/s wr, 3 op/s 2024-04-26T04:53:05.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:04 smithi104 ceph-mon[64778]: pgmap v222: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail; 9.2 KiB/s wr, 3 op/s 2024-04-26T04:53:07.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:06 smithi090 ceph-mon[86550]: pgmap v223: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:07.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:06 smithi104 ceph-mon[64778]: pgmap v223: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:09.063 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:08 smithi104 ceph-mon[64778]: pgmap v224: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:09.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:08 smithi090 ceph-mon[86550]: pgmap v224: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:10 smithi090 ceph-mon[86550]: pgmap v225: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:10 smithi104 ceph-mon[64778]: pgmap v225: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:12.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.017+0000 7f2a31de8700 1 -- 172.21.15.90:0/2068545022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c100e80 msgr2=0x7f2a2c104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.017+0000 7f2a31de8700 1 --2- 172.21.15.90:0/2068545022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c100e80 0x7f2a2c104e50 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f2a20009910 tx=0x7f2a20009c20 comp rx=0 tx=0).stop 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.018+0000 7f2a31de8700 1 -- 172.21.15.90:0/2068545022 shutdown_connections 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.018+0000 7f2a31de8700 1 --2- 172.21.15.90:0/2068545022 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c100e80 0x7f2a2c104e50 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.018+0000 7f2a31de8700 1 --2- 172.21.15.90:0/2068545022 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a2c1004e0 0x7f2a2c1008b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.018+0000 7f2a31de8700 1 -- 172.21.15.90:0/2068545022 >> 172.21.15.90:0/2068545022 conn(0x7f2a2c0fbda0 msgr2=0x7f2a2c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.018+0000 7f2a31de8700 1 -- 172.21.15.90:0/2068545022 shutdown_connections 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.018+0000 7f2a31de8700 1 -- 172.21.15.90:0/2068545022 wait complete. 2024-04-26T04:53:12.022 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.018+0000 7f2a31de8700 1 Processor -- start 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a31de8700 1 -- start start 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a31de8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c1004e0 0x7f2a2c073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a31de8700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a2c100e80 0x7f2a2c073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a31de8700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c073ad0 con 0x7f2a2c1004e0 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a31de8700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a2c073c10 con 0x7f2a2c100e80 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c1004e0 0x7f2a2c073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2b7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c1004e0 0x7f2a2c073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40208/0 (socket says 172.21.15.90:40208) 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2b7fe700 1 -- 172.21.15.90:0/670082453 learned_addr learned my addr 172.21.15.90:0/670082453 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2affd700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a2c100e80 0x7f2a2c073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2b7fe700 1 -- 172.21.15.90:0/670082453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a2c100e80 msgr2=0x7f2a2c073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2b7fe700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a2c100e80 0x7f2a2c073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2b7fe700 1 -- 172.21.15.90:0/670082453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a200095f0 con 0x7f2a2c1004e0 2024-04-26T04:53:12.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.019+0000 7f2a2b7fe700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c1004e0 0x7f2a2c073050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00d8d0 tx=0x7f2a1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:12.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.020+0000 7f2a28ff9700 1 -- 172.21.15.90:0/670082453 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c009940 con 0x7f2a2c1004e0 2024-04-26T04:53:12.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.020+0000 7f2a28ff9700 1 -- 172.21.15.90:0/670082453 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2a1c010460 con 0x7f2a2c1004e0 2024-04-26T04:53:12.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.020+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a2c1a20e0 con 0x7f2a2c1004e0 2024-04-26T04:53:12.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.020+0000 7f2a28ff9700 1 -- 172.21.15.90:0/670082453 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2a1c00f5f0 con 0x7f2a2c1004e0 2024-04-26T04:53:12.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.020+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a2c1a24f0 con 0x7f2a2c1004e0 2024-04-26T04:53:12.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.021+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a2c108850 con 0x7f2a2c1004e0 2024-04-26T04:53:12.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.021+0000 7f2a28ff9700 1 -- 172.21.15.90:0/670082453 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2a1c0105d0 con 0x7f2a2c1004e0 2024-04-26T04:53:12.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.022+0000 7f2a28ff9700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2a1406e200 0x7f2a140706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:12.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.022+0000 7f2a28ff9700 1 -- 172.21.15.90:0/670082453 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2a1c08e1a0 con 0x7f2a2c1004e0 2024-04-26T04:53:12.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.022+0000 7f2a2affd700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2a1406e200 0x7f2a140706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:12.025 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.022+0000 7f2a2affd700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2a1406e200 0x7f2a140706b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c074840 tx=0x7f2a2001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:12.027 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.027+0000 7f2a28ff9700 1 -- 172.21.15.90:0/670082453 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2a1c057480 con 0x7f2a2c1004e0 2024-04-26T04:53:12.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.247+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a2c061190 con 0x7f2a1406e200 2024-04-26T04:53:12.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.249+0000 7f2a28ff9700 1 -- 172.21.15.90:0/670082453 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7f2a2c061190 con 0x7f2a1406e200 2024-04-26T04:53:12.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2a1406e200 msgr2=0x7f2a140706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:12.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2a1406e200 0x7f2a140706b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2a2c074840 tx=0x7f2a2001a040 comp rx=0 tx=0).stop 2024-04-26T04:53:12.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c1004e0 msgr2=0x7f2a2c073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:12.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c1004e0 0x7f2a2c073050 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f2a1c00d8d0 tx=0x7f2a1c00dc90 comp rx=0 tx=0).stop 2024-04-26T04:53:12.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 shutdown_connections 2024-04-26T04:53:12.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2a1406e200 0x7f2a140706b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:12.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2a2c1004e0 0x7f2a2c073050 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:12.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 --2- 172.21.15.90:0/670082453 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2a2c100e80 0x7f2a2c073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:12.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 >> 172.21.15.90:0/670082453 conn(0x7f2a2c0fbda0 msgr2=0x7f2a2c0fe180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:12.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 shutdown_connections 2024-04-26T04:53:12.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:12.251+0000 7f2a31de8700 1 -- 172.21.15.90:0/670082453 wait complete. 2024-04-26T04:53:12.845 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:12 smithi090 ceph-mon[86550]: pgmap v226: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:12.961 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch ps' 2024-04-26T04:53:13.167 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:53:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:12 smithi104 ceph-mon[64778]: pgmap v226: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:13.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:13 smithi090 ceph-mon[86550]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:14.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:13 smithi104 ceph-mon[64778]: from='client.34476 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:14.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 -- 172.21.15.90:0/3072884422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 msgr2=0x7fe4201008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:14.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 --2- 172.21.15.90:0/3072884422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe4201008b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe41800b0a0 tx=0x7fe41800b3b0 comp rx=0 tx=0).stop 2024-04-26T04:53:14.906 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 -- 172.21.15.90:0/3072884422 shutdown_connections 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 --2- 172.21.15.90:0/3072884422 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420100e80 0x7fe420104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 --2- 172.21.15.90:0/3072884422 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe4201008b0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 -- 172.21.15.90:0/3072884422 >> 172.21.15.90:0/3072884422 conn(0x7fe4200fbda0 msgr2=0x7fe4200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 -- 172.21.15.90:0/3072884422 shutdown_connections 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.903+0000 7fe424c32700 1 -- 172.21.15.90:0/3072884422 wait complete. 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe424c32700 1 Processor -- start 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe424c32700 1 -- start start 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe424c32700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe420073050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe424c32700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420100e80 0x7fe420073590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe424c32700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420073ad0 con 0x7fe4201004e0 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe424c32700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe420073c10 con 0x7fe420100e80 2024-04-26T04:53:14.907 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe41e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe420073050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe41e59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe420073050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40222/0 (socket says 172.21.15.90:40222) 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe41e59c700 1 -- 172.21.15.90:0/2856788737 learned_addr learned my addr 172.21.15.90:0/2856788737 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe41dd9b700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420100e80 0x7fe420073590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe41e59c700 1 -- 172.21.15.90:0/2856788737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420100e80 msgr2=0x7fe420073590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe41e59c700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420100e80 0x7fe420073590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.904+0000 7fe41e59c700 1 -- 172.21.15.90:0/2856788737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe418009d00 con 0x7fe4201004e0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.905+0000 7fe41e59c700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe420073050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe418000f80 tx=0x7fe41800bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.905+0000 7fe40f7fe700 1 -- 172.21.15.90:0/2856788737 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe41800e040 con 0x7fe4201004e0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.905+0000 7fe40f7fe700 1 -- 172.21.15.90:0/2856788737 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe418003c10 con 0x7fe4201004e0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.905+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe420073e90 con 0x7fe4201004e0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.905+0000 7fe40f7fe700 1 -- 172.21.15.90:0/2856788737 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe41801d680 con 0x7fe4201004e0 2024-04-26T04:53:14.908 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.905+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4201a23f0 con 0x7fe4201004e0 2024-04-26T04:53:14.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.906+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4201087f0 con 0x7fe4201004e0 2024-04-26T04:53:14.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.906+0000 7fe40f7fe700 1 -- 172.21.15.90:0/2856788737 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe418019070 con 0x7fe4201004e0 2024-04-26T04:53:14.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.907+0000 7fe40f7fe700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe40806e200 0x7fe4080706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:14.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.907+0000 7fe40f7fe700 1 -- 172.21.15.90:0/2856788737 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe41808f5e0 con 0x7fe4201004e0 2024-04-26T04:53:14.909 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.907+0000 7fe41dd9b700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe40806e200 0x7fe4080706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:14.910 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.907+0000 7fe41dd9b700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe40806e200 0x7fe4080706b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe420074840 tx=0x7fe414009260 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:14.912 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:14.911+0000 7fe40f7fe700 1 -- 172.21.15.90:0/2856788737 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe418058940 con 0x7fe4201004e0 2024-04-26T04:53:14.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:14 smithi090 ceph-mon[86550]: pgmap v227: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:15.123 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.122+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe420061190 con 0x7fe40806e200 2024-04-26T04:53:15.133 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.132+0000 7fe40f7fe700 1 -- 172.21.15.90:0/2856788737 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7fe420061190 con 0x7fe40806e200 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (8m) 23s ago 18m 24.3M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (36s) 23s ago 19m 7792k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 6c9e14bbff14 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (30s) 28s ago 18m 6916k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9cef35eaf52 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (5m) 23s ago 18m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (5m) 28s ago 17m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (7m) 23s ago 18m 79.9M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (85s) 23s ago 13m 69.1M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a01df3d7e4d4 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (105s) 23s ago 13m 72.9M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 4bcc1e9fbb69 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (51s) 28s ago 13m 16.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 0dbd9167e2e1 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (69s) 28s ago 13m 15.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 7d741ad3ecf3 2024-04-26T04:53:15.134 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (9m) 23s ago 20m 510M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (8m) 28s ago 17m 451M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (6m) 23s ago 20m 58.3M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (6m) 28s ago 17m 47.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (8m) 23s ago 18m 16.5M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (8m) 28s ago 17m 14.6M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (4m) 23s ago 17m 98.6M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (4m) 23s ago 16m 114M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (3m) 23s ago 16m 91.2M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (3m) 28s ago 15m 120M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d860fd1e9069 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (2m) 28s ago 15m 97.5M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 fbfd5c1c44eb 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (2m) 28s ago 14m 95.6M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 64a9be8458bb 2024-04-26T04:53:15.135 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (8m) 23s ago 18m 72.0M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:53:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe40806e200 msgr2=0x7fe4080706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe40806e200 0x7fe4080706b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe420074840 tx=0x7fe414009260 comp rx=0 tx=0).stop 2024-04-26T04:53:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 msgr2=0x7fe420073050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe420073050 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe418000f80 tx=0x7fe41800bc60 comp rx=0 tx=0).stop 2024-04-26T04:53:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 shutdown_connections 2024-04-26T04:53:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe40806e200 0x7fe4080706b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:15.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe4201004e0 0x7fe420073050 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 --2- 172.21.15.90:0/2856788737 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe420100e80 0x7fe420073590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 >> 172.21.15.90:0/2856788737 conn(0x7fe4200fbda0 msgr2=0x7fe4200fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 shutdown_connections 2024-04-26T04:53:15.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:15.135+0000 7fe424c32700 1 -- 172.21.15.90:0/2856788737 wait complete. 2024-04-26T04:53:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:14 smithi104 ceph-mon[64778]: pgmap v227: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:15.892 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch upgrade status' 2024-04-26T04:53:16.112 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:53:17.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:16 smithi090 ceph-mon[86550]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:17.205 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:16 smithi090 ceph-mon[86550]: pgmap v228: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:17.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:16 smithi104 ceph-mon[64778]: from='client.34480 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:17.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:16 smithi104 ceph-mon[64778]: pgmap v228: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:17.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3312756657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ff3c0 msgr2=0x7fc8c80ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:17.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3312756657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ff3c0 0x7fc8c80ff790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc009910 tx=0x7fc8bc009c20 comp rx=0 tx=0).stop 2024-04-26T04:53:17.816 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3312756657 shutdown_connections 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3312756657 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c80ffd60 0x7fc8c8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3312756657 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ff3c0 0x7fc8c80ff790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3312756657 >> 172.21.15.90:0/3312756657 conn(0x7fc8c80fac80 msgr2=0x7fc8c80fd090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3312756657 shutdown_connections 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.813+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3312756657 wait complete. 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8ceb7c700 1 Processor -- start 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8ceb7c700 1 -- start start 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8ceb7c700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c80ff3c0 0x7fc8c8196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8ceb7c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ffd60 0x7fc8c8196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8ceb7c700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c8196c40 con 0x7fc8c80ffd60 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8ceb7c700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c819a9d0 con 0x7fc8c80ff3c0 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8c7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ffd60 0x7fc8c8196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:17.817 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8c7fff700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ffd60 0x7fc8c8196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40154/0 (socket says 172.21.15.90:40154) 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8c7fff700 1 -- 172.21.15.90:0/3505969441 learned_addr learned my addr 172.21.15.90:0/3505969441 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.814+0000 7fc8cc918700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c80ff3c0 0x7fc8c8196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8c7fff700 1 -- 172.21.15.90:0/3505969441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c80ff3c0 msgr2=0x7fc8c8196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8c7fff700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c80ff3c0 0x7fc8c8196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8c7fff700 1 -- 172.21.15.90:0/3505969441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8bc0095f0 con 0x7fc8c80ffd60 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8c7fff700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ffd60 0x7fc8c8196560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b480 tx=0x7fc8b400b790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8c5ffb700 1 -- 172.21.15.90:0/3505969441 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8b4011800 con 0x7fc8c80ffd60 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8c5ffb700 1 -- 172.21.15.90:0/3505969441 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc8b4011e20 con 0x7fc8c80ffd60 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8c819acb0 con 0x7fc8c80ffd60 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8c5ffb700 1 -- 172.21.15.90:0/3505969441 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8b4010ea0 con 0x7fc8c80ffd60 2024-04-26T04:53:17.818 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.815+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8c819b200 con 0x7fc8c80ffd60 2024-04-26T04:53:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.816+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8c804f2a0 con 0x7fc8c80ffd60 2024-04-26T04:53:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.816+0000 7fc8c5ffb700 1 -- 172.21.15.90:0/3505969441 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc8b4010380 con 0x7fc8c80ffd60 2024-04-26T04:53:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.817+0000 7fc8c5ffb700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc8b806e200 0x7fc8b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:17.819 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.817+0000 7fc8c5ffb700 1 -- 172.21.15.90:0/3505969441 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc8b408dc90 con 0x7fc8c80ffd60 2024-04-26T04:53:17.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.817+0000 7fc8cc918700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc8b806e200 0x7fc8b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:17.820 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.817+0000 7fc8cc918700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc8b806e200 0x7fc8b80706b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005f50 tx=0x7fc8bc01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:17.823 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:17.822+0000 7fc8c5ffb700 1 -- 172.21.15.90:0/3505969441 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc8b4056f70 con 0x7fc8c80ffd60 2024-04-26T04:53:18.042 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.041+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}) v1 -- 0x7fc8c819b4f0 con 0x7fc8b806e200 2024-04-26T04:53:18.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.042+0000 7fc8c5ffb700 1 -- 172.21.15.90:0/3505969441 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+175 (secure 0 0 0) 0x7fc8c819b4f0 con 0x7fc8b806e200 2024-04-26T04:53:18.043 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:53:18.043 INFO:teuthology.orchestra.run.smithi090.stdout: "target_image": null, 2024-04-26T04:53:18.043 INFO:teuthology.orchestra.run.smithi090.stdout: "in_progress": false, 2024-04-26T04:53:18.043 INFO:teuthology.orchestra.run.smithi090.stdout: "which": "", 2024-04-26T04:53:18.044 INFO:teuthology.orchestra.run.smithi090.stdout: "services_complete": [], 2024-04-26T04:53:18.044 INFO:teuthology.orchestra.run.smithi090.stdout: "progress": null, 2024-04-26T04:53:18.044 INFO:teuthology.orchestra.run.smithi090.stdout: "message": "", 2024-04-26T04:53:18.044 INFO:teuthology.orchestra.run.smithi090.stdout: "is_paused": false 2024-04-26T04:53:18.044 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:53:18.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc8b806e200 msgr2=0x7fc8b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:18.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc8b806e200 0x7fc8b80706b0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc005f50 tx=0x7fc8bc01a040 comp rx=0 tx=0).stop 2024-04-26T04:53:18.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ffd60 msgr2=0x7fc8c8196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:18.045 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ffd60 0x7fc8c8196560 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc8b400b480 tx=0x7fc8b400b790 comp rx=0 tx=0).stop 2024-04-26T04:53:18.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 shutdown_connections 2024-04-26T04:53:18.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc8b806e200 0x7fc8b80706b0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:18.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8c80ff3c0 0x7fc8c8196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:18.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 --2- 172.21.15.90:0/3505969441 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8c80ffd60 0x7fc8c8196560 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:18.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 >> 172.21.15.90:0/3505969441 conn(0x7fc8c80fac80 msgr2=0x7fc8c8103590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:18.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 shutdown_connections 2024-04-26T04:53:18.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:18.044+0000 7fc8ceb7c700 1 -- 172.21.15.90:0/3505969441 wait complete. 2024-04-26T04:53:18.714 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph health detail' 2024-04-26T04:53:18.922 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:53:18.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:18 smithi090 ceph-mon[86550]: pgmap v229: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:18 smithi104 ceph-mon[64778]: pgmap v229: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:20.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:19 smithi104 ceph-mon[64778]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:20.063 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:19 smithi090 ceph-mon[86550]: from='client.34484 -' entity='client.admin' cmd=[{"prefix": "orch upgrade status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:53:20.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 -- 172.21.15.90:0/3074756034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 msgr2=0x7fa61c100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:20.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 --2- 172.21.15.90:0/3074756034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100040 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fa610009910 tx=0x7fa610009c20 comp rx=0 tx=0).stop 2024-04-26T04:53:20.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 -- 172.21.15.90:0/3074756034 shutdown_connections 2024-04-26T04:53:20.535 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 --2- 172.21.15.90:0/3074756034 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100040 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 --2- 172.21.15.90:0/3074756034 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa61c104130 0x7fa61c104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 -- 172.21.15.90:0/3074756034 >> 172.21.15.90:0/3074756034 conn(0x7fa61c075130 msgr2=0x7fa61c075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 -- 172.21.15.90:0/3074756034 shutdown_connections 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.532+0000 7fa623415700 1 -- 172.21.15.90:0/3074756034 wait complete. 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa623415700 1 Processor -- start 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa623415700 1 -- start start 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa623415700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa623415700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa61c104130 0x7fa61c101370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa623415700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c1018b0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa623415700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c1019f0 con 0x7fa61c104130 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6211b1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6211b1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40174/0 (socket says 172.21.15.90:40174) 2024-04-26T04:53:20.536 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6211b1700 1 -- 172.21.15.90:0/875341624 learned_addr learned my addr 172.21.15.90:0/875341624 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6209b0700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa61c104130 0x7fa61c101370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6211b1700 1 -- 172.21.15.90:0/875341624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa61c104130 msgr2=0x7fa61c101370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6211b1700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa61c104130 0x7fa61c101370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6211b1700 1 -- 172.21.15.90:0/875341624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100095f0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.533+0000 7fa6211b1700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100e30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6140060b0 tx=0x7fa61400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.534+0000 7fa61a7fc700 1 -- 172.21.15.90:0/875341624 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa61400dda0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.534+0000 7fa61a7fc700 1 -- 172.21.15.90:0/875341624 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa61400f040 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.534+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c1a1ff0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.534+0000 7fa61a7fc700 1 -- 172.21.15.90:0/875341624 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa6140146f0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.537 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.534+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c1a2400 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.535+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c1086a0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.535+0000 7fa61a7fc700 1 -- 172.21.15.90:0/875341624 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa614015400 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.536+0000 7fa61a7fc700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa60406e200 0x7fa6040706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:20.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.536+0000 7fa6209b0700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa60406e200 0x7fa6040706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:20.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.536+0000 7fa61a7fc700 1 -- 172.21.15.90:0/875341624 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa61408da90 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.538 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.536+0000 7fa6209b0700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa60406e200 0x7fa6040706b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa61c102620 tx=0x7fa61001a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:20.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.541+0000 7fa61a7fc700 1 -- 172.21.15.90:0/875341624 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa614056ea0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.825+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "health", "detail": "detail"} v 0) v1 -- 0x7fa61c04f2a0 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.826 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.825+0000 7fa61a7fc700 1 -- 172.21.15.90:0/875341624 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "health", "detail": "detail"}]=0 v0) v1 ==== 74+0+10 (secure 0 0 0) 0x7fa61405a560 con 0x7fa61c0ffbd0 2024-04-26T04:53:20.827 INFO:teuthology.orchestra.run.smithi090.stdout:HEALTH_OK 2024-04-26T04:53:20.828 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa60406e200 msgr2=0x7fa6040706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa60406e200 0x7fa6040706b0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa61c102620 tx=0x7fa61001a040 comp rx=0 tx=0).stop 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 msgr2=0x7fa61c100e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100e30 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa6140060b0 tx=0x7fa61400d3b0 comp rx=0 tx=0).stop 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 shutdown_connections 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa60406e200 0x7fa6040706b0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa61c0ffbd0 0x7fa61c100e30 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 --2- 172.21.15.90:0/875341624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa61c104130 0x7fa61c101370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 >> 172.21.15.90:0/875341624 conn(0x7fa61c075130 msgr2=0x7fa61c0fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 shutdown_connections 2024-04-26T04:53:20.829 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:20.827+0000 7fa623415700 1 -- 172.21.15.90:0/875341624 wait complete. 2024-04-26T04:53:21.127 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:20 smithi090 ceph-mon[86550]: pgmap v230: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:20 smithi104 ceph-mon[64778]: pgmap v230: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:21.454 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions' 2024-04-26T04:53:21.663 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:53:22.037 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:21 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/875341624' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:53:22.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:21 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/875341624' entity='client.admin' cmd=[{"prefix": "health", "detail": "detail"}]: dispatch 2024-04-26T04:53:23.029 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:22 smithi090 ceph-mon[86550]: pgmap v231: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 -- 172.21.15.90:0/2324257329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc0ffbd0 msgr2=0x7f75cc100040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 --2- 172.21.15.90:0/2324257329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc0ffbd0 0x7f75cc100040 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f75bc009910 tx=0x7f75bc009c20 comp rx=0 tx=0).stop 2024-04-26T04:53:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 -- 172.21.15.90:0/2324257329 shutdown_connections 2024-04-26T04:53:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 --2- 172.21.15.90:0/2324257329 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc0ffbd0 0x7f75cc100040 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 --2- 172.21.15.90:0/2324257329 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75cc104130 0x7f75cc104500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:23.246 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 -- 172.21.15.90:0/2324257329 >> 172.21.15.90:0/2324257329 conn(0x7f75cc075130 msgr2=0x7f75cc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 -- 172.21.15.90:0/2324257329 shutdown_connections 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.243+0000 7f75d392d700 1 -- 172.21.15.90:0/2324257329 wait complete. 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d392d700 1 Processor -- start 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d392d700 1 -- start start 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d392d700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75cc0ffbd0 0x7f75cc196ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d392d700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc104130 0x7f75cc1971e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d392d700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc197800 con 0x7f75cc104130 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d392d700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75cc197940 con 0x7f75cc0ffbd0 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d0ec8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc104130 0x7f75cc1971e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d0ec8700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc104130 0x7f75cc1971e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:40204/0 (socket says 172.21.15.90:40204) 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d0ec8700 1 -- 172.21.15.90:0/1452550166 learned_addr learned my addr 172.21.15.90:0/1452550166 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:53:23.247 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d16c9700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75cc0ffbd0 0x7f75cc196ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d0ec8700 1 -- 172.21.15.90:0/1452550166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75cc0ffbd0 msgr2=0x7f75cc196ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d0ec8700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75cc0ffbd0 0x7f75cc196ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d0ec8700 1 -- 172.21.15.90:0/1452550166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75bc0095f0 con 0x7f75cc104130 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d16c9700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75cc0ffbd0 0x7f75cc196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.244+0000 7f75d0ec8700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc104130 0x7f75cc1971e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00bd70 tx=0x7f75bc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.245+0000 7f75c27fc700 1 -- 172.21.15.90:0/1452550166 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc01d070 con 0x7f75cc104130 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.245+0000 7f75c27fc700 1 -- 172.21.15.90:0/1452550166 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f75bc00f460 con 0x7f75cc104130 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.245+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75cc19fd10 con 0x7f75cc104130 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.245+0000 7f75c27fc700 1 -- 172.21.15.90:0/1452550166 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f75bc017630 con 0x7f75cc104130 2024-04-26T04:53:23.248 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.245+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75cc1a0120 con 0x7f75cc104130 2024-04-26T04:53:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.246+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75cc04f2a0 con 0x7f75cc104130 2024-04-26T04:53:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.246+0000 7f75c27fc700 1 -- 172.21.15.90:0/1452550166 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f75bc00f5d0 con 0x7f75cc104130 2024-04-26T04:53:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.247+0000 7f75c27fc700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f75b806e200 0x7f75b80706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:53:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.247+0000 7f75c27fc700 1 -- 172.21.15.90:0/1452550166 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f75bc08f6b0 con 0x7f75cc104130 2024-04-26T04:53:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.247+0000 7f75d16c9700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f75b806e200 0x7f75b80706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:53:23.249 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.247+0000 7f75d16c9700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f75b806e200 0x7f75b80706b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005fd0 tx=0x7f75c8005dc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:53:23.252 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.251+0000 7f75c27fc700 1 -- 172.21.15.90:0/1452550166 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f75bc058a10 con 0x7f75cc104130 2024-04-26T04:53:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:22 smithi104 ceph-mon[64778]: pgmap v231: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.539+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f75cc1a0490 con 0x7f75cc104130 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.540+0000 7f75c27fc700 1 -- 172.21.15.90:0/1452550166 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f75bc05c0d0 con 0x7f75cc104130 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:53:23.541 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 14 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:53:23.542 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:53:23.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f75b806e200 msgr2=0x7f75b80706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:23.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f75b806e200 0x7f75b80706b0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f75c8005fd0 tx=0x7f75c8005dc0 comp rx=0 tx=0).stop 2024-04-26T04:53:23.543 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc104130 msgr2=0x7f75cc1971e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc104130 0x7f75cc1971e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f75bc00bd70 tx=0x7f75bc00be50 comp rx=0 tx=0).stop 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 shutdown_connections 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f75b806e200 0x7f75b80706b0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f75cc0ffbd0 0x7f75cc196ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 --2- 172.21.15.90:0/1452550166 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f75cc104130 0x7f75cc1971e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 >> 172.21.15.90:0/1452550166 conn(0x7f75cc075130 msgr2=0x7f75cc0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 shutdown_connections 2024-04-26T04:53:23.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:53:23.542+0000 7f75d392d700 1 -- 172.21.15.90:0/1452550166 wait complete. 2024-04-26T04:53:24.116 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:23 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1452550166' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:53:24.139 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'echo "wait for servicemap items w/ changing names to refresh"' 2024-04-26T04:53:24.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:23 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1452550166' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:53:24.343 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:53:25.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:24 smithi090 ceph-mon[86550]: pgmap v232: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:25.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:24 smithi104 ceph-mon[64778]: pgmap v232: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:25.949 INFO:teuthology.orchestra.run.smithi090.stdout:wait for servicemap items w/ changing names to refresh 2024-04-26T04:53:25.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:26.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:26.676 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'sleep 60' 2024-04-26T04:53:26.883 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:53:27.128 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:26 smithi090 ceph-mon[86550]: pgmap v233: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:26 smithi104 ceph-mon[64778]: pgmap v233: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:29.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:28 smithi090 ceph-mon[86550]: pgmap v234: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:29.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:28 smithi104 ceph-mon[64778]: pgmap v234: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:31.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:30 smithi090 ceph-mon[86550]: pgmap v235: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:31.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:30 smithi104 ceph-mon[64778]: pgmap v235: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:33.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:32 smithi090 ceph-mon[86550]: pgmap v236: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:33.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:32 smithi104 ceph-mon[64778]: pgmap v236: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:35.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:34 smithi090 ceph-mon[86550]: pgmap v237: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:35.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:34 smithi104 ceph-mon[64778]: pgmap v237: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:37.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:36 smithi090 ceph-mon[86550]: pgmap v238: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:37.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:36 smithi104 ceph-mon[64778]: pgmap v238: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:39.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:38 smithi090 ceph-mon[86550]: pgmap v239: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:39.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:38 smithi104 ceph-mon[64778]: pgmap v239: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:41.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:40 smithi090 ceph-mon[86550]: pgmap v240: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:41.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:41.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:40 smithi104 ceph-mon[64778]: pgmap v240: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:41.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:43.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:42 smithi090 ceph-mon[86550]: pgmap v241: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:42 smithi104 ceph-mon[64778]: pgmap v241: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:45.179 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:44 smithi104 ceph-mon[64778]: pgmap v242: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:45.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:44 smithi090 ceph-mon[86550]: pgmap v242: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:46 smithi090 ceph-mon[86550]: pgmap v243: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:46 smithi104 ceph-mon[64778]: pgmap v243: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:49.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:48 smithi090 ceph-mon[86550]: pgmap v244: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:48 smithi104 ceph-mon[64778]: pgmap v244: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:50 smithi090 ceph-mon[86550]: pgmap v245: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:50 smithi104 ceph-mon[64778]: pgmap v245: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:53.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:52 smithi090 ceph-mon[86550]: pgmap v246: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:52 smithi104 ceph-mon[64778]: pgmap v246: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:55.014 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:54 smithi090 ceph-mon[86550]: pgmap v247: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:55.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:54 smithi104 ceph-mon[64778]: pgmap v247: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:56.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:53:57.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:56 smithi090 ceph-mon[86550]: pgmap v248: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:56 smithi104 ceph-mon[64778]: pgmap v248: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:58.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:57 smithi090 ceph-mon[86550]: pgmap v249: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:58.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:57 smithi104 ceph-mon[64778]: pgmap v249: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:53:59.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:53:59.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:53:59.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:58 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:53:59.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-26T04:53:59.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:53:59.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:58 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:54:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:59 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:59 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:54:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:59 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:54:00.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:53:59 smithi090 ceph-mon[86550]: pgmap v250: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:00.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:59 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-26T04:54:00.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:59 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-26T04:54:00.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:59 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' 2024-04-26T04:54:00.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:53:59 smithi104 ceph-mon[64778]: pgmap v250: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:03.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:02 smithi090 ceph-mon[86550]: pgmap v251: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:03.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:02 smithi104 ceph-mon[64778]: pgmap v251: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:05.124 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:04 smithi090 ceph-mon[86550]: pgmap v252: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:05.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:04 smithi104 ceph-mon[64778]: pgmap v252: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:07.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:06 smithi090 ceph-mon[86550]: pgmap v253: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:07.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:06 smithi104 ceph-mon[64778]: pgmap v253: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:09.063 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:08 smithi104 ceph-mon[64778]: pgmap v254: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:09.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:08 smithi090 ceph-mon[86550]: pgmap v254: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:10 smithi090 ceph-mon[86550]: pgmap v255: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:11.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:10 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:10 smithi104 ceph-mon[64778]: pgmap v255: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:11.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:10 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:13.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:12 smithi090 ceph-mon[86550]: pgmap v256: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:13.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:12 smithi104 ceph-mon[64778]: pgmap v256: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:15.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:14 smithi090 ceph-mon[86550]: pgmap v257: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:15.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:14 smithi104 ceph-mon[64778]: pgmap v257: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:17.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:16 smithi090 ceph-mon[86550]: pgmap v258: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:17.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:16 smithi104 ceph-mon[64778]: pgmap v258: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:19.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:18 smithi090 ceph-mon[86550]: pgmap v259: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:19.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:18 smithi104 ceph-mon[64778]: pgmap v259: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:21.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:20 smithi090 ceph-mon[86550]: pgmap v260: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:21.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:20 smithi104 ceph-mon[64778]: pgmap v260: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:23.222 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:22 smithi090 ceph-mon[86550]: pgmap v261: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:23.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:22 smithi104 ceph-mon[64778]: pgmap v261: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:25.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:24 smithi090 ceph-mon[86550]: pgmap v262: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:25.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:24 smithi104 ceph-mon[64778]: pgmap v262: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:26.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:25 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:26.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:25 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:27.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:26 smithi090 ceph-mon[86550]: pgmap v263: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:27.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:26 smithi104 ceph-mon[64778]: pgmap v263: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:28.907 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:28 smithi090 ceph-mon[86550]: pgmap v264: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:29.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:28 smithi104 ceph-mon[64778]: pgmap v264: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:29.455 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph orch ps' 2024-04-26T04:54:29.673 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:54:31.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:30 smithi104 ceph-mon[64778]: pgmap v265: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:31.200 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:30 smithi090 ceph-mon[86550]: pgmap v265: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:31.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 -- 172.21.15.90:0/64744177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ffb30 msgr2=0x7efe4410a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:31.411 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 --2- 172.21.15.90:0/64744177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ffb30 0x7efe4410a5a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7efe2c009910 tx=0x7efe2c009c20 comp rx=0 tx=0).stop 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 -- 172.21.15.90:0/64744177 shutdown_connections 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 --2- 172.21.15.90:0/64744177 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ffb30 0x7efe4410a5a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 --2- 172.21.15.90:0/64744177 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe440ff220 0x7efe440ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 -- 172.21.15.90:0/64744177 >> 172.21.15.90:0/64744177 conn(0x7efe440747e0 msgr2=0x7efe44074be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 -- 172.21.15.90:0/64744177 shutdown_connections 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.408+0000 7efe48cba700 1 -- 172.21.15.90:0/64744177 wait complete. 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe48cba700 1 Processor -- start 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe48cba700 1 -- start start 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe48cba700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ff220 0x7efe44100400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe48cba700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe440ffb30 0x7efe44100940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe48cba700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe44100e80 con 0x7efe440ff220 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe48cba700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe44100fc0 con 0x7efe440ffb30 2024-04-26T04:54:31.412 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe4259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ff220 0x7efe44100400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe4259c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ff220 0x7efe44100400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50074/0 (socket says 172.21.15.90:50074) 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe4259c700 1 -- 172.21.15.90:0/4079076629 learned_addr learned my addr 172.21.15.90:0/4079076629 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe39d9b700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe440ffb30 0x7efe44100940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe4259c700 1 -- 172.21.15.90:0/4079076629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe440ffb30 msgr2=0x7efe44100940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe4259c700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe440ffb30 0x7efe44100940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.409+0000 7efe4259c700 1 -- 172.21.15.90:0/4079076629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c0095f0 con 0x7efe440ff220 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.410+0000 7efe4259c700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ff220 0x7efe44100400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe3400d8d0 tx=0x7efe3400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.410+0000 7efe3bfff700 1 -- 172.21.15.90:0/4079076629 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe3400f860 con 0x7efe440ff220 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.410+0000 7efe3bfff700 1 -- 172.21.15.90:0/4079076629 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7efe3400fe80 con 0x7efe440ff220 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.410+0000 7efe3bfff700 1 -- 172.21.15.90:0/4079076629 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7efe340095d0 con 0x7efe440ff220 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.410+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe441a1ed0 con 0x7efe440ff220 2024-04-26T04:54:31.413 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.410+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe441a22e0 con 0x7efe440ff220 2024-04-26T04:54:31.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.411+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe44107ca0 con 0x7efe440ff220 2024-04-26T04:54:31.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.411+0000 7efe3bfff700 1 -- 172.21.15.90:0/4079076629 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7efe34009810 con 0x7efe440ff220 2024-04-26T04:54:31.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.412+0000 7efe3bfff700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efe3006e200 0x7efe300706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:31.414 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.412+0000 7efe3bfff700 1 -- 172.21.15.90:0/4079076629 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7efe3408e080 con 0x7efe440ff220 2024-04-26T04:54:31.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.412+0000 7efe39d9b700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efe3006e200 0x7efe300706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:31.415 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.412+0000 7efe39d9b700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efe3006e200 0x7efe300706b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe44101bf0 tx=0x7efe2c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:31.417 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.417+0000 7efe3bfff700 1 -- 172.21.15.90:0/4079076629 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7efe34057360 con 0x7efe440ff220 2024-04-26T04:54:31.632 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.631+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 --> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7efe44061190 con 0x7efe3006e200 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.642+0000 7efe3bfff700 1 -- 172.21.15.90:0/4079076629 <== mgr.34104 v2:172.21.15.90:6800/1611686821 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+3840 (secure 0 0 0) 0x7efe44061190 con 0x7efe3006e200 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:alertmanager.smithi090 smithi090 *:9093,9094 running (9m) 99s ago 20m 24.3M - 0.25.0 c8568f914cd2 dfa457364e6e 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi090 smithi090 running (112s) 99s ago 20m 7792k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 6c9e14bbff14 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-exporter.smithi104 smithi104 running (107s) 104s ago 19m 6916k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9cef35eaf52 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi090 smithi090 running (6m) 99s ago 20m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 079a04d1f48d 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:crash.smithi104 smithi104 running (6m) 104s ago 19m 6665k - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 90e760816ca6 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:grafana.smithi090 smithi090 *:3000 running (8m) 99s ago 19m 79.9M - 9.4.12 2bacad6d85d8 f5f47f84cf4e 2024-04-26T04:54:31.643 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.dbkjuk smithi090 running (2m) 99s ago 14m 69.1M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a01df3d7e4d4 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi090.ehputv smithi090 running (3m) 99s ago 14m 72.9M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 4bcc1e9fbb69 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.ndiusl smithi104 running (2m) 104s ago 14m 16.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 0dbd9167e2e1 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:mds.cephfs.smithi104.wpntdi smithi104 running (2m) 104s ago 14m 15.0M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 7d741ad3ecf3 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi090.iwrvwa smithi090 *:8443,9283,8765 running (10m) 99s ago 21m 510M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a9a4f36a1c0c 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:mgr.smithi104.wwoewi smithi104 *:8443,9283,8765 running (10m) 104s ago 19m 451M - 19.0.0-2447-gb22e2ebd 87e5280d5ae8 672bebec7364 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi090 smithi090 running (7m) 99s ago 21m 58.3M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 e31d5b589b5c 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:mon.smithi104 smithi104 running (7m) 104s ago 19m 47.8M 2048M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 11c4dd69e454 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi090 smithi090 *:9100 running (9m) 99s ago 20m 16.5M - 1.5.0 0da6a335fe13 79e69e4ee509 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:node-exporter.smithi104 smithi104 *:9100 running (9m) 104s ago 19m 14.6M - 1.5.0 0da6a335fe13 9a2fac8ae98e 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:osd.0 smithi090 running (6m) 99s ago 18m 98.6M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 514106f42dee 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:osd.1 smithi090 running (5m) 99s ago 17m 114M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 264b38fc9f31 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:osd.2 smithi090 running (5m) 99s ago 17m 91.2M 1216M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 a6f8b02d7e7d 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:osd.3 smithi104 running (4m) 104s ago 16m 120M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 d860fd1e9069 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:osd.4 smithi104 running (3m) 104s ago 16m 97.5M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 fbfd5c1c44eb 2024-04-26T04:54:31.644 INFO:teuthology.orchestra.run.smithi090.stdout:osd.5 smithi104 running (3m) 104s ago 16m 95.6M 2240M 19.0.0-2447-gb22e2ebd 87e5280d5ae8 64a9be8458bb 2024-04-26T04:54:31.645 INFO:teuthology.orchestra.run.smithi090.stdout:prometheus.smithi090 smithi090 *:9095 running (9m) 99s ago 19m 72.0M - 2.43.0 a07b618ecd1d 3d80778fcca0 2024-04-26T04:54:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efe3006e200 msgr2=0x7efe300706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efe3006e200 0x7efe300706b0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7efe44101bf0 tx=0x7efe2c01a040 comp rx=0 tx=0).stop 2024-04-26T04:54:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ff220 msgr2=0x7efe44100400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:31.645 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ff220 0x7efe44100400 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7efe3400d8d0 tx=0x7efe3400dbe0 comp rx=0 tx=0).stop 2024-04-26T04:54:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 shutdown_connections 2024-04-26T04:54:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7efe3006e200 0x7efe300706b0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7efe440ff220 0x7efe44100400 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 --2- 172.21.15.90:0/4079076629 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7efe440ffb30 0x7efe44100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 >> 172.21.15.90:0/4079076629 conn(0x7efe440747e0 msgr2=0x7efe44104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 shutdown_connections 2024-04-26T04:54:31.646 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:31.644+0000 7efe48cba700 1 -- 172.21.15.90:0/4079076629 wait complete. 2024-04-26T04:54:32.271 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions' 2024-04-26T04:54:32.483 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:54:33.158 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:32 smithi090 ceph-mon[86550]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:33.158 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:32 smithi090 ceph-mon[86550]: pgmap v266: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:33.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:32 smithi104 ceph-mon[64778]: from='client.34496 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-26T04:54:33.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:32 smithi104 ceph-mon[64778]: pgmap v266: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:34.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 -- 172.21.15.90:0/1119865620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c1004e0 msgr2=0x7f105c1008b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:34.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 --2- 172.21.15.90:0/1119865620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c1004e0 0x7f105c1008b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f104c009910 tx=0x7f104c009c20 comp rx=0 tx=0).stop 2024-04-26T04:54:34.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 -- 172.21.15.90:0/1119865620 shutdown_connections 2024-04-26T04:54:34.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 --2- 172.21.15.90:0/1119865620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f105c100e80 0x7f105c104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:34.001 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 --2- 172.21.15.90:0/1119865620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c1004e0 0x7f105c1008b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 -- 172.21.15.90:0/1119865620 >> 172.21.15.90:0/1119865620 conn(0x7f105c0fbda0 msgr2=0x7f105c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 -- 172.21.15.90:0/1119865620 shutdown_connections 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.998+0000 7f1064546700 1 -- 172.21.15.90:0/1119865620 wait complete. 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1064546700 1 Processor -- start 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1064546700 1 -- start start 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1064546700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f105c1004e0 0x7f105c195ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1064546700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c100e80 0x7f105c196430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1064546700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c196b10 con 0x7f105c100e80 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1064546700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105c19a8a0 con 0x7f105c1004e0 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1061ae1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c100e80 0x7f105c196430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1061ae1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c100e80 0x7f105c196430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50100/0 (socket says 172.21.15.90:50100) 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1061ae1700 1 -- 172.21.15.90:0/3849088409 learned_addr learned my addr 172.21.15.90:0/3849088409 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:34.002 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f10622e2700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f105c1004e0 0x7f105c195ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1061ae1700 1 -- 172.21.15.90:0/3849088409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f105c1004e0 msgr2=0x7f105c195ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1061ae1700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f105c1004e0 0x7f105c195ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:33.999+0000 7f1061ae1700 1 -- 172.21.15.90:0/3849088409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f104c0095f0 con 0x7f105c100e80 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.000+0000 7f1061ae1700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c100e80 0x7f105c196430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f105800d900 tx=0x7f105800dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.000+0000 7f10537fe700 1 -- 172.21.15.90:0/3849088409 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1058009900 con 0x7f105c100e80 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.000+0000 7f10537fe700 1 -- 172.21.15.90:0/3849088409 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1058010460 con 0x7f105c100e80 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.000+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105c19ab80 con 0x7f105c100e80 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.000+0000 7f10537fe700 1 -- 172.21.15.90:0/3849088409 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f105800f5f0 con 0x7f105c100e80 2024-04-26T04:54:34.003 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.000+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105c19b0d0 con 0x7f105c100e80 2024-04-26T04:54:34.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.001+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105c1087f0 con 0x7f105c100e80 2024-04-26T04:54:34.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.001+0000 7f10537fe700 1 -- 172.21.15.90:0/3849088409 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f10580105d0 con 0x7f105c100e80 2024-04-26T04:54:34.004 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.002+0000 7f10537fe700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f104806e200 0x7f10480706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:34.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.002+0000 7f10537fe700 1 -- 172.21.15.90:0/3849088409 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f105808dd60 con 0x7f105c100e80 2024-04-26T04:54:34.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.002+0000 7f10622e2700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f104806e200 0x7f10480706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:34.005 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.002+0000 7f10622e2700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f104806e200 0x7f10480706b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f104c006010 tx=0x7f104c01a040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:34.007 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.006+0000 7f10537fe700 1 -- 172.21.15.90:0/3849088409 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f1058057040 con 0x7f105c100e80 2024-04-26T04:54:34.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.288+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f105c19b2f0 con 0x7f105c100e80 2024-04-26T04:54:34.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.289+0000 7f10537fe700 1 -- 172.21.15.90:0/3849088409 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f105805a700 con 0x7f105c100e80 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout:{ 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "mon": { 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "mgr": { 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 2 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "osd": { 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 6 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "mds": { 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 4 2024-04-26T04:54:34.291 INFO:teuthology.orchestra.run.smithi090.stdout: }, 2024-04-26T04:54:34.292 INFO:teuthology.orchestra.run.smithi090.stdout: "overall": { 2024-04-26T04:54:34.292 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)": 14 2024-04-26T04:54:34.292 INFO:teuthology.orchestra.run.smithi090.stdout: } 2024-04-26T04:54:34.292 INFO:teuthology.orchestra.run.smithi090.stdout:} 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f104806e200 msgr2=0x7f10480706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f104806e200 0x7f10480706b0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f104c006010 tx=0x7f104c01a040 comp rx=0 tx=0).stop 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c100e80 msgr2=0x7f105c196430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c100e80 0x7f105c196430 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f105800d900 tx=0x7f105800dc10 comp rx=0 tx=0).stop 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 shutdown_connections 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f104806e200 0x7f10480706b0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f105c1004e0 0x7f105c195ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 --2- 172.21.15.90:0/3849088409 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f105c100e80 0x7f105c196430 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 >> 172.21.15.90:0/3849088409 conn(0x7f105c0fbda0 msgr2=0x7f105c0fe1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 shutdown_connections 2024-04-26T04:54:34.293 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:34.291+0000 7f1064546700 1 -- 172.21.15.90:0/3849088409 wait complete. 2024-04-26T04:54:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:34 smithi090 ceph-mon[86550]: pgmap v267: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:34.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:34 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3849088409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:35.177 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions | jq -e '"'"'.overall | length == 1'"'"'' 2024-04-26T04:54:35.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:34 smithi104 ceph-mon[64778]: pgmap v267: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:35.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:34 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3849088409' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:35.387 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:54:36.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 -- 172.21.15.90:0/177979879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ff350 msgr2=0x7fbfd80ff720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:36.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/177979879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ff350 0x7fbfd80ff720 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4009910 tx=0x7fbfd4009c20 comp rx=0 tx=0).stop 2024-04-26T04:54:36.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 -- 172.21.15.90:0/177979879 shutdown_connections 2024-04-26T04:54:36.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/177979879 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd80ffc60 0x7fbfd810a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:36.987 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/177979879 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ff350 0x7fbfd80ff720 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 -- 172.21.15.90:0/177979879 >> 172.21.15.90:0/177979879 conn(0x7fbfd80748b0 msgr2=0x7fbfd8074cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 -- 172.21.15.90:0/177979879 shutdown_connections 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 -- 172.21.15.90:0/177979879 wait complete. 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 Processor -- start 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.984+0000 7fbfe00d6700 1 -- start start 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfe00d6700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd80ff350 0x7fbfd8195f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfe00d6700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ffc60 0x7fbfd8196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfe00d6700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd8196b50 con 0x7fbfd80ffc60 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfe00d6700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfd819a8e0 con 0x7fbfd80ff350 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdd671700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ffc60 0x7fbfd8196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:36.988 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdd671700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ffc60 0x7fbfd8196470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:50118/0 (socket says 172.21.15.90:50118) 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdd671700 1 -- 172.21.15.90:0/510142299 learned_addr learned my addr 172.21.15.90:0/510142299 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdde72700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd80ff350 0x7fbfd8195f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdd671700 1 -- 172.21.15.90:0/510142299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd80ff350 msgr2=0x7fbfd8195f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdd671700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd80ff350 0x7fbfd8195f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdd671700 1 -- 172.21.15.90:0/510142299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfd40095f0 con 0x7fbfd80ffc60 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfdd671700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ffc60 0x7fbfd8196470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800b4b0 tx=0x7fbfc800b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfceffd700 1 -- 172.21.15.90:0/510142299 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc8011800 con 0x7fbfd80ffc60 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfceffd700 1 -- 172.21.15.90:0/510142299 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbfc8011e20 con 0x7fbfd80ffc60 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfd819abc0 con 0x7fbfd80ffc60 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.985+0000 7fbfceffd700 1 -- 172.21.15.90:0/510142299 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbfc800f2e0 con 0x7fbfd80ffc60 2024-04-26T04:54:36.989 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.986+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfd819b110 con 0x7fbfd80ffc60 2024-04-26T04:54:36.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.986+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfd8107dd0 con 0x7fbfd80ffc60 2024-04-26T04:54:36.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.987+0000 7fbfceffd700 1 -- 172.21.15.90:0/510142299 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbfc8010380 con 0x7fbfd80ffc60 2024-04-26T04:54:36.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.987+0000 7fbfceffd700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbfc406e200 0x7fbfc40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:36.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.987+0000 7fbfdde72700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbfc406e200 0x7fbfc40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:36.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.987+0000 7fbfceffd700 1 -- 172.21.15.90:0/510142299 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbfc808dca0 con 0x7fbfd80ffc60 2024-04-26T04:54:36.990 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.988+0000 7fbfdde72700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbfc406e200 0x7fbfc40706b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4006010 tx=0x7fbfd400b540 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:36.993 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:36.992+0000 7fbfceffd700 1 -- 172.21.15.90:0/510142299 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbfc8092070 con 0x7fbfd80ffc60 2024-04-26T04:54:37.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:36 smithi090 ceph-mon[86550]: pgmap v268: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:37.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:36 smithi104 ceph-mon[64778]: pgmap v268: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:37.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.279+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7fbfd804f2a0 con 0x7fbfd80ffc60 2024-04-26T04:54:37.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.279+0000 7fbfceffd700 1 -- 172.21.15.90:0/510142299 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7fbfc8057000 con 0x7fbfd80ffc60 2024-04-26T04:54:37.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbfc406e200 msgr2=0x7fbfc40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:37.283 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbfc406e200 0x7fbfc40706b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fbfd4006010 tx=0x7fbfd400b540 comp rx=0 tx=0).stop 2024-04-26T04:54:37.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ffc60 msgr2=0x7fbfd8196470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:37.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ffc60 0x7fbfd8196470 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fbfc800b4b0 tx=0x7fbfc800b870 comp rx=0 tx=0).stop 2024-04-26T04:54:37.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 shutdown_connections 2024-04-26T04:54:37.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbfc406e200 0x7fbfc40706b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:37.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbfd80ff350 0x7fbfd8195f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:37.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 --2- 172.21.15.90:0/510142299 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbfd80ffc60 0x7fbfd8196470 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:37.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 >> 172.21.15.90:0/510142299 conn(0x7fbfd80748b0 msgr2=0x7fbfd8104f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:37.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 shutdown_connections 2024-04-26T04:54:37.285 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:37.281+0000 7fbfe00d6700 1 -- 172.21.15.90:0/510142299 wait complete. 2024-04-26T04:54:37.293 INFO:teuthology.orchestra.run.smithi090.stdout:true 2024-04-26T04:54:37.956 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -e sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a -- bash -c 'ceph versions | jq -e '"'"'.overall | keys'"'"' | grep $sha1' 2024-04-26T04:54:38.162 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:54:38.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:37 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/510142299' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:38.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:37 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/510142299' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:38.942 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:38 smithi090 ceph-mon[86550]: pgmap v269: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:39.162 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:38 smithi104 ceph-mon[64778]: pgmap v269: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:39.800 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3226738477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104130 msgr2=0x7f25cc104500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3226738477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104130 0x7f25cc104500 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f25bc009910 tx=0x7f25bc009c20 comp rx=0 tx=0).stop 2024-04-26T04:54:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3226738477 shutdown_connections 2024-04-26T04:54:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3226738477 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25cc0ffbd0 0x7f25cc100040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3226738477 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc104130 0x7f25cc104500 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3226738477 >> 172.21.15.90:0/3226738477 conn(0x7f25cc075130 msgr2=0x7f25cc075530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3226738477 shutdown_connections 2024-04-26T04:54:39.801 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.799+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3226738477 wait complete. 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25d0fd1700 1 Processor -- start 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25d0fd1700 1 -- start start 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25d0fd1700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc0ffbd0 0x7f25cc195e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25d0fd1700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25cc104130 0x7f25cc196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25d0fd1700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc196a50 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25d0fd1700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25cc19a7e0 con 0x7f25cc104130 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25ca59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc0ffbd0 0x7f25cc195e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25ca59c700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc0ffbd0 0x7f25cc195e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55378/0 (socket says 172.21.15.90:55378) 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25ca59c700 1 -- 172.21.15.90:0/3286239522 learned_addr learned my addr 172.21.15.90:0/3286239522 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.800+0000 7f25c9d9b700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25cc104130 0x7f25cc196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:39.803 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25ca59c700 1 -- 172.21.15.90:0/3286239522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25cc104130 msgr2=0x7f25cc196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25ca59c700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25cc104130 0x7f25cc196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25ca59c700 1 -- 172.21.15.90:0/3286239522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc0095f0 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25ca59c700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc0ffbd0 0x7f25cc195e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f25bc006010 tx=0x7f25bc00be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25bb7fe700 1 -- 172.21.15.90:0/3286239522 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25bc01d070 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25bb7fe700 1 -- 172.21.15.90:0/3286239522 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f25bc00f460 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25bb7fe700 1 -- 172.21.15.90:0/3286239522 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f25bc017630 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25cc19aa60 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.804 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.801+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25cc19af50 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.802+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25cc04f2a0 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.803+0000 7f25bb7fe700 1 -- 172.21.15.90:0/3286239522 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f25bc00f5d0 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.803+0000 7f25bb7fe700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f25b406e200 0x7f25b40706b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:39.805 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.803+0000 7f25bb7fe700 1 -- 172.21.15.90:0/3286239522 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f25bc08f640 con 0x7f25cc0ffbd0 2024-04-26T04:54:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.803+0000 7f25c9d9b700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f25b406e200 0x7f25b40706b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:39.806 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.804+0000 7f25c9d9b700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f25b406e200 0x7f25b40706b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005950 tx=0x7f25c00058e0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:39.809 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:39.808+0000 7f25bb7fe700 1 -- 172.21.15.90:0/3286239522 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f25bc0589e0 con 0x7f25cc0ffbd0 2024-04-26T04:54:40.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.095+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "versions"} v 0) v1 -- 0x7f25cc19b230 con 0x7f25cc0ffbd0 2024-04-26T04:54:40.096 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.095+0000 7f25bb7fe700 1 -- 172.21.15.90:0/3286239522 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "versions"}]=0 v0) v1 ==== 56+0+623 (secure 0 0 0) 0x7f25bc05c0a0 con 0x7f25cc0ffbd0 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f25b406e200 msgr2=0x7f25b40706b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f25b406e200 0x7f25b40706b0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f25c0005950 tx=0x7f25c00058e0 comp rx=0 tx=0).stop 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc0ffbd0 msgr2=0x7f25cc195e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc0ffbd0 0x7f25cc195e30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f25bc006010 tx=0x7f25bc00be20 comp rx=0 tx=0).stop 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 shutdown_connections 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f25b406e200 0x7f25b40706b0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f25cc0ffbd0 0x7f25cc195e30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 --2- 172.21.15.90:0/3286239522 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f25cc104130 0x7f25cc196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 >> 172.21.15.90:0/3286239522 conn(0x7f25cc075130 msgr2=0x7f25cc0685e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 shutdown_connections 2024-04-26T04:54:40.099 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:40.097+0000 7f25d0fd1700 1 -- 172.21.15.90:0/3286239522 wait complete. 2024-04-26T04:54:40.109 INFO:teuthology.orchestra.run.smithi090.stdout: "ceph version 19.0.0-2447-gb22e2ebd (b22e2ebdeb24376882b7bda2a7329c8cccc2276a) squid (dev)" 2024-04-26T04:54:40.734 DEBUG:teuthology.parallel:result is None 2024-04-26T04:54:40.734 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-26T04:54:40.745 INFO:tasks.cephadm:Running commands on role host.a host ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:54:40.745 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm --image quay.io/ceph/ceph:v18.2.0 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f018e902-0385-11ef-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-26T04:54:40.952 INFO:teuthology.orchestra.run.smithi090.stderr:Inferring config /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/mon.smithi090/config 2024-04-26T04:54:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:40 smithi090 ceph-mon[86550]: pgmap v270: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:40 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3286239522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:40.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:40 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:40 smithi104 ceph-mon[64778]: pgmap v270: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:40 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3286239522' entity='client.admin' cmd=[{"prefix": "versions"}]: dispatch 2024-04-26T04:54:41.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:40 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:42.516 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.512+0000 7fe5869ae700 1 -- 172.21.15.90:0/2185220046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 msgr2=0x7fe580104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.512+0000 7fe5869ae700 1 --2- 172.21.15.90:0/2185220046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe580104e30 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fe570009960 tx=0x7fe570009c70 comp rx=0 tx=0).stop 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.513+0000 7fe5869ae700 1 -- 172.21.15.90:0/2185220046 shutdown_connections 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.513+0000 7fe5869ae700 1 --2- 172.21.15.90:0/2185220046 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe580104e30 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.513+0000 7fe5869ae700 1 --2- 172.21.15.90:0/2185220046 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5801004c0 0x7fe580100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.513+0000 7fe5869ae700 1 -- 172.21.15.90:0/2185220046 >> 172.21.15.90:0/2185220046 conn(0x7fe5800fbd80 msgr2=0x7fe5800fe190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.514+0000 7fe5869ae700 1 -- 172.21.15.90:0/2185220046 shutdown_connections 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.514+0000 7fe5869ae700 1 -- 172.21.15.90:0/2185220046 wait complete. 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe5869ae700 1 Processor -- start 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe5869ae700 1 -- start start 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe5869ae700 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5801004c0 0x7fe580074820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe5869ae700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe58006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe5869ae700 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe580074f00 con 0x7fe580100e60 2024-04-26T04:54:42.517 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe5869ae700 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe58006ff70 con 0x7fe5801004c0 2024-04-26T04:54:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe57f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe58006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe57f7fe700 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe58006fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55396/0 (socket says 172.21.15.90:55396) 2024-04-26T04:54:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe57f7fe700 1 -- 172.21.15.90:0/3560655383 learned_addr learned my addr 172.21.15.90:0/3560655383 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.515+0000 7fe57ffff700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5801004c0 0x7fe580074820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe57f7fe700 1 -- 172.21.15.90:0/3560655383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5801004c0 msgr2=0x7fe580074820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe57f7fe700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5801004c0 0x7fe580074820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:42.518 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe57f7fe700 1 -- 172.21.15.90:0/3560655383 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5700095f0 con 0x7fe580100e60 2024-04-26T04:54:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe57f7fe700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe58006fa30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe570005250 tx=0x7fe5700048e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe57d7fa700 1 -- 172.21.15.90:0/3560655383 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe57001d070 con 0x7fe580100e60 2024-04-26T04:54:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe57d7fa700 1 -- 172.21.15.90:0/3560655383 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe57000be00 con 0x7fe580100e60 2024-04-26T04:54:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5800701f0 con 0x7fe580100e60 2024-04-26T04:54:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe57d7fa700 1 -- 172.21.15.90:0/3560655383 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe5700217c0 con 0x7fe580100e60 2024-04-26T04:54:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.516+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5800706e0 con 0x7fe580100e60 2024-04-26T04:54:42.519 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.517+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe580108830 con 0x7fe580100e60 2024-04-26T04:54:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.518+0000 7fe57d7fa700 1 -- 172.21.15.90:0/3560655383 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe57002b440 con 0x7fe580100e60 2024-04-26T04:54:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.519+0000 7fe57d7fa700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe56806e250 0x7fe568070700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.519+0000 7fe57d7fa700 1 -- 172.21.15.90:0/3560655383 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe57008f7c0 con 0x7fe580100e60 2024-04-26T04:54:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.519+0000 7fe57ffff700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe56806e250 0x7fe568070700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:42.520 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.519+0000 7fe57ffff700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe56806e250 0x7fe568070700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe574006fd0 tx=0x7fe574009190 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:42.525 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.524+0000 7fe57d7fa700 1 -- 172.21.15.90:0/3560655383 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe570058aa0 con 0x7fe580100e60 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.771+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fe58004f2a0 con 0x7fe580100e60 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.772+0000 7fe57d7fa700 1 -- 172.21.15.90:0/3560655383 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 39 v39) v1 ==== 76+0+1951 (secure 0 0 0) 0x7fe57005c160 con 0x7fe580100e60 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stdout:e39 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stdout:legacy client fscid: 1 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:42.773 INFO:teuthology.orchestra.run.smithi090.stdout:Filesystem 'cephfs' (1) 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:fs_name cephfs 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:epoch 39 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:flags 12 joinable allow_snaps allow_multimds_snaps 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:created 2024-04-26T04:39:43.362117+0000 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:modified 2024-04-26T04:52:56.986658+0000 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:tableserver 0 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:root 0 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:session_timeout 60 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:session_autoclose 300 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:max_file_size 1099511627776 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:max_xattr_size 65536 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:required_client_features {} 2024-04-26T04:54:42.774 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure 0 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:last_failure_osd_epoch 81 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:max_mds 2 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:in 0,1 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:up {0=34380,1=34412} 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:failed 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:damaged 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:stopped 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:data_pools [3] 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:metadata_pool 2 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:inline_data disabled 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:balancer 2024-04-26T04:54:42.775 INFO:teuthology.orchestra.run.smithi090.stdout:bal_rank_mask -1 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout:standby_count_wanted 1 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout:qdb_cluster leader: 34380 members: 34412,34380 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.ehputv{0:34380} state up:active seq 22 join_fscid=1 addr [v2:172.21.15.90:6826/340161219,v1:172.21.15.90:6827/340161219] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi090.dbkjuk{1:34412} state up:active seq 17 join_fscid=1 addr [v2:172.21.15.90:6828/291277420,v1:172.21.15.90:6829/291277420] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout:Standby daemons: 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.wpntdi{-1:44291} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.104:6824/2998394587,v1:172.21.15.104:6825/2998394587] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:54:42.776 INFO:teuthology.orchestra.run.smithi090.stdout:[mds.cephfs.smithi104.ndiusl{-1:44319} state up:standby seq 1 join_fscid=1 addr [v2:172.21.15.104:6826/1280197644,v1:172.21.15.104:6827/1280197644] compat {c=[1],r=[1],i=[1fff]}] 2024-04-26T04:54:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe56806e250 msgr2=0x7fe568070700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe56806e250 0x7fe568070700 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe574006fd0 tx=0x7fe574009190 comp rx=0 tx=0).stop 2024-04-26T04:54:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 msgr2=0x7fe58006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe58006fa30 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe570005250 tx=0x7fe5700048e0 comp rx=0 tx=0).stop 2024-04-26T04:54:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 shutdown_connections 2024-04-26T04:54:42.777 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe56806e250 0x7fe568070700 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe5801004c0 0x7fe580074820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 --2- 172.21.15.90:0/3560655383 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe580100e60 0x7fe58006fa30 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 >> 172.21.15.90:0/3560655383 conn(0x7fe5800fbd80 msgr2=0x7fe5800fd420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 shutdown_connections 2024-04-26T04:54:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:42.775+0000 7fe5869ae700 1 -- 172.21.15.90:0/3560655383 wait complete. 2024-04-26T04:54:42.778 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 39 2024-04-26T04:54:43.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:42 smithi090 ceph-mon[86550]: pgmap v271: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:43.068 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:42 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3560655383' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:54:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:42 smithi104 ceph-mon[64778]: pgmap v271: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:43.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:42 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3560655383' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-26T04:54:43.418 INFO:teuthology.run_tasks:Running task fs.post_upgrade_checks... 2024-04-26T04:54:43.429 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:54:43.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.534+0000 7f08ca636640 1 -- 172.21.15.90:0/678966620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 msgr2=0x7f08c41096b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:43.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.534+0000 7f08ca636640 1 --2- 172.21.15.90:0/678966620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c41096b0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009880 tx=0x7f08b802bd70 comp rx=0 tx=0).stop 2024-04-26T04:54:43.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 -- 172.21.15.90:0/678966620 shutdown_connections 2024-04-26T04:54:43.539 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 --2- 172.21.15.90:0/678966620 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c41096b0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 --2- 172.21.15.90:0/678966620 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08c41008b0 0x7f08c4100c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 -- 172.21.15.90:0/678966620 >> 172.21.15.90:0/678966620 conn(0x7f08c4075270 msgr2=0x7f08c4075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 -- 172.21.15.90:0/678966620 shutdown_connections 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 -- 172.21.15.90:0/678966620 wait complete. 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 Processor -- start 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 -- start start 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08c41008b0 0x7f08c4195350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c4195890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c4195f20 con 0x7f08c41011c0 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.535+0000 7f08ca636640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c4199cf0 con 0x7f08c41008b0 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c4195890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c37fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c4195890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55412/0 (socket says 172.21.15.90:55412) 2024-04-26T04:54:43.540 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c37fe640 1 -- 172.21.15.90:0/4003607005 learned_addr learned my addr 172.21.15.90:0/4003607005 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c3fff640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08c41008b0 0x7f08c4195350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c37fe640 1 -- 172.21.15.90:0/4003607005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08c41008b0 msgr2=0x7f08c4195350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c37fe640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08c41008b0 0x7f08c4195350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c37fe640 1 -- 172.21.15.90:0/4003607005 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b8009560 con 0x7f08c41011c0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c3fff640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08c41008b0 0x7f08c4195350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c37fe640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c4195890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009230 tx=0x7f08b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c17fa640 1 -- 172.21.15.90:0/4003607005 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08b80037c0 con 0x7f08c41011c0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c17fa640 1 -- 172.21.15.90:0/4003607005 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f08b8004aa0 con 0x7f08c41011c0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c4199f70 con 0x7f08c41011c0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08c17fa640 1 -- 172.21.15.90:0/4003607005 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f08b803e450 con 0x7f08c41011c0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.536+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c419a4d0 con 0x7f08c41011c0 2024-04-26T04:54:43.541 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.537+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c404e000 con 0x7f08c41011c0 2024-04-26T04:54:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.538+0000 7f08c17fa640 1 -- 172.21.15.90:0/4003607005 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f08b8005060 con 0x7f08c41011c0 2024-04-26T04:54:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.538+0000 7f08c17fa640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f08ac06e230 0x7f08ac0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.538+0000 7f08c17fa640 1 -- 172.21.15.90:0/4003607005 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f08b80b10a0 con 0x7f08c41011c0 2024-04-26T04:54:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.538+0000 7f08c3fff640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f08ac06e230 0x7f08ac0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:43.542 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.538+0000 7f08c3fff640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f08ac06e230 0x7f08ac0706e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009930 tx=0x7f08b4008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:43.544 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.544+0000 7f08c17fa640 1 -- 172.21.15.90:0/4003607005 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f08b807a380 con 0x7f08c41011c0 2024-04-26T04:54:43.720 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.718+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08c410c8e0 con 0x7f08c41011c0 2024-04-26T04:54:43.721 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.719+0000 7f08c17fa640 1 -- 172.21.15.90:0/4003607005 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16775 (secure 0 0 0) 0x7f08b807da40 con 0x7f08c41011c0 2024-04-26T04:54:43.721 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:43.722 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":85,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","created":"2024-04-26T04:32:57.928472+0000","modified":"2024-04-26T04:52:17.939202+0000","last_up_change":"2024-04-26T04:51:19.550115+0000","last_in_change":"2024-04-26T04:38:13.654124+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:37:19.554153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:39:42.011282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:39:42.594560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57491e88-0d1c-41ac-a1f5-3662e5c0a467","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6803","nonce":2303726208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6805","nonce":2303726208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6809","nonce":2303726208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6807","nonce":2303726208}]},"public_addr":"172.21.15.90:6803/2303726208","cluster_addr":"172.21.15.90:6805/2303726208","heartbeat_back_addr":"172.21.15.90:6809/2303726208","heartbeat_front_addr":"172.21.15.90:6807/2303726208","state":["exists","up"]},{"osd":1,"uuid":"fe237f1b-a589-452c-8e5e-98a9a31593eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6811","nonce":2770756421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6813","nonce":2770756421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6817","nonce":2770756421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6815","nonce":2770756421}]},"public_addr":"172.21.15.90:6811/2770756421","cluster_addr":"172.21.15.90:6813/2770756421","heartbeat_back_addr":"172.21.15.90:6817/2770756421","heartbeat_front_addr":"172.21.15.90:6815/2770756421","state":["exists","up"]},{"osd":2,"uuid":"7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6819","nonce":1256593709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6821","nonce":1256593709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6825","nonce":1256593709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6823","nonce":1256593709}]},"public_addr":"172.21.15.90:6819/1256593709","cluster_addr":"172.21.15.90:6821/1256593709","heartbeat_back_addr":"172.21.15.90:6825/1256593709","heartbeat_front_addr":"172.21.15.90:6823/1256593709","state":["exists","up"]},{"osd":3,"uuid":"62adc04c-9e7e-4ee5-bb5f-42004b2a615d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6801","nonce":2129064810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6803","nonce":2129064810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6807","nonce":2129064810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6805","nonce":2129064810}]},"public_addr":"172.21.15.104:6801/2129064810","cluster_addr":"172.21.15.104:6803/2129064810","heartbeat_back_addr":"172.21.15.104:6807/2129064810","heartbeat_front_addr":"172.21.15.104:6805/2129064810","state":["exists","up"]},{"osd":4,"uuid":"a4ef2729-84e1-4d2d-868b-3da5fefeb800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6809","nonce":319614461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6811","nonce":319614461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6815","nonce":319614461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6813","nonce":319614461}]},"public_addr":"172.21.15.104:6809/319614461","cluster_addr":"172.21.15.104:6811/319614461","heartbeat_back_addr":"172.21.15.104:6815/319614461","heartbeat_front_addr":"172.21.15.104:6813/319614461","state":["exists","up"]},{"osd":5,"uuid":"2fa80b6f-ab69-4e6e-9a5f-7e73a898c300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6817","nonce":1501294989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6819","nonce":1501294989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6823","nonce":1501294989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6821","nonce":1501294989}]},"public_addr":"172.21.15.104:6817/1501294989","cluster_addr":"172.21.15.104:6819/1501294989","heartbeat_back_addr":"172.21.15.104:6823/1501294989","heartbeat_front_addr":"172.21.15.104:6821/1501294989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:19.861796+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:48.482295+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:17.028265+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:40.489573+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:05.503518+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:30.925548+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6828/370183803":"2024-04-27T04:51:20.563404+0000","172.21.15.90:6800/1235211382":"2024-04-27T04:43:18.835636+0000","172.21.15.104:6824/1058172006":"2024-04-27T04:51:20.563430+0000","172.21.15.90:0/748209074":"2024-04-27T04:33:57.088900+0000","172.21.15.104:6829/680795988":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6801/1235211382":"2024-04-27T04:43:18.835636+0000","172.21.15.90:0/2705188709":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6801/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.104:0/2711199308":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/1814853898":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/4265628059":"2024-04-27T04:46:55.630300+0000","172.21.15.90:0/2763746313":"2024-04-27T04:43:18.835636+0000","172.21.15.90:0/2473529454":"2024-04-27T04:33:24.670558+0000","172.21.15.104:6828/680795988":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6800/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/2342166800":"2024-04-27T04:33:57.088900+0000","172.21.15.104:0/1222914910":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6829/370183803":"2024-04-27T04:51:20.563404+0000","172.21.15.104:0/2266011983":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/3418315403":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2310050885":"2024-04-27T04:33:57.088900+0000","172.21.15.90:0/2803863568":"2024-04-27T04:46:55.630300+0000","172.21.15.90:0/424889384":"2024-04-27T04:33:24.670558+0000","172.21.15.90:6800/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.104:6825/1058172006":"2024-04-27T04:51:20.563430+0000","172.21.15.90:0/2846076831":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6801/3458990238":"2024-04-27T04:46:55.630300+0000","172.21.15.90:6801/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6801/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/3214112922":"2024-04-27T04:35:03.230209+0000","172.21.15.104:0/3572733912":"2024-04-27T04:44:00.133458+0000","172.21.15.104:0/3577745345":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/50635771":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/4285557765":"2024-04-27T04:46:55.630300+0000","172.21.15.104:0/204837581":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6826/3871906279":"2024-04-27T04:40:00.786348+0000","172.21.15.90:0/1198706928":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6827/3871906279":"2024-04-27T04:40:00.786348+0000","172.21.15.90:0/1376440705":"2024-04-27T04:46:55.630300+0000","172.21.15.90:6800/3458990238":"2024-04-27T04:46:55.630300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:54:43.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f08ac06e230 msgr2=0x7f08ac0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:43.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f08ac06e230 0x7f08ac0706e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f08b4009930 tx=0x7f08b4008040 comp rx=0 tx=0).stop 2024-04-26T04:54:43.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 msgr2=0x7f08c4195890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c4195890 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009230 tx=0x7f08b8004270 comp rx=0 tx=0).stop 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 shutdown_connections 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f08ac06e230 0x7f08ac0706e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f08c41011c0 0x7f08c4195890 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 --2- 172.21.15.90:0/4003607005 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f08c41008b0 0x7f08c4195350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 >> 172.21.15.90:0/4003607005 conn(0x7f08c4075270 msgr2=0x7f08c4103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 shutdown_connections 2024-04-26T04:54:43.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.722+0000 7f08ca636640 1 -- 172.21.15.90:0/4003607005 wait complete. 2024-04-26T04:54:43.734 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:37:19.554153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:39:42.011282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:39:42.594560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T04:54:43.735 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T04:54:43.837 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- 172.21.15.90:0/2641515564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580ff000 msgr2=0x7f1858109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 --2- 172.21.15.90:0/2641515564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580ff000 0x7f1858109a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f18480098b0 tx=0x7f184802bcd0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- 172.21.15.90:0/2641515564 shutdown_connections 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 --2- 172.21.15.90:0/2641515564 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580ff000 0x7f1858109a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 --2- 172.21.15.90:0/2641515564 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18580fe6f0 0x7f18580feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- 172.21.15.90:0/2641515564 >> 172.21.15.90:0/2641515564 conn(0x7f1858073d20 msgr2=0x7f1858074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- 172.21.15.90:0/2641515564 shutdown_connections 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- 172.21.15.90:0/2641515564 wait complete. 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 Processor -- start 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- start start 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580fe6f0 0x7f1858195310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:43.838 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18580ff000 0x7f1858195850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1858195ee0 con 0x7f18580fe6f0 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185f792640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1858199cb0 con 0x7f18580ff000 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185d52e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580fe6f0 0x7f1858195310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185d52e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580fe6f0 0x7f1858195310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55436/0 (socket says 172.21.15.90:55436) 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185d52e640 1 -- 172.21.15.90:0/565564932 learned_addr learned my addr 172.21.15.90:0/565564932 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185cd2d640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18580ff000 0x7f1858195850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185d52e640 1 -- 172.21.15.90:0/565564932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18580ff000 msgr2=0x7f1858195850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.834+0000 7f185d52e640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18580ff000 0x7f1858195850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.835+0000 7f185d52e640 1 -- 172.21.15.90:0/565564932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1848009560 con 0x7f18580fe6f0 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.835+0000 7f185d52e640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580fe6f0 0x7f1858195310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f185400d8d0 tx=0x7f185400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.835+0000 7f184e7fc640 1 -- 172.21.15.90:0/565564932 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f185400bea0 con 0x7f18580fe6f0 2024-04-26T04:54:43.839 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.835+0000 7f184e7fc640 1 -- 172.21.15.90:0/565564932 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f1854009c00 con 0x7f18580fe6f0 2024-04-26T04:54:43.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.835+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1858199f90 con 0x7f18580fe6f0 2024-04-26T04:54:43.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.835+0000 7f184e7fc640 1 -- 172.21.15.90:0/565564932 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f185400f710 con 0x7f18580fe6f0 2024-04-26T04:54:43.840 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.835+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185819a4c0 con 0x7f18580fe6f0 2024-04-26T04:54:43.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.837+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185804e000 con 0x7f18580fe6f0 2024-04-26T04:54:43.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.837+0000 7f184e7fc640 1 -- 172.21.15.90:0/565564932 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f185400f990 con 0x7f18580fe6f0 2024-04-26T04:54:43.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.838+0000 7f184e7fc640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f184406e170 0x7f1844070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:43.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.838+0000 7f185cd2d640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f184406e170 0x7f1844070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:43.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.838+0000 7f184e7fc640 1 -- 172.21.15.90:0/565564932 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f185408df20 con 0x7f18580fe6f0 2024-04-26T04:54:43.841 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.838+0000 7f185cd2d640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f184406e170 0x7f1844070620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1848006010 tx=0x7f184802cb30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:43.843 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:43.843+0000 7f184e7fc640 1 -- 172.21.15.90:0/565564932 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f18540c2740 con 0x7f18580fe6f0 2024-04-26T04:54:44.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.019+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f185810c810 con 0x7f18580fe6f0 2024-04-26T04:54:44.021 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.020+0000 7f184e7fc640 1 -- 172.21.15.90:0/565564932 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f185405ad30 con 0x7f18580fe6f0 2024-04-26T04:54:44.021 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-26T04:54:44.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f184406e170 msgr2=0x7f1844070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f184406e170 0x7f1844070620 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1848006010 tx=0x7f184802cb30 comp rx=0 tx=0).stop 2024-04-26T04:54:44.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580fe6f0 msgr2=0x7f1858195310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580fe6f0 0x7f1858195310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f185400d8d0 tx=0x7f185400dd90 comp rx=0 tx=0).stop 2024-04-26T04:54:44.023 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 shutdown_connections 2024-04-26T04:54:44.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f18580ff000 0x7f1858195850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f184406e170 0x7f1844070620 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 --2- 172.21.15.90:0/565564932 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f18580fe6f0 0x7f1858195310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 >> 172.21.15.90:0/565564932 conn(0x7f1858073d20 msgr2=0x7f18581042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:44.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 shutdown_connections 2024-04-26T04:54:44.024 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.022+0000 7f185f792640 1 -- 172.21.15.90:0/565564932 wait complete. 2024-04-26T04:54:44.032 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:43 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/4003607005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:54:44.034 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T04:54:44.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- 172.21.15.90:0/4081402414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 msgr2=0x7fe1f80fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/4081402414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f80fea20 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe1f4009880 tx=0x7fe1f402bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- 172.21.15.90:0/4081402414 shutdown_connections 2024-04-26T04:54:44.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/4081402414 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f80fea20 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.136 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/4081402414 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f8104590 0x7fe1f8104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- 172.21.15.90:0/4081402414 >> 172.21.15.90:0/4081402414 conn(0x7fe1f80743d0 msgr2=0x7fe1f80747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- 172.21.15.90:0/4081402414 shutdown_connections 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- 172.21.15.90:0/4081402414 wait complete. 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 Processor -- start 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- start start 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f8195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f8104590 0x7fe1f8195950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8195fe0 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.133+0000 7fe1ff78b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f8199db0 con 0x7fe1f8104590 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fd527640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f8195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fd527640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f8195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55462/0 (socket says 172.21.15.90:55462) 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fd527640 1 -- 172.21.15.90:0/2181059779 learned_addr learned my addr 172.21.15.90:0/2181059779 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:44.137 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fcd26640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f8104590 0x7fe1f8195950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fd527640 1 -- 172.21.15.90:0/2181059779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f8104590 msgr2=0x7fe1f8195950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fd527640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f8104590 0x7fe1f8195950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fd527640 1 -- 172.21.15.90:0/2181059779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f4009560 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1fd527640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f8195410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800b420 tx=0x7fe1e800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1ee7fc640 1 -- 172.21.15.90:0/2181059779 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e8011800 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1ee7fc640 1 -- 172.21.15.90:0/2181059779 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fe1e8011e20 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1ee7fc640 1 -- 172.21.15.90:0/2181059779 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fe1e800f2e0 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f819a090 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.134+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f819a5f0 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.138 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.135+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f804e000 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.136+0000 7fe1ee7fc640 1 -- 172.21.15.90:0/2181059779 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fe1e8010380 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.136+0000 7fe1ee7fc640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe1e406e170 0x7fe1e4070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.136+0000 7fe1ee7fc640 1 -- 172.21.15.90:0/2181059779 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fe1e808daf0 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.136+0000 7fe1fcd26640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe1e406e170 0x7fe1e4070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:44.139 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.136+0000 7fe1fcd26640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe1e406e170 0x7fe1e4070620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8196a20 tx=0x7fe1f4000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:44.142 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.141+0000 7fe1ee7fc640 1 -- 172.21.15.90:0/2181059779 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fe1e8056dd0 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:43 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/4003607005' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:54:44.315 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.315+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7fe1f8196760 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.316 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.315+0000 7fe1ee7fc640 1 -- 172.21.15.90:0/2181059779 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fe1e805a490 con 0x7fe1f80fe5b0 2024-04-26T04:54:44.316 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:54:44.318 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe1e406e170 msgr2=0x7fe1e4070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe1e406e170 0x7fe1e4070620 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe1f8196a20 tx=0x7fe1f4000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 msgr2=0x7fe1f8195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f8195410 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fe1e800b420 tx=0x7fe1e800b8e0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 shutdown_connections 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fe1f8104590 0x7fe1f8195950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fe1e406e170 0x7fe1e4070620 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 --2- 172.21.15.90:0/2181059779 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fe1f80fe5b0 0x7fe1f8195410 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 >> 172.21.15.90:0/2181059779 conn(0x7fe1f80743d0 msgr2=0x7fe1f8067450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 shutdown_connections 2024-04-26T04:54:44.319 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.317+0000 7fe1ff78b640 1 -- 172.21.15.90:0/2181059779 wait complete. 2024-04-26T04:54:44.329 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T04:54:44.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 -- 172.21.15.90:0/1201506624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594101030 msgr2=0x7fb594109700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.430 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 --2- 172.21.15.90:0/1201506624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594101030 0x7fb594109700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0098b0 tx=0x7fb57c02bd60 comp rx=0 tx=0).stop 2024-04-26T04:54:44.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 -- 172.21.15.90:0/1201506624 shutdown_connections 2024-04-26T04:54:44.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 --2- 172.21.15.90:0/1201506624 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594101030 0x7fb594109700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.431 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 --2- 172.21.15.90:0/1201506624 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb594100720 0x7fb594100af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 -- 172.21.15.90:0/1201506624 >> 172.21.15.90:0/1201506624 conn(0x7fb5940fa070 msgr2=0x7fb5940fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 -- 172.21.15.90:0/1201506624 shutdown_connections 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.427+0000 7fb5993b2640 1 -- 172.21.15.90:0/1201506624 wait complete. 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb5993b2640 1 Processor -- start 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb5993b2640 1 -- start start 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb5993b2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594100720 0x7fb594195370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb5993b2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb594101030 0x7fb5941958b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb5993b2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594195f40 con 0x7fb594100720 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb5993b2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb594199d10 con 0x7fb594101030 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb592ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594100720 0x7fb594195370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb592ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594100720 0x7fb594195370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55472/0 (socket says 172.21.15.90:55472) 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb592ffd640 1 -- 172.21.15.90:0/2992868247 learned_addr learned my addr 172.21.15.90:0/2992868247 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:44.432 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb592ffd640 1 -- 172.21.15.90:0/2992868247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb594101030 msgr2=0x7fb5941958b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb592ffd640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb594101030 0x7fb5941958b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.428+0000 7fb592ffd640 1 -- 172.21.15.90:0/2992868247 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c009560 con 0x7fb594100720 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.429+0000 7fb592ffd640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594100720 0x7fb594195370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d8d0 tx=0x7fb58400dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.429+0000 7fb590ff9640 1 -- 172.21.15.90:0/2992868247 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58400bea0 con 0x7fb594100720 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.429+0000 7fb590ff9640 1 -- 172.21.15.90:0/2992868247 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb584009c00 con 0x7fb594100720 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.429+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb594199ff0 con 0x7fb594100720 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.429+0000 7fb590ff9640 1 -- 172.21.15.90:0/2992868247 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb58400f710 con 0x7fb594100720 2024-04-26T04:54:44.433 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.429+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59419a550 con 0x7fb594100720 2024-04-26T04:54:44.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.430+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb59404e000 con 0x7fb594100720 2024-04-26T04:54:44.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.430+0000 7fb590ff9640 1 -- 172.21.15.90:0/2992868247 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb58400f990 con 0x7fb594100720 2024-04-26T04:54:44.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.430+0000 7fb590ff9640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb58006e170 0x7fb580070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.431+0000 7fb590ff9640 1 -- 172.21.15.90:0/2992868247 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb58408df20 con 0x7fb594100720 2024-04-26T04:54:44.434 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.431+0000 7fb58bfff640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb58006e170 0x7fb580070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:44.435 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.431+0000 7fb58bfff640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb58006e170 0x7fb580070620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb57c036040 tx=0x7fb57c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:44.436 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.435+0000 7fb590ff9640 1 -- 172.21.15.90:0/2992868247 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb5840c2740 con 0x7fb594100720 2024-04-26T04:54:44.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.608+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7fb594106d20 con 0x7fb594100720 2024-04-26T04:54:44.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.609+0000 7fb590ff9640 1 -- 172.21.15.90:0/2992868247 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7fb58405ad30 con 0x7fb594100720 2024-04-26T04:54:44.610 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:54:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb58006e170 msgr2=0x7fb580070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb58006e170 0x7fb580070620 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fb57c036040 tx=0x7fb57c000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594100720 msgr2=0x7fb594195370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594100720 0x7fb594195370 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb58400d8d0 tx=0x7fb58400dd90 comp rx=0 tx=0).stop 2024-04-26T04:54:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 shutdown_connections 2024-04-26T04:54:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb594101030 0x7fb5941958b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb58006e170 0x7fb580070620 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 --2- 172.21.15.90:0/2992868247 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb594100720 0x7fb594195370 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 >> 172.21.15.90:0/2992868247 conn(0x7fb5940fa070 msgr2=0x7fb594103ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 shutdown_connections 2024-04-26T04:54:44.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.611+0000 7fb5993b2640 1 -- 172.21.15.90:0/2992868247 wait complete. 2024-04-26T04:54:44.623 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-26T04:54:44.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 -- 172.21.15.90:0/2242130440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 msgr2=0x7ff130105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 --2- 172.21.15.90:0/2242130440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130105a70 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009880 tx=0x7ff12c02bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 -- 172.21.15.90:0/2242130440 shutdown_connections 2024-04-26T04:54:44.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 --2- 172.21.15.90:0/2242130440 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1300ff690 0x7ff1300ffb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.722 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 --2- 172.21.15.90:0/2242130440 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130105a70 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 -- 172.21.15.90:0/2242130440 >> 172.21.15.90:0/2242130440 conn(0x7ff1300fb1d0 msgr2=0x7ff1300fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 -- 172.21.15.90:0/2242130440 shutdown_connections 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.719+0000 7ff137547640 1 -- 172.21.15.90:0/2242130440 wait complete. 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff137547640 1 Processor -- start 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff137547640 1 -- start start 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff137547640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1300ff690 0x7ff130195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff137547640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff137547640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130195d50 con 0x7ff1301056a0 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff137547640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff130195ec0 con 0x7ff1300ff690 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff134ae2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:44.723 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff134ae2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130195750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55494/0 (socket says 172.21.15.90:55494) 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff134ae2640 1 -- 172.21.15.90:0/2641951124 learned_addr learned my addr 172.21.15.90:0/2641951124 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff134ae2640 1 -- 172.21.15.90:0/2641951124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1300ff690 msgr2=0x7ff130195210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff134ae2640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1300ff690 0x7ff130195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff134ae2640 1 -- 172.21.15.90:0/2641951124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff12c009560 con 0x7ff1301056a0 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.720+0000 7ff134ae2640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130195750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c8a0 tx=0x7ff12000cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.721+0000 7ff1267fc640 1 -- 172.21.15.90:0/2641951124 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff120007b80 con 0x7ff1301056a0 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.721+0000 7ff1267fc640 1 -- 172.21.15.90:0/2641951124 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7ff120007ce0 con 0x7ff1301056a0 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.721+0000 7ff1267fc640 1 -- 172.21.15.90:0/2641951124 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7ff120018810 con 0x7ff1301056a0 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.721+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff130199d50 con 0x7ff1301056a0 2024-04-26T04:54:44.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.721+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13019a330 con 0x7ff1301056a0 2024-04-26T04:54:44.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.722+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13004e000 con 0x7ff1301056a0 2024-04-26T04:54:44.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.722+0000 7ff1267fc640 1 -- 172.21.15.90:0/2641951124 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7ff120018970 con 0x7ff1301056a0 2024-04-26T04:54:44.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.722+0000 7ff1267fc640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff11c06e230 0x7ff11c0706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:44.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.723+0000 7ff1267fc640 1 -- 172.21.15.90:0/2641951124 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7ff12008e2f0 con 0x7ff1301056a0 2024-04-26T04:54:44.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.723+0000 7ff1352e3640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff11c06e230 0x7ff11c0706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:44.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.723+0000 7ff1352e3640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff11c06e230 0x7ff11c0706e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff12c036040 tx=0x7ff12c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:44.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.727+0000 7ff1267fc640 1 -- 172.21.15.90:0/2641951124 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7ff120057650 con 0x7ff1301056a0 2024-04-26T04:54:44.943 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.942+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff130196560 con 0x7ff1301056a0 2024-04-26T04:54:44.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.943+0000 7ff1267fc640 1 -- 172.21.15.90:0/2641951124 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+5230 (secure 0 0 0) 0x7ff12005ad10 con 0x7ff1301056a0 2024-04-26T04:54:44.944 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:44.944 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:56.986658+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":0,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":1,"incarnation":33,"state":"up:active","state_seq":17,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34380,"qdb_cluster":[34412,34380]},"id":1}]} 2024-04-26T04:54:44.944 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 39 2024-04-26T04:54:44.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff11c06e230 msgr2=0x7ff11c0706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff11c06e230 0x7ff11c0706e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7ff12c036040 tx=0x7ff12c000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 msgr2=0x7ff130195750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:44.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130195750 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7ff12000c8a0 tx=0x7ff12000cd60 comp rx=0 tx=0).stop 2024-04-26T04:54:44.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 shutdown_connections 2024-04-26T04:54:44.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7ff11c06e230 0x7ff11c0706e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7ff1301056a0 0x7ff130195750 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 --2- 172.21.15.90:0/2641951124 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7ff1300ff690 0x7ff130195210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:44.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 >> 172.21.15.90:0/2641951124 conn(0x7ff1300fb1d0 msgr2=0x7ff1300fc940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:44.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 shutdown_connections 2024-04-26T04:54:44.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:44.945+0000 7ff137547640 1 -- 172.21.15.90:0/2641951124 wait complete. 2024-04-26T04:54:44.955 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:44 smithi090 ceph-mon[86550]: pgmap v272: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:44.956 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:44 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/565564932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:54:44.956 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:44 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2181059779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:54:44.956 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:44 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2992868247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:54:44.957 DEBUG:tasks.fs:checking fs fscid=1,name=cephfs state = {'epoch': 17, 'max_mds': 2, 'flags': 18} 2024-04-26T04:54:44.957 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 18 2024-04-26T04:54:45.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 -- 172.21.15.90:0/754682138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 msgr2=0x7f0f50104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/754682138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50104990 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009880 tx=0x7f0f4c02bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 -- 172.21.15.90:0/754682138 shutdown_connections 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/754682138 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f500fe590 0x7f0f500fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/754682138 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50104990 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 -- 172.21.15.90:0/754682138 >> 172.21.15.90:0/754682138 conn(0x7f0f500fa070 msgr2=0x7f0f500fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 -- 172.21.15.90:0/754682138 shutdown_connections 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.054+0000 7f0f57e8f640 1 -- 172.21.15.90:0/754682138 wait complete. 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f57e8f640 1 Processor -- start 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f57e8f640 1 -- start start 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f57e8f640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f500fe590 0x7f0f50194220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f57e8f640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50194760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f57e8f640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50194df0 con 0x7f0f501045c0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f57e8f640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f50198bc0 con 0x7f0f500fe590 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f5542a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50194760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f5542a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50194760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55502/0 (socket says 172.21.15.90:55502) 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f5542a640 1 -- 172.21.15.90:0/3231307049 learned_addr learned my addr 172.21.15.90:0/3231307049 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f5542a640 1 -- 172.21.15.90:0/3231307049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f500fe590 msgr2=0x7f0f50194220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f5542a640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f500fe590 0x7f0f50194220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f5542a640 1 -- 172.21.15.90:0/3231307049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c009560 con 0x7f0f501045c0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f5542a640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50194760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0f400078f0 tx=0x7f0f40007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f46ffd640 1 -- 172.21.15.90:0/3231307049 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f40011070 con 0x7f0f501045c0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.055+0000 7f0f46ffd640 1 -- 172.21.15.90:0/3231307049 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0f4000c2e0 con 0x7f0f501045c0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.056+0000 7f0f46ffd640 1 -- 172.21.15.90:0/3231307049 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0f4000e940 con 0x7f0f501045c0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.056+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f50198e40 con 0x7f0f501045c0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.056+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f501993a0 con 0x7f0f501045c0 2024-04-26T04:54:45.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.057+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5004e000 con 0x7f0f501045c0 2024-04-26T04:54:45.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.057+0000 7f0f46ffd640 1 -- 172.21.15.90:0/3231307049 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0f4000ec00 con 0x7f0f501045c0 2024-04-26T04:54:45.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.057+0000 7f0f46ffd640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f3c06e170 0x7f0f3c070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.058+0000 7f0f55c2b640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f3c06e170 0x7f0f3c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.058+0000 7f0f46ffd640 1 -- 172.21.15.90:0/3231307049 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0f4008f950 con 0x7f0f501045c0 2024-04-26T04:54:45.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.058+0000 7f0f55c2b640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f3c06e170 0x7f0f3c070620 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c036040 tx=0x7f0f4c000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:45.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.062+0000 7f0f46ffd640 1 -- 172.21.15.90:0/3231307049 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0f40058c30 con 0x7f0f501045c0 2024-04-26T04:54:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:44 smithi104 ceph-mon[64778]: pgmap v272: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:44 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/565564932' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:54:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:44 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2181059779' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:54:45.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:44 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2992868247' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:54:45.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.274+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 18, "format": "json"} v 0) v1 -- 0x7f0f5010b930 con 0x7f0f501045c0 2024-04-26T04:54:45.276 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.275+0000 7f0f46ffd640 1 -- 172.21.15.90:0/3231307049 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 18, "format": "json"}]=0 dumped fsmap epoch 18 v39) v1 ==== 107+0+4873 (secure 0 0 0) 0x7f0f4005c2f0 con 0x7f0f501045c0 2024-04-26T04:54:45.276 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:45.276 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":18,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/650878075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":650878075},{"type":"v1","addr":"172.21.15.90:6827","nonce":650878075}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":18,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:41:46.361891+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi090.dbkjuk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6829/370183803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":370183803},{"type":"v1","addr":"172.21.15.90:6829","nonce":370183803}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi104.wpntdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6825/1058172006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1058172006},{"type":"v1","addr":"172.21.15.104:6825","nonce":1058172006}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:45.276 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 18 2024-04-26T04:54:45.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f3c06e170 msgr2=0x7f0f3c070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f3c06e170 0x7f0f3c070620 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c036040 tx=0x7f0f4c000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 msgr2=0x7f0f50194760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50194760 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f0f400078f0 tx=0x7f0f40007db0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 shutdown_connections 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0f3c06e170 0x7f0f3c070620 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0f501045c0 0x7f0f50194760 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 --2- 172.21.15.90:0/3231307049 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0f500fe590 0x7f0f50194220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 >> 172.21.15.90:0/3231307049 conn(0x7f0f500fa070 msgr2=0x7f0f500fb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 shutdown_connections 2024-04-26T04:54:45.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.277+0000 7f0f57e8f640 1 -- 172.21.15.90:0/3231307049 wait complete. 2024-04-26T04:54:45.289 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 19 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 -- 172.21.15.90:0/1568731644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 msgr2=0x7fc7d8109640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 --2- 172.21.15.90:0/1568731644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d8109640 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009880 tx=0x7fc7c802bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 -- 172.21.15.90:0/1568731644 shutdown_connections 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 --2- 172.21.15.90:0/1568731644 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d8109640 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 --2- 172.21.15.90:0/1568731644 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8100840 0x7fc7d8100c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 -- 172.21.15.90:0/1568731644 >> 172.21.15.90:0/1568731644 conn(0x7fc7d80fa0b0 msgr2=0x7fc7d80fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 -- 172.21.15.90:0/1568731644 shutdown_connections 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 -- 172.21.15.90:0/1568731644 wait complete. 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 Processor -- start 2024-04-26T04:54:45.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.386+0000 7fc7de96e640 1 -- start start 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7de96e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8100840 0x7fc7d80fe490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7de96e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d80fe9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7de96e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80fef10 con 0x7fc7d8101150 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7de96e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d80ff080 con 0x7fc7d8100840 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d80fe9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d77fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d80fe9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55532/0 (socket says 172.21.15.90:55532) 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d77fe640 1 -- 172.21.15.90:0/4274578277 learned_addr learned my addr 172.21.15.90:0/4274578277 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d7fff640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8100840 0x7fc7d80fe490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d77fe640 1 -- 172.21.15.90:0/4274578277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8100840 msgr2=0x7fc7d80fe490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d77fe640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8100840 0x7fc7d80fe490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d77fe640 1 -- 172.21.15.90:0/4274578277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c8009560 con 0x7fc7d8101150 2024-04-26T04:54:45.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d77fe640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d80fe9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009230 tx=0x7fc7c802c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d57fa640 1 -- 172.21.15.90:0/4274578277 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c8004520 con 0x7fc7d8101150 2024-04-26T04:54:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d80ff2a0 con 0x7fc7d8101150 2024-04-26T04:54:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d57fa640 1 -- 172.21.15.90:0/4274578277 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc7c8003ab0 con 0x7fc7d8101150 2024-04-26T04:54:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7d57fa640 1 -- 172.21.15.90:0/4274578277 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc7c803e450 con 0x7fc7d8101150 2024-04-26T04:54:45.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.387+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d81a19c0 con 0x7fc7d8101150 2024-04-26T04:54:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.388+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d804e000 con 0x7fc7d8101150 2024-04-26T04:54:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.389+0000 7fc7d57fa640 1 -- 172.21.15.90:0/4274578277 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc7c8004f60 con 0x7fc7d8101150 2024-04-26T04:54:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.389+0000 7fc7d57fa640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc7c006e170 0x7fc7c0070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.389+0000 7fc7d57fa640 1 -- 172.21.15.90:0/4274578277 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc7c80b0c90 con 0x7fc7d8101150 2024-04-26T04:54:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.389+0000 7fc7d7fff640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc7c006e170 0x7fc7c0070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.390+0000 7fc7d7fff640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc7c006e170 0x7fc7c0070620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009890 tx=0x7fc7cc008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:45.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.394+0000 7fc7d57fa640 1 -- 172.21.15.90:0/4274578277 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc7c8079f70 con 0x7fc7d8101150 2024-04-26T04:54:45.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.610+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 19, "format": "json"} v 0) v1 -- 0x7fc7d8106c30 con 0x7fc7d8101150 2024-04-26T04:54:45.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.610+0000 7fc7d57fa640 1 -- 172.21.15.90:0/4274578277 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 19, "format": "json"}]=0 dumped fsmap epoch 19 v39) v1 ==== 107+0+4874 (secure 0 0 0) 0x7fc7c807d630 con 0x7fc7d8101150 2024-04-26T04:54:45.612 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:45.612 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":19,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/650878075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":650878075},{"type":"v1","addr":"172.21.15.90:6827","nonce":650878075}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":19,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:42:57.992878+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi090.dbkjuk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6829/370183803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":370183803},{"type":"v1","addr":"172.21.15.90:6829","nonce":370183803}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi104.wpntdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6825/1058172006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1058172006},{"type":"v1","addr":"172.21.15.104:6825","nonce":1058172006}]},"join_fscid":1,"export_targets":[0],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:45.612 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 19 2024-04-26T04:54:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc7c006e170 msgr2=0x7fc7c0070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc7c006e170 0x7fc7c0070620 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7cc009890 tx=0x7fc7cc008040 comp rx=0 tx=0).stop 2024-04-26T04:54:45.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 msgr2=0x7fc7d80fe9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d80fe9d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7c8009230 tx=0x7fc7c802c7c0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 shutdown_connections 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc7c006e170 0x7fc7c0070620 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc7d8101150 0x7fc7d80fe9d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 --2- 172.21.15.90:0/4274578277 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc7d8100840 0x7fc7d80fe490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 >> 172.21.15.90:0/4274578277 conn(0x7fc7d80fa0b0 msgr2=0x7fc7d8103e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 shutdown_connections 2024-04-26T04:54:45.615 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.613+0000 7fc7de96e640 1 -- 172.21.15.90:0/4274578277 wait complete. 2024-04-26T04:54:45.627 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 20 2024-04-26T04:54:45.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/844535640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 msgr2=0x7f8c180ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/844535640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c180ffce0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8c00009880 tx=0x7f8c0002bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/844535640 shutdown_connections 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/844535640 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c180ffce0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/844535640 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c18105880 0x7f8c18105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/844535640 >> 172.21.15.90:0/844535640 conn(0x7f8c180fb370 msgr2=0x7f8c180fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/844535640 shutdown_connections 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.723+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/844535640 wait complete. 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c1e1dd640 1 Processor -- start 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c1e1dd640 1 -- start start 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c1e1dd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c18195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c1e1dd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c18105880 0x7f8c18195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c1e1dd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18196150 con 0x7f8c180ff870 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c1e1dd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c18199f20 con 0x7f8c18105880 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c18195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c177fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c18195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55556/0 (socket says 172.21.15.90:55556) 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c177fe640 1 -- 172.21.15.90:0/1617879400 learned_addr learned my addr 172.21.15.90:0/1617879400 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.724+0000 7f8c0ffff640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c18105880 0x7f8c18195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c177fe640 1 -- 172.21.15.90:0/1617879400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c18105880 msgr2=0x7f8c18195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c177fe640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c18105880 0x7f8c18195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c177fe640 1 -- 172.21.15.90:0/1617879400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c00009560 con 0x7f8c180ff870 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c177fe640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c18195580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800c8f0 tx=0x7f8c0800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c157fa640 1 -- 172.21.15.90:0/1617879400 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c080041f0 con 0x7f8c180ff870 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c1819a200 con 0x7f8c180ff870 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c157fa640 1 -- 172.21.15.90:0/1617879400 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8c08007e70 con 0x7f8c180ff870 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c157fa640 1 -- 172.21.15.90:0/1617879400 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8c080049d0 con 0x7f8c180ff870 2024-04-26T04:54:45.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.725+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c1819a760 con 0x7f8c180ff870 2024-04-26T04:54:45.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.726+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c1804e000 con 0x7f8c180ff870 2024-04-26T04:54:45.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.726+0000 7f8c157fa640 1 -- 172.21.15.90:0/1617879400 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8c08004350 con 0x7f8c180ff870 2024-04-26T04:54:45.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.727+0000 7f8c157fa640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8c0406e170 0x7f8c04070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:45.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.727+0000 7f8c157fa640 1 -- 172.21.15.90:0/1617879400 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8c0808dff0 con 0x7f8c180ff870 2024-04-26T04:54:45.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.727+0000 7f8c0ffff640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8c0406e170 0x7f8c04070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:45.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.727+0000 7f8c0ffff640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8c0406e170 0x7f8c04070620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c18196b90 tx=0x7f8c00000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:45.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.731+0000 7f8c157fa640 1 -- 172.21.15.90:0/1617879400 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8c08057270 con 0x7f8c180ff870 2024-04-26T04:54:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.947+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 20, "format": "json"} v 0) v1 -- 0x7f8c18066860 con 0x7f8c180ff870 2024-04-26T04:54:45.948 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.947+0000 7f8c157fa640 1 -- 172.21.15.90:0/1617879400 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 20, "format": "json"}]=0 dumped fsmap epoch 20 v39) v1 ==== 107+0+4873 (secure 0 0 0) 0x7f8c0805a930 con 0x7f8c180ff870 2024-04-26T04:54:45.949 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:45.949 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":20,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/650878075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":650878075},{"type":"v1","addr":"172.21.15.90:6827","nonce":650878075}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:46:27.679399+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi090.dbkjuk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6829/370183803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":370183803},{"type":"v1","addr":"172.21.15.90:6829","nonce":370183803}]},"join_fscid":1,"export_targets":[1],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi104.wpntdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6825/1058172006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1058172006},{"type":"v1","addr":"172.21.15.104:6825","nonce":1058172006}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:45.949 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 20 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8c0406e170 msgr2=0x7f8c04070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8c0406e170 0x7f8c04070620 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f8c18196b90 tx=0x7f8c00000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 msgr2=0x7f8c18195580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c18195580 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f8c0800c8f0 tx=0x7f8c0800cdb0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 shutdown_connections 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8c18105880 0x7f8c18195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8c0406e170 0x7f8c04070620 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 --2- 172.21.15.90:0/1617879400 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8c180ff870 0x7f8c18195580 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 >> 172.21.15.90:0/1617879400 conn(0x7f8c180fb370 msgr2=0x7f8c180fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 shutdown_connections 2024-04-26T04:54:45.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:45.949+0000 7f8c1e1dd640 1 -- 172.21.15.90:0/1617879400 wait complete. 2024-04-26T04:54:45.961 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 21 2024-04-26T04:54:46.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.058+0000 7f666c18b640 1 -- 172.21.15.90:0/1946960115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66640ff690 msgr2=0x7f66640ffb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.058+0000 7f666c18b640 1 --2- 172.21.15.90:0/1946960115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66640ff690 0x7f66640ffb00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f6654009880 tx=0x7f665402bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.058+0000 7f666c18b640 1 -- 172.21.15.90:0/1946960115 shutdown_connections 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.058+0000 7f666c18b640 1 --2- 172.21.15.90:0/1946960115 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66640ff690 0x7f66640ffb00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.058+0000 7f666c18b640 1 --2- 172.21.15.90:0/1946960115 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66641056a0 0x7f6664105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.058+0000 7f666c18b640 1 -- 172.21.15.90:0/1946960115 >> 172.21.15.90:0/1946960115 conn(0x7f66640fb1d0 msgr2=0x7f66640fd5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 -- 172.21.15.90:0/1946960115 shutdown_connections 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 -- 172.21.15.90:0/1946960115 wait complete. 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 Processor -- start 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 -- start start 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66640ff690 0x7f6664195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66641056a0 0x7f66641957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664195dd0 con 0x7f66641056a0 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f666c18b640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6664195f40 con 0x7f66640ff690 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f6669726640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66641056a0 0x7f66641957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f6669726640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66641056a0 0x7f66641957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55582/0 (socket says 172.21.15.90:55582) 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f6669726640 1 -- 172.21.15.90:0/3187388227 learned_addr learned my addr 172.21.15.90:0/3187388227 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.059+0000 7f6669f27640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66640ff690 0x7f6664195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f6669726640 1 -- 172.21.15.90:0/3187388227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66640ff690 msgr2=0x7f6664195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f6669726640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66640ff690 0x7f6664195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f6669726640 1 -- 172.21.15.90:0/3187388227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6654009560 con 0x7f66641056a0 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f6669726640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66641056a0 0x7f66641957d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6654009230 tx=0x7f6654004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f665affd640 1 -- 172.21.15.90:0/3187388227 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f66540035f0 con 0x7f66641056a0 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f665affd640 1 -- 172.21.15.90:0/3187388227 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6654003c10 con 0x7f66641056a0 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6664199d70 con 0x7f66641056a0 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f665affd640 1 -- 172.21.15.90:0/3187388227 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6654034440 con 0x7f66641056a0 2024-04-26T04:54:46.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.060+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f666419a2f0 con 0x7f66641056a0 2024-04-26T04:54:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.061+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f666404e000 con 0x7f66641056a0 2024-04-26T04:54:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.061+0000 7f665affd640 1 -- 172.21.15.90:0/3187388227 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6654004fd0 con 0x7f66641056a0 2024-04-26T04:54:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.062+0000 7f665affd640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f665006e170 0x7f6650070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.062+0000 7f665affd640 1 -- 172.21.15.90:0/3187388227 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f66540b0620 con 0x7f66641056a0 2024-04-26T04:54:46.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.062+0000 7f6669f27640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f665006e170 0x7f6650070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.062+0000 7f6669f27640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f665006e170 0x7f6650070620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6660005c40 tx=0x7f6660005bd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.067+0000 7f665affd640 1 -- 172.21.15.90:0/3187388227 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6654079900 con 0x7f66641056a0 2024-04-26T04:54:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:45 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2641951124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:54:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:45 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3231307049' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-26T04:54:46.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:45 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/4274578277' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-26T04:54:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:45 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2641951124' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-26T04:54:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:45 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3231307049' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 18, "format": "json"}]: dispatch 2024-04-26T04:54:46.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:45 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/4274578277' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 19, "format": "json"}]: dispatch 2024-04-26T04:54:46.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.277+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 21, "format": "json"} v 0) v1 -- 0x7f66641965e0 con 0x7f66641056a0 2024-04-26T04:54:46.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.278+0000 7f665affd640 1 -- 172.21.15.90:0/3187388227 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 21, "format": "json"}]=0 dumped fsmap epoch 21 v39) v1 ==== 107+0+4872 (secure 0 0 0) 0x7f665407cfc0 con 0x7f66641056a0 2024-04-26T04:54:46.279 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:46.279 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":21,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/650878075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":650878075},{"type":"v1","addr":"172.21.15.90:6827","nonce":650878075}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":21,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:46:31.691772+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":46,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{"mds_0":14486,"mds_1":24273},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14486":{"gid":14486,"name":"cephfs.smithi090.dbkjuk","rank":0,"incarnation":13,"state":"up:active","state_seq":6,"addr":"172.21.15.90:6829/370183803","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":370183803},{"type":"v1","addr":"172.21.15.90:6829","nonce":370183803}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24273":{"gid":24273,"name":"cephfs.smithi104.wpntdi","rank":1,"incarnation":7,"state":"up:active","state_seq":4,"addr":"172.21.15.104:6825/1058172006","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1058172006},{"type":"v1","addr":"172.21.15.104:6825","nonce":1058172006}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:46.279 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 21 2024-04-26T04:54:46.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f665006e170 msgr2=0x7f6650070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f665006e170 0x7f6650070620 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f6660005c40 tx=0x7f6660005bd0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66641056a0 msgr2=0x7f66641957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66641056a0 0x7f66641957d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6654009230 tx=0x7f6654004040 comp rx=0 tx=0).stop 2024-04-26T04:54:46.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 shutdown_connections 2024-04-26T04:54:46.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f665006e170 0x7f6650070620 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.281 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f66641056a0 0x7f66641957d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 --2- 172.21.15.90:0/3187388227 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f66640ff690 0x7f6664195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 >> 172.21.15.90:0/3187388227 conn(0x7f66640fb1d0 msgr2=0x7f66640fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:46.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 shutdown_connections 2024-04-26T04:54:46.282 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.280+0000 7f666c18b640 1 -- 172.21.15.90:0/3187388227 wait complete. 2024-04-26T04:54:46.292 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 22 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 -- 172.21.15.90:0/1176367688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 msgr2=0x7f6f7c0fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 --2- 172.21.15.90:0/1176367688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c0fea20 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009880 tx=0x7f6f7002bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 -- 172.21.15.90:0/1176367688 shutdown_connections 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 --2- 172.21.15.90:0/1176367688 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c0fea20 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 --2- 172.21.15.90:0/1176367688 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7c104590 0x7f6f7c104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 -- 172.21.15.90:0/1176367688 >> 172.21.15.90:0/1176367688 conn(0x7f6f7c0743d0 msgr2=0x7f6f7c0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 -- 172.21.15.90:0/1176367688 shutdown_connections 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.388+0000 7f6f815e7640 1 -- 172.21.15.90:0/1176367688 wait complete. 2024-04-26T04:54:46.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f815e7640 1 Processor -- start 2024-04-26T04:54:46.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f815e7640 1 -- start start 2024-04-26T04:54:46.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f815e7640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f815e7640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7c104590 0x7f6f7c1957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f815e7640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c195e50 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f815e7640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f7c199c20 con 0x7f6f7c104590 2024-04-26T04:54:46.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f7affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f7affd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c195280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55596/0 (socket says 172.21.15.90:55596) 2024-04-26T04:54:46.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f7affd640 1 -- 172.21.15.90:0/2909868490 learned_addr learned my addr 172.21.15.90:0/2909868490 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.389+0000 7f6f7a7fc640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7c104590 0x7f6f7c1957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f7affd640 1 -- 172.21.15.90:0/2909868490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7c104590 msgr2=0x7f6f7c1957c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f7affd640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7c104590 0x7f6f7c1957c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f7affd640 1 -- 172.21.15.90:0/2909868490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f70009560 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f7affd640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c195280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00db80 tx=0x7f6f6c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f63fff640 1 -- 172.21.15.90:0/2909868490 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6c00de40 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f63fff640 1 -- 172.21.15.90:0/2909868490 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6f6c003e30 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f7c199f00 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f63fff640 1 -- 172.21.15.90:0/2909868490 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6f6c010830 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.390+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f7c19a460 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.391+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f7c04e000 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.391+0000 7f6f63fff640 1 -- 172.21.15.90:0/2909868490 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6f6c004020 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.392+0000 7f6f63fff640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6f6406e170 0x7f6f64070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.392+0000 7f6f63fff640 1 -- 172.21.15.90:0/2909868490 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6f6c08e580 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.392+0000 7f6f7a7fc640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6f6406e170 0x7f6f64070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.395 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.392+0000 7f6f7a7fc640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6f6406e170 0x7f6f64070620 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c196890 tx=0x7f6f70000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.396+0000 7f6f63fff640 1 -- 172.21.15.90:0/2909868490 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6f6c057860 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.609+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 22, "format": "json"} v 0) v1 -- 0x7f6f7c1965d0 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.610+0000 7f6f63fff640 1 -- 172.21.15.90:0/2909868490 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 22, "format": "json"}]=0 dumped fsmap epoch 22 v39) v1 ==== 107+0+3293 (secure 0 0 0) 0x7f6f6c017cf0 con 0x7f6f7c0fe5b0 2024-04-26T04:54:46.611 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:46.611 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":22,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/650878075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":650878075},{"type":"v1","addr":"172.21.15.90:6827","nonce":650878075}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:46.611 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 22 2024-04-26T04:54:46.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.611+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6f6406e170 msgr2=0x7f6f64070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6f6406e170 0x7f6f64070620 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c196890 tx=0x7f6f70000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 msgr2=0x7f6f7c195280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c195280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f6f6c00db80 tx=0x7f6f6c00b630 comp rx=0 tx=0).stop 2024-04-26T04:54:46.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 shutdown_connections 2024-04-26T04:54:46.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6f7c104590 0x7f6f7c1957c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6f6406e170 0x7f6f64070620 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 --2- 172.21.15.90:0/2909868490 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6f7c0fe5b0 0x7f6f7c195280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 >> 172.21.15.90:0/2909868490 conn(0x7f6f7c0743d0 msgr2=0x7f6f7c1025f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:46.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 shutdown_connections 2024-04-26T04:54:46.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.612+0000 7f6f815e7640 1 -- 172.21.15.90:0/2909868490 wait complete. 2024-04-26T04:54:46.624 DEBUG:tasks.fs:max_mds not reduced in epoch 22 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:46.624 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 23 2024-04-26T04:54:46.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 -- 172.21.15.90:0/3290059988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 msgr2=0x7fb450105c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 --2- 172.21.15.90:0/3290059988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450105c50 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb440009880 tx=0x7fb44002bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 -- 172.21.15.90:0/3290059988 shutdown_connections 2024-04-26T04:54:46.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 --2- 172.21.15.90:0/3290059988 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4500ff870 0x7fb4500ffce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 --2- 172.21.15.90:0/3290059988 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450105c50 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 -- 172.21.15.90:0/3290059988 >> 172.21.15.90:0/3290059988 conn(0x7fb4500fb370 msgr2=0x7fb4500fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 -- 172.21.15.90:0/3290059988 shutdown_connections 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.721+0000 7fb455458640 1 -- 172.21.15.90:0/3290059988 wait complete. 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb455458640 1 Processor -- start 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb455458640 1 -- start start 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb455458640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4500ff870 0x7fb450072560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb455458640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450072aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb455458640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb450072fe0 con 0x7fb450105880 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb455458640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb450073150 con 0x7fb4500ff870 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb447fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450072aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb447fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450072aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55628/0 (socket says 172.21.15.90:55628) 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb447fff640 1 -- 172.21.15.90:0/14841666 learned_addr learned my addr 172.21.15.90:0/14841666 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb447fff640 1 -- 172.21.15.90:0/14841666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4500ff870 msgr2=0x7fb450072560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb447fff640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4500ff870 0x7fb450072560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.722+0000 7fb447fff640 1 -- 172.21.15.90:0/14841666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb440009560 con 0x7fb450105880 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.723+0000 7fb447fff640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450072aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb43800d8d0 tx=0x7fb43800dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.723+0000 7fb44cff9640 1 -- 172.21.15.90:0/14841666 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb438009c00 con 0x7fb450105880 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.723+0000 7fb44cff9640 1 -- 172.21.15.90:0/14841666 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fb438009d60 con 0x7fb450105880 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.723+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4500733d0 con 0x7fb450105880 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.723+0000 7fb44cff9640 1 -- 172.21.15.90:0/14841666 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fb4380049d0 con 0x7fb450105880 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.723+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4501a1a70 con 0x7fb450105880 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.724+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb45004e000 con 0x7fb450105880 2024-04-26T04:54:46.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.724+0000 7fb44cff9640 1 -- 172.21.15.90:0/14841666 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fb438004c90 con 0x7fb450105880 2024-04-26T04:54:46.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.725+0000 7fb44cff9640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb43c06e170 0x7fb43c070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:46.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.725+0000 7fb44effd640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb43c06e170 0x7fb43c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:46.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.725+0000 7fb44cff9640 1 -- 172.21.15.90:0/14841666 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fb43808dd90 con 0x7fb450105880 2024-04-26T04:54:46.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.725+0000 7fb44effd640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb43c06e170 0x7fb43c070620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb440036040 tx=0x7fb440000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:46.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.729+0000 7fb44cff9640 1 -- 172.21.15.90:0/14841666 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fb438057070 con 0x7fb450105880 2024-04-26T04:54:46.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.949+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 23, "format": "json"} v 0) v1 -- 0x7fb450066860 con 0x7fb450105880 2024-04-26T04:54:46.951 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.950+0000 7fb44cff9640 1 -- 172.21.15.90:0/14841666 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 23, "format": "json"}]=0 dumped fsmap epoch 23 v39) v1 ==== 107+0+4854 (secure 0 0 0) 0x7fb43805a730 con 0x7fb450105880 2024-04-26T04:54:46.951 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:46.951 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":23,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14502,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/650878075","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":650878075},{"type":"v1","addr":"172.21.15.90:6827","nonce":650878075}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":14},{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34374,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/1614774470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1614774470},{"type":"v1","addr":"172.21.15.90:6829","nonce":1614774470}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44269,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/1188215025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1188215025},{"type":"v1","addr":"172.21.15.104:6825","nonce":1188215025}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:46.952 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 23 2024-04-26T04:54:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb43c06e170 msgr2=0x7fb43c070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb43c06e170 0x7fb43c070620 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fb440036040 tx=0x7fb440000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 msgr2=0x7fb450072aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450072aa0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb43800d8d0 tx=0x7fb43800dd90 comp rx=0 tx=0).stop 2024-04-26T04:54:46.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 shutdown_connections 2024-04-26T04:54:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fb43c06e170 0x7fb43c070620 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fb450105880 0x7fb450072aa0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 --2- 172.21.15.90:0/14841666 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fb4500ff870 0x7fb450072560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 >> 172.21.15.90:0/14841666 conn(0x7fb4500fb370 msgr2=0x7fb4500fcd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 shutdown_connections 2024-04-26T04:54:46.954 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:46.952+0000 7fb455458640 1 -- 172.21.15.90:0/14841666 wait complete. 2024-04-26T04:54:46.964 DEBUG:tasks.fs:max_mds not reduced in epoch 23 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:46.964 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 24 2024-04-26T04:54:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 -- 172.21.15.90:0/2357495451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 msgr2=0x7fbb38100df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/2357495451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38100df0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fbb340098b0 tx=0x7fbb3402bcd0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 -- 172.21.15.90:0/2357495451 shutdown_connections 2024-04-26T04:54:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/2357495451 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb38101330 0x7fbb38109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/2357495451 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38100df0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 -- 172.21.15.90:0/2357495451 >> 172.21.15.90:0/2357495451 conn(0x7fbb380fa270 msgr2=0x7fbb380fc680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 -- 172.21.15.90:0/2357495451 shutdown_connections 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.062+0000 7fbb3fb16640 1 -- 172.21.15.90:0/2357495451 wait complete. 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3fb16640 1 Processor -- start 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3fb16640 1 -- start start 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3fb16640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38195650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3fb16640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb38101330 0x7fbb38195b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3fb16640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb38196220 con 0x7fbb38100a20 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3fb16640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb38199ff0 con 0x7fbb38101330 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3d8b2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38195650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3d8b2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38195650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55640/0 (socket says 172.21.15.90:55640) 2024-04-26T04:54:47.066 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3d8b2640 1 -- 172.21.15.90:0/3101873763 learned_addr learned my addr 172.21.15.90:0/3101873763 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3d8b2640 1 -- 172.21.15.90:0/3101873763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb38101330 msgr2=0x7fbb38195b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3d8b2640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb38101330 0x7fbb38195b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3d8b2640 1 -- 172.21.15.90:0/3101873763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34009560 con 0x7fbb38100a20 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb3d8b2640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38195650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb3402c210 tx=0x7fbb34004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.063+0000 7fbb2effd640 1 -- 172.21.15.90:0/3101873763 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb340037c0 con 0x7fbb38100a20 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.064+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3819a270 con 0x7fbb38100a20 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.064+0000 7fbb2effd640 1 -- 172.21.15.90:0/3101873763 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fbb34003de0 con 0x7fbb38100a20 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.064+0000 7fbb2effd640 1 -- 172.21.15.90:0/3101873763 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fbb3403e680 con 0x7fbb38100a20 2024-04-26T04:54:47.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.064+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3819a7d0 con 0x7fbb38100a20 2024-04-26T04:54:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.064+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb3804e000 con 0x7fbb38100a20 2024-04-26T04:54:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.065+0000 7fbb2effd640 1 -- 172.21.15.90:0/3101873763 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fbb34004dd0 con 0x7fbb38100a20 2024-04-26T04:54:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.065+0000 7fbb2effd640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbb2406e170 0x7fbb24070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.065+0000 7fbb2effd640 1 -- 172.21.15.90:0/3101873763 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fbb340b0e90 con 0x7fbb38100a20 2024-04-26T04:54:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.065+0000 7fbb3d0b1640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbb2406e170 0x7fbb24070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.066+0000 7fbb3d0b1640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbb2406e170 0x7fbb24070620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb38196c60 tx=0x7fbb28006c90 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.070+0000 7fbb2effd640 1 -- 172.21.15.90:0/3101873763 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fbb3407a290 con 0x7fbb38100a20 2024-04-26T04:54:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:46 smithi090 ceph-mon[86550]: pgmap v273: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:46 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1617879400' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-26T04:54:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:46 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3187388227' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-26T04:54:47.204 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:46 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2909868490' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-26T04:54:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:46 smithi104 ceph-mon[64778]: pgmap v273: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:46 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1617879400' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 20, "format": "json"}]: dispatch 2024-04-26T04:54:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:46 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3187388227' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 21, "format": "json"}]: dispatch 2024-04-26T04:54:47.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:46 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2909868490' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 22, "format": "json"}]: dispatch 2024-04-26T04:54:47.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.286+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 24, "format": "json"} v 0) v1 -- 0x7fbb38066860 con 0x7fbb38100a20 2024-04-26T04:54:47.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.286+0000 7fbb2effd640 1 -- 172.21.15.90:0/3101873763 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 24, "format": "json"}]=0 dumped fsmap epoch 24 v39) v1 ==== 107+0+4078 (secure 0 0 0) 0x7fbb34043090 con 0x7fbb38100a20 2024-04-26T04:54:47.287 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:47.288 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34374,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/1614774470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1614774470},{"type":"v1","addr":"172.21.15.90:6829","nonce":1614774470}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":44269,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/1188215025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1188215025},{"type":"v1","addr":"172.21.15.104:6825","nonce":1188215025}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:47.288 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 24 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbb2406e170 msgr2=0x7fbb24070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbb2406e170 0x7fbb24070620 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fbb38196c60 tx=0x7fbb28006c90 comp rx=0 tx=0).stop 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 msgr2=0x7fbb38195650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38195650 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fbb3402c210 tx=0x7fbb34004270 comp rx=0 tx=0).stop 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 shutdown_connections 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fbb38101330 0x7fbb38195b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fbb2406e170 0x7fbb24070620 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.290 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 --2- 172.21.15.90:0/3101873763 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fbb38100a20 0x7fbb38195650 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.288+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 >> 172.21.15.90:0/3101873763 conn(0x7fbb380fa270 msgr2=0x7fbb38104000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.289+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 shutdown_connections 2024-04-26T04:54:47.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.289+0000 7fbb3fb16640 1 -- 172.21.15.90:0/3101873763 wait complete. 2024-04-26T04:54:47.301 DEBUG:tasks.fs:max_mds not reduced in epoch 24 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:47.301 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 25 2024-04-26T04:54:47.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 -- 172.21.15.90:0/1067097660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8740fe590 msgr2=0x7fc8740fea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 --2- 172.21.15.90:0/1067097660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8740fe590 0x7fc8740fea00 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc868009880 tx=0x7fc86802bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 -- 172.21.15.90:0/1067097660 shutdown_connections 2024-04-26T04:54:47.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 --2- 172.21.15.90:0/1067097660 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8740fe590 0x7fc8740fea00 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 --2- 172.21.15.90:0/1067097660 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8741045c0 0x7fc874104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.401 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 -- 172.21.15.90:0/1067097660 >> 172.21.15.90:0/1067097660 conn(0x7fc8740fa070 msgr2=0x7fc8740fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 -- 172.21.15.90:0/1067097660 shutdown_connections 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 -- 172.21.15.90:0/1067097660 wait complete. 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.398+0000 7fc8793ee640 1 Processor -- start 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8793ee640 1 -- start start 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8793ee640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8740fe590 0x7fc8741960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8793ee640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8741045c0 0x7fc874196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8793ee640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874196c30 con 0x7fc8741045c0 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8793ee640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc874196d70 con 0x7fc8740fe590 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8727fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8741045c0 0x7fc874196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8727fc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8741045c0 0x7fc874196610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:55644/0 (socket says 172.21.15.90:55644) 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8727fc640 1 -- 172.21.15.90:0/2952783218 learned_addr learned my addr 172.21.15.90:0/2952783218 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:47.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc872ffd640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8740fe590 0x7fc8741960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8727fc640 1 -- 172.21.15.90:0/2952783218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8740fe590 msgr2=0x7fc8741960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8727fc640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8740fe590 0x7fc8741960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8727fc640 1 -- 172.21.15.90:0/2952783218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc868009560 con 0x7fc8741045c0 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc872ffd640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8740fe590 0x7fc8741960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8727fc640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8741045c0 0x7fc874196610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc868009230 tx=0x7fc868004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc85bfff640 1 -- 172.21.15.90:0/2952783218 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc8680035f0 con 0x7fc8741045c0 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc85bfff640 1 -- 172.21.15.90:0/2952783218 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fc868003c10 con 0x7fc8741045c0 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc87419f140 con 0x7fc8741045c0 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc85bfff640 1 -- 172.21.15.90:0/2952783218 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fc868005670 con 0x7fc8741045c0 2024-04-26T04:54:47.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.399+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc87419f5e0 con 0x7fc8741045c0 2024-04-26T04:54:47.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.400+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc87404e000 con 0x7fc8741045c0 2024-04-26T04:54:47.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.401+0000 7fc85bfff640 1 -- 172.21.15.90:0/2952783218 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fc868004dd0 con 0x7fc8741045c0 2024-04-26T04:54:47.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.401+0000 7fc85bfff640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc85c06e120 0x7fc85c0705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.401+0000 7fc872ffd640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc85c06e120 0x7fc85c0705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.401+0000 7fc85bfff640 1 -- 172.21.15.90:0/2952783218 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fc8680b0590 con 0x7fc8741045c0 2024-04-26T04:54:47.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.402+0000 7fc872ffd640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc85c06e120 0x7fc85c0705d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0058c0 tx=0x7fc86c005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.407 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.406+0000 7fc85bfff640 1 -- 172.21.15.90:0/2952783218 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fc868079870 con 0x7fc8741045c0 2024-04-26T04:54:47.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.618+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 25, "format": "json"} v 0) v1 -- 0x7fc874068950 con 0x7fc8741045c0 2024-04-26T04:54:47.621 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.618+0000 7fc85bfff640 1 -- 172.21.15.90:0/2952783218 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 25, "format": "json"}]=0 dumped fsmap epoch 25 v39) v1 ==== 107+0+4922 (secure 0 0 0) 0x7fc86807cf30 con 0x7fc8741045c0 2024-04-26T04:54:47.622 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:47.622 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":25,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34374,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/1614774470","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":1614774470},{"type":"v1","addr":"172.21.15.90:6829","nonce":1614774470}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44269,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/1188215025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1188215025},{"type":"v1","addr":"172.21.15.104:6825","nonce":1188215025}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 25 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc85c06e120 msgr2=0x7fc85c0705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc85c06e120 0x7fc85c0705d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0058c0 tx=0x7fc86c005850 comp rx=0 tx=0).stop 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8741045c0 msgr2=0x7fc874196610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8741045c0 0x7fc874196610 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc868009230 tx=0x7fc868004040 comp rx=0 tx=0).stop 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 shutdown_connections 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fc85c06e120 0x7fc85c0705d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fc8741045c0 0x7fc874196610 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 --2- 172.21.15.90:0/2952783218 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fc8740fe590 0x7fc8741960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.620+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 >> 172.21.15.90:0/2952783218 conn(0x7fc8740fa070 msgr2=0x7fc8740fba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.621+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 shutdown_connections 2024-04-26T04:54:47.624 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.621+0000 7fc8793ee640 1 -- 172.21.15.90:0/2952783218 wait complete. 2024-04-26T04:54:47.633 DEBUG:tasks.fs:max_mds not reduced in epoch 25 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:47.633 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 26 2024-04-26T04:54:47.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 -- 172.21.15.90:0/3954869607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540fe670 msgr2=0x7f95540fea40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 --2- 172.21.15.90:0/3954869607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540fe670 0x7f95540fea40 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f9544009880 tx=0x7f954402bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 -- 172.21.15.90:0/3954869607 shutdown_connections 2024-04-26T04:54:47.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 --2- 172.21.15.90:0/3954869607 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95540ff010 0x7f9554102fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.733 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 --2- 172.21.15.90:0/3954869607 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540fe670 0x7f95540fea40 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 -- 172.21.15.90:0/3954869607 >> 172.21.15.90:0/3954869607 conn(0x7f95540f9f00 msgr2=0x7f95540fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 -- 172.21.15.90:0/3954869607 shutdown_connections 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.730+0000 7f955359e640 1 -- 172.21.15.90:0/3954869607 wait complete. 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f955359e640 1 Processor -- start 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f955359e640 1 -- start start 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f955359e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95540fe670 0x7f9554197470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f955359e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540ff010 0x7f95541979b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f955359e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9554198040 con 0x7f95540ff010 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f955359e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f955419be10 con 0x7f95540fe670 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f954bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540ff010 0x7f95541979b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f954bfff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540ff010 0x7f95541979b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44176/0 (socket says 172.21.15.90:44176) 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f954bfff640 1 -- 172.21.15.90:0/3370079526 learned_addr learned my addr 172.21.15.90:0/3370079526 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:47.734 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f955259c640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95540fe670 0x7f9554197470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f954bfff640 1 -- 172.21.15.90:0/3370079526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95540fe670 msgr2=0x7f9554197470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f954bfff640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95540fe670 0x7f9554197470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.731+0000 7f954bfff640 1 -- 172.21.15.90:0/3370079526 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9544009560 con 0x7f95540ff010 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.732+0000 7f954bfff640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540ff010 0x7f95541979b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f953c00d6d0 tx=0x7f953c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.732+0000 7f954b7fe640 1 -- 172.21.15.90:0/3370079526 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953c0051f0 con 0x7f95540ff010 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.732+0000 7f954b7fe640 1 -- 172.21.15.90:0/3370079526 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f953c009ce0 con 0x7f95540ff010 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.732+0000 7f954b7fe640 1 -- 172.21.15.90:0/3370079526 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f953c004e40 con 0x7f95540ff010 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.732+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f955419c0f0 con 0x7f95540ff010 2024-04-26T04:54:47.735 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.732+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f955419c650 con 0x7f95540ff010 2024-04-26T04:54:47.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.733+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f955404e000 con 0x7f95540ff010 2024-04-26T04:54:47.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.733+0000 7f954b7fe640 1 -- 172.21.15.90:0/3370079526 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f953c005350 con 0x7f95540ff010 2024-04-26T04:54:47.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.734+0000 7f954b7fe640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f954006e120 0x7f95400705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:47.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.734+0000 7f955259c640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f954006e120 0x7f95400705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:47.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.734+0000 7f954b7fe640 1 -- 172.21.15.90:0/3370079526 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f953c08dde0 con 0x7f95540ff010 2024-04-26T04:54:47.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.734+0000 7f955259c640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f954006e120 0x7f95400705d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9544006010 tx=0x7f9544005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:47.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.738+0000 7f954b7fe640 1 -- 172.21.15.90:0/3370079526 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f953c057110 con 0x7f95540ff010 2024-04-26T04:54:47.955 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.954+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 26, "format": "json"} v 0) v1 -- 0x7f95541987c0 con 0x7f95540ff010 2024-04-26T04:54:47.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.955+0000 7f954b7fe640 1 -- 172.21.15.90:0/3370079526 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 26, "format": "json"}]=0 dumped fsmap epoch 26 v39) v1 ==== 107+0+4143 (secure 0 0 0) 0x7f953c017090 con 0x7f95540ff010 2024-04-26T04:54:47.956 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:47.956 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":26,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":44269,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/1188215025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1188215025},{"type":"v1","addr":"172.21.15.104:6825","nonce":1188215025}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:47.956 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 26 2024-04-26T04:54:47.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f954006e120 msgr2=0x7f95400705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f954006e120 0x7f95400705d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f9544006010 tx=0x7f9544005dd0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540ff010 msgr2=0x7f95541979b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:47.958 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540ff010 0x7f95541979b0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f953c00d6d0 tx=0x7f953c00db90 comp rx=0 tx=0).stop 2024-04-26T04:54:47.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 shutdown_connections 2024-04-26T04:54:47.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f954006e120 0x7f95400705d0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f95540ff010 0x7f95541979b0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 --2- 172.21.15.90:0/3370079526 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f95540fe670 0x7f9554197470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:47.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 >> 172.21.15.90:0/3370079526 conn(0x7f95540f9f00 msgr2=0x7f9554101fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:47.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 shutdown_connections 2024-04-26T04:54:47.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:47.957+0000 7f955359e640 1 -- 172.21.15.90:0/3370079526 wait complete. 2024-04-26T04:54:47.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:47 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/14841666' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-26T04:54:47.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:47 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3101873763' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-26T04:54:47.968 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:47 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2952783218' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-26T04:54:47.969 DEBUG:tasks.fs:max_mds not reduced in epoch 26 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:47.969 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 27 2024-04-26T04:54:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/3437215239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd00fe7f0 msgr2=0x7f8cd00fec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/3437215239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd00fe7f0 0x7f8cd00fec60 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009880 tx=0x7f8cc402bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/3437215239 shutdown_connections 2024-04-26T04:54:48.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/3437215239 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd00fe7f0 0x7f8cd00fec60 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/3437215239 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cd0104800 0x7f8cd0104bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/3437215239 >> 172.21.15.90:0/3437215239 conn(0x7f8cd00fa2f0 msgr2=0x7f8cd00fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/3437215239 shutdown_connections 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.066+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/3437215239 wait complete. 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8cd4fb2640 1 Processor -- start 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8cd4fb2640 1 -- start start 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8cd4fb2640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cd00fe7f0 0x7f8cd019b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8cd4fb2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd0104800 0x7f8cd019b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8cd4fb2640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd019bfc0 con 0x7f8cd0104800 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8cd4fb2640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd01954c0 con 0x7f8cd00fe7f0 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8ccdd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd0104800 0x7f8cd019b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8ccdd9b640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd0104800 0x7f8cd019b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44192/0 (socket says 172.21.15.90:44192) 2024-04-26T04:54:48.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8ccdd9b640 1 -- 172.21.15.90:0/1303656790 learned_addr learned my addr 172.21.15.90:0/1303656790 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8cce59c640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cd00fe7f0 0x7f8cd019b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8ccdd9b640 1 -- 172.21.15.90:0/1303656790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cd00fe7f0 msgr2=0x7f8cd019b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8ccdd9b640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cd00fe7f0 0x7f8cd019b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8ccdd9b640 1 -- 172.21.15.90:0/1303656790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc4009560 con 0x7f8cd0104800 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.067+0000 7f8ccdd9b640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd0104800 0x7f8cd019b930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009230 tx=0x7f8cc4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.068+0000 7f8cbf7fe640 1 -- 172.21.15.90:0/1303656790 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cc40035f0 con 0x7f8cd0104800 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.068+0000 7f8cbf7fe640 1 -- 172.21.15.90:0/1303656790 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f8cc4003c10 con 0x7f8cd0104800 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.068+0000 7f8cbf7fe640 1 -- 172.21.15.90:0/1303656790 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f8cc4034440 con 0x7f8cd0104800 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.068+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd0195740 con 0x7f8cd0104800 2024-04-26T04:54:48.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.068+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd0195c40 con 0x7f8cd0104800 2024-04-26T04:54:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.069+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd004e000 con 0x7f8cd0104800 2024-04-26T04:54:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.069+0000 7f8cbf7fe640 1 -- 172.21.15.90:0/1303656790 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f8cc4004fd0 con 0x7f8cd0104800 2024-04-26T04:54:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.070+0000 7f8cbf7fe640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8cb806e170 0x7f8cb8070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.070+0000 7f8cce59c640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8cb806e170 0x7f8cb8070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.070+0000 7f8cbf7fe640 1 -- 172.21.15.90:0/1303656790 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f8cc40b0620 con 0x7f8cd0104800 2024-04-26T04:54:48.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.070+0000 7f8cce59c640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8cb806e170 0x7f8cb8070620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00ff930 tx=0x7f8cc0008040 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.074+0000 7f8cbf7fe640 1 -- 172.21.15.90:0/1303656790 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f8cc4079900 con 0x7f8cd0104800 2024-04-26T04:54:48.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:47 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/14841666' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 23, "format": "json"}]: dispatch 2024-04-26T04:54:48.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:47 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3101873763' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 24, "format": "json"}]: dispatch 2024-04-26T04:54:48.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:47 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2952783218' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 25, "format": "json"}]: dispatch 2024-04-26T04:54:48.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.287+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 27, "format": "json"} v 0) v1 -- 0x7f8cd010ba60 con 0x7f8cd0104800 2024-04-26T04:54:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.288+0000 7f8cbf7fe640 1 -- 172.21.15.90:0/1303656790 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 27, "format": "json"}]=0 dumped fsmap epoch 27 v39) v1 ==== 107+0+4987 (secure 0 0 0) 0x7f8cc407cfc0 con 0x7f8cd0104800 2024-04-26T04:54:48.289 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:48.289 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":27,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44269,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/1188215025","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":1188215025},{"type":"v1","addr":"172.21.15.104:6825","nonce":1188215025}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:48.289 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 27 2024-04-26T04:54:48.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8cb806e170 msgr2=0x7f8cb8070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8cb806e170 0x7f8cb8070620 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f8cd00ff930 tx=0x7f8cc0008040 comp rx=0 tx=0).stop 2024-04-26T04:54:48.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd0104800 msgr2=0x7f8cd019b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.291 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd0104800 0x7f8cd019b930 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8cc4009230 tx=0x7f8cc4004040 comp rx=0 tx=0).stop 2024-04-26T04:54:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 shutdown_connections 2024-04-26T04:54:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f8cb806e170 0x7f8cb8070620 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f8cd0104800 0x7f8cd019b930 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 --2- 172.21.15.90:0/1303656790 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f8cd00fe7f0 0x7f8cd019b3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 >> 172.21.15.90:0/1303656790 conn(0x7f8cd00fa2f0 msgr2=0x7f8cd0068420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 shutdown_connections 2024-04-26T04:54:48.292 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.290+0000 7f8cd4fb2640 1 -- 172.21.15.90:0/1303656790 wait complete. 2024-04-26T04:54:48.302 DEBUG:tasks.fs:max_mds not reduced in epoch 27 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:48.302 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 28 2024-04-26T04:54:48.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 -- 172.21.15.90:0/1072501807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840ff000 msgr2=0x7f9b84109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/1072501807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840ff000 0x7f9b84109a70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f9b780098b0 tx=0x7f9b7802bcd0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 -- 172.21.15.90:0/1072501807 shutdown_connections 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/1072501807 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840ff000 0x7f9b84109a70 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/1072501807 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b840fe6f0 0x7f9b840feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 -- 172.21.15.90:0/1072501807 >> 172.21.15.90:0/1072501807 conn(0x7f9b84073d20 msgr2=0x7f9b84074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 -- 172.21.15.90:0/1072501807 shutdown_connections 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.399+0000 7f9b88fdd640 1 -- 172.21.15.90:0/1072501807 wait complete. 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b88fdd640 1 Processor -- start 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b88fdd640 1 -- start start 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b88fdd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840fe6f0 0x7f9b84195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b88fdd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b840ff000 0x7f9b841958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.403 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b88fdd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84195f50 con 0x7f9b840fe6f0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b88fdd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b84199d20 con 0x7f9b840ff000 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b8259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840fe6f0 0x7f9b84195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b8259c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840fe6f0 0x7f9b84195380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44206/0 (socket says 172.21.15.90:44206) 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b8259c640 1 -- 172.21.15.90:0/212408716 learned_addr learned my addr 172.21.15.90:0/212408716 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b8259c640 1 -- 172.21.15.90:0/212408716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b840ff000 msgr2=0x7f9b841958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b8259c640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b840ff000 0x7f9b841958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b8259c640 1 -- 172.21.15.90:0/212408716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b78009560 con 0x7f9b840fe6f0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.400+0000 7f9b8259c640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840fe6f0 0x7f9b84195380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00db80 tx=0x7f9b7c00b630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.401+0000 7f9b737fe640 1 -- 172.21.15.90:0/212408716 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7c00de40 con 0x7f9b840fe6f0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.401+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b8419a000 con 0x7f9b840fe6f0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.401+0000 7f9b737fe640 1 -- 172.21.15.90:0/212408716 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9b7c003e30 con 0x7f9b840fe6f0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.401+0000 7f9b737fe640 1 -- 172.21.15.90:0/212408716 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9b7c010830 con 0x7f9b840fe6f0 2024-04-26T04:54:48.404 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.401+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b8419a560 con 0x7f9b840fe6f0 2024-04-26T04:54:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.402+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b8404e000 con 0x7f9b840fe6f0 2024-04-26T04:54:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.402+0000 7f9b737fe640 1 -- 172.21.15.90:0/212408716 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9b7c004020 con 0x7f9b840fe6f0 2024-04-26T04:54:48.405 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.402+0000 7f9b737fe640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9b6c06e170 0x7f9b6c070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.402+0000 7f9b737fe640 1 -- 172.21.15.90:0/212408716 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9b7c08e520 con 0x7f9b840fe6f0 2024-04-26T04:54:48.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.403+0000 7f9b81d9b640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9b6c06e170 0x7f9b6c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.406 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.403+0000 7f9b81d9b640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9b6c06e170 0x7f9b6c070620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b78036040 tx=0x7f9b78000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.408 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.407+0000 7f9b737fe640 1 -- 172.21.15.90:0/212408716 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9b7c057800 con 0x7f9b840fe6f0 2024-04-26T04:54:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.622+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 28, "format": "json"} v 0) v1 -- 0x7f9b841070e0 con 0x7f9b840fe6f0 2024-04-26T04:54:48.623 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.622+0000 7f9b737fe640 1 -- 172.21.15.90:0/212408716 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 28, "format": "json"}]=0 dumped fsmap epoch 28 v39) v1 ==== 107+0+4205 (secure 0 0 0) 0x7f9b7c0042d0 con 0x7f9b840fe6f0 2024-04-26T04:54:48.624 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:48.624 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:48.624 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 28 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9b6c06e170 msgr2=0x7f9b6c070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9b6c06e170 0x7f9b6c070620 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f9b78036040 tx=0x7f9b78000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840fe6f0 msgr2=0x7f9b84195380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840fe6f0 0x7f9b84195380 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f9b7c00db80 tx=0x7f9b7c00b630 comp rx=0 tx=0).stop 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 shutdown_connections 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9b840ff000 0x7f9b841958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9b6c06e170 0x7f9b6c070620 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 --2- 172.21.15.90:0/212408716 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9b840fe6f0 0x7f9b84195380 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.626 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 >> 172.21.15.90:0/212408716 conn(0x7f9b84073d20 msgr2=0x7f9b841042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 shutdown_connections 2024-04-26T04:54:48.627 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.624+0000 7f9b88fdd640 1 -- 172.21.15.90:0/212408716 wait complete. 2024-04-26T04:54:48.637 DEBUG:tasks.fs:max_mds not reduced in epoch 28 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:48.637 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 29 2024-04-26T04:54:48.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 -- 172.21.15.90:0/3753438263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 msgr2=0x7f6b340fea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 --2- 172.21.15.90:0/3753438263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b340fea20 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f6b28009880 tx=0x7f6b2802bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 -- 172.21.15.90:0/3753438263 shutdown_connections 2024-04-26T04:54:48.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 --2- 172.21.15.90:0/3753438263 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b340fea20 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.736 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 --2- 172.21.15.90:0/3753438263 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b34104590 0x7f6b34104960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 -- 172.21.15.90:0/3753438263 >> 172.21.15.90:0/3753438263 conn(0x7f6b340743d0 msgr2=0x7f6b340747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 -- 172.21.15.90:0/3753438263 shutdown_connections 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.733+0000 7f6b3a975640 1 -- 172.21.15.90:0/3753438263 wait complete. 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b3a975640 1 Processor -- start 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b3a975640 1 -- start start 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b3a975640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b34195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b3a975640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b34104590 0x7f6b341957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b3a975640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34195dd0 con 0x7f6b340fe5b0 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b3a975640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34195f40 con 0x7f6b34104590 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b33fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b34195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b33fff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b34195290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44228/0 (socket says 172.21.15.90:44228) 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b33fff640 1 -- 172.21.15.90:0/2165604738 learned_addr learned my addr 172.21.15.90:0/2165604738 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b337fe640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b34104590 0x7f6b341957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.737 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b33fff640 1 -- 172.21.15.90:0/2165604738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b34104590 msgr2=0x7f6b341957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b33fff640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b34104590 0x7f6b341957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b33fff640 1 -- 172.21.15.90:0/2165604738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b28009560 con 0x7f6b340fe5b0 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.734+0000 7f6b33fff640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b34195290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c8a0 tx=0x7f6b2400cd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.735+0000 7f6b317fa640 1 -- 172.21.15.90:0/2165604738 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b24007b80 con 0x7f6b340fe5b0 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.735+0000 7f6b317fa640 1 -- 172.21.15.90:0/2165604738 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f6b24007ce0 con 0x7f6b340fe5b0 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.735+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b34199dd0 con 0x7f6b340fe5b0 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.735+0000 7f6b317fa640 1 -- 172.21.15.90:0/2165604738 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f6b24018810 con 0x7f6b340fe5b0 2024-04-26T04:54:48.738 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.735+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b3419a380 con 0x7f6b340fe5b0 2024-04-26T04:54:48.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.736+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3404e000 con 0x7f6b340fe5b0 2024-04-26T04:54:48.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.736+0000 7f6b317fa640 1 -- 172.21.15.90:0/2165604738 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f6b24018a80 con 0x7f6b340fe5b0 2024-04-26T04:54:48.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.736+0000 7f6b317fa640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6b1c06e170 0x7f6b1c070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:48.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.737+0000 7f6b317fa640 1 -- 172.21.15.90:0/2165604738 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f6b2408e2f0 con 0x7f6b340fe5b0 2024-04-26T04:54:48.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.737+0000 7f6b337fe640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6b1c06e170 0x7f6b1c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:48.739 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.737+0000 7f6b337fe640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6b1c06e170 0x7f6b1c070620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b28006010 tx=0x7f6b28005b50 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:48.742 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.741+0000 7f6b317fa640 1 -- 172.21.15.90:0/2165604738 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f6b240575d0 con 0x7f6b340fe5b0 2024-04-26T04:54:48.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.952+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 29, "format": "json"} v 0) v1 -- 0x7f6b341965e0 con 0x7f6b340fe5b0 2024-04-26T04:54:48.953 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.953+0000 7f6b317fa640 1 -- 172.21.15.90:0/2165604738 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 29, "format": "json"}]=0 dumped fsmap epoch 29 v39) v1 ==== 107+0+5055 (secure 0 0 0) 0x7f6b2405ac90 con 0x7f6b340fe5b0 2024-04-26T04:54:48.954 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:48.954 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24283,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":2,"addr":"172.21.15.104:6827/2655656935","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":2655656935},{"type":"v1","addr":"172.21.15.104:6827","nonce":2655656935}]},"join_fscid":1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:48.954 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 29 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6b1c06e170 msgr2=0x7f6b1c070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6b1c06e170 0x7f6b1c070620 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6b28006010 tx=0x7f6b28005b50 comp rx=0 tx=0).stop 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 msgr2=0x7f6b34195290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b34195290 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c8a0 tx=0x7f6b2400cd60 comp rx=0 tx=0).stop 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 shutdown_connections 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f6b34104590 0x7f6b341957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f6b1c06e170 0x7f6b1c070620 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.956 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 --2- 172.21.15.90:0/2165604738 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f6b340fe5b0 0x7f6b34195290 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:48.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 >> 172.21.15.90:0/2165604738 conn(0x7f6b340743d0 msgr2=0x7f6b34067330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:48.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 shutdown_connections 2024-04-26T04:54:48.957 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:48.955+0000 7f6b3a975640 1 -- 172.21.15.90:0/2165604738 wait complete. 2024-04-26T04:54:48.966 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:48 smithi090 ceph-mon[86550]: pgmap v274: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:48.966 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:48 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3370079526' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-26T04:54:48.966 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:48 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1303656790' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-26T04:54:48.966 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:48 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/212408716' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-26T04:54:48.967 DEBUG:tasks.fs:max_mds not reduced in epoch 29 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:48.967 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 30 2024-04-26T04:54:49.067 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 -- 172.21.15.90:0/2904808467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b8104800 msgr2=0x7f43b8104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 --2- 172.21.15.90:0/2904808467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b8104800 0x7f43b8104bd0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009c20 tx=0x7f43b002c1b0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 -- 172.21.15.90:0/2904808467 shutdown_connections 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 --2- 172.21.15.90:0/2904808467 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b80fe7f0 0x7f43b80fec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 --2- 172.21.15.90:0/2904808467 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b8104800 0x7f43b8104bd0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 -- 172.21.15.90:0/2904808467 >> 172.21.15.90:0/2904808467 conn(0x7f43b80fa2f0 msgr2=0x7f43b80fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 -- 172.21.15.90:0/2904808467 shutdown_connections 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.064+0000 7f43beda9640 1 -- 172.21.15.90:0/2904808467 wait complete. 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43beda9640 1 Processor -- start 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43beda9640 1 -- start start 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43beda9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b80fe7f0 0x7f43b81943c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43beda9640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8104800 0x7f43b8194900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43beda9640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8194f90 con 0x7f43b80fe7f0 2024-04-26T04:54:49.068 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43beda9640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43b8198d60 con 0x7f43b8104800 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43bcb45640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b80fe7f0 0x7f43b81943c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43bcb45640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b80fe7f0 0x7f43b81943c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44248/0 (socket says 172.21.15.90:44248) 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43bcb45640 1 -- 172.21.15.90:0/763601342 learned_addr learned my addr 172.21.15.90:0/763601342 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43b7fff640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8104800 0x7f43b8194900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43bcb45640 1 -- 172.21.15.90:0/763601342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8104800 msgr2=0x7f43b8194900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43bcb45640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8104800 0x7f43b8194900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43bcb45640 1 -- 172.21.15.90:0/763601342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43b0009900 con 0x7f43b80fe7f0 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43b7fff640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8104800 0x7f43b8194900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43bcb45640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b80fe7f0 0x7f43b81943c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009430 tx=0x7f43b0032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.065+0000 7f43b5ffb640 1 -- 172.21.15.90:0/763601342 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b0032ed0 con 0x7f43b80fe7f0 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.066+0000 7f43b5ffb640 1 -- 172.21.15.90:0/763601342 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f43b0003e30 con 0x7f43b80fe7f0 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.066+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43b8198fe0 con 0x7f43b80fe7f0 2024-04-26T04:54:49.069 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.066+0000 7f43b5ffb640 1 -- 172.21.15.90:0/763601342 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f43b003f440 con 0x7f43b80fe7f0 2024-04-26T04:54:49.070 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.066+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43b81994e0 con 0x7f43b80fe7f0 2024-04-26T04:54:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.067+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43b804e000 con 0x7f43b80fe7f0 2024-04-26T04:54:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.067+0000 7f43b5ffb640 1 -- 172.21.15.90:0/763601342 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f43b003b040 con 0x7f43b80fe7f0 2024-04-26T04:54:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.067+0000 7f43b5ffb640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f43a406e170 0x7f43a4070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.068+0000 7f43b5ffb640 1 -- 172.21.15.90:0/763601342 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f43b00b0e40 con 0x7f43b80fe7f0 2024-04-26T04:54:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.068+0000 7f43b7fff640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f43a406e170 0x7f43a4070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.071 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.068+0000 7f43b7fff640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f43a406e170 0x7f43a4070620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005d00 tx=0x7f43a8005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.072+0000 7f43b5ffb640 1 -- 172.21.15.90:0/763601342 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f43b007a1e0 con 0x7f43b80fe7f0 2024-04-26T04:54:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:48 smithi104 ceph-mon[64778]: pgmap v274: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:48 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3370079526' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 26, "format": "json"}]: dispatch 2024-04-26T04:54:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:48 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1303656790' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 27, "format": "json"}]: dispatch 2024-04-26T04:54:49.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:48 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/212408716' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 28, "format": "json"}]: dispatch 2024-04-26T04:54:49.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.282+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 30, "format": "json"} v 0) v1 -- 0x7f43b8066860 con 0x7f43b80fe7f0 2024-04-26T04:54:49.284 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.283+0000 7f43b5ffb640 1 -- 172.21.15.90:0/763601342 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 30, "format": "json"}]=0 dumped fsmap epoch 30 v39) v1 ==== 107+0+4273 (secure 0 0 0) 0x7f43b007d8a0 con 0x7f43b80fe7f0 2024-04-26T04:54:49.284 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:49.285 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:49.285 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 30 2024-04-26T04:54:49.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f43a406e170 msgr2=0x7f43a4070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f43a406e170 0x7f43a4070620 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005d00 tx=0x7f43a8005c70 comp rx=0 tx=0).stop 2024-04-26T04:54:49.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b80fe7f0 msgr2=0x7f43b81943c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.287 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b80fe7f0 0x7f43b81943c0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009430 tx=0x7f43b0032b60 comp rx=0 tx=0).stop 2024-04-26T04:54:49.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 shutdown_connections 2024-04-26T04:54:49.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f43b8104800 0x7f43b8194900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f43a406e170 0x7f43a4070620 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 --2- 172.21.15.90:0/763601342 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f43b80fe7f0 0x7f43b81943c0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 >> 172.21.15.90:0/763601342 conn(0x7f43b80fa2f0 msgr2=0x7f43b80fbcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.288 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 shutdown_connections 2024-04-26T04:54:49.289 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.285+0000 7f43beda9640 1 -- 172.21.15.90:0/763601342 wait complete. 2024-04-26T04:54:49.297 DEBUG:tasks.fs:max_mds not reduced in epoch 30 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:49.298 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 31 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.393+0000 7f78db30a640 1 -- 172.21.15.90:0/3772936744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 msgr2=0x7f78d4104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.393+0000 7f78db30a640 1 --2- 172.21.15.90:0/3772936744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4104960 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009c20 tx=0x7f78d002c1b0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.393+0000 7f78db30a640 1 -- 172.21.15.90:0/3772936744 shutdown_connections 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.393+0000 7f78db30a640 1 --2- 172.21.15.90:0/3772936744 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d40fe5b0 0x7f78d40fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.393+0000 7f78db30a640 1 --2- 172.21.15.90:0/3772936744 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4104960 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.393+0000 7f78db30a640 1 -- 172.21.15.90:0/3772936744 >> 172.21.15.90:0/3772936744 conn(0x7f78d40743d0 msgr2=0x7f78d40747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 -- 172.21.15.90:0/3772936744 shutdown_connections 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 -- 172.21.15.90:0/3772936744 wait complete. 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 Processor -- start 2024-04-26T04:54:49.397 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 -- start start 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d40fe5b0 0x7f78d41941c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4194700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d4194d90 con 0x7f78d4104590 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78db30a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d4198b60 con 0x7f78d40fe5b0 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78d88a5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4194700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78d88a5640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4194700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44274/0 (socket says 172.21.15.90:44274) 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78d88a5640 1 -- 172.21.15.90:0/105609627 learned_addr learned my addr 172.21.15.90:0/105609627 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.394+0000 7f78d90a6640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d40fe5b0 0x7f78d41941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78d88a5640 1 -- 172.21.15.90:0/105609627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d40fe5b0 msgr2=0x7f78d41941c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78d88a5640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d40fe5b0 0x7f78d41941c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78d88a5640 1 -- 172.21.15.90:0/105609627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d0009900 con 0x7f78d4104590 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78d88a5640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4194700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f78c400d6f0 tx=0x7f78c400dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.398 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78ca7fc640 1 -- 172.21.15.90:0/105609627 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c4009a30 con 0x7f78d4104590 2024-04-26T04:54:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78ca7fc640 1 -- 172.21.15.90:0/105609627 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f78c40054e0 con 0x7f78d4104590 2024-04-26T04:54:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78ca7fc640 1 -- 172.21.15.90:0/105609627 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f78c4010460 con 0x7f78d4104590 2024-04-26T04:54:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d4198e40 con 0x7f78d4104590 2024-04-26T04:54:49.399 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.395+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d4199420 con 0x7f78d4104590 2024-04-26T04:54:49.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.396+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d404e000 con 0x7f78d4104590 2024-04-26T04:54:49.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.396+0000 7f78ca7fc640 1 -- 172.21.15.90:0/105609627 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f78c4009b90 con 0x7f78d4104590 2024-04-26T04:54:49.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.397+0000 7f78ca7fc640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f78c006e170 0x7f78c0070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.397+0000 7f78ca7fc640 1 -- 172.21.15.90:0/105609627 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f78c408dbd0 con 0x7f78d4104590 2024-04-26T04:54:49.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.397+0000 7f78d90a6640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f78c006e170 0x7f78c0070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.400 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.397+0000 7f78d90a6640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f78c006e170 0x7f78c0070620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000f80 tx=0x7f78d0009840 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.402 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.402+0000 7f78ca7fc640 1 -- 172.21.15.90:0/105609627 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f78c4056fe0 con 0x7f78d4104590 2024-04-26T04:54:49.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.612+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 31, "format": "json"} v 0) v1 -- 0x7f78d4195510 con 0x7f78d4104590 2024-04-26T04:54:49.614 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.613+0000 7f78ca7fc640 1 -- 172.21.15.90:0/105609627 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 31, "format": "json"}]=0 dumped fsmap epoch 31 v39) v1 ==== 107+0+5123 (secure 0 0 0) 0x7f78c4020090 con 0x7f78d4104590 2024-04-26T04:54:49.614 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:49.614 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":31,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:51:20.563438+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:49.614 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 31 2024-04-26T04:54:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f78c006e170 msgr2=0x7f78c0070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f78c006e170 0x7f78c0070620 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f78d0000f80 tx=0x7f78d0009840 comp rx=0 tx=0).stop 2024-04-26T04:54:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 msgr2=0x7f78d4194700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4194700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f78c400d6f0 tx=0x7f78c400dbb0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 shutdown_connections 2024-04-26T04:54:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f78c006e170 0x7f78c0070620 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.616 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f78d4104590 0x7f78d4194700 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 --2- 172.21.15.90:0/105609627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f78d40fe5b0 0x7f78d41941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 >> 172.21.15.90:0/105609627 conn(0x7f78d40743d0 msgr2=0x7f78d40fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 shutdown_connections 2024-04-26T04:54:49.617 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.615+0000 7f78db30a640 1 -- 172.21.15.90:0/105609627 wait complete. 2024-04-26T04:54:49.627 DEBUG:tasks.fs:max_mds not reduced in epoch 31 as fs was failed for carrying out rapid multi-rank mds upgrade 2024-04-26T04:54:49.627 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 32 2024-04-26T04:54:49.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- 172.21.15.90:0/1019091216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f08104590 msgr2=0x7f4f08104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 --2- 172.21.15.90:0/1019091216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f08104590 0x7f4f08104960 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009880 tx=0x7f4f0402bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- 172.21.15.90:0/1019091216 shutdown_connections 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 --2- 172.21.15.90:0/1019091216 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f080fe5b0 0x7f4f080fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 --2- 172.21.15.90:0/1019091216 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f08104590 0x7f4f08104960 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- 172.21.15.90:0/1019091216 >> 172.21.15.90:0/1019091216 conn(0x7f4f080743d0 msgr2=0x7f4f080747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- 172.21.15.90:0/1019091216 shutdown_connections 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- 172.21.15.90:0/1019091216 wait complete. 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 Processor -- start 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- start start 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f080fe5b0 0x7f4f081941a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f08104590 0x7f4f081946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08194d70 con 0x7f4f080fe5b0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.724+0000 7f4f10021640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f08198b40 con 0x7f4f08104590 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0ddbd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f080fe5b0 0x7f4f081941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0ddbd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f080fe5b0 0x7f4f081941a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44290/0 (socket says 172.21.15.90:44290) 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0ddbd640 1 -- 172.21.15.90:0/1838681687 learned_addr learned my addr 172.21.15.90:0/1838681687 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0d5bc640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f08104590 0x7f4f081946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0ddbd640 1 -- 172.21.15.90:0/1838681687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f08104590 msgr2=0x7f4f081946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0ddbd640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f08104590 0x7f4f081946e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0ddbd640 1 -- 172.21.15.90:0/1838681687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f04009560 con 0x7f4f080fe5b0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f0ddbd640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f080fe5b0 0x7f4f081941a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009230 tx=0x7f4f04004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4efeffd640 1 -- 172.21.15.90:0/1838681687 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f040035f0 con 0x7f4f080fe5b0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4efeffd640 1 -- 172.21.15.90:0/1838681687 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f4f04003c10 con 0x7f4f080fe5b0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f08198dc0 con 0x7f4f080fe5b0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4efeffd640 1 -- 172.21.15.90:0/1838681687 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f4f04034440 con 0x7f4f080fe5b0 2024-04-26T04:54:49.729 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.725+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f08199340 con 0x7f4f080fe5b0 2024-04-26T04:54:49.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.726+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f0804e000 con 0x7f4f080fe5b0 2024-04-26T04:54:49.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.727+0000 7f4efeffd640 1 -- 172.21.15.90:0/1838681687 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f4f04004fd0 con 0x7f4f080fe5b0 2024-04-26T04:54:49.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.727+0000 7f4efeffd640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4ef406e170 0x7f4ef4070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:49.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.727+0000 7f4efeffd640 1 -- 172.21.15.90:0/1838681687 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f4f040b0620 con 0x7f4f080fe5b0 2024-04-26T04:54:49.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.727+0000 7f4f0d5bc640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4ef406e170 0x7f4ef4070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:49.731 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.727+0000 7f4f0d5bc640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4ef406e170 0x7f4ef4070620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f081957b0 tx=0x7f4ef800a4c0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:49.732 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.732+0000 7f4efeffd640 1 -- 172.21.15.90:0/1838681687 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f4f04079900 con 0x7f4f080fe5b0 2024-04-26T04:54:49.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.945+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 32, "format": "json"} v 0) v1 -- 0x7f4f081954f0 con 0x7f4f080fe5b0 2024-04-26T04:54:49.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.946+0000 7f4efeffd640 1 -- 172.21.15.90:0/1838681687 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 32, "format": "json"}]=0 dumped fsmap epoch 32 v39) v1 ==== 107+0+5122 (secure 0 0 0) 0x7f4f0407cfc0 con 0x7f4f080fe5b0 2024-04-26T04:54:49.947 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:49.947 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":32,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":25},{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":27},{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":32,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:31.234225+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":2,"in":[0,1],"up":{},"failed":[0,1],"damaged":[],"stopped":[],"info":{},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:49.948 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 32 2024-04-26T04:54:49.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4ef406e170 msgr2=0x7f4ef4070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4ef406e170 0x7f4ef4070620 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f4f081957b0 tx=0x7f4ef800a4c0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.949 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f080fe5b0 msgr2=0x7f4f081941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f080fe5b0 0x7f4f081941a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f4f04009230 tx=0x7f4f04004040 comp rx=0 tx=0).stop 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 shutdown_connections 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f4f08104590 0x7f4f081946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f4ef406e170 0x7f4ef4070620 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 --2- 172.21.15.90:0/1838681687 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f4f080fe5b0 0x7f4f081941a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 >> 172.21.15.90:0/1838681687 conn(0x7f4f080743d0 msgr2=0x7f4f080fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 shutdown_connections 2024-04-26T04:54:49.950 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:49.948+0000 7f4f10021640 1 -- 172.21.15.90:0/1838681687 wait complete. 2024-04-26T04:54:49.960 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 33 2024-04-26T04:54:50.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- 172.21.15.90:0/1074523244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238104770 msgr2=0x7f9238104b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 --2- 172.21.15.90:0/1074523244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238104770 0x7f9238104b40 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9230009c20 tx=0x7f923002c1b0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- 172.21.15.90:0/1074523244 shutdown_connections 2024-04-26T04:54:50.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 --2- 172.21.15.90:0/1074523244 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f92380fe760 0x7f92380febd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 --2- 172.21.15.90:0/1074523244 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9238104770 0x7f9238104b40 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- 172.21.15.90:0/1074523244 >> 172.21.15.90:0/1074523244 conn(0x7f92380fa260 msgr2=0x7f92380fc670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- 172.21.15.90:0/1074523244 shutdown_connections 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- 172.21.15.90:0/1074523244 wait complete. 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 Processor -- start 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- start start 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92380fe760 0x7f9238196410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9238104770 0x7f9238196950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238196e90 con 0x7f92380fe760 2024-04-26T04:54:50.061 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.057+0000 7f923eef4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9238197000 con 0x7f9238104770 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923cc90640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92380fe760 0x7f9238196410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923cc90640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92380fe760 0x7f9238196410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44298/0 (socket says 172.21.15.90:44298) 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923cc90640 1 -- 172.21.15.90:0/500507542 learned_addr learned my addr 172.21.15.90:0/500507542 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f9237fff640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9238104770 0x7f9238196950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923cc90640 1 -- 172.21.15.90:0/500507542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9238104770 msgr2=0x7f9238196950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923cc90640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9238104770 0x7f9238196950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923cc90640 1 -- 172.21.15.90:0/500507542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9230009900 con 0x7f92380fe760 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923cc90640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92380fe760 0x7f9238196410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9230032b60 tx=0x7f9230032b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f9235ffb640 1 -- 172.21.15.90:0/500507542 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9230034820 con 0x7f92380fe760 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f9235ffb640 1 -- 172.21.15.90:0/500507542 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9230003e30 con 0x7f92380fe760 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9238197280 con 0x7f92380fe760 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f9235ffb640 1 -- 172.21.15.90:0/500507542 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f923003f440 con 0x7f92380fe760 2024-04-26T04:54:50.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.058+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923819f7f0 con 0x7f92380fe760 2024-04-26T04:54:50.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.059+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923804e000 con 0x7f92380fe760 2024-04-26T04:54:50.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.060+0000 7f9235ffb640 1 -- 172.21.15.90:0/500507542 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f923003b040 con 0x7f92380fe760 2024-04-26T04:54:50.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.060+0000 7f9235ffb640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f922406e170 0x7f9224070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.063 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.060+0000 7f9235ffb640 1 -- 172.21.15.90:0/500507542 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f92300b0ce0 con 0x7f92380fe760 2024-04-26T04:54:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.060+0000 7f9237fff640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f922406e170 0x7f9224070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.064 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.060+0000 7f9237fff640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f922406e170 0x7f9224070620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9228005e10 tx=0x7f9228005da0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:50.065 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.065+0000 7f9235ffb640 1 -- 172.21.15.90:0/500507542 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9230079fc0 con 0x7f92380fe760 2024-04-26T04:54:50.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:49 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2165604738' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-26T04:54:50.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:49 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/763601342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-26T04:54:50.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:49 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/105609627' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-26T04:54:50.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:49 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2165604738' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 29, "format": "json"}]: dispatch 2024-04-26T04:54:50.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:49 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/763601342' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 30, "format": "json"}]: dispatch 2024-04-26T04:54:50.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:49 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/105609627' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 31, "format": "json"}]: dispatch 2024-04-26T04:54:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.275+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 33, "format": "json"} v 0) v1 -- 0x7f9238197280 con 0x7f92380fe760 2024-04-26T04:54:50.277 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.276+0000 7f9235ffb640 1 -- 172.21.15.90:0/500507542 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 33, "format": "json"}]=0 dumped fsmap epoch 33 v39) v1 ==== 107+0+5213 (secure 0 0 0) 0x7f9238197280 con 0x7f92380fe760 2024-04-26T04:54:50.278 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:50.278 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":33,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:32.242183+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":1,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:50.278 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 33 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f922406e170 msgr2=0x7f9224070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f922406e170 0x7f9224070620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f9228005e10 tx=0x7f9228005da0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92380fe760 msgr2=0x7f9238196410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92380fe760 0x7f9238196410 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9230032b60 tx=0x7f9230032b90 comp rx=0 tx=0).stop 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 shutdown_connections 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9238104770 0x7f9238196950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f922406e170 0x7f9224070620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 --2- 172.21.15.90:0/500507542 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f92380fe760 0x7f9238196410 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 >> 172.21.15.90:0/500507542 conn(0x7f92380fa260 msgr2=0x7f92380fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 shutdown_connections 2024-04-26T04:54:50.280 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.278+0000 7f923eef4640 1 -- 172.21.15.90:0/500507542 wait complete. 2024-04-26T04:54:50.290 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 34 2024-04-26T04:54:50.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 -- 172.21.15.90:0/2404411948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 msgr2=0x7f58e40febb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 --2- 172.21.15.90:0/2404411948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e40febb0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009880 tx=0x7f58d402bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 -- 172.21.15.90:0/2404411948 shutdown_connections 2024-04-26T04:54:50.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 --2- 172.21.15.90:0/2404411948 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e40febb0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 --2- 172.21.15.90:0/2404411948 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58e4104750 0x7f58e4104b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 -- 172.21.15.90:0/2404411948 >> 172.21.15.90:0/2404411948 conn(0x7f58e40fa200 msgr2=0x7f58e40fc610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 -- 172.21.15.90:0/2404411948 shutdown_connections 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.387+0000 7f58ebefc640 1 -- 172.21.15.90:0/2404411948 wait complete. 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58ebefc640 1 Processor -- start 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58ebefc640 1 -- start start 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58ebefc640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e4194400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58ebefc640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58e4104750 0x7f58e4194940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58ebefc640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e4194fd0 con 0x7f58e40fe740 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58ebefc640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e4198da0 con 0x7f58e4104750 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9c98640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e4194400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9c98640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e4194400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44314/0 (socket says 172.21.15.90:44314) 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9c98640 1 -- 172.21.15.90:0/373771955 learned_addr learned my addr 172.21.15.90:0/373771955 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:50.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9497640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58e4104750 0x7f58e4194940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9c98640 1 -- 172.21.15.90:0/373771955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58e4104750 msgr2=0x7f58e4194940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9c98640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58e4104750 0x7f58e4194940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9c98640 1 -- 172.21.15.90:0/373771955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d4009560 con 0x7f58e40fe740 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58e9c98640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e4194400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f58e000d8d0 tx=0x7f58e000dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58daffd640 1 -- 172.21.15.90:0/373771955 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e000bea0 con 0x7f58e40fe740 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58daffd640 1 -- 172.21.15.90:0/373771955 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f58e0009c00 con 0x7f58e40fe740 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e4199080 con 0x7f58e40fe740 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.388+0000 7f58daffd640 1 -- 172.21.15.90:0/373771955 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f58e000f790 con 0x7f58e40fe740 2024-04-26T04:54:50.392 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.389+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e41995b0 con 0x7f58e40fe740 2024-04-26T04:54:50.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.389+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e404e000 con 0x7f58e40fe740 2024-04-26T04:54:50.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.390+0000 7f58daffd640 1 -- 172.21.15.90:0/373771955 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f58e000fa50 con 0x7f58e40fe740 2024-04-26T04:54:50.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.390+0000 7f58daffd640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f58d006e170 0x7f58d0070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.390+0000 7f58daffd640 1 -- 172.21.15.90:0/373771955 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f58e008e0c0 con 0x7f58e40fe740 2024-04-26T04:54:50.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.390+0000 7f58e9497640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f58d006e170 0x7f58d0070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.393 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.391+0000 7f58e9497640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f58d006e170 0x7f58d0070620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58d4006010 tx=0x7f58d4005dd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:50.396 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.395+0000 7f58daffd640 1 -- 172.21.15.90:0/373771955 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f58e00573a0 con 0x7f58e40fe740 2024-04-26T04:54:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.606+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 34, "format": "json"} v 0) v1 -- 0x7f58e4066860 con 0x7f58e40fe740 2024-04-26T04:54:50.607 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.606+0000 7f58daffd640 1 -- 172.21.15.90:0/373771955 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 34, "format": "json"}]=0 dumped fsmap epoch 34 v39) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f58e005aa60 con 0x7f58e40fe740 2024-04-26T04:54:50.608 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:50.608 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":34,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:38.433787+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":0,"incarnation":33,"state":"up:replay","state_seq":1,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":1,"incarnation":33,"state":"up:resolve","state_seq":14,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:50.608 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 34 2024-04-26T04:54:50.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f58d006e170 msgr2=0x7f58d0070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f58d006e170 0x7f58d0070620 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f58d4006010 tx=0x7f58d4005dd0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 msgr2=0x7f58e4194400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.610 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e4194400 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f58e000d8d0 tx=0x7f58e000dd90 comp rx=0 tx=0).stop 2024-04-26T04:54:50.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 shutdown_connections 2024-04-26T04:54:50.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f58e4104750 0x7f58e4194940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f58d006e170 0x7f58d0070620 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 --2- 172.21.15.90:0/373771955 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f58e40fe740 0x7f58e4194400 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 >> 172.21.15.90:0/373771955 conn(0x7f58e40fa200 msgr2=0x7f58e40fbbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:50.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 shutdown_connections 2024-04-26T04:54:50.611 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.609+0000 7f58ebefc640 1 -- 172.21.15.90:0/373771955 wait complete. 2024-04-26T04:54:50.621 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 35 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- 172.21.15.90:0/640580678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b681013d0 msgr2=0x7f2b681098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 --2- 172.21.15.90:0/640580678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b681013d0 0x7f2b681098c0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f2b500098b0 tx=0x7f2b5002bd60 comp rx=0 tx=0).stop 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- 172.21.15.90:0/640580678 shutdown_connections 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 --2- 172.21.15.90:0/640580678 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b681013d0 0x7f2b681098c0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 --2- 172.21.15.90:0/640580678 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b68100ac0 0x7f2b68100e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- 172.21.15.90:0/640580678 >> 172.21.15.90:0/640580678 conn(0x7f2b680fa2f0 msgr2=0x7f2b680fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- 172.21.15.90:0/640580678 shutdown_connections 2024-04-26T04:54:50.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- 172.21.15.90:0/640580678 wait complete. 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 Processor -- start 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- start start 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b68100ac0 0x7f2b68195580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b681013d0 0x7f2b68195ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68196150 con 0x7f2b68100ac0 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.721+0000 7f2b6f036640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b68199f20 con 0x7f2b681013d0 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6cdd2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b68100ac0 0x7f2b68195580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6cdd2640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b68100ac0 0x7f2b68195580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44330/0 (socket says 172.21.15.90:44330) 2024-04-26T04:54:50.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6cdd2640 1 -- 172.21.15.90:0/2176750252 learned_addr learned my addr 172.21.15.90:0/2176750252 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b67fff640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b681013d0 0x7f2b68195ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6cdd2640 1 -- 172.21.15.90:0/2176750252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b681013d0 msgr2=0x7f2b68195ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6cdd2640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b681013d0 0x7f2b68195ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6cdd2640 1 -- 172.21.15.90:0/2176750252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b50009560 con 0x7f2b68100ac0 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b67fff640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b681013d0 0x7f2b68195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6cdd2640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b68100ac0 0x7f2b68195580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000ea00 tx=0x7f2b6000eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b65ffb640 1 -- 172.21.15.90:0/2176750252 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b6000cd50 con 0x7f2b68100ac0 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b65ffb640 1 -- 172.21.15.90:0/2176750252 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f2b60005020 con 0x7f2b68100ac0 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b6819a200 con 0x7f2b68100ac0 2024-04-26T04:54:50.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b65ffb640 1 -- 172.21.15.90:0/2176750252 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f2b600116a0 con 0x7f2b68100ac0 2024-04-26T04:54:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.722+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b6819a760 con 0x7f2b68100ac0 2024-04-26T04:54:50.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.723+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b6804e000 con 0x7f2b68100ac0 2024-04-26T04:54:50.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.724+0000 7f2b65ffb640 1 -- 172.21.15.90:0/2176750252 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f2b60005190 con 0x7f2b68100ac0 2024-04-26T04:54:50.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.724+0000 7f2b65ffb640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2b5406e170 0x7f2b54070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:50.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.724+0000 7f2b65ffb640 1 -- 172.21.15.90:0/2176750252 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f2b6008e5f0 con 0x7f2b68100ac0 2024-04-26T04:54:50.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.724+0000 7f2b67fff640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2b5406e170 0x7f2b54070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:50.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.724+0000 7f2b67fff640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2b5406e170 0x7f2b54070620 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005ec0 tx=0x7f2b50005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:50.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.729+0000 7f2b65ffb640 1 -- 172.21.15.90:0/2176750252 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f2b60057830 con 0x7f2b68100ac0 2024-04-26T04:54:50.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.941+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 35, "format": "json"} v 0) v1 -- 0x7f2b68066860 con 0x7f2b68100ac0 2024-04-26T04:54:50.944 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.942+0000 7f2b65ffb640 1 -- 172.21.15.90:0/2176750252 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 35, "format": "json"}]=0 dumped fsmap epoch 35 v39) v1 ==== 107+0+5217 (secure 0 0 0) 0x7f2b6005aef0 con 0x7f2b68100ac0 2024-04-26T04:54:50.944 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:50.944 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":35,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":35,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:41.023615+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":0,"incarnation":33,"state":"up:resolve","state_seq":19,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":1,"incarnation":33,"state":"up:resolve","state_seq":14,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 35 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2b5406e170 msgr2=0x7f2b54070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2b5406e170 0x7f2b54070620 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f2b50005ec0 tx=0x7f2b50005fd0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b68100ac0 msgr2=0x7f2b68195580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b68100ac0 0x7f2b68195580 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000ea00 tx=0x7f2b6000eec0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 shutdown_connections 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f2b681013d0 0x7f2b68195ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.946 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f2b5406e170 0x7f2b54070620 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 --2- 172.21.15.90:0/2176750252 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f2b68100ac0 0x7f2b68195580 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 >> 172.21.15.90:0/2176750252 conn(0x7f2b680fa2f0 msgr2=0x7f2b68104100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 shutdown_connections 2024-04-26T04:54:50.947 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:50.944+0000 7f2b6f036640 1 -- 172.21.15.90:0/2176750252 wait complete. 2024-04-26T04:54:50.957 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 36 2024-04-26T04:54:51.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.053+0000 7f2403b4e640 1 -- 172.21.15.90:0/4002693457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 msgr2=0x7f23fc104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.053+0000 7f2403b4e640 1 --2- 172.21.15.90:0/4002693457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc104960 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009c20 tx=0x7f23f802c1b0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.053+0000 7f2403b4e640 1 -- 172.21.15.90:0/4002693457 shutdown_connections 2024-04-26T04:54:51.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.053+0000 7f2403b4e640 1 --2- 172.21.15.90:0/4002693457 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23fc0fe5b0 0x7f23fc0fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.053+0000 7f2403b4e640 1 --2- 172.21.15.90:0/4002693457 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc104960 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.053+0000 7f2403b4e640 1 -- 172.21.15.90:0/4002693457 >> 172.21.15.90:0/4002693457 conn(0x7f23fc0743d0 msgr2=0x7f23fc0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 -- 172.21.15.90:0/4002693457 shutdown_connections 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 -- 172.21.15.90:0/4002693457 wait complete. 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 Processor -- start 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 -- start start 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23fc0fe5b0 0x7f23fc195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc195f00 con 0x7f23fc104590 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f2403b4e640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23fc199cd0 con 0x7f23fc0fe5b0 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f24010e9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f24010e9640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44358/0 (socket says 172.21.15.90:44358) 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f24010e9640 1 -- 172.21.15.90:0/1912026123 learned_addr learned my addr 172.21.15.90:0/1912026123 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:51.058 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.054+0000 7f24018ea640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23fc0fe5b0 0x7f23fc195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f24010e9640 1 -- 172.21.15.90:0/1912026123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23fc0fe5b0 msgr2=0x7f23fc195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f24010e9640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23fc0fe5b0 0x7f23fc195330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f24010e9640 1 -- 172.21.15.90:0/1912026123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23f8009900 con 0x7f23fc104590 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f24010e9640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc195870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c8f0 tx=0x7f23ec00cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f23f2ffd640 1 -- 172.21.15.90:0/1912026123 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ec007b80 con 0x7f23fc104590 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f23f2ffd640 1 -- 172.21.15.90:0/1912026123 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f23ec0054e0 con 0x7f23fc104590 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23fc199fb0 con 0x7f23fc104590 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f23f2ffd640 1 -- 172.21.15.90:0/1912026123 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f23ec00f450 con 0x7f23fc104590 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.055+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23fc19a510 con 0x7f23fc104590 2024-04-26T04:54:51.059 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.056+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23fc04e000 con 0x7f23fc104590 2024-04-26T04:54:51.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.056+0000 7f23f2ffd640 1 -- 172.21.15.90:0/1912026123 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f23ec004f60 con 0x7f23fc104590 2024-04-26T04:54:51.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.057+0000 7f23f2ffd640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f23e806e170 0x7f23e8070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.057+0000 7f23f2ffd640 1 -- 172.21.15.90:0/1912026123 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f23ec08dff0 con 0x7f23fc104590 2024-04-26T04:54:51.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.057+0000 7f24018ea640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f23e806e170 0x7f23e8070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.060 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.057+0000 7f24018ea640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f23e806e170 0x7f23e8070620 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f23f8033000 tx=0x7f23f8005f10 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:51.062 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.062+0000 7f23f2ffd640 1 -- 172.21.15.90:0/1912026123 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f23ec0572d0 con 0x7f23fc104590 2024-04-26T04:54:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:50 smithi090 ceph-mon[86550]: pgmap v275: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:50 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1838681687' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-26T04:54:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:50 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/500507542' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-26T04:54:51.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:50 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/373771955' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-26T04:54:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:50 smithi104 ceph-mon[64778]: pgmap v275: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:50 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1838681687' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 32, "format": "json"}]: dispatch 2024-04-26T04:54:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:50 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/500507542' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 33, "format": "json"}]: dispatch 2024-04-26T04:54:51.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:50 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/373771955' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 34, "format": "json"}]: dispatch 2024-04-26T04:54:51.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.274+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 36, "format": "json"} v 0) v1 -- 0x7f23fc196680 con 0x7f23fc104590 2024-04-26T04:54:51.275 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.275+0000 7f23f2ffd640 1 -- 172.21.15.90:0/1912026123 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 36, "format": "json"}]=0 dumped fsmap epoch 36 v39) v1 ==== 107+0+5221 (secure 0 0 0) 0x7f23ec05a990 con 0x7f23fc104590 2024-04-26T04:54:51.276 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:51.276 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:41.990689+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":0,"incarnation":33,"state":"up:reconnect","state_seq":20,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":1,"incarnation":33,"state":"up:reconnect","state_seq":15,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:51.276 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 36 2024-04-26T04:54:51.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.276+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f23e806e170 msgr2=0x7f23e8070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.276+0000 7f2403b4e640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f23e806e170 0x7f23e8070620 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f23f8033000 tx=0x7f23f8005f10 comp rx=0 tx=0).stop 2024-04-26T04:54:51.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.276+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 msgr2=0x7f23fc195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.276+0000 7f2403b4e640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc195870 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f23ec00c8f0 tx=0x7f23ec00cdb0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.277+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 shutdown_connections 2024-04-26T04:54:51.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.277+0000 7f2403b4e640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f23e806e170 0x7f23e8070620 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.278 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.277+0000 7f2403b4e640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f23fc104590 0x7f23fc195870 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.277+0000 7f2403b4e640 1 --2- 172.21.15.90:0/1912026123 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f23fc0fe5b0 0x7f23fc195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.277+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 >> 172.21.15.90:0/1912026123 conn(0x7f23fc0743d0 msgr2=0x7f23fc067330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:51.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.277+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 shutdown_connections 2024-04-26T04:54:51.279 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.277+0000 7f2403b4e640 1 -- 172.21.15.90:0/1912026123 wait complete. 2024-04-26T04:54:51.289 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 37 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 -- 172.21.15.90:0/1950714300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 msgr2=0x7f51fc0fea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 --2- 172.21.15.90:0/1950714300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc0fea00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f51f0009880 tx=0x7f51f002bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 -- 172.21.15.90:0/1950714300 shutdown_connections 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 --2- 172.21.15.90:0/1950714300 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc0fea00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 --2- 172.21.15.90:0/1950714300 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc1045c0 0x7f51fc104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 -- 172.21.15.90:0/1950714300 >> 172.21.15.90:0/1950714300 conn(0x7f51fc0fa0b0 msgr2=0x7f51fc0fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 -- 172.21.15.90:0/1950714300 shutdown_connections 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 -- 172.21.15.90:0/1950714300 wait complete. 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.385+0000 7f5201c16640 1 Processor -- start 2024-04-26T04:54:51.388 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f5201c16640 1 -- start start 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f5201c16640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc193ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f5201c16640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc1045c0 0x7f51fc194400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f5201c16640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc194940 con 0x7f51fc0fe590 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f5201c16640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51fc194ab0 con 0x7f51fc1045c0 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc193ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51fb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc193ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44368/0 (socket says 172.21.15.90:44368) 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51fb7fe640 1 -- 172.21.15.90:0/1777856347 learned_addr learned my addr 172.21.15.90:0/1777856347 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51fb7fe640 1 -- 172.21.15.90:0/1777856347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc1045c0 msgr2=0x7f51fc194400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51fb7fe640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc1045c0 0x7f51fc194400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51fb7fe640 1 -- 172.21.15.90:0/1777856347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51f0009560 con 0x7f51fc0fe590 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51fb7fe640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc193ec0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d6f0 tx=0x7f51ec00dbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51f8ff9640 1 -- 172.21.15.90:0/1777856347 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ec009a30 con 0x7f51fc0fe590 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f51f8ff9640 1 -- 172.21.15.90:0/1777856347 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f51ec0054e0 con 0x7f51fc0fe590 2024-04-26T04:54:51.389 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.386+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51fc1a1530 con 0x7f51fc0fe590 2024-04-26T04:54:51.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.387+0000 7f51f8ff9640 1 -- 172.21.15.90:0/1777856347 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f51ec010460 con 0x7f51fc0fe590 2024-04-26T04:54:51.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.387+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51fc1a18a0 con 0x7f51fc0fe590 2024-04-26T04:54:51.390 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.387+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51fc0fb590 con 0x7f51fc0fe590 2024-04-26T04:54:51.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.388+0000 7f51f8ff9640 1 -- 172.21.15.90:0/1777856347 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f51ec009b90 con 0x7f51fc0fe590 2024-04-26T04:54:51.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.388+0000 7f51f8ff9640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f51e406e170 0x7f51e4070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.388+0000 7f51f8ff9640 1 -- 172.21.15.90:0/1777856347 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f51ec08dbf0 con 0x7f51fc0fe590 2024-04-26T04:54:51.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.388+0000 7f51faffd640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f51e406e170 0x7f51e4070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.391 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.389+0000 7f51faffd640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f51e406e170 0x7f51e4070620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036040 tx=0x7f51f0000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:51.394 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.393+0000 7f51f8ff9640 1 -- 172.21.15.90:0/1777856347 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f51ec056f50 con 0x7f51fc0fe590 2024-04-26T04:54:51.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.608+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 37, "format": "json"} v 0) v1 -- 0x7f51fc04e000 con 0x7f51fc0fe590 2024-04-26T04:54:51.609 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.608+0000 7f51f8ff9640 1 -- 172.21.15.90:0/1777856347 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 37, "format": "json"}]=0 dumped fsmap epoch 37 v39) v1 ==== 107+0+5215 (secure 0 0 0) 0x7f51ec020090 con 0x7f51fc0fe590 2024-04-26T04:54:51.610 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:51.610 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":37,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":37,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:42.955176+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":0,"incarnation":33,"state":"up:rejoin","state_seq":21,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":1,"incarnation":33,"state":"up:rejoin","state_seq":16,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":0,"qdb_cluster":[]},"id":1}]} 2024-04-26T04:54:51.610 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 37 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f51e406e170 msgr2=0x7f51e4070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f51e406e170 0x7f51e4070620 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f51f0036040 tx=0x7f51f0000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 msgr2=0x7f51fc193ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc193ec0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f51ec00d6f0 tx=0x7f51ec00dbb0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 shutdown_connections 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f51fc1045c0 0x7f51fc194400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f51e406e170 0x7f51e4070620 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 --2- 172.21.15.90:0/1777856347 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f51fc0fe590 0x7f51fc193ec0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.612 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 >> 172.21.15.90:0/1777856347 conn(0x7f51fc0fa0b0 msgr2=0x7f51fc0fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:51.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 shutdown_connections 2024-04-26T04:54:51.613 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.611+0000 7f5201c16640 1 -- 172.21.15.90:0/1777856347 wait complete. 2024-04-26T04:54:51.623 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 38 2024-04-26T04:54:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.721+0000 7f16a6adb640 1 -- 172.21.15.90:0/1411974870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00ff000 msgr2=0x7f16a0109a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.724 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.721+0000 7f16a6adb640 1 --2- 172.21.15.90:0/1411974870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00ff000 0x7f16a0109a70 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f16900098b0 tx=0x7f169002bcd0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.721+0000 7f16a6adb640 1 -- 172.21.15.90:0/1411974870 shutdown_connections 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.721+0000 7f16a6adb640 1 --2- 172.21.15.90:0/1411974870 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00ff000 0x7f16a0109a70 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.721+0000 7f16a6adb640 1 --2- 172.21.15.90:0/1411974870 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a00fe6f0 0x7f16a00feac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 -- 172.21.15.90:0/1411974870 >> 172.21.15.90:0/1411974870 conn(0x7f16a0073d20 msgr2=0x7f16a0074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 -- 172.21.15.90:0/1411974870 shutdown_connections 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 -- 172.21.15.90:0/1411974870 wait complete. 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 Processor -- start 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 -- start start 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00fe6f0 0x7f16a0195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a00ff000 0x7f16a01958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0195f30 con 0x7f16a00fe6f0 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a6adb640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16a0199d00 con 0x7f16a00ff000 2024-04-26T04:54:51.725 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a4877640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00fe6f0 0x7f16a0195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a4877640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00fe6f0 0x7f16a0195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44384/0 (socket says 172.21.15.90:44384) 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.722+0000 7f16a4877640 1 -- 172.21.15.90:0/2290139950 learned_addr learned my addr 172.21.15.90:0/2290139950 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f169ffff640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a00ff000 0x7f16a01958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f16a4877640 1 -- 172.21.15.90:0/2290139950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a00ff000 msgr2=0x7f16a01958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f16a4877640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a00ff000 0x7f16a01958a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f16a4877640 1 -- 172.21.15.90:0/2290139950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1690009560 con 0x7f16a00fe6f0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f16a4877640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00fe6f0 0x7f16a0195360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f169800ba50 tx=0x7f169800bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f169dffb640 1 -- 172.21.15.90:0/2290139950 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f169800cb30 con 0x7f16a00fe6f0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f169dffb640 1 -- 172.21.15.90:0/2290139950 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f169800cc90 con 0x7f16a00fe6f0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16a0199fe0 con 0x7f16a00fe6f0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f169dffb640 1 -- 172.21.15.90:0/2290139950 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f1698007740 con 0x7f16a00fe6f0 2024-04-26T04:54:51.726 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.723+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16a019a540 con 0x7f16a00fe6f0 2024-04-26T04:54:51.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.724+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16a004e000 con 0x7f16a00fe6f0 2024-04-26T04:54:51.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.724+0000 7f169dffb640 1 -- 172.21.15.90:0/2290139950 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f1698014b90 con 0x7f16a00fe6f0 2024-04-26T04:54:51.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.725+0000 7f169dffb640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f168c06e170 0x7f168c070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:51.727 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.725+0000 7f169dffb640 1 -- 172.21.15.90:0/2290139950 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f169808dbb0 con 0x7f16a00fe6f0 2024-04-26T04:54:51.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.725+0000 7f169ffff640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f168c06e170 0x7f168c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:51.728 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.725+0000 7f169ffff640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f168c06e170 0x7f168c070620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1690036040 tx=0x7f1690000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:51.730 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.729+0000 7f169dffb640 1 -- 172.21.15.90:0/2290139950 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f169805aa10 con 0x7f16a00fe6f0 2024-04-26T04:54:51.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.941+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "fs dump", "epoch": 38, "format": "json"} v 0) v1 -- 0x7f16a01070e0 con 0x7f16a00fe6f0 2024-04-26T04:54:51.942 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.941+0000 7f169dffb640 1 -- 172.21.15.90:0/2290139950 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "epoch": 38, "format": "json"}]=0 dumped fsmap epoch 38 v39) v1 ==== 107+0+5230 (secure 0 0 0) 0x7f169805a5a0 con 0x7f16a00fe6f0 2024-04-26T04:54:51.943 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:51.943 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":44291,"name":"cephfs.smithi104.wpntdi","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6825/2998394587","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6824","nonce":2998394587},{"type":"v1","addr":"172.21.15.104:6825","nonce":2998394587}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29},{"gid":44319,"name":"cephfs.smithi104.ndiusl","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.104:6827/1280197644","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6826","nonce":1280197644},{"type":"v1","addr":"172.21.15.104:6827","nonce":1280197644}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31}],"filesystems":[{"mdsmap":{"epoch":38,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-26T04:39:43.362117+0000","modified":"2024-04-26T04:52:44.956698+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":81,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":2,"in":[0,1],"up":{"mds_0":34380,"mds_1":34412},"failed":[],"damaged":[],"stopped":[],"info":{"gid_34380":{"gid":34380,"name":"cephfs.smithi090.ehputv","rank":0,"incarnation":33,"state":"up:active","state_seq":22,"addr":"172.21.15.90:6827/340161219","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6826","nonce":340161219},{"type":"v1","addr":"172.21.15.90:6827","nonce":340161219}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_34412":{"gid":34412,"name":"cephfs.smithi090.dbkjuk","rank":1,"incarnation":33,"state":"up:active","state_seq":17,"addr":"172.21.15.90:6829/291277420","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6828","nonce":291277420},{"type":"v1","addr":"172.21.15.90:6829","nonce":291277420}]},"join_fscid":1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":34380,"qdb_cluster":[34380,34412]},"id":1}]} 2024-04-26T04:54:51.943 INFO:teuthology.orchestra.run.smithi090.stderr:dumped fsmap epoch 38 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f168c06e170 msgr2=0x7f168c070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f168c06e170 0x7f168c070620 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f1690036040 tx=0x7f1690000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00fe6f0 msgr2=0x7f16a0195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00fe6f0 0x7f16a0195360 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f169800ba50 tx=0x7f169800bf10 comp rx=0 tx=0).stop 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 shutdown_connections 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f16a00ff000 0x7f16a01958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f168c06e170 0x7f168c070620 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 --2- 172.21.15.90:0/2290139950 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f16a00fe6f0 0x7f16a0195360 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 >> 172.21.15.90:0/2290139950 conn(0x7f16a0073d20 msgr2=0x7f16a01042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 shutdown_connections 2024-04-26T04:54:51.945 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:51.943+0000 7f16a6adb640 1 -- 172.21.15.90:0/2290139950 wait complete. 2024-04-26T04:54:51.956 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-26T04:54:51.970 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-26T04:54:51.970 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:54:51.970 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T04:54:51.987 DEBUG:teuthology.orchestra.run.smithi090:> set -ex 2024-04-26T04:54:51.987 DEBUG:teuthology.orchestra.run.smithi090:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T04:54:52.043 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:54:52.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/2962035584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 msgr2=0x7f9bf0104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/2962035584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0104b80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f9be0009880 tx=0x7f9be002bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/2962035584 shutdown_connections 2024-04-26T04:54:52.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/2962035584 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf00fe7a0 0x7f9bf00fec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/2962035584 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0104b80 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.182 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/2962035584 >> 172.21.15.90:0/2962035584 conn(0x7f9bf00fa2c0 msgr2=0x7f9bf00fc6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/2962035584 shutdown_connections 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.179+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/2962035584 wait complete. 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9bf5eaf640 1 Processor -- start 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9bf5eaf640 1 -- start start 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9bf5eaf640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf00fe7a0 0x7f9bf0194330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9bf5eaf640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0194870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9bf5eaf640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0194f00 con 0x7f9bf01047b0 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9bf5eaf640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bf0198cd0 con 0x7f9bf00fe7a0 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9beeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0194870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9beeffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0194870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44398/0 (socket says 172.21.15.90:44398) 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9beeffd640 1 -- 172.21.15.90:0/3199353062 learned_addr learned my addr 172.21.15.90:0/3199353062 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9bef7fe640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf00fe7a0 0x7f9bf0194330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9beeffd640 1 -- 172.21.15.90:0/3199353062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf00fe7a0 msgr2=0x7f9bf0194330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9beeffd640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf00fe7a0 0x7f9bf0194330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.180+0000 7f9beeffd640 1 -- 172.21.15.90:0/3199353062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be0009560 con 0x7f9bf01047b0 2024-04-26T04:54:52.183 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.181+0000 7f9beeffd640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0194870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9be400c810 tx=0x7f9be400ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:52.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.181+0000 7f9becff9640 1 -- 172.21.15.90:0/3199353062 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be40041f0 con 0x7f9bf01047b0 2024-04-26T04:54:52.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.181+0000 7f9becff9640 1 -- 172.21.15.90:0/3199353062 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f9be4007e70 con 0x7f9bf01047b0 2024-04-26T04:54:52.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.181+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bf0198fb0 con 0x7f9bf01047b0 2024-04-26T04:54:52.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.181+0000 7f9becff9640 1 -- 172.21.15.90:0/3199353062 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f9be40049d0 con 0x7f9bf01047b0 2024-04-26T04:54:52.184 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.181+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bf0199590 con 0x7f9bf01047b0 2024-04-26T04:54:52.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.182+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bf004e000 con 0x7f9bf01047b0 2024-04-26T04:54:52.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.182+0000 7f9becff9640 1 -- 172.21.15.90:0/3199353062 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f9be40079b0 con 0x7f9bf01047b0 2024-04-26T04:54:52.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.183+0000 7f9becff9640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9bd806e230 0x7f9bd80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.183+0000 7f9becff9640 1 -- 172.21.15.90:0/3199353062 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f9be408dfc0 con 0x7f9bf01047b0 2024-04-26T04:54:52.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.183+0000 7f9bef7fe640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9bd806e230 0x7f9bd80706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.185 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.183+0000 7f9bef7fe640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9bd806e230 0x7f9bd80706e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9be0036040 tx=0x7f9be0005fd0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:52.188 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.187+0000 7f9becff9640 1 -- 172.21.15.90:0/3199353062 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f9be40572a0 con 0x7f9bf01047b0 2024-04-26T04:54:52.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:51 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2176750252' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-26T04:54:52.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:51 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1912026123' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-26T04:54:52.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:51 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1777856347' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-26T04:54:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:51 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2176750252' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 35, "format": "json"}]: dispatch 2024-04-26T04:54:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:51 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1912026123' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 36, "format": "json"}]: dispatch 2024-04-26T04:54:52.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:51 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1777856347' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 37, "format": "json"}]: dispatch 2024-04-26T04:54:52.365 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.365+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bf010b970 con 0x7f9bf01047b0 2024-04-26T04:54:52.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.366+0000 7f9becff9640 1 -- 172.21.15.90:0/3199353062 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16775 (secure 0 0 0) 0x7f9be40791a0 con 0x7f9bf01047b0 2024-04-26T04:54:52.367 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:52.367 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":85,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","created":"2024-04-26T04:32:57.928472+0000","modified":"2024-04-26T04:52:17.939202+0000","last_up_change":"2024-04-26T04:51:19.550115+0000","last_in_change":"2024-04-26T04:38:13.654124+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:37:19.554153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:39:42.011282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:39:42.594560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57491e88-0d1c-41ac-a1f5-3662e5c0a467","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6803","nonce":2303726208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6805","nonce":2303726208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6809","nonce":2303726208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6807","nonce":2303726208}]},"public_addr":"172.21.15.90:6803/2303726208","cluster_addr":"172.21.15.90:6805/2303726208","heartbeat_back_addr":"172.21.15.90:6809/2303726208","heartbeat_front_addr":"172.21.15.90:6807/2303726208","state":["exists","up"]},{"osd":1,"uuid":"fe237f1b-a589-452c-8e5e-98a9a31593eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6811","nonce":2770756421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6813","nonce":2770756421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6817","nonce":2770756421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6815","nonce":2770756421}]},"public_addr":"172.21.15.90:6811/2770756421","cluster_addr":"172.21.15.90:6813/2770756421","heartbeat_back_addr":"172.21.15.90:6817/2770756421","heartbeat_front_addr":"172.21.15.90:6815/2770756421","state":["exists","up"]},{"osd":2,"uuid":"7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6819","nonce":1256593709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6821","nonce":1256593709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6825","nonce":1256593709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6823","nonce":1256593709}]},"public_addr":"172.21.15.90:6819/1256593709","cluster_addr":"172.21.15.90:6821/1256593709","heartbeat_back_addr":"172.21.15.90:6825/1256593709","heartbeat_front_addr":"172.21.15.90:6823/1256593709","state":["exists","up"]},{"osd":3,"uuid":"62adc04c-9e7e-4ee5-bb5f-42004b2a615d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6801","nonce":2129064810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6803","nonce":2129064810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6807","nonce":2129064810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6805","nonce":2129064810}]},"public_addr":"172.21.15.104:6801/2129064810","cluster_addr":"172.21.15.104:6803/2129064810","heartbeat_back_addr":"172.21.15.104:6807/2129064810","heartbeat_front_addr":"172.21.15.104:6805/2129064810","state":["exists","up"]},{"osd":4,"uuid":"a4ef2729-84e1-4d2d-868b-3da5fefeb800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6809","nonce":319614461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6811","nonce":319614461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6815","nonce":319614461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6813","nonce":319614461}]},"public_addr":"172.21.15.104:6809/319614461","cluster_addr":"172.21.15.104:6811/319614461","heartbeat_back_addr":"172.21.15.104:6815/319614461","heartbeat_front_addr":"172.21.15.104:6813/319614461","state":["exists","up"]},{"osd":5,"uuid":"2fa80b6f-ab69-4e6e-9a5f-7e73a898c300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6817","nonce":1501294989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6819","nonce":1501294989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6823","nonce":1501294989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6821","nonce":1501294989}]},"public_addr":"172.21.15.104:6817/1501294989","cluster_addr":"172.21.15.104:6819/1501294989","heartbeat_back_addr":"172.21.15.104:6823/1501294989","heartbeat_front_addr":"172.21.15.104:6821/1501294989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:19.861796+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:48.482295+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:17.028265+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:40.489573+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:05.503518+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:30.925548+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6828/370183803":"2024-04-27T04:51:20.563404+0000","172.21.15.90:6800/1235211382":"2024-04-27T04:43:18.835636+0000","172.21.15.104:6824/1058172006":"2024-04-27T04:51:20.563430+0000","172.21.15.90:0/748209074":"2024-04-27T04:33:57.088900+0000","172.21.15.104:6829/680795988":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6801/1235211382":"2024-04-27T04:43:18.835636+0000","172.21.15.90:0/2705188709":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6801/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.104:0/2711199308":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/1814853898":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/4265628059":"2024-04-27T04:46:55.630300+0000","172.21.15.90:0/2763746313":"2024-04-27T04:43:18.835636+0000","172.21.15.90:0/2473529454":"2024-04-27T04:33:24.670558+0000","172.21.15.104:6828/680795988":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6800/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/2342166800":"2024-04-27T04:33:57.088900+0000","172.21.15.104:0/1222914910":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6829/370183803":"2024-04-27T04:51:20.563404+0000","172.21.15.104:0/2266011983":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/3418315403":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2310050885":"2024-04-27T04:33:57.088900+0000","172.21.15.90:0/2803863568":"2024-04-27T04:46:55.630300+0000","172.21.15.90:0/424889384":"2024-04-27T04:33:24.670558+0000","172.21.15.90:6800/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.104:6825/1058172006":"2024-04-27T04:51:20.563430+0000","172.21.15.90:0/2846076831":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6801/3458990238":"2024-04-27T04:46:55.630300+0000","172.21.15.90:6801/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6801/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/3214112922":"2024-04-27T04:35:03.230209+0000","172.21.15.104:0/3572733912":"2024-04-27T04:44:00.133458+0000","172.21.15.104:0/3577745345":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/50635771":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/4285557765":"2024-04-27T04:46:55.630300+0000","172.21.15.104:0/204837581":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6826/3871906279":"2024-04-27T04:40:00.786348+0000","172.21.15.90:0/1198706928":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6827/3871906279":"2024-04-27T04:40:00.786348+0000","172.21.15.90:0/1376440705":"2024-04-27T04:46:55.630300+0000","172.21.15.90:6800/3458990238":"2024-04-27T04:46:55.630300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9bd806e230 msgr2=0x7f9bd80706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9bd806e230 0x7f9bd80706e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f9be0036040 tx=0x7f9be0005fd0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 msgr2=0x7f9bf0194870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0194870 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f9be400c810 tx=0x7f9be400ccd0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 shutdown_connections 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f9bd806e230 0x7f9bd80706e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f9bf01047b0 0x7f9bf0194870 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 --2- 172.21.15.90:0/3199353062 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f9bf00fe7a0 0x7f9bf0194330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 >> 172.21.15.90:0/3199353062 conn(0x7f9bf00fa2c0 msgr2=0x7f9bf00fbaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 shutdown_connections 2024-04-26T04:54:52.370 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.368+0000 7f9bf5eaf640 1 -- 172.21.15.90:0/3199353062 wait complete. 2024-04-26T04:54:52.380 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:37:19.554153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:39:42.011282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:39:42.594560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T04:54:52.380 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 -- 172.21.15.90:0/3920165707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 msgr2=0x7fd2fc0ffce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 --2- 172.21.15.90:0/3920165707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc0ffce0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec009880 tx=0x7fd2ec02bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 -- 172.21.15.90:0/3920165707 shutdown_connections 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 --2- 172.21.15.90:0/3920165707 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc0ffce0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 --2- 172.21.15.90:0/3920165707 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2fc105880 0x7fd2fc105c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 -- 172.21.15.90:0/3920165707 >> 172.21.15.90:0/3920165707 conn(0x7fd2fc0fb370 msgr2=0x7fd2fc0fd780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 -- 172.21.15.90:0/3920165707 shutdown_connections 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.479+0000 7fd303924640 1 -- 172.21.15.90:0/3920165707 wait complete. 2024-04-26T04:54:52.482 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd303924640 1 Processor -- start 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd303924640 1 -- start start 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd303924640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc195560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd303924640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2fc105880 0x7fd2fc195aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd303924640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc1960a0 con 0x7fd2fc0ff870 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd303924640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2fc196210 con 0x7fd2fc105880 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd3016c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc195560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd3016c0640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc195560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44418/0 (socket says 172.21.15.90:44418) 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd3016c0640 1 -- 172.21.15.90:0/3927402408 learned_addr learned my addr 172.21.15.90:0/3927402408 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd3016c0640 1 -- 172.21.15.90:0/3927402408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2fc105880 msgr2=0x7fd2fc195aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd3016c0640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2fc105880 0x7fd2fc195aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd3016c0640 1 -- 172.21.15.90:0/3927402408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2ec009560 con 0x7fd2fc0ff870 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.480+0000 7fd3016c0640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc195560 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c8f0 tx=0x7fd2f800cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.481+0000 7fd2f27fc640 1 -- 172.21.15.90:0/3927402408 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f80041f0 con 0x7fd2fc0ff870 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.481+0000 7fd2f27fc640 1 -- 172.21.15.90:0/3927402408 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fd2f8007e10 con 0x7fd2fc0ff870 2024-04-26T04:54:52.483 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.481+0000 7fd2f27fc640 1 -- 172.21.15.90:0/3927402408 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fd2f800f450 con 0x7fd2fc0ff870 2024-04-26T04:54:52.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.481+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2fc19a0a0 con 0x7fd2fc0ff870 2024-04-26T04:54:52.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.481+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2fc19a600 con 0x7fd2fc0ff870 2024-04-26T04:54:52.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.482+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2fc04e000 con 0x7fd2fc0ff870 2024-04-26T04:54:52.484 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.482+0000 7fd2f27fc640 1 -- 172.21.15.90:0/3927402408 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fd2f8007950 con 0x7fd2fc0ff870 2024-04-26T04:54:52.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.482+0000 7fd2f27fc640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd2e806e170 0x7fd2e8070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.483+0000 7fd300ebf640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd2e806e170 0x7fd2e8070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.483+0000 7fd2f27fc640 1 -- 172.21.15.90:0/3927402408 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fd2f8013070 con 0x7fd2fc0ff870 2024-04-26T04:54:52.485 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.483+0000 7fd300ebf640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd2e806e170 0x7fd2e8070620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec036040 tx=0x7fd2ec000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:52.488 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.487+0000 7fd2f27fc640 1 -- 172.21.15.90:0/3927402408 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fd2f80575c0 con 0x7fd2fc0ff870 2024-04-26T04:54:52.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.660+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd2fc10ca40 con 0x7fd2fc0ff870 2024-04-26T04:54:52.662 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.661+0000 7fd2f27fc640 1 -- 172.21.15.90:0/3927402408 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd2f805ac80 con 0x7fd2fc0ff870 2024-04-26T04:54:52.662 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-26T04:54:52.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd2e806e170 msgr2=0x7fd2e8070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd2e806e170 0x7fd2e8070620 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fd2ec036040 tx=0x7fd2ec000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 msgr2=0x7fd2fc195560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc195560 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fd2f800c8f0 tx=0x7fd2f800cdb0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.664 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 shutdown_connections 2024-04-26T04:54:52.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fd2fc105880 0x7fd2fc195aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fd2e806e170 0x7fd2e8070620 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 --2- 172.21.15.90:0/3927402408 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fd2fc0ff870 0x7fd2fc195560 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 >> 172.21.15.90:0/3927402408 conn(0x7fd2fc0fb370 msgr2=0x7fd2fc0fcd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:52.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 shutdown_connections 2024-04-26T04:54:52.665 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.663+0000 7fd303924640 1 -- 172.21.15.90:0/3927402408 wait complete. 2024-04-26T04:54:52.675 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T04:54:52.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 -- 172.21.15.90:0/1570492695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec067350 msgr2=0x7f0fec0677c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1570492695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec067350 0x7f0fec0677c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009880 tx=0x7f0fe002bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.775 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 -- 172.21.15.90:0/1570492695 shutdown_connections 2024-04-26T04:54:52.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1570492695 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec067350 0x7f0fec0677c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1570492695 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fec105690 0x7f0fec105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 -- 172.21.15.90:0/1570492695 >> 172.21.15.90:0/1570492695 conn(0x7f0fec0743d0 msgr2=0x7f0fec0747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:52.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 -- 172.21.15.90:0/1570492695 shutdown_connections 2024-04-26T04:54:52.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 -- 172.21.15.90:0/1570492695 wait complete. 2024-04-26T04:54:52.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 Processor -- start 2024-04-26T04:54:52.776 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.774+0000 7f0ff2386640 1 -- start start 2024-04-26T04:54:52.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0ff2386640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fec067350 0x7f0fec1952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0ff2386640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec105690 0x7f0fec1957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.778 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0ff2386640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec195e80 con 0x7f0fec105690 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0ff2386640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fec199c50 con 0x7f0fec067350 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0feb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec105690 0x7f0fec1957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0feb7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec105690 0x7f0fec1957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44434/0 (socket says 172.21.15.90:44434) 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0feb7fe640 1 -- 172.21.15.90:0/1022934978 learned_addr learned my addr 172.21.15.90:0/1022934978 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0febfff640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fec067350 0x7f0fec1952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0feb7fe640 1 -- 172.21.15.90:0/1022934978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fec067350 msgr2=0x7f0fec1952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0feb7fe640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fec067350 0x7f0fec1952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0feb7fe640 1 -- 172.21.15.90:0/1022934978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fe0009560 con 0x7f0fec105690 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0febfff640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fec067350 0x7f0fec1952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0feb7fe640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec105690 0x7f0fec1957f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009230 tx=0x7f0fe0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0fe97fa640 1 -- 172.21.15.90:0/1022934978 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe00035f0 con 0x7f0fec105690 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fec199ed0 con 0x7f0fec105690 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0fe97fa640 1 -- 172.21.15.90:0/1022934978 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f0fe0003c10 con 0x7f0fec105690 2024-04-26T04:54:52.779 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0fe97fa640 1 -- 172.21.15.90:0/1022934978 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f0fe0034440 con 0x7f0fec105690 2024-04-26T04:54:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.775+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fec19a450 con 0x7f0fec105690 2024-04-26T04:54:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.777+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fec04e000 con 0x7f0fec105690 2024-04-26T04:54:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.777+0000 7f0fe97fa640 1 -- 172.21.15.90:0/1022934978 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f0fe0004fd0 con 0x7f0fec105690 2024-04-26T04:54:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.777+0000 7f0fe97fa640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0fd406e170 0x7f0fd4070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.777+0000 7f0fe97fa640 1 -- 172.21.15.90:0/1022934978 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f0fe00b0620 con 0x7f0fec105690 2024-04-26T04:54:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.777+0000 7f0febfff640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0fd406e170 0x7f0fd4070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:52.780 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.778+0000 7f0febfff640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0fd406e170 0x7f0fd4070620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc005fd0 tx=0x7f0fdc005ea0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:52.783 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.782+0000 7f0fe97fa640 1 -- 172.21.15.90:0/1022934978 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f0fe0079900 con 0x7f0fec105690 2024-04-26T04:54:52.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.958+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f0fec196600 con 0x7f0fec105690 2024-04-26T04:54:52.959 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.958+0000 7f0fe97fa640 1 -- 172.21.15.90:0/1022934978 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f0fe007cfc0 con 0x7f0fec105690 2024-04-26T04:54:52.959 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:54:52.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0fd406e170 msgr2=0x7f0fd4070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0fd406e170 0x7f0fd4070620 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc005fd0 tx=0x7f0fdc005ea0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec105690 msgr2=0x7f0fec1957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:52.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec105690 0x7f0fec1957f0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f0fe0009230 tx=0x7f0fe0004040 comp rx=0 tx=0).stop 2024-04-26T04:54:52.961 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 shutdown_connections 2024-04-26T04:54:52.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f0fd406e170 0x7f0fd4070620 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f0fec105690 0x7f0fec1957f0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 --2- 172.21.15.90:0/1022934978 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f0fec067350 0x7f0fec1952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:52.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 >> 172.21.15.90:0/1022934978 conn(0x7f0fec0743d0 msgr2=0x7f0fec0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:52.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 shutdown_connections 2024-04-26T04:54:52.962 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:52.960+0000 7f0ff2386640 1 -- 172.21.15.90:0/1022934978 wait complete. 2024-04-26T04:54:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:52 smithi090 ceph-mon[86550]: pgmap v276: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:52 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2290139950' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-26T04:54:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:52 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3199353062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:54:52.971 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:52 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3927402408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:54:52.972 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 -- 172.21.15.90:0/3413555200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a0104800 msgr2=0x7f79a0104bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 --2- 172.21.15.90:0/3413555200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a0104800 0x7f79a0104bd0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7988009880 tx=0x7f798802bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 -- 172.21.15.90:0/3413555200 shutdown_connections 2024-04-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 --2- 172.21.15.90:0/3413555200 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a00fe7f0 0x7f79a00fec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.072 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 --2- 172.21.15.90:0/3413555200 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a0104800 0x7f79a0104bd0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 -- 172.21.15.90:0/3413555200 >> 172.21.15.90:0/3413555200 conn(0x7f79a00fa2f0 msgr2=0x7f79a00fc700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 -- 172.21.15.90:0/3413555200 shutdown_connections 2024-04-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 -- 172.21.15.90:0/3413555200 wait complete. 2024-04-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 Processor -- start 2024-04-26T04:54:53.073 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.071+0000 7f79a66ec640 1 -- start start 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f79a66ec640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a00fe7f0 0x7f79a0196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f79a66ec640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a0104800 0x7f79a01968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f79a66ec640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a0196e80 con 0x7f79a00fe7f0 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f79a66ec640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79a0196ff0 con 0x7f79a0104800 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a00fe7f0 0x7f79a0196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799ffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a00fe7f0 0x7f79a0196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44444/0 (socket says 172.21.15.90:44444) 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799ffff640 1 -- 172.21.15.90:0/462397131 learned_addr learned my addr 172.21.15.90:0/462397131 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799f7fe640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a0104800 0x7f79a01968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799ffff640 1 -- 172.21.15.90:0/462397131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a0104800 msgr2=0x7f79a01968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799ffff640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a0104800 0x7f79a01968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.075 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799ffff640 1 -- 172.21.15.90:0/462397131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7988009560 con 0x7f79a00fe7f0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799ffff640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a00fe7f0 0x7f79a0196370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f798802c220 tx=0x7f7988004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f799d7fa640 1 -- 172.21.15.90:0/462397131 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f79880035f0 con 0x7f79a00fe7f0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79a019f3e0 con 0x7f79a00fe7f0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.072+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79a019f880 con 0x7f79a00fe7f0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.073+0000 7f799d7fa640 1 -- 172.21.15.90:0/462397131 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7988003c10 con 0x7f79a00fe7f0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.073+0000 7f799d7fa640 1 -- 172.21.15.90:0/462397131 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7988034440 con 0x7f79a00fe7f0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.073+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7980005280 con 0x7f79a00fe7f0 2024-04-26T04:54:53.076 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.074+0000 7f799d7fa640 1 -- 172.21.15.90:0/462397131 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f79880345a0 con 0x7f79a00fe7f0 2024-04-26T04:54:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.074+0000 7f799d7fa640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f798c06e170 0x7f798c070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.074+0000 7f799d7fa640 1 -- 172.21.15.90:0/462397131 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f79880b04a0 con 0x7f79a00fe7f0 2024-04-26T04:54:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.074+0000 7f799f7fe640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f798c06e170 0x7f798c070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.077 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.075+0000 7f799f7fe640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f798c06e170 0x7f798c070620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7990005fd0 tx=0x7f7990005e20 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:53.080 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.079+0000 7f799d7fa640 1 -- 172.21.15.90:0/462397131 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f7988079780 con 0x7f79a00fe7f0 2024-04-26T04:54:53.253 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.252+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f7980005ab0 con 0x7f79a00fe7f0 2024-04-26T04:54:53.254 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.253+0000 7f799d7fa640 1 -- 172.21.15.90:0/462397131 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f798807ce40 con 0x7f79a00fe7f0 2024-04-26T04:54:53.254 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.254+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f798c06e170 msgr2=0x7f798c070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.254+0000 7f79a66ec640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f798c06e170 0x7f798c070620 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f7990005fd0 tx=0x7f7990005e20 comp rx=0 tx=0).stop 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a00fe7f0 msgr2=0x7f79a0196370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a00fe7f0 0x7f79a0196370 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f798802c220 tx=0x7f7988004040 comp rx=0 tx=0).stop 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 shutdown_connections 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f79a0104800 0x7f79a01968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f798c06e170 0x7f798c070620 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 --2- 172.21.15.90:0/462397131 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f79a00fe7f0 0x7f79a0196370 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 >> 172.21.15.90:0/462397131 conn(0x7f79a00fa2f0 msgr2=0x7f79a00fbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:53.256 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 shutdown_connections 2024-04-26T04:54:53.257 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.255+0000 7f79a66ec640 1 -- 172.21.15.90:0/462397131 wait complete. 2024-04-26T04:54:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:52 smithi104 ceph-mon[64778]: pgmap v276: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:52 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2290139950' entity='client.admin' cmd=[{"prefix": "fs dump", "epoch": 38, "format": "json"}]: dispatch 2024-04-26T04:54:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:52 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3199353062' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:54:53.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:52 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3927402408' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:54:53.267 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T04:54:53.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 -- 172.21.15.90:0/2782780555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 msgr2=0x7f20d00fea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 --2- 172.21.15.90:0/2782780555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d00fea10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f20c4009880 tx=0x7f20c402bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 -- 172.21.15.90:0/2782780555 shutdown_connections 2024-04-26T04:54:53.366 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 --2- 172.21.15.90:0/2782780555 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d00fea10 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 --2- 172.21.15.90:0/2782780555 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20d01045d0 0x7f20d01049a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 -- 172.21.15.90:0/2782780555 >> 172.21.15.90:0/2782780555 conn(0x7f20d0075270 msgr2=0x7f20d0075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 -- 172.21.15.90:0/2782780555 shutdown_connections 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.363+0000 7f20d6b55640 1 -- 172.21.15.90:0/2782780555 wait complete. 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d6b55640 1 Processor -- start 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d6b55640 1 -- start start 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d6b55640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d0195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d6b55640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20d01045d0 0x7f20d01957a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d6b55640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0195e30 con 0x7f20d00fe5a0 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d6b55640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20d0199c00 con 0x7f20d01045d0 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d48f1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d0195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d48f1640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d0195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44466/0 (socket says 172.21.15.90:44466) 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.364+0000 7f20d48f1640 1 -- 172.21.15.90:0/3755226368 learned_addr learned my addr 172.21.15.90:0/3755226368 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:53.367 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20d48f1640 1 -- 172.21.15.90:0/3755226368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20d01045d0 msgr2=0x7f20d01957a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20d48f1640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20d01045d0 0x7f20d01957a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20d48f1640 1 -- 172.21.15.90:0/3755226368 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20c4009560 con 0x7f20d00fe5a0 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20d48f1640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d0195260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b420 tx=0x7f20c000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20cdffb640 1 -- 172.21.15.90:0/3755226368 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20c0011800 con 0x7f20d00fe5a0 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20cdffb640 1 -- 172.21.15.90:0/3755226368 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f20c0011e20 con 0x7f20d00fe5a0 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20d0199ee0 con 0x7f20d00fe5a0 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20cdffb640 1 -- 172.21.15.90:0/3755226368 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f20c000f2e0 con 0x7f20d00fe5a0 2024-04-26T04:54:53.368 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.365+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20d019a4c0 con 0x7f20d00fe5a0 2024-04-26T04:54:53.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.366+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20d004e000 con 0x7f20d00fe5a0 2024-04-26T04:54:53.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.366+0000 7f20cdffb640 1 -- 172.21.15.90:0/3755226368 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f20c0010380 con 0x7f20d00fe5a0 2024-04-26T04:54:53.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.367+0000 7f20cdffb640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f20b806e230 0x7f20b80706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.367+0000 7f20cdffb640 1 -- 172.21.15.90:0/3755226368 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f20c008daf0 con 0x7f20d00fe5a0 2024-04-26T04:54:53.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.367+0000 7f20cffff640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f20b806e230 0x7f20b80706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.369 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.367+0000 7f20cffff640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f20b806e230 0x7f20b80706e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20c4036040 tx=0x7f20c4000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:53.372 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.371+0000 7f20cdffb640 1 -- 172.21.15.90:0/3755226368 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f20c0056dd0 con 0x7f20d00fe5a0 2024-04-26T04:54:53.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.560+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f20d010b950 con 0x7f20d00fe5a0 2024-04-26T04:54:53.561 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.560+0000 7f20cdffb640 1 -- 172.21.15.90:0/3755226368 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v85) v1 ==== 81+0+2433 (secure 0 0 0) 0x7f20c005a490 con 0x7f20d00fe5a0 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6828/370183803 2024-04-27T04:51:20.563404+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1235211382 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6824/1058172006 2024-04-27T04:51:20.563430+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/748209074 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6829/680795988 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1235211382 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2705188709 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1230348805 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/2711199308 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1814853898 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:53.562 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/4265628059 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2763746313 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2473529454 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6828/680795988 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1230348805 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2342166800 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/1222914910 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6829/370183803 2024-04-27T04:51:20.563404+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/2266011983 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3418315403 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2310050885 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2803863568 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/424889384 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2176954768 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:53.563 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6825/1058172006 2024-04-27T04:51:20.563430+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2846076831 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3458990238 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2176954768 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2647874706 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2647874706 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3214112922 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/3572733912 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/3577745345 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/50635771 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/4285557765 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/204837581 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6826/3871906279 2024-04-27T04:40:00.786348+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1198706928 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6827/3871906279 2024-04-27T04:40:00.786348+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1376440705 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:53.564 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3458990238 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:53.565 INFO:teuthology.orchestra.run.smithi090.stderr:listed 41 entries 2024-04-26T04:54:53.565 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f20b806e230 msgr2=0x7f20b80706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f20b806e230 0x7f20b80706e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f20c4036040 tx=0x7f20c4000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 msgr2=0x7f20d0195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d0195260 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f20c000b420 tx=0x7f20c000b8e0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 shutdown_connections 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f20d01045d0 0x7f20d01957a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f20b806e230 0x7f20b80706e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 --2- 172.21.15.90:0/3755226368 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f20d00fe5a0 0x7f20d0195260 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 >> 172.21.15.90:0/3755226368 conn(0x7f20d0075270 msgr2=0x7f20d00fd100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 shutdown_connections 2024-04-26T04:54:53.566 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.562+0000 7f20d6b55640 1 -- 172.21.15.90:0/3755226368 wait complete. 2024-04-26T04:54:53.575 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-26T04:54:53.575 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:53.575 DEBUG:teuthology.orchestra.run.smithi090:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-26T04:54:53.599 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:54:53.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:53.599 DEBUG:teuthology.orchestra.run.smithi090:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-26T04:54:53.678 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:54:53.678 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T04:54:53.699 DEBUG:teuthology.orchestra.run.smithi104:> set -ex 2024-04-26T04:54:53.699 DEBUG:teuthology.orchestra.run.smithi104:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-26T04:54:53.758 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-26T04:54:53.856 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 -- 172.21.15.90:0/1481073630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9501045d0 msgr2=0x7fa9501049a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 --2- 172.21.15.90:0/1481073630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9501045d0 0x7fa9501049a0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fa940009880 tx=0x7fa94002bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 -- 172.21.15.90:0/1481073630 shutdown_connections 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 --2- 172.21.15.90:0/1481073630 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9500fe5a0 0x7fa9500fea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 --2- 172.21.15.90:0/1481073630 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9501045d0 0x7fa9501049a0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 -- 172.21.15.90:0/1481073630 >> 172.21.15.90:0/1481073630 conn(0x7fa950075270 msgr2=0x7fa950075670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 -- 172.21.15.90:0/1481073630 shutdown_connections 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 -- 172.21.15.90:0/1481073630 wait complete. 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.855+0000 7fa955af4640 1 Processor -- start 2024-04-26T04:54:53.857 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa955af4640 1 -- start start 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa955af4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9500fe5a0 0x7fa9501940d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa955af4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9501045d0 0x7fa950194610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa955af4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950194ca0 con 0x7fa9500fe5a0 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa955af4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa950198a70 con 0x7fa9501045d0 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9500fe5a0 0x7fa9501940d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94f7fe640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9500fe5a0 0x7fa9501940d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44472/0 (socket says 172.21.15.90:44472) 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94f7fe640 1 -- 172.21.15.90:0/169777665 learned_addr learned my addr 172.21.15.90:0/169777665 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94effd640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9501045d0 0x7fa950194610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94f7fe640 1 -- 172.21.15.90:0/169777665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9501045d0 msgr2=0x7fa950194610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94f7fe640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9501045d0 0x7fa950194610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94f7fe640 1 -- 172.21.15.90:0/169777665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa940009560 con 0x7fa9500fe5a0 2024-04-26T04:54:53.860 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.856+0000 7fa94f7fe640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9500fe5a0 0x7fa9501940d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa940009230 tx=0x7fa940004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.857+0000 7fa94cff9640 1 -- 172.21.15.90:0/169777665 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa9400035f0 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.857+0000 7fa94cff9640 1 -- 172.21.15.90:0/169777665 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7fa940003c10 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.857+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa950198cf0 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.857+0000 7fa94cff9640 1 -- 172.21.15.90:0/169777665 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7fa940005670 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.857+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa950199270 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.858+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95004e000 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.858+0000 7fa94cff9640 1 -- 172.21.15.90:0/169777665 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7fa940004dd0 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.858+0000 7fa94cff9640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa93806e1d0 0x7fa938070680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.859+0000 7fa94cff9640 1 -- 172.21.15.90:0/169777665 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7fa9400b0590 con 0x7fa9500fe5a0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.859+0000 7fa94effd640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa93806e1d0 0x7fa938070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:53.861 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.859+0000 7fa94effd640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa93806e1d0 0x7fa938070680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa944005fd0 tx=0x7fa944005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:53.864 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:53.863+0000 7fa94cff9640 1 -- 172.21.15.90:0/169777665 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7fa940079870 con 0x7fa9500fe5a0 2024-04-26T04:54:54.043 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.042+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9500008d0 con 0x7fa9500fe5a0 2024-04-26T04:54:54.044 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.043+0000 7fa94cff9640 1 -- 172.21.15.90:0/169777665 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) v1 ==== 74+0+16775 (secure 0 0 0) 0x7fa94007cf30 con 0x7fa9500fe5a0 2024-04-26T04:54:54.044 INFO:teuthology.orchestra.run.smithi090.stdout: 2024-04-26T04:54:54.045 INFO:teuthology.orchestra.run.smithi090.stdout:{"epoch":85,"fsid":"f018e902-0385-11ef-bc93-c7b262605968","created":"2024-04-26T04:32:57.928472+0000","modified":"2024-04-26T04:52:17.939202+0000","last_up_change":"2024-04-26T04:51:19.550115+0000","last_in_change":"2024-04-26T04:38:13.654124+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":20,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":6,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-26T04:37:19.554153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":6,"score_stable":6,"optimal_score":0.5,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs.cephfs.meta","create_time":"2024-04-26T04:39:42.011282+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.690000057220459,"score_stable":1.690000057220459,"optimal_score":1,"raw_score_acting":1.690000057220459,"raw_score_stable":1.690000057220459,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs.cephfs.data","create_time":"2024-04-26T04:39:42.594560+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"45","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"57491e88-0d1c-41ac-a1f5-3662e5c0a467","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":9,"last_clean_end":50,"up_from":54,"up_thru":79,"down_at":51,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6802","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6803","nonce":2303726208}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6804","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6805","nonce":2303726208}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6808","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6809","nonce":2303726208}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6806","nonce":2303726208},{"type":"v1","addr":"172.21.15.90:6807","nonce":2303726208}]},"public_addr":"172.21.15.90:6803/2303726208","cluster_addr":"172.21.15.90:6805/2303726208","heartbeat_back_addr":"172.21.15.90:6809/2303726208","heartbeat_front_addr":"172.21.15.90:6807/2303726208","state":["exists","up"]},{"osd":1,"uuid":"fe237f1b-a589-452c-8e5e-98a9a31593eb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":14,"last_clean_end":55,"up_from":59,"up_thru":79,"down_at":56,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6810","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6811","nonce":2770756421}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6812","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6813","nonce":2770756421}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6816","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6817","nonce":2770756421}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6814","nonce":2770756421},{"type":"v1","addr":"172.21.15.90:6815","nonce":2770756421}]},"public_addr":"172.21.15.90:6811/2770756421","cluster_addr":"172.21.15.90:6813/2770756421","heartbeat_back_addr":"172.21.15.90:6817/2770756421","heartbeat_front_addr":"172.21.15.90:6815/2770756421","state":["exists","up"]},{"osd":2,"uuid":"7a8ea94c-6ca4-4e1e-aa2b-3bdf069f2cc2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":19,"last_clean_end":60,"up_from":64,"up_thru":79,"down_at":61,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6818","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6819","nonce":1256593709}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6820","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6821","nonce":1256593709}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6824","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6825","nonce":1256593709}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.90:6822","nonce":1256593709},{"type":"v1","addr":"172.21.15.90:6823","nonce":1256593709}]},"public_addr":"172.21.15.90:6819/1256593709","cluster_addr":"172.21.15.90:6821/1256593709","heartbeat_back_addr":"172.21.15.90:6825/1256593709","heartbeat_front_addr":"172.21.15.90:6823/1256593709","state":["exists","up"]},{"osd":3,"uuid":"62adc04c-9e7e-4ee5-bb5f-42004b2a615d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":26,"last_clean_end":65,"up_from":69,"up_thru":79,"down_at":66,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6800","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6801","nonce":2129064810}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6802","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6803","nonce":2129064810}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6806","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6807","nonce":2129064810}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6804","nonce":2129064810},{"type":"v1","addr":"172.21.15.104:6805","nonce":2129064810}]},"public_addr":"172.21.15.104:6801/2129064810","cluster_addr":"172.21.15.104:6803/2129064810","heartbeat_back_addr":"172.21.15.104:6807/2129064810","heartbeat_front_addr":"172.21.15.104:6805/2129064810","state":["exists","up"]},{"osd":4,"uuid":"a4ef2729-84e1-4d2d-868b-3da5fefeb800","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":34,"last_clean_end":70,"up_from":74,"up_thru":79,"down_at":71,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6808","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6809","nonce":319614461}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6810","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6811","nonce":319614461}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6814","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6815","nonce":319614461}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6812","nonce":319614461},{"type":"v1","addr":"172.21.15.104:6813","nonce":319614461}]},"public_addr":"172.21.15.104:6809/319614461","cluster_addr":"172.21.15.104:6811/319614461","heartbeat_back_addr":"172.21.15.104:6815/319614461","heartbeat_front_addr":"172.21.15.104:6813/319614461","state":["exists","up"]},{"osd":5,"uuid":"2fa80b6f-ab69-4e6e-9a5f-7e73a898c300","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":39,"last_clean_end":75,"up_from":79,"up_thru":79,"down_at":76,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6816","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6817","nonce":1501294989}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6818","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6819","nonce":1501294989}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6822","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6823","nonce":1501294989}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.104:6820","nonce":1501294989},{"type":"v1","addr":"172.21.15.104:6821","nonce":1501294989}]},"public_addr":"172.21.15.104:6817/1501294989","cluster_addr":"172.21.15.104:6819/1501294989","heartbeat_back_addr":"172.21.15.104:6823/1501294989","heartbeat_front_addr":"172.21.15.104:6821/1501294989","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:19.861796+0000","dead_epoch":50},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:36:48.482295+0000","dead_epoch":55},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:17.028265+0000","dead_epoch":60},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:37:40.489573+0000","dead_epoch":65},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:05.503518+0000","dead_epoch":70},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-26T04:38:30.925548+0000","dead_epoch":75}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.90:6828/370183803":"2024-04-27T04:51:20.563404+0000","172.21.15.90:6800/1235211382":"2024-04-27T04:43:18.835636+0000","172.21.15.104:6824/1058172006":"2024-04-27T04:51:20.563430+0000","172.21.15.90:0/748209074":"2024-04-27T04:33:57.088900+0000","172.21.15.104:6829/680795988":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6801/1235211382":"2024-04-27T04:43:18.835636+0000","172.21.15.90:0/2705188709":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6801/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.104:0/2711199308":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/1814853898":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/4265628059":"2024-04-27T04:46:55.630300+0000","172.21.15.90:0/2763746313":"2024-04-27T04:43:18.835636+0000","172.21.15.90:0/2473529454":"2024-04-27T04:33:24.670558+0000","172.21.15.104:6828/680795988":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6800/1230348805":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/2342166800":"2024-04-27T04:33:57.088900+0000","172.21.15.104:0/1222914910":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6829/370183803":"2024-04-27T04:51:20.563404+0000","172.21.15.104:0/2266011983":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/3418315403":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/2310050885":"2024-04-27T04:33:57.088900+0000","172.21.15.90:0/2803863568":"2024-04-27T04:46:55.630300+0000","172.21.15.90:0/424889384":"2024-04-27T04:33:24.670558+0000","172.21.15.90:6800/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.104:6825/1058172006":"2024-04-27T04:51:20.563430+0000","172.21.15.90:0/2846076831":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6801/3458990238":"2024-04-27T04:46:55.630300+0000","172.21.15.90:6801/2176954768":"2024-04-27T04:33:57.088900+0000","172.21.15.90:6800/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:6801/2647874706":"2024-04-27T04:35:03.230209+0000","172.21.15.90:0/3214112922":"2024-04-27T04:35:03.230209+0000","172.21.15.104:0/3572733912":"2024-04-27T04:44:00.133458+0000","172.21.15.104:0/3577745345":"2024-04-27T04:44:00.133458+0000","172.21.15.90:0/50635771":"2024-04-27T04:33:24.670558+0000","172.21.15.90:0/4285557765":"2024-04-27T04:46:55.630300+0000","172.21.15.104:0/204837581":"2024-04-27T04:44:00.133458+0000","172.21.15.90:6826/3871906279":"2024-04-27T04:40:00.786348+0000","172.21.15.90:0/1198706928":"2024-04-27T04:43:18.835636+0000","172.21.15.90:6827/3871906279":"2024-04-27T04:40:00.786348+0000","172.21.15.90:0/1376440705":"2024-04-27T04:46:55.630300+0000","172.21.15.90:6800/3458990238":"2024-04-27T04:46:55.630300+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-26T04:54:54.046 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa93806e1d0 msgr2=0x7fa938070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa93806e1d0 0x7fa938070680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa944005fd0 tx=0x7fa944005850 comp rx=0 tx=0).stop 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9500fe5a0 msgr2=0x7fa9501940d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9500fe5a0 0x7fa9501940d0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa940009230 tx=0x7fa940004040 comp rx=0 tx=0).stop 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 shutdown_connections 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7fa9501045d0 0x7fa950194610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7fa93806e1d0 0x7fa938070680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 --2- 172.21.15.90:0/169777665 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7fa9500fe5a0 0x7fa9501940d0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 >> 172.21.15.90:0/169777665 conn(0x7fa950075270 msgr2=0x7fa9500fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 shutdown_connections 2024-04-26T04:54:54.047 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.045+0000 7fa955af4640 1 -- 172.21.15.90:0/169777665 wait complete. 2024-04-26T04:54:54.057 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-26T04:37:19.554153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6, 'score_stable': 6, 'optimal_score': 0.5, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs.cephfs.meta', 'create_time': '2024-04-26T04:39:42.011282+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.690000057220459, 'score_stable': 1.690000057220459, 'optimal_score': 1, 'raw_score_acting': 1.690000057220459, 'raw_score_stable': 1.690000057220459, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs.cephfs.data', 'create_time': '2024-04-26T04:39:42.594560+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '45', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-26T04:54:54.058 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-26T04:54:54.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 -- 172.21.15.90:0/1050873758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7498104590 msgr2=0x7f7498104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 --2- 172.21.15.90:0/1050873758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7498104590 0x7f7498104960 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7494009c20 tx=0x7f749402c1b0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 -- 172.21.15.90:0/1050873758 shutdown_connections 2024-04-26T04:54:54.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 --2- 172.21.15.90:0/1050873758 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f74980fe5b0 0x7f74980fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 --2- 172.21.15.90:0/1050873758 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f7498104590 0x7f7498104960 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 -- 172.21.15.90:0/1050873758 >> 172.21.15.90:0/1050873758 conn(0x7f74980743d0 msgr2=0x7f74980747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:54.160 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 -- 172.21.15.90:0/1050873758 shutdown_connections 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.156+0000 7f74a02f4640 1 -- 172.21.15.90:0/1050873758 wait complete. 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f74a02f4640 1 Processor -- start 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f74a02f4640 1 -- start start 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f74a02f4640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74980fe5b0 0x7f74981952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f74a02f4640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7498104590 0x7f7498195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f74a02f4640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7498195e90 con 0x7f74980fe5b0 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f74a02f4640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7498199c60 con 0x7f7498104590 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f749e090640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74980fe5b0 0x7f74981952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f749e090640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74980fe5b0 0x7f74981952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44482/0 (socket says 172.21.15.90:44482) 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f749e090640 1 -- 172.21.15.90:0/2835328579 learned_addr learned my addr 172.21.15.90:0/2835328579 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.157+0000 7f749d88f640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7498104590 0x7f7498195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f749e090640 1 -- 172.21.15.90:0/2835328579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7498104590 msgr2=0x7f7498195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.161 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f749e090640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7498104590 0x7f7498195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f749e090640 1 -- 172.21.15.90:0/2835328579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7494009900 con 0x7f74980fe5b0 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f749e090640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74980fe5b0 0x7f74981952c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7494037040 tx=0x7f7494032b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f748f7fe640 1 -- 172.21.15.90:0/2835328579 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f7494034820 con 0x7f74980fe5b0 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f748f7fe640 1 -- 172.21.15.90:0/2835328579 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f7494003e30 con 0x7f74980fe5b0 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7498199ee0 con 0x7f74980fe5b0 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f748f7fe640 1 -- 172.21.15.90:0/2835328579 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f749403f440 con 0x7f74980fe5b0 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.158+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f749819a460 con 0x7f74980fe5b0 2024-04-26T04:54:54.162 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.159+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749804e000 con 0x7f74980fe5b0 2024-04-26T04:54:54.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.159+0000 7f748f7fe640 1 -- 172.21.15.90:0/2835328579 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f749403b040 con 0x7f74980fe5b0 2024-04-26T04:54:54.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.160+0000 7f748f7fe640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f748406e170 0x7f7484070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.160+0000 7f749d88f640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f748406e170 0x7f7484070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.160+0000 7f748f7fe640 1 -- 172.21.15.90:0/2835328579 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f74940b0fe0 con 0x7f74980fe5b0 2024-04-26T04:54:54.163 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.160+0000 7f749d88f640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f748406e170 0x7f7484070620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7488005fd0 tx=0x7f7488005850 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:54.165 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.164+0000 7f748f7fe640 1 -- 172.21.15.90:0/2835328579 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f749407a2c0 con 0x7f74980fe5b0 2024-04-26T04:54:54.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:53 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/1022934978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:54:54.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:53 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/462397131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:54:54.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:53 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/3755226368' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T04:54:54.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:53 smithi090 ceph-mon[86550]: pgmap v277: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:53 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/1022934978' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:54:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:53 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/462397131' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:54:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:53 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/3755226368' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T04:54:54.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:53 smithi104 ceph-mon[64778]: pgmap v277: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:54.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.340+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f749810b840 con 0x7f74980fe5b0 2024-04-26T04:54:54.341 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.340+0000 7f748f7fe640 1 -- 172.21.15.90:0/2835328579 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v85) v1 ==== 93+0+10 (secure 0 0 0) 0x7f749407d980 con 0x7f74980fe5b0 2024-04-26T04:54:54.342 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 1 2024-04-26T04:54:54.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.342+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f748406e170 msgr2=0x7f7484070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.342+0000 7f74a02f4640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f748406e170 0x7f7484070620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f7488005fd0 tx=0x7f7488005850 comp rx=0 tx=0).stop 2024-04-26T04:54:54.344 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.342+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74980fe5b0 msgr2=0x7f74981952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.342+0000 7f74a02f4640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74980fe5b0 0x7f74981952c0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f7494037040 tx=0x7f7494032b60 comp rx=0 tx=0).stop 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.343+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 shutdown_connections 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.343+0000 7f74a02f4640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f7498104590 0x7f7498195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.343+0000 7f74a02f4640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f748406e170 0x7f7484070620 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.343+0000 7f74a02f4640 1 --2- 172.21.15.90:0/2835328579 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f74980fe5b0 0x7f74981952c0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.343+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 >> 172.21.15.90:0/2835328579 conn(0x7f74980743d0 msgr2=0x7f7498067310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.343+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 shutdown_connections 2024-04-26T04:54:54.345 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.343+0000 7f74a02f4640 1 -- 172.21.15.90:0/2835328579 wait complete. 2024-04-26T04:54:54.355 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.meta pg_num 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 -- 172.21.15.90:0/64386539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53481045c0 msgr2=0x7f5348104990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 --2- 172.21.15.90:0/64386539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53481045c0 0x7f5348104990 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f5338009880 tx=0x7f533802bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 -- 172.21.15.90:0/64386539 shutdown_connections 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 --2- 172.21.15.90:0/64386539 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53480fe590 0x7f53480fea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 --2- 172.21.15.90:0/64386539 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53481045c0 0x7f5348104990 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 -- 172.21.15.90:0/64386539 >> 172.21.15.90:0/64386539 conn(0x7f53480fa070 msgr2=0x7f53480fc480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 -- 172.21.15.90:0/64386539 shutdown_connections 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.454+0000 7f534d91a640 1 -- 172.21.15.90:0/64386539 wait complete. 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f534d91a640 1 Processor -- start 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f534d91a640 1 -- start start 2024-04-26T04:54:54.459 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f534d91a640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53480fe590 0x7f5348100170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f534d91a640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53481045c0 0x7f53481006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f534d91a640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5348100bf0 con 0x7f53480fe590 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f534d91a640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5348100d60 con 0x7f53481045c0 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f5346ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53480fe590 0x7f5348100170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f5346ffd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53480fe590 0x7f5348100170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44502/0 (socket says 172.21.15.90:44502) 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f5346ffd640 1 -- 172.21.15.90:0/2077998727 learned_addr learned my addr 172.21.15.90:0/2077998727 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f533ffff640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53481045c0 0x7f53481006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f5346ffd640 1 -- 172.21.15.90:0/2077998727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53481045c0 msgr2=0x7f53481006b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f5346ffd640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53481045c0 0x7f53481006b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f5346ffd640 1 -- 172.21.15.90:0/2077998727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5338009560 con 0x7f53480fe590 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.455+0000 7f533ffff640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53481045c0 0x7f53481006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.456+0000 7f5346ffd640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53480fe590 0x7f5348100170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5338009230 tx=0x7f5338004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.456+0000 7f534c918640 1 -- 172.21.15.90:0/2077998727 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f53380035f0 con 0x7f53480fe590 2024-04-26T04:54:54.460 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.456+0000 7f534c918640 1 -- 172.21.15.90:0/2077998727 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f5338003c10 con 0x7f53480fe590 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.456+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5348100f80 con 0x7f53480fe590 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.456+0000 7f534c918640 1 -- 172.21.15.90:0/2077998727 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f5338034440 con 0x7f53480fe590 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.456+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53481a1810 con 0x7f53480fe590 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.457+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f534804e000 con 0x7f53480fe590 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.457+0000 7f534c918640 1 -- 172.21.15.90:0/2077998727 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f5338004fd0 con 0x7f53480fe590 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.458+0000 7f534c918640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f533406e170 0x7f5334070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.458+0000 7f533ffff640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f533406e170 0x7f5334070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.458+0000 7f534c918640 1 -- 172.21.15.90:0/2077998727 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f53380b0530 con 0x7f53480fe590 2024-04-26T04:54:54.461 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.458+0000 7f533ffff640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f533406e170 0x7f5334070620 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5348101960 tx=0x7f5330005d30 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:54.463 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.462+0000 7f534c918640 1 -- 172.21.15.90:0/2077998727 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f5338079890 con 0x7f53480fe590 2024-04-26T04:54:54.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.636+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"} v 0) v1 -- 0x7f534810b750 con 0x7f53480fe590 2024-04-26T04:54:54.637 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.636+0000 7f534c918640 1 -- 172.21.15.90:0/2077998727 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f533807cf50 con 0x7f53480fe590 2024-04-26T04:54:54.638 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f533406e170 msgr2=0x7f5334070620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f533406e170 0x7f5334070620 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f5348101960 tx=0x7f5330005d30 comp rx=0 tx=0).stop 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53480fe590 msgr2=0x7f5348100170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53480fe590 0x7f5348100170 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f5338009230 tx=0x7f5338004040 comp rx=0 tx=0).stop 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 shutdown_connections 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f53481045c0 0x7f53481006b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f533406e170 0x7f5334070620 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 --2- 172.21.15.90:0/2077998727 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f53480fe590 0x7f5348100170 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.640 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 >> 172.21.15.90:0/2077998727 conn(0x7f53480fa070 msgr2=0x7f5348068360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:54.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 shutdown_connections 2024-04-26T04:54:54.641 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.638+0000 7f534d91a640 1 -- 172.21.15.90:0/2077998727 wait complete. 2024-04-26T04:54:54.651 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs.cephfs.data pg_num 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 -- 172.21.15.90:0/2573901667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3748104590 msgr2=0x7f3748104960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2573901667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3748104590 0x7f3748104960 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f3738009880 tx=0x7f373802bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 -- 172.21.15.90:0/2573901667 shutdown_connections 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2573901667 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f37480fe5b0 0x7f37480fea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2573901667 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f3748104590 0x7f3748104960 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 -- 172.21.15.90:0/2573901667 >> 172.21.15.90:0/2573901667 conn(0x7f37480743d0 msgr2=0x7f37480747d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 -- 172.21.15.90:0/2573901667 shutdown_connections 2024-04-26T04:54:54.753 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 -- 172.21.15.90:0/2573901667 wait complete. 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 Processor -- start 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.750+0000 7f374ce7d640 1 -- start start 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374ce7d640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37480fe5b0 0x7f37481941a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374ce7d640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3748104590 0x7f37481946e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374ce7d640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748194d70 con 0x7f37480fe5b0 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374ce7d640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3748198b40 con 0x7f3748104590 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37480fe5b0 0x7f37481941a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374659c640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37480fe5b0 0x7f37481941a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44516/0 (socket says 172.21.15.90:44516) 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374659c640 1 -- 172.21.15.90:0/2319732419 learned_addr learned my addr 172.21.15.90:0/2319732419 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f373ffff640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3748104590 0x7f37481946e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374659c640 1 -- 172.21.15.90:0/2319732419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3748104590 msgr2=0x7f37481946e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374659c640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3748104590 0x7f37481946e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.754 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374659c640 1 -- 172.21.15.90:0/2319732419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3738009560 con 0x7f37480fe5b0 2024-04-26T04:54:54.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374659c640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37480fe5b0 0x7f37481941a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3738009230 tx=0x7f3738004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:54.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f373f7fe640 1 -- 172.21.15.90:0/2319732419 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f37380035f0 con 0x7f37480fe5b0 2024-04-26T04:54:54.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f373f7fe640 1 -- 172.21.15.90:0/2319732419 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f3738003c10 con 0x7f37480fe5b0 2024-04-26T04:54:54.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3748198dc0 con 0x7f37480fe5b0 2024-04-26T04:54:54.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f373f7fe640 1 -- 172.21.15.90:0/2319732419 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f3738005670 con 0x7f37480fe5b0 2024-04-26T04:54:54.755 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.751+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37481992c0 con 0x7f37480fe5b0 2024-04-26T04:54:54.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.752+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f374804e000 con 0x7f37480fe5b0 2024-04-26T04:54:54.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.753+0000 7f373f7fe640 1 -- 172.21.15.90:0/2319732419 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f3738004dd0 con 0x7f37480fe5b0 2024-04-26T04:54:54.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.753+0000 7f373f7fe640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f373406e120 0x7f37340705d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:54.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.753+0000 7f373f7fe640 1 -- 172.21.15.90:0/2319732419 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f37380b0590 con 0x7f37480fe5b0 2024-04-26T04:54:54.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.753+0000 7f373ffff640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f373406e120 0x7f37340705d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:54.756 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.754+0000 7f373ffff640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f373406e120 0x7f37340705d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3730005fd0 tx=0x7f3730005c70 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:54.759 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.758+0000 7f373f7fe640 1 -- 172.21.15.90:0/2319732419 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f3738079870 con 0x7f37480fe5b0 2024-04-26T04:54:54.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.932+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"} v 0) v1 -- 0x7f37481954f0 con 0x7f37480fe5b0 2024-04-26T04:54:54.934 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.933+0000 7f373f7fe640 1 -- 172.21.15.90:0/2319732419 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]=0 v85) v1 ==== 107+0+11 (secure 0 0 0) 0x7f373807cf30 con 0x7f37480fe5b0 2024-04-26T04:54:54.934 INFO:teuthology.orchestra.run.smithi090.stdout:pg_num: 32 2024-04-26T04:54:54.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f373406e120 msgr2=0x7f37340705d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f373406e120 0x7f37340705d0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f3730005fd0 tx=0x7f3730005c70 comp rx=0 tx=0).stop 2024-04-26T04:54:54.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37480fe5b0 msgr2=0x7f37481941a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:54.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37480fe5b0 0x7f37481941a0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3738009230 tx=0x7f3738004040 comp rx=0 tx=0).stop 2024-04-26T04:54:54.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 shutdown_connections 2024-04-26T04:54:54.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f3748104590 0x7f37481946e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.936 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f373406e120 0x7f37340705d0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 --2- 172.21.15.90:0/2319732419 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f37480fe5b0 0x7f37481941a0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:54.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 >> 172.21.15.90:0/2319732419 conn(0x7f37480743d0 msgr2=0x7f37480fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:54.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 shutdown_connections 2024-04-26T04:54:54.937 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:54.935+0000 7f374ce7d640 1 -- 172.21.15.90:0/2319732419 wait complete. 2024-04-26T04:54:54.947 DEBUG:teuthology.orchestra.run.smithi090:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-26T04:54:55.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 -- 172.21.15.90:0/1925610627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80fe7c0 msgr2=0x7f48c80feb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:55.051 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/1925610627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80fe7c0 0x7f48c80feb90 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f48b8009880 tx=0x7f48b802bce0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 -- 172.21.15.90:0/1925610627 shutdown_connections 2024-04-26T04:54:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/1925610627 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48c80ff0d0 0x7f48c8109b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/1925610627 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80fe7c0 0x7f48c80feb90 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 -- 172.21.15.90:0/1925610627 >> 172.21.15.90:0/1925610627 conn(0x7f48c8073d20 msgr2=0x7f48c8074120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 -- 172.21.15.90:0/1925610627 shutdown_connections 2024-04-26T04:54:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 -- 172.21.15.90:0/1925610627 wait complete. 2024-04-26T04:54:55.052 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.045+0000 7f48cdfdd640 1 Processor -- start 2024-04-26T04:54:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48cdfdd640 1 -- start start 2024-04-26T04:54:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48cdfdd640 1 --2- >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48c80fe7c0 0x7f48c8195280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48cdfdd640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80ff0d0 0x7f48c81957c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48cdfdd640 1 -- --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8195e50 con 0x7f48c80ff0d0 2024-04-26T04:54:55.053 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48cdfdd640 1 -- --> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48c8199c20 con 0x7f48c80fe7c0 2024-04-26T04:54:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48bffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80ff0d0 0x7f48c81957c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48bffff640 1 --2- >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80ff0d0 0x7f48c81957c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.90:3300/0 says I am v2:172.21.15.90:44534/0 (socket says 172.21.15.90:44534) 2024-04-26T04:54:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48bffff640 1 -- 172.21.15.90:0/4229896365 learned_addr learned my addr 172.21.15.90:0/4229896365 (peer_addr_for_me v2:172.21.15.90:0/0) 2024-04-26T04:54:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48bffff640 1 -- 172.21.15.90:0/4229896365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48c80fe7c0 msgr2=0x7f48c8195280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48bffff640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48c80fe7c0 0x7f48c8195280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48bffff640 1 -- 172.21.15.90:0/4229896365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48b8009560 con 0x7f48c80ff0d0 2024-04-26T04:54:55.054 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48bffff640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80ff0d0 0x7f48c81957c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f48b000b420 tx=0x7f48b000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48c57fa640 1 -- 172.21.15.90:0/4229896365 <== mon.0 v2:172.21.15.90:3300/0 1 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b0011800 con 0x7f48c80ff0d0 2024-04-26T04:54:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48c8199f00 con 0x7f48c80ff0d0 2024-04-26T04:54:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48c57fa640 1 -- 172.21.15.90:0/4229896365 <== mon.0 v2:172.21.15.90:3300/0 2 ==== config(27 keys) v1 ==== 1109+0+0 (secure 0 0 0) 0x7f48b0011e20 con 0x7f48c80ff0d0 2024-04-26T04:54:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48c57fa640 1 -- 172.21.15.90:0/4229896365 <== mon.0 v2:172.21.15.90:3300/0 3 ==== mon_map magic: 0 v1 ==== 357+0+0 (secure 0 0 0) 0x7f48b000f2e0 con 0x7f48c80ff0d0 2024-04-26T04:54:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.046+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48c819a460 con 0x7f48c80ff0d0 2024-04-26T04:54:55.055 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.047+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48c804e000 con 0x7f48c80ff0d0 2024-04-26T04:54:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.048+0000 7f48c57fa640 1 -- 172.21.15.90:0/4229896365 <== mon.0 v2:172.21.15.90:3300/0 4 ==== mgrmap(e 38) v1 ==== 92203+0+0 (secure 0 0 0) 0x7f48b0010380 con 0x7f48c80ff0d0 2024-04-26T04:54:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.048+0000 7f48c57fa640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f48b406e230 0x7f48b40706e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-26T04:54:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.048+0000 7f48c57fa640 1 -- 172.21.15.90:0/4229896365 <== mon.0 v2:172.21.15.90:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6586+0+0 (secure 0 0 0) 0x7f48b008daf0 con 0x7f48c80ff0d0 2024-04-26T04:54:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.048+0000 7f48c77fe640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f48b406e230 0x7f48b40706e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-26T04:54:55.056 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.049+0000 7f48c77fe640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f48b406e230 0x7f48b40706e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48b802efd0 tx=0x7f48b8000bc0 comp rx=0 tx=0).ready entity=mgr.34104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-26T04:54:55.057 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.053+0000 7f48c57fa640 1 -- 172.21.15.90:0/4229896365 <== mon.0 v2:172.21.15.90:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+186147 (secure 0 0 0) 0x7f48b0056dd0 con 0x7f48c80ff0d0 2024-04-26T04:54:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:54 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/169777665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:54:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:54 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2835328579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:54:55.221 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:54 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2077998727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:54:55.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.244+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 --> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f48c81071b0 con 0x7f48c80ff0d0 2024-04-26T04:54:55.245 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.244+0000 7f48c57fa640 1 -- 172.21.15.90:0/4229896365 <== mon.0 v2:172.21.15.90:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 41 entries v85) v1 ==== 81+0+2433 (secure 0 0 0) 0x7f48b005a490 con 0x7f48c80ff0d0 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6828/370183803 2024-04-27T04:51:20.563404+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1235211382 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6824/1058172006 2024-04-27T04:51:20.563430+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/748209074 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6829/680795988 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1235211382 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2705188709 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/1230348805 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/2711199308 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1814853898 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/4265628059 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:55.246 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2763746313 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2473529454 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6828/680795988 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/1230348805 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2342166800 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/1222914910 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6829/370183803 2024-04-27T04:51:20.563404+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/2266011983 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3418315403 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2310050885 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2803863568 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/424889384 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2176954768 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:6825/1058172006 2024-04-27T04:51:20.563430+0000 2024-04-26T04:54:55.247 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/2846076831 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/3458990238 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2176954768 2024-04-27T04:33:57.088900+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/2647874706 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6801/2647874706 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/3214112922 2024-04-27T04:35:03.230209+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/3572733912 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/3577745345 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/50635771 2024-04-27T04:33:24.670558+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/4285557765 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.104:0/204837581 2024-04-27T04:44:00.133458+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6826/3871906279 2024-04-27T04:40:00.786348+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1198706928 2024-04-27T04:43:18.835636+0000 2024-04-26T04:54:55.248 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6827/3871906279 2024-04-27T04:40:00.786348+0000 2024-04-26T04:54:55.249 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:0/1376440705 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:55.249 INFO:teuthology.orchestra.run.smithi090.stdout:172.21.15.90:6800/3458990238 2024-04-27T04:46:55.630300+0000 2024-04-26T04:54:55.249 INFO:teuthology.orchestra.run.smithi090.stderr:listed 41 entries 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f48b406e230 msgr2=0x7f48b40706e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f48b406e230 0x7f48b40706e0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f48b802efd0 tx=0x7f48b8000bc0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80ff0d0 msgr2=0x7f48c81957c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80ff0d0 0x7f48c81957c0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f48b000b420 tx=0x7f48b000b8e0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 shutdown_connections 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:6800/1611686821,v1:172.21.15.90:6801/1611686821] conn(0x7f48b406e230 0x7f48b40706e0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.90:3300/0,v1:172.21.15.90:6789/0] conn(0x7f48c80ff0d0 0x7f48c81957c0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.246+0000 7f48cdfdd640 1 --2- 172.21.15.90:0/4229896365 >> [v2:172.21.15.104:3300/0,v1:172.21.15.104:6789/0] conn(0x7f48c80fe7c0 0x7f48c8195280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.247+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 >> 172.21.15.90:0/4229896365 conn(0x7f48c8073d20 msgr2=0x7f48c8104380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.247+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 shutdown_connections 2024-04-26T04:54:55.250 INFO:teuthology.orchestra.run.smithi090.stderr:2024-04-26T04:54:55.247+0000 7f48cdfdd640 1 -- 172.21.15.90:0/4229896365 wait complete. 2024-04-26T04:54:55.259 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-04-26T04:54:55.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.259 DEBUG:teuthology.orchestra.run.smithi104:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-04-26T04:54:55.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:54 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/169777665' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-26T04:54:55.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:54 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2835328579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-26T04:54:55.262 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:54 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2077998727' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.meta", "var": "pg_num"}]: dispatch 2024-04-26T04:54:55.292 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi104.front.sepia.ceph.com 2024-04-26T04:54:55.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.292 DEBUG:teuthology.orchestra.run.smithi104:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-04-26T04:54:55.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.360 DEBUG:teuthology.orchestra.run.smithi090:> ip netns list 2024-04-26T04:54:55.375 INFO:teuthology.orchestra.run.smithi090.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-26T04:54:55.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.376 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-26T04:54:55.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.440 DEBUG:teuthology.orchestra.run.smithi090:> sudo ip link delete ceph-brx 2024-04-26T04:54:55.546 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.546 DEBUG:teuthology.orchestra.run.smithi104:> ip netns list 2024-04-26T04:54:55.564 INFO:teuthology.orchestra.run.smithi104.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-04-26T04:54:55.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.564 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-04-26T04:54:55.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-26T04:54:55.633 DEBUG:teuthology.orchestra.run.smithi104:> sudo ip link delete ceph-brx 2024-04-26T04:54:55.731 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-26T04:54:55.742 INFO:tasks.cephadm:Teardown begin 2024-04-26T04:54:55.742 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:54:55.767 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:54:55.793 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-26T04:54:55.793 DEBUG:teuthology.orchestra.run.smithi090:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T04:54:55.823 DEBUG:teuthology.orchestra.run.smithi104:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-26T04:54:55.849 INFO:tasks.cephadm:Stopping all daemons... 2024-04-26T04:54:55.849 INFO:tasks.cephadm.mon.smithi090:Stopping mon.smithi090... 2024-04-26T04:54:55.849 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090 2024-04-26T04:54:56.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:55 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/2319732419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:54:56.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:55 smithi090 systemd[1]: Stopping Ceph mon.smithi090 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:54:56.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:55 smithi090 ceph-mon[86550]: from='client.? 172.21.15.90:0/4229896365' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T04:54:56.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:55 smithi090 ceph-mon[86550]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:56.153 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:55 smithi090 ceph-mon[86550]: pgmap v278: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:55 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/2319732419' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs.cephfs.data", "var": "pg_num"}]: dispatch 2024-04-26T04:54:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:55 smithi104 ceph-mon[64778]: from='client.? 172.21.15.90:0/4229896365' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-26T04:54:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:55 smithi104 ceph-mon[64778]: from='mgr.34104 172.21.15.90:0/366930930' entity='mgr.smithi090.iwrvwa' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-04-26T04:54:56.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:55 smithi104 ceph-mon[64778]: pgmap v278: 65 pgs: 65 active+clean; 293 MiB data, 1.1 GiB used, 535 GiB / 536 GiB avail 2024-04-26T04:54:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:56 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[86546]: 2024-04-26T04:54:56.151+0000 7fd66350a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi090 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:54:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:56 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090[86546]: 2024-04-26T04:54:56.151+0000 7fd66350a700 -1 mon.smithi090@0(leader) e3 *** Got Signal Terminated *** 2024-04-26T04:54:56.471 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:56 smithi090 podman[131925]: 2024-04-26 04:54:56.172888416 +0000 UTC m=+0.122727916 container died e31d5b589b5cd1b13a519542b3872de1913c6a1d218cfeea84ad598fb1fdc988 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T04:54:56.788 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:56 smithi090 podman[131925]: 2024-04-26 04:54:56.569790369 +0000 UTC m=+0.519629869 container cleanup e31d5b589b5cd1b13a519542b3872de1913c6a1d218cfeea84ad598fb1fdc988 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T04:54:56.788 INFO:journalctl@ceph.mon.smithi090.smithi090.stdout:Apr 26 04:54:56 smithi090 bash[131925]: ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi090 2024-04-26T04:54:56.915 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi090.service' 2024-04-26T04:54:56.962 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:54:56.962 INFO:tasks.cephadm.mon.smithi090:Stopped mon.smithi090 2024-04-26T04:54:56.962 INFO:tasks.cephadm.mon.smithi104:Stopping mon.smithi104... 2024-04-26T04:54:56.962 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi104 2024-04-26T04:54:57.261 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:57 smithi104 systemd[1]: Stopping Ceph mon.smithi104 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:54:57.697 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:57 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104[64774]: 2024-04-26T04:54:57.403+0000 7f8391427700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.smithi104 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:54:57.697 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:57 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104[64774]: 2024-04-26T04:54:57.403+0000 7f8391427700 -1 mon.smithi104@1(peon) e3 *** Got Signal Terminated *** 2024-04-26T04:54:57.697 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:57 smithi104 podman[101892]: 2024-04-26 04:54:57.425277154 +0000 UTC m=+0.274657448 container died 11c4dd69e45404d95fa5efef1ddbaf1523edb2280db940cd517aa5ba4486da15 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:54:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:57 smithi104 podman[101892]: 2024-04-26 04:54:57.863280168 +0000 UTC m=+0.712660460 container cleanup 11c4dd69e45404d95fa5efef1ddbaf1523edb2280db940cd517aa5ba4486da15 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=) 2024-04-26T04:54:58.011 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:57 smithi104 bash[101892]: ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104 2024-04-26T04:54:58.290 INFO:journalctl@ceph.mon.smithi104.smithi104.stdout:Apr 26 04:54:58 smithi104 podman[101905]: 2024-04-26 04:54:58.03963971 +0000 UTC m=+0.610928004 container remove 11c4dd69e45404d95fa5efef1ddbaf1523edb2280db940cd517aa5ba4486da15 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-mon-smithi104, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:54:58.294 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@mon.smithi104.service' 2024-04-26T04:54:58.342 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:54:58.342 INFO:tasks.cephadm.mon.smithi104:Stopped mon.smithi104 2024-04-26T04:54:58.342 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-26T04:54:58.343 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.0 2024-04-26T04:54:58.658 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:54:58 smithi090 systemd[1]: Stopping Ceph osd.0 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:54:58.971 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:54:58 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[98097]: 2024-04-26T04:54:58.657+0000 7fcd13bb7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:54:58.971 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:54:58 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[98097]: 2024-04-26T04:54:58.657+0000 7fcd13bb7700 -1 osd.0 85 *** Got signal Terminated *** 2024-04-26T04:54:58.971 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:54:58 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0[98097]: 2024-04-26T04:54:58.657+0000 7fcd13bb7700 -1 osd.0 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:55:04.354 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:04 smithi090 podman[132059]: 2024-04-26 04:55:04.01226385 +0000 UTC m=+5.467908143 container died 514106f42deebff6c6df4f5203e16f39fb2448e19863565483f4a67bac43851d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb) 2024-04-26T04:55:04.721 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:04 smithi090 podman[132059]: 2024-04-26 04:55:04.468146342 +0000 UTC m=+5.923790624 container cleanup 514106f42deebff6c6df4f5203e16f39fb2448e19863565483f4a67bac43851d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:55:04.721 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:04 smithi090 bash[132059]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0 2024-04-26T04:55:05.089 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:04 smithi090 podman[132076]: 2024-04-26 04:55:04.762916444 +0000 UTC m=+0.749320761 container remove 514106f42deebff6c6df4f5203e16f39fb2448e19863565483f4a67bac43851d (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, GIT_CLEAN=True) 2024-04-26T04:55:05.411 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:05 smithi090 podman[132168]: 2024-04-26 04:55:05.089234059 +0000 UTC m=+0.107334422 container create c60bb831dd10c5a25f2a205634b9562399b25b44c2ededdd3abeb9a8f4e93021 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-26T04:55:05.411 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:05 smithi090 podman[132168]: 2024-04-26 04:55:05.000205584 +0000 UTC m=+0.018305947 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:05.411 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:05 smithi090 podman[132168]: 2024-04-26 04:55:05.335638378 +0000 UTC m=+0.353738737 container init c60bb831dd10c5a25f2a205634b9562399b25b44c2ededdd3abeb9a8f4e93021 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-b22e2eb, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-26T04:55:05.411 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:05 smithi090 podman[132168]: 2024-04-26 04:55:05.339001857 +0000 UTC m=+0.357102225 container start c60bb831dd10c5a25f2a205634b9562399b25b44c2ededdd3abeb9a8f4e93021 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-26T04:55:05.411 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:05 smithi090 podman[132168]: 2024-04-26 04:55:05.410639514 +0000 UTC m=+0.428739892 container attach c60bb831dd10c5a25f2a205634b9562399b25b44c2ededdd3abeb9a8f4e93021 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:55:05.721 INFO:journalctl@ceph.osd.0.smithi090.stdout:Apr 26 04:55:05 smithi090 podman[132168]: 2024-04-26 04:55:05.642463174 +0000 UTC m=+0.660563534 container died c60bb831dd10c5a25f2a205634b9562399b25b44c2ededdd3abeb9a8f4e93021 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-0-deactivate, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:55:06.246 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.0.service' 2024-04-26T04:55:06.284 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:55:06.284 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-26T04:55:06.284 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-26T04:55:06.284 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.1 2024-04-26T04:55:06.471 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:06 smithi090 systemd[1]: Stopping Ceph osd.1 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:55:06.971 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:06 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[104007]: 2024-04-26T04:55:06.620+0000 7f6af4b05700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:55:06.971 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:06 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[104007]: 2024-04-26T04:55:06.620+0000 7f6af4b05700 -1 osd.1 85 *** Got signal Terminated *** 2024-04-26T04:55:06.971 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:06 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1[104007]: 2024-04-26T04:55:06.620+0000 7f6af4b05700 -1 osd.1 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:55:12.373 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:12 smithi090 podman[132272]: 2024-04-26 04:55:12.106157819 +0000 UTC m=+5.586533915 container died 264b38fc9f3109120ebb265eee301143780e02a8f868d06beca0de5e711e361b (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T04:55:12.690 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:12 smithi090 podman[132272]: 2024-04-26 04:55:12.446875048 +0000 UTC m=+5.927251191 container cleanup 264b38fc9f3109120ebb265eee301143780e02a8f868d06beca0de5e711e361b (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, RELEASE=squid-b22e2eb) 2024-04-26T04:55:12.691 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:12 smithi090 bash[132272]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1 2024-04-26T04:55:12.971 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:12 smithi090 podman[132303]: 2024-04-26 04:55:12.690473771 +0000 UTC m=+0.575531064 container remove 264b38fc9f3109120ebb265eee301143780e02a8f868d06beca0de5e711e361b (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5) 2024-04-26T04:55:13.430 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:13 smithi090 podman[132394]: 2024-04-26 04:55:13.108382904 +0000 UTC m=+0.113391891 container create 48c229e41111ebb29966b07f88a9d72ccd864f6b9da8f8e8519cfeb990f1f79e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-26T04:55:13.430 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:13 smithi090 podman[132394]: 2024-04-26 04:55:13.013401052 +0000 UTC m=+0.018410052 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:13.430 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:13 smithi090 podman[132394]: 2024-04-26 04:55:13.338175712 +0000 UTC m=+0.343184716 container init 48c229e41111ebb29966b07f88a9d72ccd864f6b9da8f8e8519cfeb990f1f79e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:55:13.430 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:13 smithi090 podman[132394]: 2024-04-26 04:55:13.341580325 +0000 UTC m=+0.346589321 container start 48c229e41111ebb29966b07f88a9d72ccd864f6b9da8f8e8519cfeb990f1f79e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240424) 2024-04-26T04:55:13.430 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:13 smithi090 podman[132394]: 2024-04-26 04:55:13.429453427 +0000 UTC m=+0.434462423 container attach 48c229e41111ebb29966b07f88a9d72ccd864f6b9da8f8e8519cfeb990f1f79e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5) 2024-04-26T04:55:13.721 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:13 smithi090 podman[132394]: 2024-04-26 04:55:13.632325246 +0000 UTC m=+0.637334239 container died 48c229e41111ebb29966b07f88a9d72ccd864f6b9da8f8e8519cfeb990f1f79e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0) 2024-04-26T04:55:14.176 INFO:journalctl@ceph.osd.1.smithi090.stdout:Apr 26 04:55:14 smithi090 podman[132420]: 2024-04-26 04:55:14.159013644 +0000 UTC m=+0.520653047 container remove 48c229e41111ebb29966b07f88a9d72ccd864f6b9da8f8e8519cfeb990f1f79e (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-1-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:55:14.180 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.1.service' 2024-04-26T04:55:14.218 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:55:14.219 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-26T04:55:14.219 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-26T04:55:14.219 DEBUG:teuthology.orchestra.run.smithi090:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.2 2024-04-26T04:55:14.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:14 smithi090 systemd[1]: Stopping Ceph osd.2 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:55:14.971 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:14 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[109430]: 2024-04-26T04:55:14.553+0000 7faff8325700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:55:14.971 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:14 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[109430]: 2024-04-26T04:55:14.553+0000 7faff8325700 -1 osd.2 85 *** Got signal Terminated *** 2024-04-26T04:55:14.972 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:14 smithi090 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2[109430]: 2024-04-26T04:55:14.553+0000 7faff8325700 -1 osd.2 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:55:20.174 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:19 smithi090 podman[132500]: 2024-04-26 04:55:19.912983746 +0000 UTC m=+5.477070431 container died a6f8b02d7e7d2e7ba9e618ee9bbbb3180e71e330531de8e3eb0356a2ada74883 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-26T04:55:20.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:20 smithi090 podman[132500]: 2024-04-26 04:55:20.273065151 +0000 UTC m=+5.837151836 container cleanup a6f8b02d7e7d2e7ba9e618ee9bbbb3180e71e330531de8e3eb0356a2ada74883 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T04:55:20.471 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:20 smithi090 bash[132500]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2 2024-04-26T04:55:20.898 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:20 smithi090 podman[132521]: 2024-04-26 04:55:20.558052574 +0000 UTC m=+0.641393754 container remove a6f8b02d7e7d2e7ba9e618ee9bbbb3180e71e330531de8e3eb0356a2ada74883 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2) 2024-04-26T04:55:21.222 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:20 smithi090 podman[132615]: 2024-04-26 04:55:20.80088878 +0000 UTC m=+0.018408950 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:21.222 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:20 smithi090 podman[132615]: 2024-04-26 04:55:20.926025946 +0000 UTC m=+0.143546141 container create f3cffedf7d2a6229d7706a54471fcad30b971cdc121315c188a428df3957f7a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-26T04:55:21.222 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:21 smithi090 podman[132615]: 2024-04-26 04:55:21.139426126 +0000 UTC m=+0.356946299 container init f3cffedf7d2a6229d7706a54471fcad30b971cdc121315c188a428df3957f7a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb) 2024-04-26T04:55:21.222 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:21 smithi090 podman[132615]: 2024-04-26 04:55:21.143027759 +0000 UTC m=+0.360547925 container start f3cffedf7d2a6229d7706a54471fcad30b971cdc121315c188a428df3957f7a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, ceph=True, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:55:21.721 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:21 smithi090 podman[132615]: 2024-04-26 04:55:21.222116479 +0000 UTC m=+0.439636640 container attach f3cffedf7d2a6229d7706a54471fcad30b971cdc121315c188a428df3957f7a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240424) 2024-04-26T04:55:21.721 INFO:journalctl@ceph.osd.2.smithi090.stdout:Apr 26 04:55:21 smithi090 podman[132615]: 2024-04-26 04:55:21.437748281 +0000 UTC m=+0.655268433 container died f3cffedf7d2a6229d7706a54471fcad30b971cdc121315c188a428df3957f7a3 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, ceph=True, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-26T04:55:22.054 DEBUG:teuthology.orchestra.run.smithi090:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.2.service' 2024-04-26T04:55:22.091 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:55:22.091 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-26T04:55:22.092 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-26T04:55:22.092 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.3 2024-04-26T04:55:22.261 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:22 smithi104 systemd[1]: Stopping Ceph osd.3 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:55:22.761 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:22 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[74827]: 2024-04-26T04:55:22.397+0000 7f1fdb352700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:55:22.761 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:22 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[74827]: 2024-04-26T04:55:22.397+0000 7f1fdb352700 -1 osd.3 85 *** Got signal Terminated *** 2024-04-26T04:55:22.762 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:22 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3[74827]: 2024-04-26T04:55:22.397+0000 7f1fdb352700 -1 osd.3 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:55:27.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:26 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:26.672+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:27.012 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:26 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:26.639+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:28.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:27 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:27.660+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:28.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:27 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:27.616+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:28.012 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:27 smithi104 podman[102027]: 2024-04-26 04:55:27.742203932 +0000 UTC m=+5.458802724 container died d860fd1e9069b1e6738a4317122f80ef199f3569b3cb29e8a00e7174fb5f7e5a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux ) 2024-04-26T04:55:28.341 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:28 smithi104 podman[102027]: 2024-04-26 04:55:28.17267425 +0000 UTC m=+5.889273032 container cleanup d860fd1e9069b1e6738a4317122f80ef199f3569b3cb29e8a00e7174fb5f7e5a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, org.label-schema.schema-version=1.0, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=) 2024-04-26T04:55:28.341 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:28 smithi104 bash[102027]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3 2024-04-26T04:55:28.636 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:28 smithi104 podman[102045]: 2024-04-26 04:55:28.340748157 +0000 UTC m=+0.598916204 container remove d860fd1e9069b1e6738a4317122f80ef199f3569b3cb29e8a00e7174fb5f7e5a (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T04:55:28.911 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:28 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:28.635+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:28.911 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:28 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:28.664+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:28.911 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:28 smithi104 podman[102137]: 2024-04-26 04:55:28.689879957 +0000 UTC m=+0.018126674 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:28.912 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:28 smithi104 podman[102137]: 2024-04-26 04:55:28.799897321 +0000 UTC m=+0.128144022 container create 1aa3adf0e0a6abe8b747b1c77cd10a50e7d192798bd048bbde7d94a591031904 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_CLEAN=True, RELEASE=squid-b22e2eb, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T04:55:29.261 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:29 smithi104 podman[102137]: 2024-04-26 04:55:29.037777941 +0000 UTC m=+0.366024643 container init 1aa3adf0e0a6abe8b747b1c77cd10a50e7d192798bd048bbde7d94a591031904 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:55:29.261 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:29 smithi104 podman[102137]: 2024-04-26 04:55:29.040833394 +0000 UTC m=+0.369080102 container start 1aa3adf0e0a6abe8b747b1c77cd10a50e7d192798bd048bbde7d94a591031904 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424) 2024-04-26T04:55:29.262 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:29 smithi104 podman[102137]: 2024-04-26 04:55:29.118916553 +0000 UTC m=+0.447163283 container attach 1aa3adf0e0a6abe8b747b1c77cd10a50e7d192798bd048bbde7d94a591031904 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True) 2024-04-26T04:55:29.606 INFO:journalctl@ceph.osd.3.smithi104.stdout:Apr 26 04:55:29 smithi104 podman[102137]: 2024-04-26 04:55:29.338246079 +0000 UTC m=+0.666492780 container died 1aa3adf0e0a6abe8b747b1c77cd10a50e7d192798bd048bbde7d94a591031904 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=squid-b22e2eb, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-26T04:55:29.982 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:29 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:29.629+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:29.982 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:29 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:29.643+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:30.002 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.3.service' 2024-04-26T04:55:30.040 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:55:30.040 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-26T04:55:30.040 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-26T04:55:30.041 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.4 2024-04-26T04:55:30.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:30 smithi104 systemd[1]: Stopping Ceph osd.4 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:55:30.761 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:30 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:30.628+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:30.761 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:30 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:30.383+0000 7f43efa96700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:55:30.762 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:30 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:30.383+0000 7f43efa96700 -1 osd.4 85 *** Got signal Terminated *** 2024-04-26T04:55:30.762 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:30 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:30.383+0000 7f43efa96700 -1 osd.4 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:55:30.762 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:30 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:30.594+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:32.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:31 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:31.624+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:32.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:31 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:31.597+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:33.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:32 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:32.622+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:33.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:32 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:32.622+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:06.355995+0000 front 2024-04-26T04:55:06.356046+0000 (oldest deadline 2024-04-26T04:55:32.255943+0000) 2024-04-26T04:55:33.012 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:32 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:32.646+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:34.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:33 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:33.630+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:34.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:33 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:33.630+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:06.355995+0000 front 2024-04-26T04:55:06.356046+0000 (oldest deadline 2024-04-26T04:55:32.255943+0000) 2024-04-26T04:55:34.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:33 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:33.636+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:35.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:34 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:34.648+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:01.055678+0000 front 2024-04-26T04:55:01.055693+0000 (oldest deadline 2024-04-26T04:55:26.355576+0000) 2024-04-26T04:55:35.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:34 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4[80033]: 2024-04-26T04:55:34.648+0000 7f43ec88b700 -1 osd.4 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:06.355995+0000 front 2024-04-26T04:55:06.356046+0000 (oldest deadline 2024-04-26T04:55:32.255943+0000) 2024-04-26T04:55:35.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:34 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:34.600+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:36.011 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:35 smithi104 podman[102253]: 2024-04-26 04:55:35.764345626 +0000 UTC m=+5.487167231 container died fbfd5c1c44eb3edbdbb79985ee001e116d8ba5b05a63812a5a1575e528a86e14 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, org.label-schema.build-date=20240424, org.label-schema.vendor=CentOS) 2024-04-26T04:55:36.012 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:35 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:35.615+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:36.385 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 podman[102253]: 2024-04-26 04:55:36.167364944 +0000 UTC m=+5.890186582 container cleanup fbfd5c1c44eb3edbdbb79985ee001e116d8ba5b05a63812a5a1575e528a86e14 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=squid-b22e2eb, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, io.buildah.version=1.33.5) 2024-04-26T04:55:36.385 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 bash[102253]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4 2024-04-26T04:55:36.709 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 podman[102272]: 2024-04-26 04:55:36.38527676 +0000 UTC m=+0.614931776 container remove fbfd5c1c44eb3edbdbb79985ee001e116d8ba5b05a63812a5a1575e528a86e14 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424) 2024-04-26T04:55:36.709 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:36 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:36.624+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:36.709 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:36 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:36.624+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:09.850005+0000 front 2024-04-26T04:55:09.850033+0000 (oldest deadline 2024-04-26T04:55:35.750047+0000) 2024-04-26T04:55:36.989 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 podman[102359]: 2024-04-26 04:55:36.612120133 +0000 UTC m=+0.017745840 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:36.989 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 podman[102359]: 2024-04-26 04:55:36.727960604 +0000 UTC m=+0.133586306 container create 8d0972c5e17856147b3b374bf1bc0093a16436594d0b5857d542bfc270f2aa90 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.build-date=20240424, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=squid-b22e2eb, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-26T04:55:36.990 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 podman[102359]: 2024-04-26 04:55:36.922328719 +0000 UTC m=+0.327954408 container init 8d0972c5e17856147b3b374bf1bc0093a16436594d0b5857d542bfc270f2aa90 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True) 2024-04-26T04:55:36.990 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 podman[102359]: 2024-04-26 04:55:36.925424652 +0000 UTC m=+0.331050342 container start 8d0972c5e17856147b3b374bf1bc0093a16436594d0b5857d542bfc270f2aa90 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0) 2024-04-26T04:55:36.990 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:36 smithi104 podman[102359]: 2024-04-26 04:55:36.988977975 +0000 UTC m=+0.394603666 container attach 8d0972c5e17856147b3b374bf1bc0093a16436594d0b5857d542bfc270f2aa90 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, GIT_CLEAN=True, org.label-schema.build-date=20240424, ceph=True) 2024-04-26T04:55:37.261 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:37 smithi104 podman[102359]: 2024-04-26 04:55:37.230560253 +0000 UTC m=+0.636185955 container died 8d0972c5e17856147b3b374bf1bc0093a16436594d0b5857d542bfc270f2aa90 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-04-26T04:55:37.768 INFO:journalctl@ceph.osd.4.smithi104.stdout:Apr 26 04:55:37 smithi104 podman[102385]: 2024-04-26 04:55:37.75037085 +0000 UTC m=+0.515225543 container remove 8d0972c5e17856147b3b374bf1bc0093a16436594d0b5857d542bfc270f2aa90 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=squid-b22e2eb, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240424) 2024-04-26T04:55:37.768 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:37 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:37.658+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:37.768 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:37 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:37.658+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:09.850005+0000 front 2024-04-26T04:55:09.850033+0000 (oldest deadline 2024-04-26T04:55:35.750047+0000) 2024-04-26T04:55:37.773 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.4.service' 2024-04-26T04:55:37.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:55:37.811 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-26T04:55:37.811 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-26T04:55:37.811 DEBUG:teuthology.orchestra.run.smithi104:> sudo systemctl stop ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.5 2024-04-26T04:55:38.145 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:37 smithi104 systemd[1]: Stopping Ceph osd.5 for f018e902-0385-11ef-bc93-c7b262605968... 2024-04-26T04:55:38.145 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:38.144+0000 7fdd6ad52700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-26T04:55:38.145 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:38.144+0000 7fdd6ad52700 -1 osd.5 85 *** Got signal Terminated *** 2024-04-26T04:55:38.146 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:38.144+0000 7fdd6ad52700 -1 osd.5 85 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-26T04:55:39.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:38.642+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:39.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:38 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:38.642+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:09.850005+0000 front 2024-04-26T04:55:09.850033+0000 (oldest deadline 2024-04-26T04:55:35.750047+0000) 2024-04-26T04:55:40.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:39 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:39.600+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:40.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:39 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:39.600+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:09.850005+0000 front 2024-04-26T04:55:09.850033+0000 (oldest deadline 2024-04-26T04:55:35.750047+0000) 2024-04-26T04:55:41.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:40 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:40.595+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:41.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:40 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:40.595+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:09.850005+0000 front 2024-04-26T04:55:09.850033+0000 (oldest deadline 2024-04-26T04:55:35.750047+0000) 2024-04-26T04:55:42.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:41 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:41.620+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:42.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:41 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:41.620+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:09.850005+0000 front 2024-04-26T04:55:09.850033+0000 (oldest deadline 2024-04-26T04:55:35.750047+0000) 2024-04-26T04:55:43.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:42 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:42.646+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6806 osd.0 since back 2024-04-26T04:55:03.449300+0000 front 2024-04-26T04:55:03.449385+0000 (oldest deadline 2024-04-26T04:55:26.349147+0000) 2024-04-26T04:55:43.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:42 smithi104 ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5[85396]: 2024-04-26T04:55:42.646+0000 7fdd67b47700 -1 osd.5 85 heartbeat_check: no reply from 172.21.15.90:6814 osd.1 since back 2024-04-26T04:55:09.850005+0000 front 2024-04-26T04:55:09.850033+0000 (oldest deadline 2024-04-26T04:55:35.750047+0000) 2024-04-26T04:55:43.754 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:43 smithi104 podman[102463]: 2024-04-26 04:55:43.494215906 +0000 UTC m=+5.467709676 container died 64a9be8458bb7982ef15220b317e53fb21ce8b72996ae8230db67a255b141854 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, RELEASE=squid-b22e2eb, maintainer=Guillaume Abrioux ) 2024-04-26T04:55:44.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:43 smithi104 podman[102463]: 2024-04-26 04:55:43.919741054 +0000 UTC m=+5.893234818 container cleanup 64a9be8458bb7982ef15220b317e53fb21ce8b72996ae8230db67a255b141854 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.build-date=20240424, io.buildah.version=1.33.5, RELEASE=squid-b22e2eb, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-26T04:55:44.011 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:43 smithi104 bash[102463]: ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5 2024-04-26T04:55:44.511 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:44 smithi104 podman[102493]: 2024-04-26 04:55:44.112685498 +0000 UTC m=+0.616348849 container remove 64a9be8458bb7982ef15220b317e53fb21ce8b72996ae8230db67a255b141854 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5, RELEASE=squid-b22e2eb, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-26T04:55:44.816 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:44 smithi104 podman[102584]: 2024-04-26 04:55:44.521838186 +0000 UTC m=+0.105985520 container create a72deff269524aedfa24907a72ac9ab964b8315d14009bf1d23a9ca14571ff51 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240424, org.label-schema.schema-version=1.0) 2024-04-26T04:55:44.816 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:44 smithi104 podman[102584]: 2024-04-26 04:55:44.433757881 +0000 UTC m=+0.017905199 image pull 87e5280d5ae87e34552d9b448225db981ca5313b28020853b569ec2cb8ee53ae quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6 2024-04-26T04:55:44.816 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:44 smithi104 podman[102584]: 2024-04-26 04:55:44.73283293 +0000 UTC m=+0.316980231 container init a72deff269524aedfa24907a72ac9ab964b8315d14009bf1d23a9ca14571ff51 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, GIT_BRANCH=HEAD, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-04-26T04:55:44.816 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:44 smithi104 podman[102584]: 2024-04-26 04:55:44.736082624 +0000 UTC m=+0.320229927 container start a72deff269524aedfa24907a72ac9ab964b8315d14009bf1d23a9ca14571ff51 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=squid-b22e2eb, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240424, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b) 2024-04-26T04:55:44.816 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:44 smithi104 podman[102584]: 2024-04-26 04:55:44.815737142 +0000 UTC m=+0.399884448 container attach a72deff269524aedfa24907a72ac9ab964b8315d14009bf1d23a9ca14571ff51 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240424, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=squid-b22e2eb, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-04-26T04:55:45.261 INFO:journalctl@ceph.osd.5.smithi104.stdout:Apr 26 04:55:45 smithi104 podman[102584]: 2024-04-26 04:55:45.029783532 +0000 UTC m=+0.613930835 container died a72deff269524aedfa24907a72ac9ab964b8315d14009bf1d23a9ca14571ff51 (image=quay.ceph.io/ceph-ci/ceph@sha256:265587011b499c966b9dbaf00ce5756e85851e17df544df961d9e3203c8e39d6, name=ceph-f018e902-0385-11ef-bc93-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240424, GIT_COMMIT=318fe5928e744a8c3fe408a2f9ccaca3c31b552b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=squid-b22e2eb, ceph=True) 2024-04-26T04:55:45.685 DEBUG:teuthology.orchestra.run.smithi104:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f018e902-0385-11ef-bc93-c7b262605968@osd.5.service' 2024-04-26T04:55:45.726 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-26T04:55:45.726 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-26T04:55:45.726 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f018e902-0385-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T04:55:45.882 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:56:10.386 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f018e902-0385-11ef-bc93-c7b262605968 --force --keep-logs 2024-04-26T04:56:10.540 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:56:25.009 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:56:25.042 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-26T04:56:25.067 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-26T04:56:25.069 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558/remote/smithi090/crash 2024-04-26T04:56:25.070 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/crash -- . 2024-04-26T04:56:25.107 INFO:teuthology.orchestra.run.smithi090.stderr:tar: /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T04:56:25.108 INFO:teuthology.orchestra.run.smithi090.stderr:tar: Error is not recoverable: exiting now 2024-04-26T04:56:25.109 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/crash to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558/remote/smithi104/crash 2024-04-26T04:56:25.110 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/crash -- . 2024-04-26T04:56:25.132 INFO:teuthology.orchestra.run.smithi104.stderr:tar: /var/lib/ceph/f018e902-0385-11ef-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-26T04:56:25.132 INFO:teuthology.orchestra.run.smithi104.stderr:tar: Error is not recoverable: exiting now 2024-04-26T04:56:25.133 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-26T04:56:25.133 DEBUG:teuthology.orchestra.run.smithi090:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v OSD_DOWN | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v MON_DOWN | head -n 1 2024-04-26T04:56:25.192 INFO:tasks.cephadm:Compressing logs... 2024-04-26T04:56:25.192 DEBUG:teuthology.orchestra.run.smithi090:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T04:56:25.235 DEBUG:teuthology.orchestra.run.smithi104:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-26T04:56:25.260 INFO:teuthology.orchestra.run.smithi090.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.log 2024-04-26T04:56:25.260 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-26T04:56:25.260 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.2.log 2024-04-26T04:56:25.260 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.dbkjuk.log 2024-04-26T04:56:25.260 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T04:56:25.260 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.dbkjuk.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mon.smithi090.log 2024-04-26T04:56:25.260 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.1.log 2024-04-26T04:56:25.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.audit.log: /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mon.smithi090.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.0.log 2024-04-26T04:56:25.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T04:56:25.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.ehputv.log 2024-04-26T04:56:25.261 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.cephadm.log: 88.0% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T04:56:25.261 INFO:teuthology.orchestra.run.smithi090.stderr: 85.9% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T04:56:25.267 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T04:56:25.267 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.ehputv.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mgr.smithi090.iwrvwa.log 2024-04-26T04:56:25.267 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-volume.log: 91.4% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T04:56:25.267 INFO:teuthology.orchestra.run.smithi090.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi090.log 2024-04-26T04:56:25.272 INFO:teuthology.orchestra.run.smithi104.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.log 2024-04-26T04:56:25.272 INFO:teuthology.orchestra.run.smithi104.stderr:: No such file or directory 2024-04-26T04:56:25.272 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.audit.log 2024-04-26T04:56:25.272 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.5.log 2024-04-26T04:56:25.272 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.log: /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.3.log 2024-04-26T04:56:25.272 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.4.log 2024-04-26T04:56:25.272 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.cephadm.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi104.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.4.log: /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi104.ndiusl.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi104.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-volume.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr: 85.9% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi104.ndiusl.log: 88.0% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.log.gz 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mgr.smithi104.wwoewi.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi104.wpntdi.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr: 94.1% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi104.log.gz 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mgr.smithi104.wwoewi.log: gzip -5 --verbose -- /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mon.smithi104.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi104.wpntdi.log: 91.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph.audit.log.gz 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T04:56:25.273 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mon.smithi104.log: 94.0% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi104.ndiusl.log.gz 2024-04-26T04:56:25.274 INFO:teuthology.orchestra.run.smithi104.stderr: 91.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T04:56:25.279 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mgr.smithi090.iwrvwa.log: 91.7%gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-26T04:56:25.279 INFO:teuthology.orchestra.run.smithi090.stderr: -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-volume.log.gz 2024-04-26T04:56:25.279 INFO:teuthology.orchestra.run.smithi104.stderr:/var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T04:56:25.280 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi090.log: 94.1% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-client.ceph-exporter.smithi090.log.gz 2024-04-26T04:56:25.306 INFO:teuthology.orchestra.run.smithi090.stderr:/var/log/ceph/cephadm.log: 91.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-26T04:56:25.309 INFO:teuthology.orchestra.run.smithi104.stderr: 89.0% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mgr.smithi104.wwoewi.log.gz 2024-04-26T04:56:25.543 INFO:teuthology.orchestra.run.smithi104.stderr: 92.6% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mon.smithi104.log.gz 2024-04-26T04:56:25.653 INFO:teuthology.orchestra.run.smithi090.stderr: 88.4% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mgr.smithi090.iwrvwa.log.gz 2024-04-26T04:56:26.352 INFO:teuthology.orchestra.run.smithi090.stderr: 90.7% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mon.smithi090.log.gz 2024-04-26T04:56:29.120 INFO:teuthology.orchestra.run.smithi090.stderr: 93.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-26T04:56:29.219 INFO:teuthology.orchestra.run.smithi104.stderr: 93.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-26T04:56:30.262 INFO:teuthology.orchestra.run.smithi090.stderr: 95.0% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.ehputv.log.gz 2024-04-26T04:56:30.278 INFO:teuthology.orchestra.run.smithi090.stderr: 93.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-26T04:56:30.733 INFO:teuthology.orchestra.run.smithi104.stderr: 93.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-26T04:56:31.046 INFO:teuthology.orchestra.run.smithi104.stderr: 93.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-26T04:56:31.146 INFO:teuthology.orchestra.run.smithi090.stderr: 93.5% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-26T04:56:58.766 INFO:teuthology.orchestra.run.smithi104.stderr: 93.2% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi104.wpntdi.log.gz 2024-04-26T04:56:58.768 INFO:teuthology.orchestra.run.smithi104.stderr: 2024-04-26T04:56:58.768 INFO:teuthology.orchestra.run.smithi104.stderr:real 0m33.519s 2024-04-26T04:56:58.768 INFO:teuthology.orchestra.run.smithi104.stderr:user 0m47.345s 2024-04-26T04:56:58.768 INFO:teuthology.orchestra.run.smithi104.stderr:sys 0m1.675s 2024-04-26T04:57:19.853 INFO:teuthology.orchestra.run.smithi090.stderr: 93.0% -- replaced with /var/log/ceph/f018e902-0385-11ef-bc93-c7b262605968/ceph-mds.cephfs.smithi090.dbkjuk.log.gz 2024-04-26T04:57:19.855 INFO:teuthology.orchestra.run.smithi090.stderr: 2024-04-26T04:57:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:real 0m54.608s 2024-04-26T04:57:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:user 1m13.363s 2024-04-26T04:57:19.855 INFO:teuthology.orchestra.run.smithi090.stderr:sys 0m2.396s 2024-04-26T04:57:19.856 INFO:tasks.cephadm:Archiving logs... 2024-04-26T04:57:19.856 DEBUG:teuthology.misc:Transferring archived files from smithi090:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558/remote/smithi090/log 2024-04-26T04:57:19.857 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T04:57:30.828 DEBUG:teuthology.misc:Transferring archived files from smithi104:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558/remote/smithi104/log 2024-04-26T04:57:30.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-26T04:57:38.325 INFO:tasks.cephadm:Removing cluster... 2024-04-26T04:57:38.325 DEBUG:teuthology.orchestra.run.smithi090:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f018e902-0385-11ef-bc93-c7b262605968 --force 2024-04-26T04:57:38.479 INFO:teuthology.orchestra.run.smithi090.stdout:Deleting cluster with fsid: f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:57:38.839 DEBUG:teuthology.orchestra.run.smithi104:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f018e902-0385-11ef-bc93-c7b262605968 --force 2024-04-26T04:57:38.993 INFO:teuthology.orchestra.run.smithi104.stdout:Deleting cluster with fsid: f018e902-0385-11ef-bc93-c7b262605968 2024-04-26T04:57:39.321 INFO:tasks.cephadm:Removing cephadm ... 2024-04-26T04:57:39.322 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T04:57:39.337 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-26T04:57:39.352 INFO:tasks.cephadm:Teardown complete 2024-04-26T04:57:39.352 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-26T04:57:39.364 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-26T04:57:39.364 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T04:57:39.380 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-26T04:57:39.465 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-26T04:57:39.477 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-26T04:57:39.477 DEBUG:teuthology.orchestra.run.smithi090:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:57:39.479 DEBUG:teuthology.orchestra.run.smithi104:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-26T04:57:39.491 INFO:teuthology.orchestra.run.smithi090.stderr:bash: line 1: ntpq: command not found 2024-04-26T04:57:39.493 INFO:teuthology.orchestra.run.smithi104.stderr:bash: line 1: ntpq: command not found 2024-04-26T04:57:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T04:57:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:=============================================================================== 2024-04-26T04:57:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:^* hv01.front.sepia.ceph.com 3 6 377 32 +1113us[+1104us] +/- 42ms 2024-04-26T04:57:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv02.front.sepia.ceph.com 3 8 377 36 -1335us[-1344us] +/- 61ms 2024-04-26T04:57:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 34 -134us[ -143us] +/- 72ms 2024-04-26T04:57:39.496 INFO:teuthology.orchestra.run.smithi090.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:57:39.497 INFO:teuthology.orchestra.run.smithi104.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-26T04:57:39.497 INFO:teuthology.orchestra.run.smithi104.stdout:=============================================================================== 2024-04-26T04:57:39.498 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv01.front.sepia.ceph.com 3 6 377 38 +1066us[+1086us] +/- 42ms 2024-04-26T04:57:39.498 INFO:teuthology.orchestra.run.smithi104.stdout:^* hv02.front.sepia.ceph.com 3 6 377 35 -1398us[-1377us] +/- 61ms 2024-04-26T04:57:39.498 INFO:teuthology.orchestra.run.smithi104.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 35 -153us[ -153us] +/- 72ms 2024-04-26T04:57:39.498 INFO:teuthology.orchestra.run.smithi104.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-26T04:57:39.498 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-26T04:57:39.509 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-26T04:57:39.510 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-26T04:57:39.521 DEBUG:teuthology.orchestra.run.smithi090:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T04:57:39.615 DEBUG:teuthology.orchestra.run.smithi104:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-26T04:57:39.685 DEBUG:teuthology.orchestra.run.smithi090:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T04:57:39.711 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:57:39.712 DEBUG:teuthology.orchestra.run.smithi104:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-26T04:57:39.739 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:57:39.739 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-26T04:57:39.750 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-26T04:57:39.763 INFO:teuthology.task.internal:Duration was 1871.456602 seconds 2024-04-26T04:57:39.764 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-26T04:57:39.775 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-26T04:57:39.777 DEBUG:teuthology.orchestra.run.smithi090:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T04:57:39.780 DEBUG:teuthology.orchestra.run.smithi104:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-26T04:57:39.813 INFO:teuthology.orchestra.run.smithi090.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T04:57:39.815 INFO:teuthology.orchestra.run.smithi104.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-26T04:57:40.059 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-26T04:57:40.060 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi090.front.sepia.ceph.com 2024-04-26T04:57:40.060 DEBUG:teuthology.orchestra.run.smithi090:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T04:57:40.079 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi104.front.sepia.ceph.com 2024-04-26T04:57:40.079 DEBUG:teuthology.orchestra.run.smithi104:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-26T04:57:40.123 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-26T04:57:40.123 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T04:57:40.125 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-26T04:57:40.187 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-26T04:57:40.187 DEBUG:teuthology.orchestra.run.smithi090:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T04:57:40.190 DEBUG:teuthology.orchestra.run.smithi104:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-26T04:57:40.793 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-26T04:57:40.806 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-26T04:57:40.806 DEBUG:teuthology.orchestra.run.smithi090:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T04:57:40.829 DEBUG:teuthology.orchestra.run.smithi104:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-26T04:57:40.854 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-26T04:57:40.865 DEBUG:teuthology.orchestra.run.smithi090:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:57:40.871 DEBUG:teuthology.orchestra.run.smithi104:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:57:40.894 INFO:teuthology.orchestra.run.smithi090.stdout:kernel.core_pattern = core 2024-04-26T04:57:40.916 INFO:teuthology.orchestra.run.smithi104.stdout:kernel.core_pattern = core 2024-04-26T04:57:40.928 DEBUG:teuthology.orchestra.run.smithi090:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:57:40.956 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:57:40.957 DEBUG:teuthology.orchestra.run.smithi104:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-26T04:57:40.982 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-26T04:57:40.983 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-26T04:57:40.995 INFO:teuthology.task.internal:Transferring archived files... 2024-04-26T04:57:40.997 DEBUG:teuthology.misc:Transferring archived files from smithi090:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558/remote/smithi090 2024-04-26T04:57:40.997 DEBUG:teuthology.orchestra.run.smithi090:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T04:57:41.044 DEBUG:teuthology.misc:Transferring archived files from smithi104:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_02:08:13-upgrade-squid-distro-default-smithi/7674558/remote/smithi104 2024-04-26T04:57:41.045 DEBUG:teuthology.orchestra.run.smithi104:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-26T04:57:41.082 INFO:teuthology.task.internal:Removing archive directory... 2024-04-26T04:57:41.082 DEBUG:teuthology.orchestra.run.smithi090:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T04:57:41.085 DEBUG:teuthology.orchestra.run.smithi104:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-26T04:57:41.129 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-26T04:57:41.142 INFO:teuthology.task.internal:Not uploading archives. 2024-04-26T04:57:41.143 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-26T04:57:41.155 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-26T04:57:41.155 DEBUG:teuthology.orchestra.run.smithi090:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T04:57:41.157 DEBUG:teuthology.orchestra.run.smithi104:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-26T04:57:41.169 INFO:teuthology.orchestra.run.smithi090.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 04:57 /home/ubuntu/cephtest 2024-04-26T04:57:41.183 INFO:teuthology.orchestra.run.smithi104.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 26 04:57 /home/ubuntu/cephtest 2024-04-26T04:57:41.184 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-26T04:57:41.196 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-26T04:57:41.251 INFO:teuthology.run:Summary data: description: upgrade/cephfs/mds_upgrade_sequence/{bluestore-bitmap centos_9.stream conf/{client mds mgr mon osd} fail_fs/yes overrides/{ignorelist_health ignorelist_upgrade ignorelist_wrongly_marked_down pg-warn syntax} roles tasks/{0-from/reef/{v18.2.0} 1-volume/{0-create 1-ranks/2 2-allow_standby_replay/no 3-inline/no 4-verify} 2-client/kclient 3-upgrade-mgr-staggered 4-config-upgrade/{fail_fs} 5-upgrade-with-workload 6-verify}} duration: 1871.456601858139 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-26T04:57:41.251 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-26T04:57:41.334 INFO:teuthology.run:pass